Skip to content

Commit 7ba7ca1

Browse files
Thiyagu55Thiyagu55
authored
go-sql refactoring (#165)
1 parent b29b610 commit 7ba7ca1

File tree

5 files changed

+70
-147
lines changed

5 files changed

+70
-147
lines changed

‎go/go-sql/README.md‎

Lines changed: 15 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Sqlcommenter [In development]
1+
# go-sql-driver [In development]
22

33
SQLcommenter is a plugin/middleware/wrapper to augment application related information/tags with SQL Statements that can be used later to correlate user code with SQL statements.
44

@@ -7,28 +7,29 @@ SQLcommenter is a plugin/middleware/wrapper to augment application related infor
77
### Install from source
88

99
* Clone the source
10-
* In terminal go inside the client folder location where we need to import google-sqlcommenter package and enter the below commands
10+
* In terminal go inside the client folder location where we need to import sqlcommenter go-sql module and enter the below commands
1111

1212
```shell
13-
go mod edit -replace google.com/sqlcommenter=path/to/google/sqlcommenter/go
13+
go mod edit -replace google.com/sqlcommenter=path/to/google/sqlcommenter/go-sql
1414

1515
go mod tiny
16+
17+
go get google.com/sqlcommenter/gosql
1618
```
1719
### Install from github [To be added]
1820

19-
## Usages
21+
## Usage
2022

21-
### go-sql-driver
22-
Please use the sqlcommenter's default database driver to execute statements. \
23+
Please use the sqlcommenter's default go-sql database driver to execute statements.
2324
Due to inherent nature of Go, the safer way to pass information from framework to database driver is via `context`. So, it is recommended to use the context based methods of `DB` interface like `QueryContext`, `ExecContext` and `PrepareContext`.
2425

2526
```go
26-
db, err := sqlcommenter.Open("<driver>", "<connectionString>", sqlcommenter.CommenterOptions{<tag>:<bool>})
27+
db, err := gosql.Open("<driver>", "<connectionString>", sqlcommenter.CommenterOptions{<tag>:<bool>})
2728
```
2829

29-
#### Configuration
30+
### Configuration
3031

31-
Users are given control over what tags they want to append by using `sqlcommenter.CommenterOptions` struct.
32+
Users are given control over what tags they want to append by using `core.CommenterOptions` struct.
3233

3334
```go
3435
type CommenterOptions struct {
@@ -41,32 +42,15 @@ type CommenterOptions struct {
4142
}
4243
```
4344

44-
### net/http
45-
Populate the request context with sqlcommenter.AddHttpRouterTags(r) function in a custom middleware.
4645

47-
#### Note
48-
* We only support the `database/sql` driver and have provided an implementation for that.
49-
* <b>ORM related tags are added to the driver only when the tags are enabled in the commenter's driver's config and also the request context should passed to the querying functions</b>
46+
### Framework Supported
47+
* [http/net](.../../../http-net/README.md)
5048

51-
#### Example
52-
```go
53-
// middleware is used to intercept incoming HTTP calls and populate request context with commenter tags.
54-
func middleware(next http.Handler) http.Handler {
55-
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
56-
ctx := sqlcommenter.AddHttpRouterTags(r, next)
57-
next.ServeHTTP(w, r.WithContext(ctx))
58-
})
59-
}
60-
```
6149

6250
## Options
6351

6452
With Go SqlCommenter, we have configuration to choose which tags to be appended to the comment.
6553

66-
| Options | Included by default? | go-sql-orm | net/http | Notes |
67-
| --------------- | :------------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---: |
68-
| `DBDriver` | | [ go-sql-driver](https://pkg.go.dev/database/sql/driver) | |
69-
| `Action` | | | [net/http handle](https://pkg.go.dev/net/http#Handle) | |
70-
| `Route` | | | [net/http routing path](https://pkg.go.dev/github.com/gorilla/mux#Route.URLPath) | |
71-
| `Framework` | | | [net/http](https://pkg.go.dev/net/http) | |
72-
| `Opentelemetry` | | [W3C TraceContext.Traceparent](https://www.w3.org/TR/trace-context/#traceparent-field), [W3C TraceContext.Tracestate](https://www.w3.org/TR/trace-context/#tracestate-field) | [W3C TraceContext.Traceparent](https://www.w3.org/TR/trace-context/#traceparent-field), [W3C TraceContext.Tracestate](https://www.w3.org/TR/trace-context/#tracestate-field) | |
54+
| Options | Included by default? | go-sql-driver |
55+
| ---------- | -------------------- | -------------------------------------------------------- |
56+
| `DBDriver` | | [ go-sql-driver](https://pkg.go.dev/database/sql/driver) |

‎go/go-sql/go-sql.go‎

Lines changed: 13 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -18,40 +18,17 @@ import (
1818
"context"
1919
"database/sql"
2020
"fmt"
21-
"net/http"
22-
"net/url"
23-
"reflect"
24-
"runtime"
25-
"sort"
2621
"strings"
2722

28-
"go.opentelemetry.io/otel/propagation"
29-
)
30-
31-
const (
32-
route string = "route"
33-
controller string = "controller"
34-
action string = "action"
35-
framework string = "framework"
36-
driver string = "driver"
37-
traceparent string = "traceparent"
23+
"github.com/google/sqlcommenter/go/core"
3824
)
3925

4026
type DB struct {
4127
*sql.DB
42-
options CommenterOptions
43-
}
44-
45-
type CommenterOptions struct {
46-
EnableDBDriver bool
47-
EnableRoute bool
48-
EnableFramework bool
49-
EnableController bool
50-
EnableAction bool
51-
EnableTraceparent bool
28+
options core.CommenterOptions
5229
}
5330

54-
func Open(driverName string, dataSourceName string, options CommenterOptions) (*DB, error) {
31+
func Open(driverName string, dataSourceName string, options core.CommenterOptions) (*DB, error) {
5532
db, err := sql.Open(driverName, dataSourceName)
5633
return &DB{DB: db, options: options}, err
5734
}
@@ -88,53 +65,41 @@ func (db *DB) PrepareContext(ctx context.Context, query string) (*sql.Stmt, erro
8865

8966
// ***** Query Functions *****
9067

91-
// ***** Framework Functions *****
92-
93-
func AddHttpRouterTags(r *http.Request, next any) context.Context { // any type is set because we need to refrain from importing http-router package
94-
ctx := context.Background()
95-
ctx = context.WithValue(ctx, route, r.URL.Path)
96-
ctx = context.WithValue(ctx, action, getFunctionName(next))
97-
ctx = context.WithValue(ctx, framework, "net/http")
98-
return ctx
99-
}
100-
101-
// ***** Framework Functions *****
102-
10368
// ***** Commenter Functions *****
10469

10570
func (db *DB) withComment(ctx context.Context, query string) string {
10671
var commentsMap = map[string]string{}
10772
query = strings.TrimSpace(query)
10873

10974
// Sorted alphabetically
110-
if db.options.EnableAction && (ctx.Value(action) != nil) {
111-
commentsMap[action] = ctx.Value(action).(string)
75+
if db.options.EnableAction && (ctx.Value(core.Action) != nil) {
76+
commentsMap[core.Action] = ctx.Value(core.Action).(string)
11277
}
11378

11479
// `driver` information should not be coming from framework.
11580
// So, explicitly adding that here.
11681
if db.options.EnableDBDriver {
117-
commentsMap[driver] = "database/sql"
82+
commentsMap[core.Driver] = "database/sql"
11883
}
11984

120-
if db.options.EnableFramework && (ctx.Value(framework) != nil) {
121-
commentsMap[framework] = ctx.Value(framework).(string)
85+
if db.options.EnableFramework && (ctx.Value(core.Framework) != nil) {
86+
commentsMap[core.Framework] = ctx.Value(core.Framework).(string)
12287
}
12388

124-
if db.options.EnableRoute && (ctx.Value(route) != nil) {
125-
commentsMap[route] = ctx.Value(route).(string)
89+
if db.options.EnableRoute && (ctx.Value(core.Route) != nil) {
90+
commentsMap[core.Route] = ctx.Value(core.Route).(string)
12691
}
12792

12893
if db.options.EnableTraceparent {
129-
carrier := extractTraceparent(ctx)
94+
carrier := core.ExtractTraceparent(ctx)
13095
if val, ok := carrier["traceparent"]; ok {
131-
commentsMap[traceparent] = val
96+
commentsMap[core.Traceparent] = val
13297
}
13398
}
13499

135100
var commentsString string = ""
136101
if len(commentsMap) > 0 { // Converts comments map to string and appends it to query
137-
commentsString = fmt.Sprintf("/*%s*/", convertMapToComment(commentsMap))
102+
commentsString = fmt.Sprintf("/*%s*/", core.ConvertMapToComment(commentsMap))
138103
}
139104

140105
// A semicolon at the end of the SQL statement means the query ends there.
@@ -146,45 +111,3 @@ func (db *DB) withComment(ctx context.Context, query string) string {
146111
}
147112

148113
// ***** Commenter Functions *****
149-
150-
// ***** Util Functions *****
151-
152-
func encodeURL(k string) string {
153-
return url.QueryEscape(string(k))
154-
}
155-
156-
func getFunctionName(i interface{}) string {
157-
return runtime.FuncForPC(reflect.ValueOf(i).Pointer()).Name()
158-
}
159-
160-
func convertMapToComment(tags map[string]string) string {
161-
var sb strings.Builder
162-
i, sz := 0, len(tags)
163-
164-
//sort by keys
165-
sortedKeys := make([]string, 0, len(tags))
166-
for k := range tags {
167-
sortedKeys = append(sortedKeys, k)
168-
}
169-
sort.Strings(sortedKeys)
170-
171-
for _, key := range sortedKeys {
172-
if i == sz-1 {
173-
sb.WriteString(fmt.Sprintf("%s=%v", encodeURL(key), encodeURL(tags[key])))
174-
} else {
175-
sb.WriteString(fmt.Sprintf("%s=%v,", encodeURL(key), encodeURL(tags[key])))
176-
}
177-
i++
178-
}
179-
return sb.String()
180-
}
181-
182-
func extractTraceparent(ctx context.Context) propagation.MapCarrier {
183-
// Serialize the context into carrier
184-
propgator := propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{})
185-
carrier := propagation.MapCarrier{}
186-
propgator.Inject(ctx, carrier)
187-
return carrier
188-
}
189-
190-
// ***** Util Functions *****

‎go/go-sql/go-sql_test.go‎

Lines changed: 34 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,17 @@
1+
// Copyright 2022 Google LLC
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
115
package gosql
216

317
import (
@@ -7,20 +21,18 @@ import (
721
"testing"
822

923
"github.com/DATA-DOG/go-sqlmock"
24+
"github.com/google/sqlcommenter/go/core"
25+
httpnet "github.com/google/sqlcommenter/go/net/http"
1026
"go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
1127
sdktrace "go.opentelemetry.io/otel/sdk/trace"
1228
)
1329

14-
var engine, connectionParams = "mysql", "root:root@/gotest"
15-
1630
func TestDisabled(t *testing.T) {
1731
mockDB, _, err := sqlmock.New()
18-
19-
db := DB{DB: mockDB, options: CommenterOptions{}}
2032
if err != nil {
2133
t.Fatalf("MockSQL failed with unexpected error: %s", err)
2234
}
23-
35+
db := DB{DB: mockDB, options: core.CommenterOptions{}}
2436
query := "SELECT 2"
2537
if got, want := db.withComment(context.Background(), query), query; got != want {
2638
t.Errorf("db.withComment(context.Background(), %q) = %q, want = %q", query, got, want)
@@ -29,59 +41,61 @@ func TestDisabled(t *testing.T) {
2941

3042
func TestHTTP_Net(t *testing.T) {
3143
mockDB, _, err := sqlmock.New()
32-
33-
db := DB{DB: mockDB, options: CommenterOptions{EnableDBDriver: true, EnableRoute: true, EnableFramework: true}}
3444
if err != nil {
3545
t.Fatalf("MockSQL failed with unexpected error: %s", err)
3646
}
3747

38-
r, _ := http.NewRequest("GET", "hello/1", nil)
39-
ctx := AddHttpRouterTags(r, context.Background())
48+
db := DB{DB: mockDB, options: core.CommenterOptions{EnableDBDriver: true, EnableRoute: true, EnableFramework: true}}
49+
r, err := http.NewRequest("GET", "hello/1", nil)
50+
if err != nil {
51+
t.Errorf("http.NewRequest('GET', 'hello/1', nil) returned unexpected error: %v", err)
52+
}
4053

54+
ctx := core.ContextInject(r.Context(), httpnet.NewHTTPRequestExtractor(r, nil))
4155
got := db.withComment(ctx, "Select 1")
4256
want := "Select 1/*driver=database%2Fsql,framework=net%2Fhttp,route=hello%2F1*/"
43-
4457
if got != want {
45-
t.Errorf("got %q, wanted %q", got, want)
58+
t.Errorf("db.withComment(ctx, 'Select 1') got %q, wanted %q", got, want)
4659
}
4760
}
4861

4962
func TestQueryWithSemicolon(t *testing.T) {
5063
mockDB, _, err := sqlmock.New()
51-
52-
db := DB{DB: mockDB, options: CommenterOptions{EnableDBDriver: true}}
5364
if err != nil {
5465
t.Fatalf("MockSQL failed with unexpected error: %s", err)
5566
}
67+
68+
db := DB{DB: mockDB, options: core.CommenterOptions{EnableDBDriver: true}}
5669
got := db.withComment(context.Background(), "Select 1;")
5770
want := "Select 1/*driver=database%2Fsql*/;"
58-
5971
if got != want {
60-
t.Errorf("got %q, wanted %q", got, want)
72+
t.Errorf("db.withComment(context.Background(), 'Select 1;') got %q, wanted %q", got, want)
6173
}
6274
}
6375

6476
func TestOtelIntegration(t *testing.T) {
6577
mockDB, _, err := sqlmock.New()
66-
67-
db := DB{DB: mockDB, options: CommenterOptions{EnableTraceparent: true}}
6878
if err != nil {
6979
t.Fatalf("MockSQL failed with unexpected error: %s", err)
7080
}
7181

82+
db := DB{DB: mockDB, options: core.CommenterOptions{EnableTraceparent: true}}
7283
exp, _ := stdouttrace.New(stdouttrace.WithPrettyPrint())
7384
bsp := sdktrace.NewSimpleSpanProcessor(exp) // You should use batch span processor in prod
7485
tp := sdktrace.NewTracerProvider(
7586
sdktrace.WithSampler(sdktrace.AlwaysSample()),
7687
sdktrace.WithSpanProcessor(bsp),
7788
)
78-
7989
ctx, _ := tp.Tracer("").Start(context.Background(), "parent-span-name")
8090

8191
got := db.withComment(ctx, "Select 1;")
82-
r, _ := regexp.Compile("Select 1/\\*traceparent=\\d{1,2}-[a-zA-Z0-9_]{32}-[a-zA-Z0-9_]{16}-\\d{1,2}\\*/;")
92+
wantRegex := "Select 1/\\*traceparent=\\d{1,2}-[a-zA-Z0-9_]{32}-[a-zA-Z0-9_]{16}-\\d{1,2}\\*/;"
93+
r, err := regexp.Compile(wantRegex)
94+
if err != nil {
95+
t.Errorf("regex.Compile() failed with error: %v", err)
96+
}
8397

8498
if !r.MatchString(got) {
85-
t.Errorf("got %q", got)
99+
t.Errorf("%q does not match the given regex %q", got, wantRegex)
86100
}
87101
}

‎go/go-sql/go.mod‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@ module google.com/sqlcommenter/gosql
22

33
go 1.19
44

5-
require (
6-
go.opentelemetry.io/otel v1.10.0
7-
go.opentelemetry.io/otel/sdk v1.10.0
8-
)
5+
require go.opentelemetry.io/otel/sdk v1.10.0
96

107
require (
118
github.com/go-logr/logr v1.2.3 // indirect
129
github.com/go-logr/stdr v1.2.2 // indirect
10+
github.com/google/sqlcommenter/go/core v0.0.1-beta // indirect
11+
go.opentelemetry.io/otel v1.10.0 // indirect
1312
golang.org/x/sys v0.0.0-20220927170352-d9d178bc13c6 // indirect
1413
)
1514

1615
require (
1716
github.com/DATA-DOG/go-sqlmock v1.5.0
17+
github.com/google/sqlcommenter/go/net/http v0.0.1-beta
1818
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.10.0
1919
go.opentelemetry.io/otel/trace v1.10.0 // indirect
2020
)

‎go/go-sql/go.sum‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbV
77
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
88
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
99
github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
10+
github.com/google/sqlcommenter/go/core v0.0.1-beta h1:IVszEHanWVeS7UcmP8C3SHa57CmfeqMBj0QUcJ8VZ9Q=
11+
github.com/google/sqlcommenter/go/core v0.0.1-beta/go.mod h1:CZfcqmbIxngExnZ7Se6AsKNVubZhKyi54aeDJZiqTMQ=
12+
github.com/google/sqlcommenter/go/net/http v0.0.1-beta h1:7XQ6poZv+ZJwwHWQHlesq9IMsRus3G6Z9n10qAkrGqE=
13+
github.com/google/sqlcommenter/go/net/http v0.0.1-beta/go.mod h1:tVUqM1YZ/K3eRTdGzeav1GSbw+BXNdTGzSAbLW9CxAc=
1014
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
1115
github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
1216
go.opentelemetry.io/otel v1.10.0 h1:Y7DTJMR6zs1xkS/upamJYk0SxxN4C9AqRd77jmZnyY4=
@@ -17,8 +21,6 @@ go.opentelemetry.io/otel/sdk v1.10.0 h1:jZ6K7sVn04kk/3DNUdJ4mqRlGDiXAVuIG+MMENpT
1721
go.opentelemetry.io/otel/sdk v1.10.0/go.mod h1:vO06iKzD5baltJz1zarxMCNHFpUlUiOy4s65ECtn6kE=
1822
go.opentelemetry.io/otel/trace v1.10.0 h1:npQMbR8o7mum8uF95yFbOEJffhs1sbCOfDh8zAJiH5E=
1923
go.opentelemetry.io/otel/trace v1.10.0/go.mod h1:Sij3YYczqAdz+EhmGhE6TpTxUO5/F/AzrK+kxfGqySM=
20-
golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7 h1:iGu644GcxtEcrInvDsQRCwJjtCIOlT2V7IRt6ah2Whw=
21-
golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
2224
golang.org/x/sys v0.0.0-20220927170352-d9d178bc13c6 h1:cy1ko5847T/lJ45eyg/7uLprIE/amW5IXxGtEnQdYMI=
2325
golang.org/x/sys v0.0.0-20220927170352-d9d178bc13c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
2426
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=

0 commit comments

Comments
 (0)