Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[apmazure] storage queue #1107

Merged
merged 41 commits into from
Aug 24, 2021
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
12ff5c7
add azure blob storage instrumentation
stuartnelson3 Aug 13, 2021
1cc7e97
Use a "real" azure url
stuartnelson3 Aug 16, 2021
33bc894
go mod tidy
stuartnelson3 Aug 16, 2021
dae9391
update dockerfile-testing
stuartnelson3 Aug 16, 2021
1c7d182
update docs
stuartnelson3 Aug 16, 2021
e059db6
Revert "go mod tidy"
stuartnelson3 Aug 16, 2021
7795683
tidy the right mods
stuartnelson3 Aug 16, 2021
069b9e1
update go.mod:wq
stuartnelson3 Aug 17, 2021
735750d
Merge branch 'master' into azure-storage
stuartnelson3 Aug 17, 2021
01f7dea
drop version req
stuartnelson3 Aug 17, 2021
5a92a9c
Add build constraints
stuartnelson3 Aug 17, 2021
ec86c40
add apmazure doc.go
stuartnelson3 Aug 17, 2021
48d00e6
ignore modules with no buildable go files
stuartnelson3 Aug 17, 2021
c0b3aea
Revert "ignore modules with no buildable go files"
stuartnelson3 Aug 17, 2021
829ef0a
Merge branch 'master' into azure-storage
stuartnelson3 Aug 18, 2021
78c15f3
goimports
stuartnelson3 Aug 18, 2021
1c98ac7
remove dead comment
stuartnelson3 Aug 18, 2021
5fb2d47
introduce azureRPC interface
stuartnelson3 Aug 18, 2021
93edb8b
implement using correct blob storage lib
stuartnelson3 Aug 18, 2021
f8f02a4
Update instrumenting.asciidoc
stuartnelson3 Aug 19, 2021
cc953b4
update supported-tech.asciidoc
stuartnelson3 Aug 19, 2021
4bb5dc2
use GetTags method
stuartnelson3 Aug 19, 2021
730e2c5
Merge branch 'azure-storage' into azure-storage-queue
stuartnelson3 Aug 19, 2021
956c13a
create queue in newAzureRPC
stuartnelson3 Aug 19, 2021
922adef
add send and http verb tests
stuartnelson3 Aug 19, 2021
cbe0431
add test for receiving from queue
stuartnelson3 Aug 19, 2021
082da6f
pr comments
stuartnelson3 Aug 20, 2021
3de8927
Merge branch 'master' into azure-storage
stuartnelson3 Aug 20, 2021
fb2e674
Merge branch 'azure-storage' into azure-storage-queue
stuartnelson3 Aug 20, 2021
0f85740
remove comment/test for deprecated features
stuartnelson3 Aug 20, 2021
6cb3918
test span on receive
stuartnelson3 Aug 20, 2021
32b00b3
fix removing / in queue resourcename
stuartnelson3 Aug 20, 2021
4c19eda
use httpsender for testing
stuartnelson3 Aug 23, 2021
4d5638a
Merge branch 'azure-storage' into azure-storage-queue
stuartnelson3 Aug 23, 2021
dfdadce
update instrumenting and supported-tech asciidocs
stuartnelson3 Aug 23, 2021
470797e
Merge branch 'master' into azure-storage-queue
stuartnelson3 Aug 23, 2021
b8cc28c
Update module/apmazure/blob.go
stuartnelson3 Aug 24, 2021
ba1ce72
Update module/apmazure/blob.go
stuartnelson3 Aug 24, 2021
144cc6b
Merge branch 'master' into azure-storage-queue
stuartnelson3 Aug 24, 2021
7e7e879
use fakeSender in queue tests
stuartnelson3 Aug 24, 2021
f839e59
remove comments
stuartnelson3 Aug 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/instrumenting.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ that framework's API. The request context can be used for reporting <<custom-ins
* <<builtin-modules-apmelasticsearch>>
* <<builtin-modules-apmmongo>>
* <<builtin-modules-apmawssdkgo>>
* <<builtin-modules-apmazure>>

[[builtin-modules-apmhttp]]
==== module/apmhttp
Expand Down Expand Up @@ -904,3 +905,40 @@ func (s *server) handleRequest(w http.ResponseWriter, req *http.Request) {
...
}
----

[[builtin-modules-apmazure]]
==== module/apmazure
Package apmazure provides a means of instrumenting the
https://github.com/Azure/azure-pipeline-go[Azure Pipeline Go] pipeline object,
so that Azure requests are reported as spans within the current transaction.

To create spans for Azure requests, you should create a new pipeline using the
relevant service's `NewPipeline` function, like `azblob.NewPipeline`, then wrap
the `pipeline.Pipeline` with `apmazure.WrapPipeline`. The returned `Pipeline`
can be used as normal.

The following services are supported:

- Blob Storage
- Queue Storage


[source,go]
----
import (
"github.com/Azure/azure-storage-blob-go/azblob"

"go.elastic.co/apm/module/apmazure"
)

func main() {
p := azblob.NewPipeline(azblob.NewAnonymousCredential(), po)
p = apmazure.WrapPipeline(p)
u, err := url.Parse("https://my-account.blob.core.windows.net")
serviceURL := azblob.NewServiceURL(*u, p)
containerURL := serviceURL.NewContainerURL("mycontainer")
blobURL := containerURL.NewBlobURL("readme.txt")
// Use the blobURL to interact with Blob Storage
...
}
----
10 changes: 10 additions & 0 deletions docs/supported-tech.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,16 @@ https://github.com/aws/aws-sdk-go[AWS SDK Go].
See <<builtin-modules-apmawssdkgo, module/apmawssdkgo>> for more information
about AWS SDK Go instrumentation.

[float]
==== Azure Storage
We provide instrumentation for Azure Storage. This is usable with:

- github.com/Azure/azure-storage-blob-go/azblob[Azure Blob Storage]
- github.com/Azure/azure-storage-queue-go/azqueue[Azure Queue Storage]

See <<builtin-modules-apmazure, module/apmazure>> for more information
about Azure SDK Go instrumentation.

[float]
[[supported-tech-messaging-systems]]
=== Messaging Systems
Expand Down
180 changes: 180 additions & 0 deletions module/apmazure/blob.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

//go:build go1.14
// +build go1.14

package apmazure // import "go.elastic.co/apm/module/apmazure"

import (
"fmt"
"net/http"
"net/url"
"strings"

"github.com/Azure/azure-pipeline-go/pipeline"
)

type blobRPC struct {
accountName string
resourceName string
req pipeline.Request
}

func (b *blobRPC) name() string {
return fmt.Sprintf("AzureBlob %s %s", b.operation(), b.resourceName)
}

func (b *blobRPC) _type() string {
return "storage"
}

func (b *blobRPC) subtype() string {
return "azureblob"
}

func (b *blobRPC) storageAccountName() string {
return b.accountName
}

func (b *blobRPC) resource() string {
return b.resourceName
}

func (b *blobRPC) operation() string {
if b.req.Method == http.MethodDelete {
return "Delete"
}
q := b.req.URL.Query()
switch b.req.Method {
// From net/http documentation:
// For client requests, an empty string means GET.
case http.MethodGet, "":
return b.getOperation(q)
case http.MethodPost:
return b.postOperation(q)
case http.MethodHead:
return b.headOperation(q)
case http.MethodPut:
return b.putOperation(q, b.req.Header)
default:
return b.req.Method
}
}

func (b *blobRPC) getOperation(v url.Values) string {
restype := v.Get("restype")
comp := v.Get("comp")
if (restype == "" && comp == "") || comp == "blocklist" {
return "Download"
}
if restype == "container" && comp == "" {
return "GetProperties"
}

switch comp {
case "metadata":
return "GetMetadata"
case "acl":
return "GetAcl"
case "list":
if restype == "container" {
return "ListBlobs"
}
return "ListContainers"
case "tags":
if v.Get("where") != "" {
return "FindTags"
}
return "GetTags"
default:
return "unknown operation"
}
}

func (b *blobRPC) postOperation(v url.Values) string {
comp := v.Get("comp")
switch comp {
case "batch":
return "Batch"
case "query":
return "Query"
case "userdelegationkey":
return "GetUserDelegationKey"
default:
return "unknown operation"
}

}
func (b *blobRPC) headOperation(v url.Values) string {
stuartnelson3 marked this conversation as resolved.
Show resolved Hide resolved
restype := v.Get("restype")
comp := v.Get("comp")
if restype == "" && comp == "" {
return "GetProperties"
}
if restype != "container" {
return "unknown operation"
}
switch comp {
case "metadata":
return "GetMetadata"
case "acl":
return "GetAcl"
default:
return "unknown operation"
}
}
func (b *blobRPC) putOperation(v url.Values, h http.Header) string {
stuartnelson3 marked this conversation as resolved.
Show resolved Hide resolved
// header.Get canonicalizes the key, ie. x-ms-copy-source->X-Ms-Copy-Source.
// The headers used are all lowercase, so we access the map directly.
_, copySource := h["x-ms-copy-source"]
_, blobType := h["x-ms-blob-type"]
_, pageWrite := h["x-ms-page-write"]
restype := v.Get("restype")
comp := v.Get("comp")
if restype == "container" && comp == "acl" {
return "SetAcl"
}

if comp == "" && !(copySource || blobType || pageWrite) {
return "Create"
}
if copySource {
return "Copy"
}
if blobType {
return "Upload"
}
if comp == "page" && pageWrite {
return "Clear"
}

switch comp {
case "block", "blocklist", "page", "appendblock":
return "Upload"
case "copy":
return "Abort"
case "metadata":
return "SetMetadata"
case "lease", "snapshot", "undelete", "seal", "rename":
return strings.Title(comp)
case "properties", "tags", "tier", "expiry":
return "Set" + strings.Title(comp)
default:
return "unknown operation"
}
}
Loading