Skip to content

Commit

Permalink
Add instrumentation for Azure Service Bus (#1225)
Browse files Browse the repository at this point in the history
This commit adds instrumentation for Azure Service Bus when an application is 
using Microsoft.Azure.ServiceBus 3.0.0+ or Azure.Messaging.ServiceBus 7.0.0+ nuget packages.

Two IDiagnosticListener implementations, one for Microsoft.Azure.ServiceBus 
and another for Azure.Messaging.ServiceBus, create transactions and spans for received 
and sent messages:

A new transaction is created when

- one or more messages are received from a queue or topic subscription.
- a message is receive deferred from a queue or topic subscription.

A new span is created when there is a current transaction, and when

- one or more messages are sent to a queue or topic.
- one or more messages are scheduled to a queue or a topic.

The diagnostic events do not expose details about sent or received messages.
The trace ids of messages are exposed but are not currently captured in this implementation.
Messages are often received in batches, and it is possible for each message to have its
own trace id, but the APM implementation does not have a concept for capturing such
data right now. See elastic/apm#122

A terraform template file is used to create a resource group, Azure Service Bus namespace 
resource in the resource group, and set RBAC rules to allow the Service Principal that issues
the creation access to the resources. The Service Principal credentials can are sourced from
a .credentials.json file in the root of the repository for CI, and from an account authenticated
with az for local development. A default location is set within the template, but all variables 
can be passed using standard Terraform input variable conventions.

Closes #1157
  • Loading branch information
russcam authored Apr 6, 2021
1 parent d9c163f commit 63df486
Show file tree
Hide file tree
Showing 58 changed files with 2,217 additions and 40 deletions.
9 changes: 7 additions & 2 deletions .ci/docker/sdk-linux/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ RUN /bin/bash ./dotnet-install.sh --install-dir "${DOTNET_ROOT}" -version "2.1.5
RUN /bin/bash ./dotnet-install.sh --install-dir "${DOTNET_ROOT}" -version "3.0.103"
RUN /bin/bash ./dotnet-install.sh --install-dir "${DOTNET_ROOT}" -version "3.1.100"


# Install docker
RUN apt update \
&& apt-get -qq install -y apt-transport-https ca-certificates curl \
Expand All @@ -21,4 +20,10 @@ RUN apt update \
&& apt -qq update \
&& apt-get -qq install -y docker-ce docker-ce-cli containerd.io \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/*

# Install terraform
RUN curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add - \
&& apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" \
&& apt-get update \
&& apt-get install terraform
3 changes: 2 additions & 1 deletion .ci/linux/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ declare -a projectsToPublish=(
"Elastic.Apm.Extensions.Hosting"
"Elastic.Apm.GrpcClient"
"Elastic.Apm.Extensions.Logging"
"Elastic.Apm.StackExchange.Redis")
"Elastic.Apm.StackExchange.Redis"
"Elastic.Apm.Azure.ServiceBus")

for project in "${projectsToPublish[@]}"
do
Expand Down
6 changes: 6 additions & 0 deletions .ci/windows/test-tools.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,9 @@ if (!$codecov) {
dotnet tool install -g Codecov.Tool --version 1.2.0
}

# Install terraform
choco install terraform -m -y --no-progress --force -r --version=0.14.8
if ($LASTEXITCODE -ne 0) {
Write-Host "terraform installation failed."
exit 1
}
12 changes: 11 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -340,4 +340,14 @@ html_docs
build/output/

# Generated .NET core sln file
ElasticApmAgent.NetCore.sln
ElasticApmAgent.NetCore.sln

# Terraform configuration state files
.terraform
.terraform.lock.hcl
terraform.tfstate
terraform.tfstate.backup
.terraform.tfstate.lock.info

# Azure credentials file
.credentials.json
21 changes: 21 additions & 0 deletions ElasticApmAgent.sln
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,12 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Elastic.Apm.Extensions.Logg
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elastic.Apm.Extensions.Logging.Tests", "test\Elastic.Apm.Extensions.Logging.Tests\Elastic.Apm.Extensions.Logging.Tests.csproj", "{B235B13F-42AE-42DA-A3C8-20D047F38685}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elastic.Apm.Azure.ServiceBus", "src\Elastic.Apm.Azure.ServiceBus\Elastic.Apm.Azure.ServiceBus.csproj", "{1D43C8C5-4116-45C5-9F4B-56C1D926ED29}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elastic.Apm.Azure.ServiceBus.Tests", "test\Elastic.Apm.Azure.ServiceBus.Tests\Elastic.Apm.Azure.ServiceBus.Tests.csproj", "{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elastic.Apm.Azure.ServiceBus.Sample", "sample\Elastic.Apm.Azure.ServiceBus.Sample\Elastic.Apm.Azure.ServiceBus.Sample.csproj", "{27563B4E-ECB1-4F1B-B9F1-22C2C165B270}"
EndProject
Global
GlobalSection(SharedMSBuildProjectFiles) = preSolution
test\Elastic.Apm.DatabaseTests.Common\Elastic.Apm.DatabaseTests.Common.projitems*{968e1e85-e996-42de-9845-d20dae16165a}*SharedItemsImports = 5
Expand Down Expand Up @@ -324,6 +330,18 @@ Global
{B235B13F-42AE-42DA-A3C8-20D047F38685}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B235B13F-42AE-42DA-A3C8-20D047F38685}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B235B13F-42AE-42DA-A3C8-20D047F38685}.Release|Any CPU.Build.0 = Release|Any CPU
{1D43C8C5-4116-45C5-9F4B-56C1D926ED29}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{1D43C8C5-4116-45C5-9F4B-56C1D926ED29}.Debug|Any CPU.Build.0 = Debug|Any CPU
{1D43C8C5-4116-45C5-9F4B-56C1D926ED29}.Release|Any CPU.ActiveCfg = Release|Any CPU
{1D43C8C5-4116-45C5-9F4B-56C1D926ED29}.Release|Any CPU.Build.0 = Release|Any CPU
{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D}.Release|Any CPU.Build.0 = Release|Any CPU
{27563B4E-ECB1-4F1B-B9F1-22C2C165B270}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{27563B4E-ECB1-4F1B-B9F1-22C2C165B270}.Debug|Any CPU.Build.0 = Debug|Any CPU
{27563B4E-ECB1-4F1B-B9F1-22C2C165B270}.Release|Any CPU.ActiveCfg = Release|Any CPU
{27563B4E-ECB1-4F1B-B9F1-22C2C165B270}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -375,6 +393,9 @@ Global
{9AE4805D-2586-4FA5-A0D0-885264EBC565} = {267A241E-571F-458F-B04C-B6C4DE79E735}
{9BAEEF56-4061-488A-8FB8-28BDBBB26C3D} = {3734A52F-2222-454B-BF58-1BA5C1F29D77}
{B235B13F-42AE-42DA-A3C8-20D047F38685} = {267A241E-571F-458F-B04C-B6C4DE79E735}
{1D43C8C5-4116-45C5-9F4B-56C1D926ED29} = {3734A52F-2222-454B-BF58-1BA5C1F29D77}
{D9CC53B2-5F6B-434B-8689-2350F3A9FB2D} = {267A241E-571F-458F-B04C-B6C4DE79E735}
{27563B4E-ECB1-4F1B-B9F1-22C2C165B270} = {3C791D9C-6F19-4F46-B367-2EC0F818762D}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {69E02FD9-C9DE-412C-AB6B-5B8BECC6BFA5}
Expand Down
96 changes: 96 additions & 0 deletions build/terraform/azure/service_bus/test_resources.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.46.0"
}
}
}

provider "azurerm" {
features {}
}

# configuration is sourced from the following environment variables:
# ARM_CLIENT_ID
# ARM_CLIENT_SECRET
# ARM_SUBSCRIPTION_ID
# ARM_TENANT_ID
#
# See https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret
# for creating a Service Principal and Client Secret
data "azurerm_client_config" "current" {
}

resource "random_uuid" "variables" {
}

variable "resource_group" {
type = string
description = "The name of the resource group to create"
}

variable "location" {
type = string
description = "The Azure location in which to deploy resources"
default = "westus"
}

variable "servicebus_namespace" {
type = string
description = "The name of the servicebus namespace to create"
}

resource "azurerm_resource_group" "servicebus_resource_group" {
name = var.resource_group
location = var.location
}

resource "azurerm_servicebus_namespace" "servicebus_namespace" {
location = azurerm_resource_group.servicebus_resource_group.location
name = var.servicebus_namespace
resource_group_name = azurerm_resource_group.servicebus_resource_group.name
sku = "Standard"
depends_on = [azurerm_resource_group.servicebus_resource_group]
}

# random name to generate for the contributor role assignment
resource "random_uuid" "contributor_role" {
keepers = {
client_id = data.azurerm_client_config.current.client_id
}
}

resource "azurerm_role_assignment" "contributor_role" {
name = random_uuid.contributor_role.result
principal_id = data.azurerm_client_config.current.object_id
role_definition_name = "Contributor"
scope = azurerm_resource_group.servicebus_resource_group.id
depends_on = [azurerm_servicebus_namespace.servicebus_namespace]
}

# random name to generate for the contributor role assignment
resource "random_uuid" "data_owner_role" {
keepers = {
client_id = data.azurerm_client_config.current.client_id
}
}

resource "azurerm_role_assignment" "servicebus_data_owner_role" {
name = random_uuid.data_owner_role.result
principal_id = data.azurerm_client_config.current.object_id
role_definition_name = "Azure Service Bus Data Owner"
scope = azurerm_resource_group.servicebus_resource_group.id
depends_on = [azurerm_servicebus_namespace.servicebus_namespace]
}

# following role assignment, there can be a delay of up to ~1 minute
# for the assignments to propagate in Azure. You may need to introduce
# a wait before using the Azure resources created.

output "connection_string" {
value = azurerm_servicebus_namespace.servicebus_namespace.default_primary_connection_string
description = "The service bus primary connection string"
sensitive = true
}

32 changes: 31 additions & 1 deletion docs/configuration.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -892,10 +892,39 @@ When this setting is `true`, the agent will also add the header `elasticapm-trac
| `true` | Boolean
|============

[[config-messaging]]
=== Messaging configuration options

[float]
[[config-ignore-message-queues]]
==== `IgnoreMessageQueues` (added[1.10])

Used to filter out specific messaging queues/topics/exchanges from being traced. When set, sends-to and receives-from the
specified queues/topics/exchanges will be ignored.

This config accepts a comma separated string of wildcard patterns of queues/topics/exchange names which should be ignored.

The wildcard, `*`, matches zero or more characters, and matching is case insensitive by default.
Prepending an element with `(?-i)` makes the matching case sensitive.
Examples: `/foo/*/bar/*/baz*`, `*foo*`.

[options="header"]
|============
| Default | Type
| <empty string> | String
|============

[options="header"]
|============
| Environment variable name | IConfiguration or Web.config key
| `ELASTIC_APM_IGNORE_MESSAGE_QUEUES` | `ElasticApm:IgnoreMessageQueues`
|============


[[config-stacktrace]]
=== Stacktrace configuration options
[float]

[float]
[[config-application-namespaces]]
==== `ApplicationNamespaces` (added[1.5])

Expand Down Expand Up @@ -1040,6 +1069,7 @@ you must instead set the `LogLevel` for the internal APM logger under the `Loggi
| <<config-excluded-namespaces,`ExcludedNamespaces`>> | No | Stacktrace
| <<config-flush-interval,`FlushInterval`>> | No | Reporter
| <<config-global-labels,`GlobalLabels`>> | No | Core
| <<config-ignore-message-queues,`IgnoreMessageQueues`>> | Yes | Messaging, Performance
| <<config-hostname,`HostName`>> | No | Core
| <<config-log-level,`LogLevel`>> | Yes | Supportability
| <<config-max-batch-event-count,`MaxBatchEventCount`>> | No | Reporter
Expand Down
42 changes: 42 additions & 0 deletions docs/setup.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ On .NET Core the agent also supports auto instrumentation without any code chang
* <<setup-ef6>>
* <<setup-sqlclient>>
* <<setup-stackexchange-redis>>
* <<setup-azure-servicebus>>
* <<setup-general>>

[float]
Expand Down Expand Up @@ -53,6 +54,14 @@ https://www.nuget.org/packages/Elastic.Apm.SqlClient[**Elastic.Apm.SqlClient**]:

This package contains https://www.nuget.org/packages/System.Data.SqlClient[System.Data.SqlClient] and https://www.nuget.org/packages/Microsoft.Data.SqlClient[Microsoft.Data.SqlClient] monitoring related code.

https://www.nuget.org/packages/Elastic.Apm.StackExchange.Redis[**Elastic.Apm.StackExchange.Redis**]::

This packages contains instrumentation to capture spans for commands sent to redis with https://www.nuget.org/packages/StackExchange.Redis/[StackExchange.Redis] package.

https://www.nuget.org/packages/Elastic.Apm.StackExchange.Redis[**Elastic.Apm.Azure.ServiceBus**]::

This packages contains instrumentation to capture transactions and spans for messages sent and received from Azure Service Bus with https://www.nuget.org/packages/Microsoft.Azure.ServiceBus/[Microsoft.Azure.ServiceBus] and https://www.nuget.org/packages/Azure.Messaging.ServiceBus/[Azure.Messaging.ServiceBus] packages.


[[setup-dotnet-net-core]]
=== .NET Core
Expand Down Expand Up @@ -361,6 +370,39 @@ connection.UseElasticApm();
A callback is registered with the `IConnectionMultiplexer` to provide a profiling session for each transaction and span that captures redis commands
sent with `IConnectionMultiplexer`.

[[setup-azure-servicebus]]
=== Azure Service Bus

[float]
==== Quick start

Instrumentation can be enabled for Azure Service Bus by referencing https://www.nuget.org/packages/Elastic.Apm.Azure.ServiceBus[`Elastic.Apm.Azure.ServiceBus`] package and subscribing to diagnostic events
using one of the subscribers:

. If the agent is included by referencing the `Elastic.Apm.NetCoreAll` package, the subscribers will be automatically subscribed with the agent, and no further action is required.
. If you're using `Microsoft.Azure.ServiceBus`, subscribe `MicrosoftAzureServiceBusDiagnosticsSubscriber` with the agent
+
[source, csharp]
----
Agent.Subscribe(new MicrosoftAzureServiceBusDiagnosticsSubscriber());
----
. If you're using `Azure.Messaging.ServiceBus`, subscribe `AzureMessagingServiceBusDiagnosticsSubscriber` with the agent
+
[source, csharp]
----
Agent.Subscribe(new AzureMessagingServiceBusDiagnosticsSubscriber());
----

A new transaction is created when

* one or more messages are received from a queue or topic subscription.
* a message is receive deferred from a queue or topic subscription.

A new span is created when there is a current transaction, and when

* one or more messages are sent to a queue or topic.
* one or more messages are scheduled to a queue or a topic.

[[setup-general]]
=== Other .NET applications

Expand Down
32 changes: 25 additions & 7 deletions docs/supported-technologies.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Streaming is not supported. In practice this means for streaming use-cases the a
|Framework |Supported versions |Supported since agent's version

|gRPC on .NET Core
|2.23.2 and later
|2.23.2+
|1.7
|===

Expand All @@ -78,24 +78,24 @@ We support automatic instrumentation for the following data access technologies.
|Data access technology |Supported versions |Notes |Supported since agent's version

|Entity Framework (EF) Core
|2.x
|2.x+
|A DB span is automatically created for each access to underlying database performed by Entity Framework Core.
|1.0

|Entity Framework (EF) 6
|6.2 and later
|6.2+
|A DB span is automatically created for each access to underlying database performed by Entity Framework 6.
|1.2

| Elasticsearch (Elasticsearch.Net and NEST)
| 7.6.0
| 7.6.0+
| __If you're using 7.10.1 or 7.11.0, upgrade to at least 7.11.1 which fixes a bug in capture__
| 1.6.0
| 1.6

| Redis (StackExchange.Redis)
| 2.0.495
| 2.0.495+
| A DB span is automatically created for each profiled redis command peformed by StackExchange.Redis
| 1.8.0
| 1.8
|===

[float]
Expand All @@ -118,3 +118,21 @@ The spans are named after the schema `<method> <host>`, for example `GET elastic
|1.1

|===

[float]
[[supported-cloud-services]]
=== Cloud services

Automatic instrumentation for the following cloud services

|===
| Cloud services | Supported versions | Notes | Supported since agent's version

| Azure Service Bus
| 3.0.0+ for Microsoft.Azure.ServiceBus,
7.0.0+ for Azure.Messaging.ServiceBus
| A new transaction is created for received and
receive deferred messages. A new span is created for sent and scheduled messages if there's a current transaction.
| 1.9

|===
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Azure.Messaging.ServiceBus" Version="7.0.0" />
<PackageReference Include="Microsoft.Azure.ServiceBus" Version="3.0.0" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\..\src\Elastic.Apm.Azure.ServiceBus\Elastic.Apm.Azure.ServiceBus.csproj" />
</ItemGroup>

</Project>
Loading

0 comments on commit 63df486

Please sign in to comment.