Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cosmos DB support #1239

Closed
wants to merge 5 commits into from
Closed

Add Cosmos DB support #1239

wants to merge 5 commits into from

Conversation

gregkalapos
Copy link
Contributor

@gregkalapos gregkalapos commented Mar 22, 2021

Closes #1154

@gregkalapos gregkalapos self-assigned this Mar 22, 2021
@apmmachine
Copy link
Contributor

apmmachine commented Mar 22, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Branch indexing

  • Start Time: 2021-04-09T02:20:23.433+0000

  • Duration: 67 min 43 sec

  • Commit: 900763b

Test stats 🧪

Test Results
Failed 0
Passed 19246
Skipped 85
Total 19331

Trends 🧪

Image of Build Times

Image of Tests

@AlexanderWert AlexanderWert added this to the 7.13 milestone Apr 1, 2021
@russcam russcam added enhancement New feature or request v1.10.0 labels Apr 8, 2021
@AlexanderWert AlexanderWert modified the milestones: 7.13, 7.14 May 3, 2021
* CI: use the withAzureCredentials step (#1240)

* Always start a new Activity, if we're not ignoring activities. (#1228)

This commit always starts a new Activity, if we're not ignoring
activities.

If DistributedTracingData has been supplied, try to set the
ParentId of the created activity using it.

* Propagate sample rate through tracestate (#1147)

This commit propagates the sample rate of the
APM agent through the tracestate header, and
uses the sample rate received in incoming tracestate
as the sample rate for non-root transactions and spans.

Introduce TraceState type to model tracestate,
allowing incoming headers to be added and sample
rate to be set. Move TraceState validation from
TraceContext onto TraceState.

Change Transaction and Span SampleRate to double?
to allow a value to be omitted when incoming
Trace context does not contain a tracestate header
or the header does not contain the es vendor s attribute for
a non-root transaction.

When an activity is created when a transaction is started, use the tracestate
from the activity, if present, but set the sample rate from the sampler and
mutate tracestate if the es vendor is present. A transaction created without
explicitly passed DistributedTracingData is considered to be a root transaction,
so the sample rate is governed by the sampler.

Closes #1021

* Wait and retry applying IIS changes (#1245)

This commit waits 500 milliseconds between retrying IIS changes, 
and tries for up to 10 seconds.

* Change NetCoreFact to NetCoreAndNet5Fact (#1243)

With this tests marked with NetCoreAndNet5Fact will run on .NET 5 (and on .NET Core)

* Bump version to 1.9.0 and add changelog (#1249)

* Add Azure App Service Metadata gherkin spec (#1221)

This commit adds the scenario implementations
for testing the collection of Azure App Service Metadata.

EnvironmentHelper is changed to IEnvironmentVariables,
to introduce a test seam for environment variables, to isolate
the manipulation of environment variables to tests.

Update AzureAppServiceMetadataProvider to handle the
additional test case.

Co-authored-by: Russ Cam <[email protected]>
Co-authored-by: Gergely Kalapos <[email protected]>

* Add instrumentation for Azure Service Bus (#1225)

This commit adds instrumentation for Azure Service Bus when an application is 
using Microsoft.Azure.ServiceBus 3.0.0+ or Azure.Messaging.ServiceBus 7.0.0+ nuget packages.

Two IDiagnosticListener implementations, one for Microsoft.Azure.ServiceBus 
and another for Azure.Messaging.ServiceBus, create transactions and spans for received 
and sent messages:

A new transaction is created when

- one or more messages are received from a queue or topic subscription.
- a message is receive deferred from a queue or topic subscription.

A new span is created when there is a current transaction, and when

- one or more messages are sent to a queue or topic.
- one or more messages are scheduled to a queue or a topic.

The diagnostic events do not expose details about sent or received messages.
The trace ids of messages are exposed but are not currently captured in this implementation.
Messages are often received in batches, and it is possible for each message to have its
own trace id, but the APM implementation does not have a concept for capturing such
data right now. See elastic/apm#122

A terraform template file is used to create a resource group, Azure Service Bus namespace 
resource in the resource group, and set RBAC rules to allow the Service Principal that issues
the creation access to the resources. The Service Principal credentials can are sourced from
a .credentials.json file in the root of the repository for CI, and from an account authenticated
with az for local development. A default location is set within the template, but all variables 
can be passed using standard Terraform input variable conventions.

Closes #1157

* Read recording key in AbstractConfigurationWithEnvFallbackReader (#1252)

This commit fixes a bug whereby the Recording configuration value was reading
from the wrong configuration/environment key.

Fixes #1250

* Azure storage integration (#1247)

This commit provides integrations for Azure storage, 
with the newer Azure.Storage.Blobs, Azure.Storage.Queues 
and Azure.Storage.Files.Shares nuget packages.

Closes #1156
Closes #1155

* synchronize json schema specs (#1257)

Co-authored-by: apmmachine <[email protected]>

* Internalize Newtonsoft.Json (#1241)

* Add Newtonsoft.Json source code and make it compile

* Newtonsoft.Json: apply .editorconfig

* Make Newtonsoft.Json internal

* Newtonsoft.Json: handle nullable

* Fix release build

* Update Loader.cs

Do not load `Newtonsoft.Json.dll`, since it's part of the agent code now.

* Update FilterTests.cs

* Add NOTICE.txt file

* Fix Elastic.Apm.Tests.ServerCertificateTests

* Add compiler constants (#10)

And attribute `Outcome` with `StringEnumConverter`

* Update NOTICE

* Update AzureCredentials.cs

Co-authored-by: Russ Cam <[email protected]>

* synchronize json schema specs (#1260)

Co-authored-by: apmmachine <[email protected]>

* Move error capturing to Error event handler (#1259)

This commit moves the capturing of exceptions to an
Error event handler. This handler runs before an
Application_Error event handler that may be defined
on the HttpApplication derived type in Global.asax.cs,
which may perform some action with errors that affects
collecting them in the EndRequest handler e.g.
calling Server.ClearError();

* Revert "Update dotnet sdk 5.0 docker image (#1149)" (#1262)

This reverts commit 03d963c.

* CosmosDb terraform template

* Update configuration.asciidoc (#1266)

Co-authored-by: Brandon Morelli <[email protected]>

* Avoid array allocation in DiagnosticInitializer (#1268)

This commit avoids the array allocations when creating an instance of
DiagnosticInitializer. All but one instantiation pass a single diagnostic
listener.

* Update configuration.asciidoc (#1270)

* Update CloudProviderSteps.cs (#1269)

* EnableBuffering when extracting request body (#1265)

This commit enables buffering when extracting the request
body, either from form values or from the body stream

* CosmosDb Integration tests

This commit adds integration tests for Microsoft.Azure.Cosmos
and Microsoft.Azure.DocumentDB.Core CosmosDb client packages

* Add MongoDb support (#1215)

* Add Elastic.Apm.MongoDb + tests

This is a port of https://github.com/vhatsura/elastic-apm-mongo

All the initial work was done by @vhatsura

* Use DockerInDocker for CI

* Update Elastic.Apm.SqlClient.Tests.csproj

* Update CI scripts for MongoDb

* Add MongoDb to NetCoreAll

Add documentation, remove unneeded reflection

* Update setup.asciidoc

Document MongoDb

* Use TestEnvironment.Docker.Containers.Mongo

* MongoDb: Adapt namespaces

* Update dotnet.bat

Remove mongo test

* MongoDb Adapt namespace

* Review feedback

* Review feedback

* MongoDb: adapt headers and add notice.txt and license.txt

* Update MongoDiagnosticListener.cs

* Review

* Add NOTICE to Build.props and remove `.txt` extension

* Update deploy.sh

* Add msbuild target to concatenate LICENSE files

* Comment .props and .targets files

* Concatenate license files: add new line between files

* Update supported-technologies.asciidoc

* Implement Error exception cause (#1277)

This commit implements the cause property on
the exception of an APM error. Cause can capture
chained exceptions, which are the chain of
inner exceptions for .NET.

Avoid the allocation of an array and
copying of stacktrace framew when a stacktrace limit
> 0 is specified.

Closes #1267

* Create HTTP spans for known services (#1258)

This commit adds the ability to register IHttpSpanTracers
to be able to create HTTP spans for known services.

The HttpDiagnosticListener enumerates registered tracers
passing details about the HTTP request, which a tracer
can determine whether it is a match for a known service.
If it is a match, the tracer has the opportunity to create a
span. If none of the tracers determine it to be a match,
the HttpDiagnosticListener may create a "normal" HTTP span
for the HTTP request.

Since IHttpSpanTracers are dependent on a subscribed
HttpDiagnosticListener, adding a tracer checks to see if one
is subscribed, and subscribes one if not. When the HttpDiagnosticListener
is subscribed by a tracer, it won't create "normal" HTTP spans
for a HTTP request if none of the tracers determine it to be a match.
Explicitly subscribing HttpDiagnosticListener will create "normal"
HTTP spans, or in the event that a HttpDiagnosticListener is
already subscribed, will set configuration to create "normal" HTTP spans
when a tracer does not match. Such an implementation doesn't expose 
the implementation details of span capture to a user.

* Use DotNet.Testcontainers for SQL Server docker tests (#1272)

This commit uses DotNet.Testcontainers instead
of TestEnvironment.Docker for SQL Server docker tests. 
Version 1.0.0 is used as support for net461 is needed.

This change is made as CI has started to throw an
exception on docker container setup related to
duplicate port mappings (1433 in this case).

* Refactor CosmosDb integration

This commit refactors the CosmosDb integration
as an IHttpSpanTracer implementation

Co-authored-by: Victor Martinez <[email protected]>
Co-authored-by: Gergely Kalapos <[email protected]>
Co-authored-by: apmmachine <[email protected]>
Co-authored-by: apmmachine <[email protected]>
Co-authored-by: Rich Lander <[email protected]>
Co-authored-by: leahleahy <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
Co-authored-by: zhangq <[email protected]>
@gregkalapos
Copy link
Contributor Author

As discussed on zoom, it's better to open a new PR than trying to rebase this.

Opened #1342

gregkalapos added a commit that referenced this pull request Jun 18, 2021
* Add CosmosDb support + tests

Prior PR: #1239

* Create test_resources.tf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[META 410] Instrumentation for CosmosDB
4 participants