[BUG] {PayloadSenderV2} Failed sending event error

[NicolasREY69330]

APM Agent version

The version of the Elastic.Apm nuget package used
<PackageReference Include="Elastic.Apm.NetCoreAll" Version="1.12.0" />

Environment

Operating system and version:
Windows 10 + WSL2 (Ubuntu) / Docker Desktop
.NET Framework/Core name and version (e.g. .NET 4.6.1, NET Core 3.1.100) :
.net 6
Application Target Framework(s) (e.g. net461, netcoreapp3.1):
.net 6

Describe the bug

From a background worker, when polling my DB with a simple SELECT * ... using Dapper (2.0.123) I've got the following error :

[14:42:17 ERR] {PayloadSenderV2} Failed sending event. Events intake API absolute URL: http://apm:8200/intake/v2/events. APM Server response: status code: BadRequest, content: 
{"accepted":9,"errors":[{"message":"validation error: span: context: destination: service: 'name' required","document":"{\"span\":{\"action\":\"query\",\"context\":{\"db\":{\"instance\":\"mydb\",\"statement\":\"SELECT TOP (@Max) * FROM [MyTable] WHERE Error = 0 ORDER BY Id ASC\",\"type\":\"sql\"},\"destination\":{\"address\":\"mydbaddress.database.windows.net\",\"port\":1433,\"service\":{\"resource\":\"mssql\"}}},\"duration\":2.0,\"id\":\"4529ac4f63d659b1\",\"name\":\"SELECT TOP (@Max) * FROM [MyTable] WHERE Error = 0 ORDER BY Id ASC\",\"outcome\":\"success\",\"parent_id\":\"bb2a7f2cceb34b23\",\"sample_rate\":1.0,\"subtype\":\"mssql\",\"timestamp\":1636728137224970,\"trace_id\":\"f33f66acdfac8cbe5fc6c1169fcc89ca\",\"transaction_id\":\"bb2a7f2cceb34b23\",\"type\":\"db\"}}"}]}

My compose

version: '3.7'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9200:9200"
    volumes:
      - data01:/usr/share/elasticsearch/data

  apm:
    image: docker.elastic.co/apm/apm-server:7.12.0
    container_name: apm
    cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]
    cap_drop: ["ALL"]
    command: >
       apm-server -e
         -E apm-server.rum.enabled=true
         -E setup.kibana.host=kibana:5601
         -E setup.template.settings.index.number_of_replicas=0
         -E apm-server.kibana.enabled=true
         -E apm-server.kibana.host=kibana:5601
         -E output.elasticsearch.hosts=["elasticsearch:9200"]
    ports:
      - "8200:8200"
    expose:
      - "8200"
    links:
      - elasticsearch
    depends_on:
      - elasticsearch
      - kibana

  kibana:
    image: docker.elastic.co/kibana/kibana:7.12.0
    container_name: kibana
    environment:
      - ELASTICSEARCH_URL=http://elasticsearch:9200
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

To Reproduce

Steps to reproduce the behavior:

Simplified code :

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            // Prevent APM singleton implicit creation with default parameters
            // https://www.elastic.co/guide/en/apm/agent/dotnet/1.x/troubleshooting.html#double-agent-initialization
            await Task.Delay(5 * 1000, stoppingToken);

            using var scope = _scopeFactory.CreateScope();
            var scopedService = scope.ServiceProvider.GetRequiredService<IMyScopedRepository>();

            while (!stoppingToken.IsCancellationRequested)
            {
                var checkPeriodInSeconds = _settings.CurrentValue.CheckPeriodInSeconds;

                await Agent.Tracer
                .CaptureTransaction("GettingRecords", "SqlServer", async (t) =>
                {
                            var myRecords = await scopedService.GetAllAsync(100); 
                            // this is where an error is produced. My repo basically does something like :
                            // connectionString = configuration.GetConnectionString("mySQLServerDB");
                            // const string sql = @"SELECT TOP (@Max) * FROM [MyTable] WHERE Error = 0  ORDER BY Id ASC";
                            // using IDbConnection dbConnection = new SqlConnection(connectionString );
                            // return await dbConnection.QueryAsync<myRecords>(sql, new { Max = max }, commandTimeout: 60 * 3);
                });
            }
                // Delay
                await Task.Delay(checkPeriodInSeconds * 1000, stoppingToken);
        }

Behavior

With previous version (1.11.1) I get no error

[russcam]

The error is

span: context: destination: service: 'name' required"

which looks to be related to #1520. Can you try running against APM server 7.15.2, @NicolasREY69330?

[russcam]

The agent should now be sending span.context.destination.service.name and span.context.destination.service.type as empty strings by default. I'll open a PR to address this.

[NicolasREY69330]

@russcam @gregkalapos thanks well done!