`ignore_tags` parameter not working as expected in OpenMetrics integration

[fabn]

Reference config line: conf.yaml.example#L287

Problem Description

We are using the ignore_tags parameter in our OpenMetrics integration configuration to filter out certain tags, but it appears that the filtering is not being applied correctly.

The Datadog Agent configuration on our dev cluster includes these values for check config

{
  "openmetrics": {
    "instances": [
      {
        "openmetrics_endpoint": "http://%%host%%:9394/metrics",
        "namespace": "sidekiq",
        "raw_metric_prefix": "ruby_sidekiq_",
        "metrics": [
          ".*"
        ],
        "exclude_metrics": [
          "^ruby_.*"
        ],
        "tag_by_endpoint": false,
        "ignore_tags": [
          "aws_*:*",
          "security-group:*",
          "availability-zone:*",
          "image:*",
          "karpenter.*:*",
          "kube_replica_set*",
          "kube_node:*",
          "host:*",
          "name:*"
        ]
      }
    ]
  }
}

However, when running:

agent check openmetrics

We still see tags in the output that should have been filtered out, for example:

=== openmetrics check ===
Configuration provider: kubernetes-container-allinone
Configuration source: container:containerd://e66c6c9952b84113d74de5f51031c98d302212e00c0eebf2ed405c14ba0b2b81
Config for instance ID: openmetrics:sidekiq:93548d936f1896d5
exclude_metrics:
  - ^ruby_.*
ignore_tags:
  - aws_*:*
  - security-group:*
  - availability-zone:*
  - image:*
  - karpenter.*:*
  - kube_replica_set:*
  - kube_node:*
  - host:*
  - name:*
metrics:
  - .*
namespace: sidekiq
openmetrics_endpoint: http://172.32.9.195:9394/metrics
raw_metric_prefix: ruby_sidekiq_
tag_by_endpoint: false
tags:
  - git.commit.sha:"***********************************3d5bc"
  - git.repository_url:https://github.com/1000farmacie/1000farmacie
  - image_id:********@sha256:722ad3b8acc214054b7b2b54586f1675a1c167ddf5ead82a7b4632f3d4cbb183
  - image_name:***********************************************
  - image_tag:MLF-4014-35e7879
  - kube_container_name:metrics-worker
  - kube_deployment:metrics-worker
  - kube_namespace:mlf-4014
  - kube_ownerref_kind:replicaset
  - kube_qos:Burstable
  - kube_replica_set:metrics-worker-6dcc699cc6
  - kube_service:metrics-worker-service-metrics
  - pod_phase:running
  - service:metrics-worker
  - version:MLF-4014

Expected Behavior

Tags matching the patterns specified in ignore_tags should be excluded from the collected metrics at every level

Actual Behavior

Some tags matching the patterns still appear in the collected metrics, maybe the tags added by agent before sending them to datadog?

The output still contains ignored tags even when the configuration appears valid and metrics are reported in our datadog account making the metric cardinality very high.

---

We would like to confirm if ignore_tags is supported for Kubernetes container autodiscovery with OpenMetrics, or if this is a bug.