Add Labeler and instrumentation integration for custom attribute addition to HTTP request duration metrics (Flask, Django, Falcon, WSGI, ASGI)

[tammy-baylis-swi]

Description

Adds new a Python instrumentation Labeler, inspired by Go's net/http instrumentation: open-telemetry/opentelemetry-go-contrib#306. It stores KVs added to it in the current request context. The add / add_attributes can be called by user's instrumented code manually, or by custom distros. The following instrumentors have been updated in this PR so that custom attributes are read from the Labeler in context, then merged into the other attributes (new or old semconv) that are passed to the record calls of the request duration histograms:

1. opentelemetry-instrumentation-flask
2. opentelemetry-instrumentation-django
3. opentelemetry-instrumentation-falcon
4. opentelemetry-instrumentation-wsgi
5. opentelemetry-instrumentation-asgi

Example OTel Collector debug metrics from ASGI OpenTelemetryMiddleware instrumentation, with histograms including custom attributes: https://gist.github.com/tammy-baylis-swi/974ec71d5fb9e88968327b175f2b59a5

Addresses #3702, #3703

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• New unit tests
• Local installation and manual instrumentation of Flask/Falcon/Django app, as per new doc
• Local installation and manual instrumentation of web.py app (no instrumentation) with WSGI
• Local installation and manual instrumentation of Quart app (no instrumentation) with ASGI

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[emdneto]

This would be nice to have 🙌🏻

[emdneto]

If I remove the locks, test_thread_safety doesn't fail, are we missing asserts?

[tammy-baylis-swi]

I did add one new test and updated the existing test in 0bfd192, not sure if that helps.

I tried the same (comment out all with self._lock) and tried adding other tests, but none of them would fail. I'm not sure if there is a way? The tests should be using real, concurrent threads but without something like writing a C extension I don't think GIL can be released.