Make opentelemetry_metrics_exporter entrypoint support pull exporters

[aabmass]

Description

Update SDK configuration to handle when opentelemetry_metrics_exporter entrypoint points to a MetricReader, which is how we represent pull exporters e.g. Prometheus.

Fixes #3411

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added new tests cases for configuration components. Will add Prometheus support to fix #2864 and test this out concretely.

Does This PR Require a Contrib Repo Change?

• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated – N/A

[ocelotl]

This is smart, saves us from breaking changes and/or a much more complicated solution 👍

In practical terms, this change is aimed at opentelemetry-instrument users, so I think it would make sense to also add some documentation in the corresponding contrib component so that when --help is used with that command there is some information that lets the user know they can use --metrics-exporter with a metrics reader entry point as well.

[ocelotl]

BTW, I noticed that the OTEL_*_EXPORTER environment variables are defined in the API. Shouldn't they be in the SDK instead?

[aabmass]

BTW, I noticed that the OTEL_*_EXPORTER environment variables are defined in the API. Shouldn't they be in the SDK instead?

Possibly but it would be a breaking change

[aabmass]

In practical terms, this change is aimed at opentelemetry-instrument users, so I think it would make sense to also add some documentation in the corresponding contrib component so that when --help is used with that command there is some information that lets the user know they can use --metrics-exporter with a metrics reader entry point as well.

@ocelotl I took a look at the output of opentelemetry-instrument --help:

--help output

usage: opentelemetry-instrument [-h] [--logs_exporter LOGS_EXPORTER]
                                [--metrics_exporter METRICS_EXPORTER]
                                [--propagators PROPAGATORS]
                                [--context CONTEXT]
                                [--id_generator ID_GENERATOR]
                                [--meter_provider METER_PROVIDER]
                                [--tracer_provider TRACER_PROVIDER]
                                [--traces_exporter TRACES_EXPORTER]
                                [--configurator CONFIGURATOR]
                                [--disabled_instrumentations DISABLED_INSTRUMENTATIONS]
                                [--distro DISTRO]
                                [--attribute_count_limit ATTRIBUTE_COUNT_LIMIT]
                                [--attribute_value_length_limit ATTRIBUTE_VALUE_LENGTH_LIMIT]
                                [--blrp_export_timeout BLRP_EXPORT_TIMEOUT]
                                [--blrp_max_export_batch_size BLRP_MAX_EXPORT_BATCH_SIZE]
                                [--blrp_max_queue_size BLRP_MAX_QUEUE_SIZE]
                                [--blrp_schedule_delay BLRP_SCHEDULE_DELAY]
                                [--bsp_export_timeout BSP_EXPORT_TIMEOUT]
                                [--bsp_max_export_batch_size BSP_MAX_EXPORT_BATCH_SIZE]
                                [--bsp_max_queue_size BSP_MAX_QUEUE_SIZE]
                                [--bsp_schedule_delay BSP_SCHEDULE_DELAY]
                                [--event_attribute_count_limit EVENT_ATTRIBUTE_COUNT_LIMIT]
                                [--experimental_resource_detectors EXPERIMENTAL_RESOURCE_DETECTORS]
                                [--exporter_jaeger_agent_host EXPORTER_JAEGER_AGENT_HOST]
                                [--exporter_jaeger_agent_port EXPORTER_JAEGER_AGENT_PORT]
                                [--exporter_jaeger_agent_split_oversized_batches EXPORTER_JAEGER_AGENT_SPLIT_OVERSIZED_BATCHES]
                                [--exporter_jaeger_certificate EXPORTER_JAEGER_CERTIFICATE]
                                [--exporter_jaeger_endpoint EXPORTER_JAEGER_ENDPOINT]
                                [--exporter_jaeger_grpc_insecure EXPORTER_JAEGER_GRPC_INSECURE]
                                [--exporter_jaeger_password EXPORTER_JAEGER_PASSWORD]
                                [--exporter_jaeger_timeout EXPORTER_JAEGER_TIMEOUT]
                                [--exporter_jaeger_user EXPORTER_JAEGER_USER]
                                [--exporter_otlp_certificate EXPORTER_OTLP_CERTIFICATE]
                                [--exporter_otlp_compression EXPORTER_OTLP_COMPRESSION]
                                [--exporter_otlp_endpoint EXPORTER_OTLP_ENDPOINT]
                                [--exporter_otlp_headers EXPORTER_OTLP_HEADERS]
                                [--exporter_otlp_insecure EXPORTER_OTLP_INSECURE]
                                [--exporter_otlp_logs_certificate EXPORTER_OTLP_LOGS_CERTIFICATE]
                                [--exporter_otlp_logs_compression EXPORTER_OTLP_LOGS_COMPRESSION]
                                [--exporter_otlp_logs_endpoint EXPORTER_OTLP_LOGS_ENDPOINT]
                                [--exporter_otlp_logs_headers EXPORTER_OTLP_LOGS_HEADERS]
                                [--exporter_otlp_logs_insecure EXPORTER_OTLP_LOGS_INSECURE]
                                [--exporter_otlp_logs_protocol EXPORTER_OTLP_LOGS_PROTOCOL]
                                [--exporter_otlp_logs_timeout EXPORTER_OTLP_LOGS_TIMEOUT]
                                [--exporter_otlp_metrics_certificate EXPORTER_OTLP_METRICS_CERTIFICATE]
                                [--exporter_otlp_metrics_client_certificate EXPORTER_OTLP_METRICS_CLIENT_CERTIFICATE]
                                [--exporter_otlp_metrics_client_key EXPORTER_OTLP_METRICS_CLIENT_KEY]
                                [--exporter_otlp_metrics_compression EXPORTER_OTLP_METRICS_COMPRESSION]
                                [--exporter_otlp_metrics_default_histogram_aggregation EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION]
                                [--exporter_otlp_metrics_endpoint EXPORTER_OTLP_METRICS_ENDPOINT]
                                [--exporter_otlp_metrics_headers EXPORTER_OTLP_METRICS_HEADERS]
                                [--exporter_otlp_metrics_insecure EXPORTER_OTLP_METRICS_INSECURE]
                                [--exporter_otlp_metrics_protocol EXPORTER_OTLP_METRICS_PROTOCOL]
                                [--exporter_otlp_metrics_temporality_preference EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE]
                                [--exporter_otlp_metrics_timeout EXPORTER_OTLP_METRICS_TIMEOUT]
                                [--exporter_otlp_protocol EXPORTER_OTLP_PROTOCOL]
                                [--exporter_otlp_timeout EXPORTER_OTLP_TIMEOUT]
                                [--exporter_otlp_traces_certificate EXPORTER_OTLP_TRACES_CERTIFICATE]
                                [--exporter_otlp_traces_compression EXPORTER_OTLP_TRACES_COMPRESSION]
                                [--exporter_otlp_traces_endpoint EXPORTER_OTLP_TRACES_ENDPOINT]
                                [--exporter_otlp_traces_headers EXPORTER_OTLP_TRACES_HEADERS]
                                [--exporter_otlp_traces_insecure EXPORTER_OTLP_TRACES_INSECURE]
                                [--exporter_otlp_traces_protocol EXPORTER_OTLP_TRACES_PROTOCOL]
                                [--exporter_otlp_traces_timeout EXPORTER_OTLP_TRACES_TIMEOUT]
                                [--exporter_prometheus_host EXPORTER_PROMETHEUS_HOST]
                                [--exporter_prometheus_port EXPORTER_PROMETHEUS_PORT]
                                [--exporter_zipkin_endpoint EXPORTER_ZIPKIN_ENDPOINT]
                                [--exporter_zipkin_timeout EXPORTER_ZIPKIN_TIMEOUT]
                                [--link_attribute_count_limit LINK_ATTRIBUTE_COUNT_LIMIT]
                                [--log_level LOG_LEVEL]
                                [--metrics_exemplar_filter METRICS_EXEMPLAR_FILTER]
                                [--metric_export_interval METRIC_EXPORT_INTERVAL]
                                [--metric_export_timeout METRIC_EXPORT_TIMEOUT]
                                [--resource_attributes RESOURCE_ATTRIBUTES]
                                [--service_name SERVICE_NAME]
                                [--span_attribute_count_limit SPAN_ATTRIBUTE_COUNT_LIMIT]
                                [--span_attribute_value_length_limit SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT]
                                [--span_event_count_limit SPAN_EVENT_COUNT_LIMIT]
                                [--span_link_count_limit SPAN_LINK_COUNT_LIMIT]
                                [--traces_sampler TRACES_SAMPLER]
                                [--traces_sampler_arg TRACES_SAMPLER_ARG]
                                [--version]
                                command ...
opentelemetry-instrument automatically instruments a Python program and its

dependencies and then runs the program.
positional arguments:

command               Your Python application.

command_args          Arguments for your application.
options:

-h, --help            show this help message and exit

--logs_exporter LOGS_EXPORTER

--metrics_exporter METRICS_EXPORTER

--propagators PROPAGATORS

--context CONTEXT

--id_generator ID_GENERATOR

--meter_provider METER_PROVIDER

--tracer_provider TRACER_PROVIDER

--traces_exporter TRACES_EXPORTER

--configurator CONFIGURATOR

--disabled_instrumentations DISABLED_INSTRUMENTATIONS

--distro DISTRO

--attribute_count_limit ATTRIBUTE_COUNT_LIMIT

--attribute_value_length_limit ATTRIBUTE_VALUE_LENGTH_LIMIT

--blrp_export_timeout BLRP_EXPORT_TIMEOUT

--blrp_max_export_batch_size BLRP_MAX_EXPORT_BATCH_SIZE

--blrp_max_queue_size BLRP_MAX_QUEUE_SIZE

--blrp_schedule_delay BLRP_SCHEDULE_DELAY

--bsp_export_timeout BSP_EXPORT_TIMEOUT

--bsp_max_export_batch_size BSP_MAX_EXPORT_BATCH_SIZE

--bsp_max_queue_size BSP_MAX_QUEUE_SIZE

--bsp_schedule_delay BSP_SCHEDULE_DELAY

--event_attribute_count_limit EVENT_ATTRIBUTE_COUNT_LIMIT

--experimental_resource_detectors EXPERIMENTAL_RESOURCE_DETECTORS

--exporter_jaeger_agent_host EXPORTER_JAEGER_AGENT_HOST

--exporter_jaeger_agent_port EXPORTER_JAEGER_AGENT_PORT

--exporter_jaeger_agent_split_oversized_batches EXPORTER_JAEGER_AGENT_SPLIT_OVERSIZED_BATCHES

--exporter_jaeger_certificate EXPORTER_JAEGER_CERTIFICATE

--exporter_jaeger_endpoint EXPORTER_JAEGER_ENDPOINT

--exporter_jaeger_grpc_insecure EXPORTER_JAEGER_GRPC_INSECURE

--exporter_jaeger_password EXPORTER_JAEGER_PASSWORD

--exporter_jaeger_timeout EXPORTER_JAEGER_TIMEOUT

--exporter_jaeger_user EXPORTER_JAEGER_USER

--exporter_otlp_certificate EXPORTER_OTLP_CERTIFICATE

--exporter_otlp_compression EXPORTER_OTLP_COMPRESSION

--exporter_otlp_endpoint EXPORTER_OTLP_ENDPOINT

--exporter_otlp_headers EXPORTER_OTLP_HEADERS

--exporter_otlp_insecure EXPORTER_OTLP_INSECURE

--exporter_otlp_logs_certificate EXPORTER_OTLP_LOGS_CERTIFICATE

--exporter_otlp_logs_compression EXPORTER_OTLP_LOGS_COMPRESSION

--exporter_otlp_logs_endpoint EXPORTER_OTLP_LOGS_ENDPOINT

--exporter_otlp_logs_headers EXPORTER_OTLP_LOGS_HEADERS

--exporter_otlp_logs_insecure EXPORTER_OTLP_LOGS_INSECURE

--exporter_otlp_logs_protocol EXPORTER_OTLP_LOGS_PROTOCOL

--exporter_otlp_logs_timeout EXPORTER_OTLP_LOGS_TIMEOUT

--exporter_otlp_metrics_certificate EXPORTER_OTLP_METRICS_CERTIFICATE

--exporter_otlp_metrics_client_certificate EXPORTER_OTLP_METRICS_CLIENT_CERTIFICATE

--exporter_otlp_metrics_client_key EXPORTER_OTLP_METRICS_CLIENT_KEY

--exporter_otlp_metrics_compression EXPORTER_OTLP_METRICS_COMPRESSION

--exporter_otlp_metrics_default_histogram_aggregation EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION

--exporter_otlp_metrics_endpoint EXPORTER_OTLP_METRICS_ENDPOINT

--exporter_otlp_metrics_headers EXPORTER_OTLP_METRICS_HEADERS

--exporter_otlp_metrics_insecure EXPORTER_OTLP_METRICS_INSECURE

--exporter_otlp_metrics_protocol EXPORTER_OTLP_METRICS_PROTOCOL

--exporter_otlp_metrics_temporality_preference EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE

--exporter_otlp_metrics_timeout EXPORTER_OTLP_METRICS_TIMEOUT

--exporter_otlp_protocol EXPORTER_OTLP_PROTOCOL

--exporter_otlp_timeout EXPORTER_OTLP_TIMEOUT

--exporter_otlp_traces_certificate EXPORTER_OTLP_TRACES_CERTIFICATE

--exporter_otlp_traces_compression EXPORTER_OTLP_TRACES_COMPRESSION

--exporter_otlp_traces_endpoint EXPORTER_OTLP_TRACES_ENDPOINT

--exporter_otlp_traces_headers EXPORTER_OTLP_TRACES_HEADERS

--exporter_otlp_traces_insecure EXPORTER_OTLP_TRACES_INSECURE

--exporter_otlp_traces_protocol EXPORTER_OTLP_TRACES_PROTOCOL

--exporter_otlp_traces_timeout EXPORTER_OTLP_TRACES_TIMEOUT

--exporter_prometheus_host EXPORTER_PROMETHEUS_HOST

--exporter_prometheus_port EXPORTER_PROMETHEUS_PORT

--exporter_zipkin_endpoint EXPORTER_ZIPKIN_ENDPOINT

--exporter_zipkin_timeout EXPORTER_ZIPKIN_TIMEOUT

--link_attribute_count_limit LINK_ATTRIBUTE_COUNT_LIMIT

--log_level LOG_LEVEL

--metrics_exemplar_filter METRICS_EXEMPLAR_FILTER

--metric_export_interval METRIC_EXPORT_INTERVAL

--metric_export_timeout METRIC_EXPORT_TIMEOUT

--resource_attributes RESOURCE_ATTRIBUTES

--service_name SERVICE_NAME

--span_attribute_count_limit SPAN_ATTRIBUTE_COUNT_LIMIT

--span_attribute_value_length_limit SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT

--span_event_count_limit SPAN_EVENT_COUNT_LIMIT

--span_link_count_limit SPAN_LINK_COUNT_LIMIT

--traces_sampler TRACES_SAMPLER

--traces_sampler_arg TRACES_SAMPLER_ARG

--version             print version information
Optional arguments (except for --help and --version) for opentelemetry-

instrument directly correspond with OpenTelemetry environment variables. The

corresponding optional argument is formed by removing the OTEL_ or

OTEL_PYTHON_ prefix from the environment variable and lower casing the rest.

For example, the optional argument --attribute_value_length_limit corresponds

with the environment variable OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT. These

optional arguments will override the current value of the corresponding

environment variable during the execution of the command.

It doesn't have any detailed info as the flags are just coming from environment variables. I guess it would make the most sense to just document it along with the environment variable then?

[aabmass]

I updated the sphinx docs for OTEL_METRICS_EXPORTER explaining what's supported and that it's considered experimental

[aabmass]

Test failures look like census-instrumentation/opencensus-python#1219 @lzchen did we do a recent OpenCensus release?

[aabmass]

Fix in this repo for the problem #3433