Allow to customize which Elasticsearch request bodies are captured

[danielmitterdorfer]

Currently, agent implementations have a hardcoded list of Elasticsearch request paths patterns that define when to capture a request body, e.g.

• node-js agent
• Python agent
• Go agent

We use the APM agent to capture requests that Kibana server sends to Elasticsearch and would like to automatically generate Rally benchmarks from these request traces. The problem that we are facing is that request bodies are only captured for a subset of requests against Elasticsearch (basically searches, but also not all of them, e.g. PIT is missing). Instead, we'd like to be able to customize for which requests the agent captures request bodies. We don't mind if this creates a lot of data because we'd only use this feature in a setup we have tight control over.

This also very loosely relates to elastic/apm-agent-nodejs#2019 as the request bodies would be captured in context.db.statement.

[felixbarny]

Which are the URLs you'd like to capture the request body for?

We could either add them to the static list of URLs where agents capture the body or add a config option like elasticsearch_capture_body_urls where the value would be a list of wildcard matchers.

[danielmitterdorfer]

The URLs depend on what requests Kibana sends for these benchmarks and that can change as Kibana adopts new Elasticsearch APIs. Our plan was to cast a wide net so a property like elasticsearch_capture_body_urls would likely be the easiest and most flexible option.