[Request]: Update or remove Transform data with custom ingest pipelines tutorial

[colleenmcginnis]

Description

Related to #4313

With the recent addition of the Elasticsearch apm-data plugin and the updated role of the APM integration, the Transform data with custom ingest pipelines tutorial is no longer accurate. We need to decide if we should update it or remove it.

Resources

#4313

Which documentation set does this change impact?

Stateful only

Feature differences

I think this tutorial only exists in the stateful docs.

What release is this request related to?

8.15

Collaboration model

The documentation team

Point of contact.

Main contact: ?

Stakeholders: @carsonip

[bmorelli25]

@colleenmcginnis to use analytics to inform our decision here. My gut is to change this page.

[colleenmcginnis]

I spent some time with this today. I'm not sure it makes sense to make any changes to the tutorial in the Fleet docs because it's not technically about APM data and it's framed as instructions for one sample scenario:

Scenario: You have Elastic Agents collecting system metrics with the System integration.

We linked to it from the APM docs because there are some helpful concepts. Because the "scenario" explicitly says it assumes the user is working with the System integration, I think it could be confusing to add notes to the existing doc explaining when something doesn't apply to APM.

We could add some additional information in the APM docs where we link to the tutorial warning readers that some parts of it won't be relevant. Or we could create a separate tutorial specifically for APM. What do you think @bmorelli25?

[bmorelli25]

The Transform data with custom ingest pipelines tutorial, explains the following steps:

• Step 1: Create a custom ingest pipeline
• Step 2: Apply your ingest pipeline
• Step 3: Test the ingest pipeline (optional)
• Step 4: Add custom mappings
• Step 5: Test the custom mappings (optional)
• Step 6: Add an ingest pipeline for a data type

In the APM Guide, we have this tutorial: Tutorial: redact sensitive information. It has the following sections:

• Create a pipeline
• Add processors to the pipeline
• Register the pipeline
• Test the pipeline
• Create an @custom pipeline

I'm not sure it makes sense to make any changes to the tutorial in the Fleet docs because it's not technically about APM data and it's framed as instructions for one sample scenario:

I agree. Since the flow of that tutorial is similar to our APM tutorial, it doesn't actually look like we're losing much by removing the link to that content from the APM docs (in PR #4239). What if we just update our existing APM-specific scenario tutorial? It's probably pretty out of date. An update to the formatting/structure and to the content to match the updated role of the APM integration would be groovy.