[Journeys] Set traceparent for Playwright #189800

dgieselaar · 2024-08-02T13:51:32Z

Sets the traceparent for Playwright, so the trace from the test runner includes the trace events from the browser and Kibana server.

obltmachine · 2024-08-02T13:51:49Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

dgieselaar · 2024-08-03T13:05:18Z

This needs elastic/apm-agent-rum-js#1521 to really work - currently the browser transaction becomes the root transaction and the parent of the backend transaction/span, and in the Journey setup, that means that you can no longer navigate the full trace from the perspective of the test runner.

…ce-context

dgieselaar · 2024-08-27T07:58:13Z

I have verified elastic/apm-agent-rum-js#1521 works, and the initial page load is associated with the first step from the journey. However, when the page load transaction expires, and new managed transactions are started (e.g. based on an http request or a user interaction event), a new trace ID will be generated, and we can no longer see these events in the trace waterfall. I have reached out to the agent devs to see if there is a solution for that.

…ce-context

dgieselaar · 2024-08-27T10:08:36Z

Pushed a change that uses page.evaluate() to override trace.id and parent.id on every created Transaction. It's a private API so not sure if it's sustainable, will discuss with the agent folks.

afharo · 2024-08-27T18:33:14Z

packages/core/rendering/core-rendering-server-internal/src/get_apm_config.ts

@@ -45,7 +45,8 @@ export const getApmConfig = (requestPath: string) => {
      ...config,
      pageLoadTraceId: traceId,
      pageLoadSampled: sampled,
-      pageLoadSpanId: backendTransaction.ensureParentId(),


Q: should we keep this one and use the other ones only as fallbacks?

pageLoadSpanId inverses the relationship - the RUM page-load transaction becomes the parent of the transaction on the backend. I still don't get what the use case is for this (or maybe I forgot), to me it makes more sense to have the server transaction to render the HTML be the parent of the page-load transaction. What are your thoughts here?

TBH, I'm not familiar with why we did this and what ensureParentId() does. I'm mostly concerned about breaking the current behavior (I'm not sure if it's already broken 😅).

What you're saying makes a lot of sense to me. However, as I was trying to understand what this parameter does, I found this piece of documentation recommending the use of ensureParentId as the pageLoadSpanId on dynamically generated HTML.

Unless anyone is depending on the fact that the RUM transaction is the parent of the backend transaction (I honestly can't think of a use case for this), I think we can safely change this. However, I've reverted this change so we don't have to wait until elastic/apm-agent-rum-js#1521 is merged.

Both serve different purpose, For all APM agents (before Synthetics existed) RUM transaction is always considered the root.

So whenever the HTML is rendered from the Server, the Original Navigation timing span Requesting and Receiving the document will include the timing spent on the backend server which captures the time between requestStart and responseEnd. This is the part where the pageLoadSpanId helps where we can tie the frontend navigation with the backend HTML response generation.

Post Synthetics world, we need a new way to keep the RUM transaction as a child of the Synthetics one since browser is started from the synthetics side (Lots of details are in this issue - [Proposal] Crosslinking Synthetics with APM synthetics#265). This PR feat: add pageLoadParentId configuration apm-agent-rum-js#1521 solves this linking by creating a new option to override the RUM parent trace.

Hope this helps.

…ce-context

kibana-ci · 2024-08-30T07:30:56Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 739c9ea

Failed CI Steps

Metrics [docs]

✅ unchanged

History

💛 Build #230366 was flaky 52dc0c5
💔 Build #230047 failed 011cc4b
💛 Build #229694 was flaky e1311ea
💛 Build #225590 was flaky 35630ea

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…ce-context

dmlemeshko · 2024-09-03T08:47:32Z

posting update for visibility:

After running journeys on CI, we still have an error and missing kibana server traces. Example for dashboard_listing_page journey

We are waiting for the RUM agent PR with required changes to be merged.

vigneshshanmugam · 2024-09-11T19:08:31Z

packages/kbn-journeys/journey/journey_apm_config.ts

@@ -15,7 +15,7 @@ export const JOURNEY_APM_CONFIG = {
  secretToken: APM_PUBLIC_TOKEN,
  active: 'true',
  contextPropagationOnly: 'false',
-  environment: process.env.CI ? 'ci' : 'development',
+  environment: process.env.ELASTIC_APM_ENVIRONMENT || (process.env.CI ? 'ci' : 'development'),


Not sure if its required, ENV variables overrides everything by default on the agent level? If this is for RUM agent, then yes its needed.

yes, it's for the RUM agent

Have you tested it? AFAIK, the APM config function should already do this env-mapping inside Kibana precisely to forward them to the RUM agent.

If we're not doing that, we might want to fix it there instead of in the journey config. If this is something journey-specific. Then I'm all good!

The environment variables are set explicitly to the value that is defined here:

kibana/packages/kbn-journeys/journey/journey_ftr_config.ts

Line 100 in b628770

ELASTIC_APM_ENVIRONMENT: JOURNEY_APM_CONFIG.environment,

@dmlemeshko anything to add here?

vigneshshanmugam · 2024-09-11T19:18:07Z

packages/kbn-journeys/journey/journey_ftr_harness.ts

+            return;
+          }
+          win.traceIdOverrideListenerAttached = true;
+          win.elasticApm!.observe('transaction:start', (tx) => {


there is a high chance this gets changed before transaction:end gets fired. Better would be to do it at that time.

can you elaborate? what gets changed? (this sets the trace and parent id on the current transaction to the active step in the journey)

It was more of a caution, but I guess we need do this on the start to ensure the steps relationship works as expected.

…ce-context

afharo

Just a question FMI.

…ce-context

…ana into journey-ftr-trace-context

vigneshshanmugam · 2025-01-22T22:10:45Z

packages/kbn-journeys/journey/journey_ftr_harness.ts

+            return;
+          }
+          win.traceIdOverrideListenerAttached = true;
+          win.elasticApm!.observe('transaction:start', (tx) => {


It was more of a caution, but I guess we need do this on the start to ensure the steps relationship works as expected.

dgieselaar · 2025-01-23T15:39:57Z

@elasticmachine merge upstream

dmlemeshko

LGTM

dgieselaar · 2025-01-26T08:01:47Z

@elasticmachine merge upstream

dgieselaar · 2025-02-03T14:03:57Z

@elasticmachine merge upstream

dgieselaar · 2025-02-05T16:06:45Z

@elasticmachine merge upstream

kibanamachine · 2025-02-05T17:53:53Z

Starting backport for target branches: 8.18, 9.0

https://github.com/elastic/kibana/actions/runs/13163516215

elasticmachine · 2025-02-05T17:54:11Z

💚 Build Succeeded

Buildkite Build
Commit: e3b6b1e

Metrics [docs]

✅ unchanged

History

💚 Build #272742 succeeded b586c19
💛 Build #269742 was flaky e84cbfa
💚 Build #263944 succeeded bcacb7a

Sets the traceparent for Playwright, so the trace from the test runner includes the trace events from the browser and Kibana server. --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 27893f5)

kibanamachine · 2025-02-05T18:09:56Z

💚 All backports created successfully

Status	Branch	Result
✅	8.18
✅	9.0

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.18`: - [[Journeys] Set traceparent for Playwright (#189800)](#189800)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Dario Gieselaar <[email protected]>

# Backport This will backport the following commits from `main` to `9.0`: - [[Journeys] Set traceparent for Playwright (#189800)](#189800)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Dario Gieselaar <[email protected]>

Sets the traceparent for Playwright, so the trace from the test runner includes the trace events from the browser and Kibana server. --------- Co-authored-by: Elastic Machine <[email protected]>

[Journeys] Set traceparent for Playwright

35630ea

dgieselaar added release_note:skip Skip the PR/issue when compiling release notes v8.16.0 labels Aug 2, 2024

dgieselaar requested review from a team as code owners August 2, 2024 13:51

jbudz approved these changes Aug 2, 2024

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

e1311ea

…ce-context

dgieselaar added 2 commits August 27, 2024 12:07

Override trace id/parent id with page.evaluate()

dc54f76

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

011cc4b

…ce-context

dgieselaar requested a review from a team as a code owner August 27, 2024 10:07

afharo reviewed Aug 27, 2024

View reviewed changes

dgieselaar added 4 commits August 28, 2024 11:20

Fix tests

8a34ca6

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

52dc0c5

…ce-context

Use pageLoadSpanId for now

559dbbd

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

739c9ea

…ce-context

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

77a1983

…ce-context

vigneshshanmugam reviewed Sep 11, 2024

View reviewed changes

dgieselaar added 2 commits January 3, 2025 09:15

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

95891e4

…ce-context

Use pageLoadParentId again

a943791

dgieselaar added v9.0.0 backport:version Backport to applied version labels v8.18.0 and removed v8.16.0 labels Jan 3, 2025

Merge branch 'main' into journey-ftr-trace-context

bcacb7a

afharo reviewed Jan 11, 2025

View reviewed changes

dgieselaar added 4 commits January 14, 2025 12:47

Merge branch 'main' of github.com:elastic/kibana into journey-ftr-tra…

5274935

…ce-context

Clarify commented ES instrumentation

f5db545

Merge branch 'journey-ftr-trace-context' of github.com:dgieselaar/kib…

ec8d26c

…ana into journey-ftr-trace-context

Await initializeApmListener

6253f11

vigneshshanmugam approved these changes Jan 22, 2025

View reviewed changes

Merge branch 'main' into journey-ftr-trace-context

e84cbfa

dmlemeshko approved these changes Jan 24, 2025

View reviewed changes

dgieselaar requested a review from afharo January 24, 2025 13:06

dgieselaar enabled auto-merge (squash) January 24, 2025 13:06

Merge branch 'main' into journey-ftr-trace-context

9bd565d

Merge branch 'main' into journey-ftr-trace-context

b586c19

afharo approved these changes Feb 5, 2025

View reviewed changes

Merge branch 'main' into journey-ftr-trace-context

e3b6b1e

dgieselaar merged commit 27893f5 into elastic:main Feb 5, 2025
10 checks passed

kibanamachine added the v9.1.0 label Feb 5, 2025

kibanamachine mentioned this pull request Feb 5, 2025

[8.18] [Journeys] Set traceparent for Playwright (#189800) #209859

Merged

kibanamachine mentioned this pull request Feb 5, 2025

[9.0] [Journeys] Set traceparent for Playwright (#189800) #209860

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Journeys] Set traceparent for Playwright #189800

[Journeys] Set traceparent for Playwright #189800

dgieselaar commented Aug 2, 2024 •

edited by kibanamachine

Loading

obltmachine commented Aug 2, 2024

dgieselaar commented Aug 3, 2024

dgieselaar commented Aug 27, 2024

dgieselaar commented Aug 27, 2024

afharo Aug 27, 2024

dgieselaar Aug 28, 2024

afharo Aug 28, 2024

dgieselaar Aug 30, 2024

vigneshshanmugam Sep 11, 2024

kibana-ci commented Aug 30, 2024

dmlemeshko commented Sep 3, 2024

vigneshshanmugam Sep 11, 2024

dgieselaar Dec 30, 2024

afharo Jan 11, 2025 •

edited

Loading

dgieselaar Feb 3, 2025

vigneshshanmugam Sep 11, 2024

dgieselaar Jan 14, 2025

vigneshshanmugam Jan 22, 2025

afharo left a comment

vigneshshanmugam Jan 22, 2025

dgieselaar commented Jan 23, 2025

dmlemeshko left a comment

dgieselaar commented Jan 26, 2025

dgieselaar commented Feb 3, 2025

dgieselaar commented Feb 5, 2025

kibanamachine commented Feb 5, 2025

elasticmachine commented Feb 5, 2025

kibanamachine commented Feb 5, 2025

[Journeys] Set traceparent for Playwright #189800

[Journeys] Set traceparent for Playwright #189800

Conversation

dgieselaar commented Aug 2, 2024 • edited by kibanamachine Loading

obltmachine commented Aug 2, 2024

🤖 GitHub comments

dgieselaar commented Aug 3, 2024

dgieselaar commented Aug 27, 2024

dgieselaar commented Aug 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Aug 30, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

History

dmlemeshko commented Sep 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afharo Jan 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afharo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgieselaar commented Jan 23, 2025

dmlemeshko left a comment

Choose a reason for hiding this comment

dgieselaar commented Jan 26, 2025

dgieselaar commented Feb 3, 2025

dgieselaar commented Feb 5, 2025

kibanamachine commented Feb 5, 2025

elasticmachine commented Feb 5, 2025

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Feb 5, 2025

💚 All backports created successfully

Questions ?

dgieselaar commented Aug 2, 2024 •

edited by kibanamachine

Loading

afharo Jan 11, 2025 •

edited

Loading