Logflow

[Depechie]

Can you maybe show some example flow of a request that goes from frontend to backend in regards to logging?

Some context to this question, I have not installed the complete sample and I've not used Elastic before.
So I was wondering how the end result in the dashboard would be if you need to 'trace' the logs in reference to an action the user did on a SPA that touches an API and maybe does call out to other API's or backends.

I see ElasticApmTraceId as being added as a reference id? But not sure if that is carried over to other calls from API to API?

Anyhow, any feedback/sample would be nice :)

I know https://github.com/damikun/trouble-training/blob/main/Doc/Assets/CreateWebHook_Output.png shows this flow, but more details on how this came about is what I'm referring too...

[damikun]

I will add SPA logging correlation as we spoke but I'm bit out of time (but wanna do that) its good idea to include :) I have that in my app but not in workshop since i use custom SDK for that..

In the PNG you post in discussion you can see the tab LOGS there are all logs for this trace available sorted by time... for each operation...

So elastic stack helps a lot with correlation of logs and traces if you follow some rules. (this also apply for non elastic monitoring)

• You need to include span_id and trace_id in logs. (This is bit explained in Logging section of Backend) you need to write this data to your logs using some Enricher or manually for each log. This is needed for correlation and pairing which logs is for what trace..
• In case of elastic you need to flow ECS log format this helps Kibana to query between different logs sources. (all application can have different log format how fields are named or what fields are available) this ECS template ensure you have +- equal log structure between different languages and environments.. (There is also lib for ECS formatted logging in javascript https://www.elastic.co/guide/en/ecs-logging/nodejs/current/intro.html )

If you wanna quickstart logging from frontend you can also use ElasticAPM Agent for javascript (https://www.elastic.co/guide/en/apm/agent/rum-js/current/index.html) but there are some disadvantages as you go to platform specific observability and you lost power of opentelemetry...

In distributed tracing and logging all is builded by refs. All records usually has some root_id and parrentId or transactionId and they are not connected durig request just each app log this and add the context info to this logs.. The analytics software take all traces and logs and merge them to graph and tables based on this ids..

Some languages as .Net have some native support and helps you a lot with context propagation (passing root/traceid betwwen http calls) so some stoff are automatically copyed in your request between services. Unfortunately javascript does not have this support and you need to use some SDK to help you with..

[Depechie]

I guess the SPA need is not as important but more a nice to have in the long run.

Maybe a small follow up... like you stated : All records usually has some root_id and parentId or transactionId.
Can you point out the enrichers in the sample? Seems I'm overlooking the obvious, or maybe they are auto added by the tooling you are using?

[damikun]

So in example logging config I do this:

I use prepared functions from serilog but that's just few lines to do yourself with custom Enricher (log interceptor)

        logCfg = logCfg
            .Enrich.FromLogContext()
            .Enrich.WithExceptionDetails()
            .Enrich.WithMachineName()
            .Enrich.WithSpan() <-- This include span corelation info (parrent_id, root_id)
            .Enrich.WithElasticApmCorrelationInfo() <-- This include APM correlation fields in logs
            .Enrich.WithProperty(
                "Environment",
                  hostEnvironment.EnvironmentName)
            .ReadFrom.Configuration(configuration);

You need to do equal for frontend... All logging Fw supports enricher (interceptors).. I also shows how to do custom enricher using serilog (https://github.com/damikun/trouble-training/blob/main/Doc/Logging.md#custom-enricher) this shows how to add some userId to all logs..

You can do similar for parent and rootid just passing Activity.Current.RootId to specific named field.. Just make sure the field name follow some ECE or opentelemetry or W3C naming...

[damikun]

@Depechie Just for info log correlation will be soon much easier. Normally opentelemety is made to handle (traces, logs, metrics) and collector can already do that...but the logs collecting and exporting by javascript opentelemetry SDK is not ready see the picture you can find this under official github https://github.com/open-telemetry/opentelemetry-js#feature-status and you will export logs equal as traces from SPA without much to do.. but until this comes out only custom enricher or APMAgent is the way...