Cross-Cluster Tracing with Unified Trace ID in Multi-Cluster Kubernetes

[linshenfanying]

May I ask, I have multiple Kubernetes clusters, and services between these Kubernetes clusters call each other. To ensure the entire link is connected, meaning a single trace_id can be passed through services across multiple clusters, which of the following solutions can meet the requirement:

1. Deploy an OAP Server in each cluster, but use the same backend storage?
2. Use a shared OAP Server cluster for multiple Kubernetes clusters?

[wu-sheng]

If you are focusing on Java agent, you could use either shared OAP cluster ot dedicated OAP per cluster.
But considering network IP(in-cluster) could be same, you should need to set agent.cluster at agent side.
Ref to https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/configurations/

If you are using more features than just Java agent, e.g. using some MAL functions like, k8s monitoring, you can only using dedicated OAP cluster per cluster.(a)

Further, if you have same service(logic name) deployed in two clusters, a shared OAP cluster, but you can see it has conflicts with (a). You can't have both.

[linshenfanying]

Currently, it is ensured that the same set of code services will be deployed to multiple Kubernetes clusters, but we guarantee that the Service Name is unique in the OAP cluster. In this case, is it better to use the scheme? It can be utilized (e.g., for Kubernetes monitoring through MAL, etc.).

[wu-sheng]

Per OAP, cluster uniqueness is not helpful. In this way, you have to use a shared OAP cluster.
And if you need k8s monitoring, you may need another OAP cluster dedicated for that, but no agent/service trace/metric flow for that.