tracing.md

Tracing

Introduction

Tracing is a very good tool to see how trace how request processing happens in the system. There are various tools to visualize components and delays. Such as Jaeger or Zipkin.

How it works

When you use RESTful, you do not need to write a single line of code for tracing to work.

• When a request is received, Server/Lambda handler puts tracing header information to context parameter. Generates new trace ID if not received any.
• When sending a request, Client functions read tracing information from the context and make a new span.
• Send/receive logs contain compact tracing information. The exact behavior depends on Logrus log level.
• If SetOTel(true, tracerProvider) or SetOTelGrpc("host:4317", 0.01) are called, tracing is based on industry standard OpenTelemetry project. The main difference between the default and OTel is that for OTel you may define an exporter which sends traces to a collector. While the default one just propagates the headers, and relies on a service mesh to report to a collector in timely manner.

OTel can be activated using environment variables OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT instead of using SetOTel functions.

An example, tracing data propagated in variable ctx.

func handle(ctx context.Context, data struct{}) error {
    _, err := restful.NewClient().Post(ctx, "https://example.com", &data, nil)
    return err
}

Headers

RESTful's tracing supports 2 kinds of headers:

• B3 and X-B3-* headers: See Open Zipkin documentation.
• traceparent: See W3C recommendation.