Skip to content

[kbn-evals] Tracing not initialised: all trace IDs null #32

Description

@milistu

Summary

init_tracing() is never called by the SDK. Without it, every ingested score has trace_id=null, and every Kibana evaluator short-circuits to label="unavailable". Running an experiment with default settings produces no usable evaluation results.

Problem

  • init_tracing() is only called in example scripts, never from run_experiment()
  • evaluators/kibana.py:226-231 returns "unavailable" when trace_id is None
  • Nothing in the docs warns users about this

Fix

Call init_tracing(config.tracing) automatically in ElasticEvalsClient.__init__. TracingConfig.enabled already exists as the opt-out: setting ELASTIC_EVALS_TRACING_ENABLED=false in the environment disables tracing before the client is constructed. No new API parameter is needed.

Done when

  • Running run_experiment() without calling init_tracing() first does not produce trace_id=null scores
  • The opt-out (ELASTIC_EVALS_TRACING_ENABLED=false) is documented

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team:nightshift-context-and-researchNightshift Context & Research Teamkbn-evalsIssue related to the work on Kibana's LLM evaluation framework.

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions