New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

End to end test to validate functioning of Pythia all-in-one stack #20

Open

cloudronin opened this issue Sep 30, 2024 · 0 comments

Assignees

Milestone

Member

cloudronin commented Sep 30, 2024

We need to create an end to end test that can ensure that the following flow works

Client App making LLM calls instrumented and sending traces
Traces are saved in Jaeger properly with the right service name
Pythia is able to see the new incoming traces (once an only once)
Pythia is able to run the configured validators (and only those) on the incoming traces
Pythia is able to create metrics from the incoming traces
Pythia is able to write the new metrics into Prometheus
Grafana is able to read the Prometheus metrics

cloudronin added this to the v0.1 milestone

cloudronin assigned pratacosmin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment