A framework for implementing Continuous Alignment Testing (CAT) for LLM-based applications.
CAT provides the infrastructure to:
- Run and track automated tests against an AI application.
- Store and analyze AI test results over iterations.
- Monitor reliability changes in an AI application as prompts, models, code, and data evolve.
- Integrate AI tests into the CI pipeline.
- Build the AI application as an API service using Python and use any language to invoke it.
- Ensure AI applications are reliable while maintaining creativity.
- Iterate on prompts and code while measuring improvements.
- Team Recommender - AI assembles team of developers
- you next idea?
Edit Pages in docs folder to automatically deploy: 