Enhancing swagger and redocs

.env.example

@@ -16,6 +16,7 @@ ENVIRONMENT=development
 
 PROJECT_NAME="Kaapi"
 STACK_NAME=Kaapi
+API_VERSION=0.5.0
 
 #Backend
 SECRET_KEY=changethis

backend/app/api/docs/api_keys/create.md

@@ -0,0 +1,3 @@
+Create a new API key for programmatic access to the platform.
+
+The raw API key is returned **only once during creation**. Store it securely as it cannot be retrieved again. Only the key prefix will be visible in subsequent requests for security reasons.

backend/app/api/docs/api_keys/delete.md

@@ -0,0 +1,3 @@
+Delete an API key by its ID.
+
+Permanently revokes the API key. Any requests using this key will fail immediately after deletion.

backend/app/api/docs/api_keys/list.md

@@ -0,0 +1,3 @@
+List all API keys for the current project.
+
+Returns a paginated list of API keys with key prefix for security. The full key is only shown during creation and cannot be retrieved afterward.

backend/app/api/docs/collections/create.md

@@ -1,13 +1,9 @@
 Setup and configure the document store that is pertinent to the RAG
 pipeline:
 
-* Make OpenAI
-  [File](https://platform.openai.com/docs/api-reference/files)'s from
-  documents stored in the cloud (see the `documents` interface).
-* Create an OpenAI [Vector
-  Store](https://platform.openai.com/docs/api-reference/vector-stores)
-  based on those file(s).
-* [To be deprecated] Attach the Vector Store to an OpenAI
+* Create a vector store from the document IDs you received after uploading your
+  documents through the Documents module.
+* [Deprecated] Attach the Vector Store to an OpenAI
   [Assistant](https://platform.openai.com/docs/api-reference/assistants). Use
   parameters in the request body relevant to an Assistant to flesh out
   its configuration. Note that an assistant will only be created when you pass both

backend/app/api/docs/collections/delete.md

@@ -1,6 +1,6 @@
 Remove a collection from the platform. This is a two step process:
 
-1. Delete all OpenAI resources that were allocated: file(s), the Vector
+1. Delete all resources that were allocated: file(s), the Vector
    Store, and the Assistant.
 2. Delete the collection entry from the kaapi database.
 

backend/app/api/docs/collections/info.md

@@ -1,4 +1,4 @@
 Retrieve detailed information about a specific collection by its collection id. This endpoint returns the collection object including its project, organization,
 timestamps, and associated LLM service details (`llm_service_id` and `llm_service_name`).
 
-Additionally, if the `include_docs` flag in the request body is true then you will get a list of document IDs associated with a given collection as well. Note that, documents returned are not only stored by the AI platform, but also by OpenAI.
+Additionally, if the `include_docs` flag in the request body is true then you will get a list of document IDs associated with a given collection as well. Note that, documents returned are not only stored by the AI platform, but also by Vector store provider.

backend/app/api/docs/collections/list.md

@@ -1,6 +1,5 @@
-List _active_ collections -- collections that have been created but
-not deleted
+List all _active_ collections that have been created and are not deleted
 
-If a vector store was created - `llm_service_name` and `llm_service_id` in the response denote the name of the vector store (eg. 'openai vector store') and its id.
+If a vector store was created - `llm_service_name` and `llm_service_id` in the response denotes the name of the vector store (eg. 'openai vector store') and its id respectively.
 
-[To be deprecated] If an assistant was created, `llm_service_name` and `llm_service_id` in the response denotes the name of the model used in the assistant (eg. 'gpt-4o') and assistant id.
+[Deprecated] If an assistant was created, `llm_service_name` and `llm_service_id` in the response denotes the name of the model used in the assistant (eg. 'gpt-4o') and assistant id.

backend/app/api/docs/config/create.md

@@ -11,7 +11,7 @@ Configurations allow you to store and manage reusable LLM parameters
 * Provider-agnostic storage - params are passed through to the provider as-is
 
 
-**Example for the config blob: OpenAI Responses API with File Search**
+**Example for the config blob: OpenAI Responses API with File Search -**
 
 ```json
 "config_blob": {

backend/app/api/docs/credentials/create.md

@@ -0,0 +1,3 @@
+Persist new credentials for the current organization and project.
+
+Credentials are encrypted and stored securely for provider integrations (OpenAI, Langfuse, etc.). Only one credential per provider is allowed per organization-project combination.

backend/app/api/docs/credentials/delete_all.md

@@ -0,0 +1,3 @@
+Delete all credentials for current organization and project.
+
+Permanently removes all provider credentials from the current organization and project.

backend/app/api/docs/credentials/delete_provider.md

@@ -0,0 +1,3 @@
+Delete credentials for a specific provider.
+
+Permanently removes credentials for a specific provider from the current organization and project.

backend/app/api/docs/credentials/get_provider.md

@@ -0,0 +1,3 @@
+Get credentials for a specific provider.
+
+Retrieves decrypted credentials for a specific provider (e.g., `openai`, `langfuse`) for the current organization and project.

backend/app/api/docs/credentials/list.md

@@ -0,0 +1,3 @@
+Get all credentials for current organization and project.
+
+Returns list of all provider credentials associated with your organization and project.

backend/app/api/docs/credentials/update.md

@@ -0,0 +1,3 @@
+Update credentials for a specific provider.
+
+Updates existing provider credentials for the current organization and project. Provider and credential fields must be provided.

backend/app/api/docs/documents/delete.md

@@ -1,4 +1,4 @@
-Perform a soft delete of the document. A soft delete makes the
+Perform a delete of the document. This makes the
 document invisible. It does not delete the document from cloud storage
 or its information from the database.
 

backend/app/api/docs/documents/info.md

@@ -1 +1,3 @@
-Retrieve all information about a given document. If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the retrieved document. If you don't set it to true, the URL will not be included in the response.
+Retrieve all information about a given document.
+
+If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the retrieved document. If you don't set it to true, the URL will not be included in the response.

backend/app/api/docs/documents/job_info.md

@@ -1 +1,3 @@
-Get the status and details of a document transformation job. If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the transformed document if the job has been successful. If you don't set it to true, the URL will not be included in the response.
+Get the status and details of a document transformation job.
+
+If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the transformed document if the job has been successful. If you don't set it to true, the URL will not be included in the response.

backend/app/api/docs/documents/job_list.md

@@ -1 +1,3 @@
-Get the status and details of multiple document transformation jobs by IDs. If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the transformed document for successful jobs. If you don't set it to true, the URL will not be included in the response.
+Get the status and details of multiple document transformation jobs by IDs.
+
+If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the transformed document for successful jobs. If you don't set it to true, the URL will not be included in the response.

backend/app/api/docs/documents/list.md

@@ -1 +1,3 @@
-List documents uploaded to the AI platform.  If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the retrieved documents. If you don't set it to true, the URL will not be included in the response.
+List documents uploaded to the AI platform.
+
+If you set the ``include_url`` parameter to true, a signed URL will be included in the response, which is a clickable link to access the retrieved documents. If you don't set it to true, the URL will not be included in the response.

backend/app/api/docs/documents/permanent_delete.md

@@ -1,6 +1,7 @@
 This operation marks the document as deleted in the database while retaining its metadata. However, the actual file is
 permanently deleted from cloud storage (e.g., S3) and cannot be recovered. Only the database record remains for reference
 purposes.
+
 If the document is part of an active collection, those collections
 will be deleted using the collections delete interface. Noteably, this
 means all OpenAI Vector Store's and Assistant's to which this document

backend/app/api/docs/evaluation/create_evaluation.md

@@ -1,80 +1,46 @@
-Start an evaluation using OpenAI Batch API.
+Start an evaluation run using the OpenAI Batch API.
 
-This endpoint:
-1. Fetches the dataset from database and validates it has Langfuse dataset ID
-2. Creates an EvaluationRun record in the database
-3. Fetches dataset items from Langfuse
-4. Builds JSONL for batch processing (config is used as-is)
-5. Creates a batch job via the generic batch infrastructure
-6. Returns the evaluation run details with batch_job_id
+Evaluations allow you to systematically test LLM configurations against
+predefined datasets with automatic progress tracking and result collection.
 
-The batch will be processed asynchronously by Celery Beat (every 60s).
-Use GET /evaluations/{evaluation_id} to check progress.
+**Key Features:**
+* Fetches dataset items from Langfuse and creates batch processing job via OpenAI Batch API
+* Asynchronous processing with automatic progress tracking (checks every 60s)
+* Supports configuration from direct parameters or existing assistants
+* Stores results for comparison and analysis
+* Use `GET /evaluations/{evaluation_id}` to monitor progress and retrieve results of evaluation.
 
-## Request Body
-
-- **dataset_id** (required): ID of the evaluation dataset (from /evaluations/datasets)
-- **experiment_name** (required): Name for this evaluation experiment/run
-- **config** (optional): Configuration dict that will be used as-is in JSONL generation. Can include any OpenAI Responses API parameters like:
-  - model: str (e.g., "gpt-4o", "gpt-5")
-  - instructions: str
-  - tools: list (e.g., [{"type": "file_search", "vector_store_ids": [...]}])
-  - reasoning: dict (e.g., {"effort": "low"})
-  - text: dict (e.g., {"verbosity": "low"})
-  - temperature: float
-  - include: list (e.g., ["file_search_call.results"])
-  - Note: "input" will be added automatically from the dataset
-- **assistant_id** (optional): Assistant ID to fetch configuration from. If provided, configuration will be fetched from the assistant in the database. Config can be passed as empty dict {} when using assistant_id.
-
-## Example with config
+**Example: Using Direct Configuration**
 
 ```json
 {
-    "dataset_id": 123,
-    "experiment_name": "test_run",
-    "config": {
-        "model": "gpt-4.1",
-        "instructions": "You are a helpful FAQ assistant.",
-        "tools": [
-            {
-                "type": "file_search",
-                "vector_store_ids": ["vs_12345"],
-                "max_num_results": 3
-            }
-        ],
-        "include": ["file_search_call.results"]
-    }
+  "dataset_id": 123,
+  "experiment_name": "gpt4_file_search_test",
+  "config": {
+    "model": "gpt-4o",
+    "instructions": "You are a helpful FAQ assistant for farmers.",
+    "tools": [
+      {
+        "type": "file_search",
+        "vector_store_ids": ["vs_abc123"],
+        "max_num_results": 5
+      }
+    ],
+    "temperature": 0.7,
+    "include": ["file_search_call.results"]
+  }
 }
 ```
 
-## Example with assistant_id
+**Example: Using Existing Assistant**
 
 ```json
 {
-    "dataset_id": 123,
-    "experiment_name": "test_run",
-    "config": {},
-    "assistant_id": "asst_xyz"
+  "dataset_id": 123,
+  "experiment_name": "production_assistant_eval",
+  "config": {},
+  "assistant_id": "asst_xyz789"
 }
 ```
 
-## Returns
-
-EvaluationRunPublic with batch details and status:
-- id: Evaluation run ID
-- run_name: Name of the evaluation run
-- dataset_name: Name of the dataset used
-- dataset_id: ID of the dataset used
-- config: Configuration used for the evaluation
-- batch_job_id: ID of the batch job processing this evaluation
-- status: Current status (pending, running, completed, failed)
-- total_items: Total number of items being evaluated
-- completed_items: Number of items completed so far
-- results: Evaluation results (when completed)
-- error_message: Error message if failed
-
-## Error Responses
-
-- **404**: Dataset or assistant not found or not accessible
-- **400**: Missing required credentials (OpenAI or Langfuse), dataset missing Langfuse ID, or config missing required fields
-- **500**: Failed to configure API clients or start batch evaluation
+**Note:** When using `assistant_id`, configuration is fetched from the assistant in the database. You can pass `config` as an empty object `{}`.

backend/app/api/docs/evaluation/delete_dataset.md

@@ -1,18 +1,3 @@
 Delete a dataset by ID.
 
-This will remove the dataset record from the database. The CSV file in object store (if exists) will remain for audit purposes, but the dataset will no longer be accessible for creating new evaluations.
-
-## Path Parameters
-
-- **dataset_id**: ID of the dataset to delete
-
-## Returns
-
-Success message with deleted dataset details:
-- message: Confirmation message
-- dataset_id: ID of the deleted dataset
-
-## Error Responses
-
-- **404**: Dataset not found or not accessible to your organization/project
-- **400**: Dataset cannot be deleted (e.g., has active evaluation runs)
+This will remove the dataset record from the database. The CSV file in object store (if exists) will remain there for audit purposes, but the dataset will no longer be accessible for creating new evaluations.

backend/app/api/docs/evaluation/get_dataset.md

@@ -1,22 +1,3 @@
 Get details of a specific dataset by ID.
 
-Retrieves comprehensive information about a dataset including metadata, object store URL, and Langfuse integration details.
-
-## Path Parameters
-
-- **dataset_id**: ID of the dataset to retrieve
-
-## Returns
-
-DatasetUploadResponse with dataset details:
-- dataset_id: Unique identifier for the dataset
-- dataset_name: Name of the dataset (sanitized)
-- total_items: Total number of items including duplication
-- original_items: Number of original items before duplication
-- duplication_factor: Factor by which items were duplicated
-- langfuse_dataset_id: ID of the dataset in Langfuse
-- object_store_url: URL to the CSV file in object storage
-
-## Error Responses
-
-- **404**: Dataset not found or not accessible to your organization/project
+Returns comprehensive dataset information including metadata (ID, name, item counts, duplication factor), Langfuse integration details (dataset ID), and the object store URL for the CSV file.

backend/app/api/docs/evaluation/get_evaluation.md

@@ -1,37 +1,12 @@
-Get the current status of a specific evaluation run.
+Get the current status and results of a specific evaluation run by the evaluation ID along with some optional query parameters listed below.
 
-Retrieves comprehensive information about an evaluation run including its current processing status, results (if completed), and error details (if failed).
+Returns comprehensive evaluation information including processing status, configuration, progress metrics, and detailed scores with Q&A context when requested. You can check this endpoint periodically to get to know the evaluation progress. Evaluations are processed asynchronously with status checks every 60 seconds.
 
-## Path Parameters
+**Query Parameters:**
+* `get_trace_info` (optional, default: false) - Include Langfuse trace scores with Q&A context. Data is fetched from Langfuse on first request and cached for subsequent calls. Only available for completed evaluations.
+* `resync_score` (optional, default: false) - Clear cached scores and re-fetch from Langfuse. Useful when evaluators have been updated. Requires `get_trace_info=true`.
 
-- **evaluation_id**: ID of the evaluation run
-
-## Query Parameters
-
-- **get_trace_info** (optional, default: false): If true, fetch and include Langfuse trace scores with Q&A context. On first request, data is fetched from Langfuse and cached in the score column. Subsequent requests return cached data. Only available for completed evaluations.
-
-- **resync_score** (optional, default: false): If true, clear cached scores and re-fetch from Langfuse. Useful when new evaluators have been added or scores have been updated. Requires get_trace_info=true.
-
-## Returns
-
-EvaluationRunPublic with current status and results:
-- id: Evaluation run ID
-- run_name: Name of the evaluation run
-- dataset_name: Name of the dataset used
-- dataset_id: ID of the dataset used
-- config: Configuration used for the evaluation
-- batch_job_id: ID of the batch job processing this evaluation
-- status: Current status (pending, running, completed, failed)
-- total_items: Total number of items being evaluated
-- completed_items: Number of items completed so far
-- score: Evaluation scores (when get_trace_info=true and status=completed)
-- error_message: Error message if failed
-- created_at: Timestamp when the evaluation was created
-- updated_at: Timestamp when the evaluation was last updated
-
-## Score Format
-
-When `get_trace_info=true` and evaluation is completed, the `score` field contains:
+**Score Format** (`get_trace_info=true`):
 
 ```json
 {
@@ -74,16 +49,8 @@ When `get_trace_info=true` and evaluation is completed, the `score` field contai
 }
 ```
 
-**Notes:**
-- Only complete scores are included (scores where all traces have been rated)
-- Numeric values are rounded to 2 decimal places
-- NUMERIC scores show `avg` and `std` in summary
-- CATEGORICAL scores show `distribution` counts in summary
-
-## Usage
-
-Use this endpoint to poll for evaluation progress. The evaluation is processed asynchronously by Celery Beat (every 60s), so you should poll periodically to check if the status has changed to "completed" or "failed".
-
-## Error Responses
-
-- **404**: Evaluation run not found or not accessible to this organization/project
+**Score Details:**
+* NUMERIC scores include average (`avg`) and standard deviation (`std`) in summary
+* CATEGORICAL scores include distribution counts in summary
+* Only complete scores are included (all traces have been rated)
+* Numeric values are rounded to 2 decimal places

backend/app/api/docs/evaluation/list_datasets.md

@@ -1,19 +1,3 @@
 List all datasets for the current organization and project.
 
-Returns a paginated list of dataset records ordered by most recent first.
-
-## Query Parameters
-
-- **limit**: Maximum number of datasets to return (default 50, max 100)
-- **offset**: Number of datasets to skip for pagination (default 0)
-
-## Returns
-
-List of DatasetUploadResponse objects, each containing:
-- dataset_id: Unique identifier for the dataset
-- dataset_name: Name of the dataset (sanitized)
-- total_items: Total number of items including duplication
-- original_items: Number of original items before duplication
-- duplication_factor: Factor by which items were duplicated
-- langfuse_dataset_id: ID of the dataset in Langfuse
-- object_store_url: URL to the CSV file in object storage
+Returns a paginated list of datasets ordered by most recent first. Each dataset includes metadata (ID, name, item counts, duplication factor), Langfuse integration details, and object store URL.