Stayquery

Stayquery is a Python 3.12 hybrid RAG application for hospitality data. It uses SQL retrieval and vector retrieval as internal inputs to one hybrid, evidence-grounded answer. (pyproject.toml, src/stayquery/service.py, src/stayquery/cli.py, src/stayquery/api.py)

Postgres stores source records, normalized hospitality tables, document chunks, query runs, and retrieval hits. Qdrant stores a derived vector index that is rehydrated from Postgres documents before evidence is returned. (src/stayquery/models.py, src/stayquery/vector.py, src/stayquery/ingest.py)

OpenAI is used for structured query planning, embeddings, and evidence-grounded answer generation. OPENAI_API_KEY is required for default ingest, ask, and eval flows that call embeddings or answer generation. (src/stayquery/openai_client.py, src/stayquery/ingest.py, src/stayquery/config.py)

Prerequisites

• Python 3.12, managed through uv. (pyproject.toml, uv.lock)
• Docker with Compose for local Postgres and Qdrant. (docker-compose.yml)
• uv for dependency installation and command execution. (pyproject.toml, uv.lock)
• An OpenAI API key for embedding, planning, and answer generation paths. (src/stayquery/openai_client.py, src/stayquery/ingest.py)
• Aminess API settings when syncing shared source data directly from the external API. (src/stayquery/source_api.py, src/stayquery/config.py)

Quick Start za kolegicu

Ovo je najkraći Windows/PowerShell put od svježeg checkouta do RAG pitanja za Korčulu. Prije početka treba imati instaliran Docker Desktop i uv, te treba imati OpenAI API key i Aminess client_id / client_key. Tajne se upisuju samo u lokalni .env, nikad u Git.

Clone i ulazak u projekt:

git clone https://github.com/Matej23031/rag_shared.git
cd rag_shared

Ako je projekt preuzet kao ZIP umjesto clonea, uđi u folder koji sadrži pyproject.toml, README.md i .env.example.

Napravi lokalni .env:

Copy-Item .env.example .env
notepad .env

U .env popuni najmanje ove vrijednosti:

OPENAI_API_KEY=OVDJE_IDE_OPENAI_KEY
AMINESS_API_BASE_URL=https://webservices.aminess.com
AMINESS_API_CLIENT_ID=OVDJE_IDE_AMINESS_CLIENT_ID
AMINESS_API_CLIENT_KEY=OVDJE_IDE_AMINESS_CLIENT_KEY
AMINESS_API_LISTING_PATH=/api/v1/knowledge-base/listing
AMINESS_API_ITEM_PATH=/api/v1/knowledge-base/item

Ne treba ručno vaditi access_token: stayquery sync-api ga dobije automatski preko AMINESS_API_CLIENT_ID i AMINESS_API_CLIENT_KEY. Ako netko već ima gotov kratkotrajni bearer token, može ga umjesto toga staviti u AMINESS_API_KEY.

Instaliraj Python/dependencyje kroz uv:

$env:UV_CACHE_DIR=".uv-cache"
$env:UV_PYTHON_INSTALL_DIR=".uv-python"
uv python install 3.12 --install-dir .uv-python --no-registry
$env:UV_PYTHON=(Resolve-Path ".uv-python/cpython-3.12.13-windows-x86_64-none/python.exe").Path
uv sync --all-extras --managed-python

Pokreni bazu i migracije:

docker compose up -d postgres qdrant
uv run --managed-python stayquery db upgrade

Povuci Korčulu iz Aminess API-ja i napravi full ingest s embeddingom:

uv run --managed-python stayquery sync-api --destination korcula --out data/korcula

Za provjeru bez OpenAI embeddinga može se koristiti --skip-vector, ali tada RAG/vector test nije kompletan:

uv run --managed-python stayquery sync-api --destination korcula --out data/korcula --skip-vector

Testiraj RAG/LLM odgovore:

uv run --managed-python stayquery ask "Koji su objekti dostupni na Korčuli?"
uv run --managed-python stayquery ask "Koji smještaj na Korčuli je najbolji za obitelj s djecom?"
uv run --managed-python stayquery ask "Ima li Aminess Port9 restoran ili bar?"

Očekivani dobar znak: odgovor spominje konkretne objekte iz podataka i ima citate u obliku neki-file.json#section. Ako Docker javi da se ne može spojiti na engine, prvo pokreni Docker Desktop. Ako ingest kaže da nedostaje OPENAI_API_KEY, provjeri .env i ponovno pokreni komandu iz istog project foldera.

Za lokalni PolyAI ADK test pokreni Stayquery API:

uv run --managed-python stayquery serve --host 127.0.0.1 --port 8000

PolyAI action/webhook može zvati compact endpoint:

POST http://127.0.0.1:8000/polyai/ask

Body može biti jedan od ovih oblika:

{ "query": "Koji su objekti dostupni na Korčuli?" }

{ "message": "Ima li Aminess Port9 restoran ili bar?" }

Odgovor ima jednostavan oblik za čitanje u voice/chat flowu:

{
  "answer": "Tekst koji PolyAI može pročitati korisniku.",
  "sources": ["source.json#section"],
  "fallback_reason": null,
  "warnings": [],
  "query_run_id": "..."
}

Local Setup

On Windows, the existing workflow uses repo-local uv cache directories: (README.md, pyproject.toml)

$env:UV_CACHE_DIR=".uv-cache"
$env:UV_PYTHON_INSTALL_DIR=".uv-python"

Install or select Python 3.12, then install dependencies: (pyproject.toml, uv.lock)

uv python install 3.12 --install-dir .uv-python --no-registry
$env:UV_PYTHON=(Resolve-Path ".uv-python/cpython-3.12.13-windows-x86_64-none/python.exe").Path
uv sync --all-extras --managed-python

Start local services and migrate the database: (docker-compose.yml, src/stayquery/cli.py, migrations/env.py, migrations/versions/0001_initial_schema.py)

docker compose up -d postgres qdrant
uv run --managed-python stayquery db upgrade

Create .env from the committed non-secret .env.example template and fill secret values locally. Real OpenAI keys belong only in local .env files or deployment secret storage; a key-shaped value previously appeared in repository history and should be treated as exposed until the key owner confirms revocation or rotation. (.env.example, .gitignore, src/stayquery/config.py)

Common Commands

Ingest requires an explicit source directory. The tracked fake records in data/raw are available for tests and deliberate local ingestion. The current real source is the private data/novigrad dataset, and API-synced Korcula records should be written to the ignored data/korcula dataset. (.gitignore, src/stayquery/cli.py, src/stayquery/source_api.py)

Validate a selected dataset and write Postgres rows without OpenAI or Qdrant indexing: (src/stayquery/cli.py, src/stayquery/ingest.py)

uv run --managed-python stayquery ingest --data path/to/data --skip-vector

Use this local-only path for private datasets such as data/novigrad unless sending their document chunks to OpenAI for embedding has been explicitly approved.

Sync Korcula records from the external Aminess API, save the local JSON export under data/korcula, and ingest without embeddings for validation/Postgres smoke tests: (src/stayquery/cli.py, src/stayquery/source_api.py, src/stayquery/ingest.py)

uv run --managed-python stayquery sync-api --destination korcula --out data/korcula --skip-vector

Run full ingestion with embeddings and Qdrant indexing: (src/stayquery/cli.py, src/stayquery/ingest.py, src/stayquery/vector.py)

uv run --managed-python stayquery ingest --data path/to/data

The full API sync variant also indexes embeddings when OPENAI_API_KEY is configured:

uv run --managed-python stayquery sync-api --destination korcula --out data/korcula

Ask one question through the CLI: (src/stayquery/cli.py, src/stayquery/rag.py, src/stayquery/service.py)

uv run --managed-python stayquery ask "Which options are good for a family that needs beach access?"

Run the evaluation fixture: (src/stayquery/cli.py, tests/fixtures/questions.yaml, tests/test_eval_fixture.py)

uv run --managed-python stayquery eval --questions tests/fixtures/questions.yaml

Run the optional OpenAI-backed quality rubric against the private Novigrad benchmark after the dataset has been deliberately ingested and indexed locally. The expected private fixture path is ignored by Git: (src/stayquery/cli.py, src/stayquery/openai_client.py)

uv run --managed-python stayquery eval --questions data/novigrad/eval_questions.yaml --quality-rubric --judge-model gpt-4o-mini --fail-under 0.80 --report-json data/novigrad/eval-report.json

Run the HTTP API locally with Uvicorn: (src/stayquery/cli.py, src/stayquery/api.py, pyproject.toml)

uv run --managed-python stayquery serve --host 127.0.0.1 --port 8000 --reload

The API exposes GET /health and POST /ask. (src/stayquery/api.py, tests/test_api.py)

Testing

Run the test suite with pytest through uv: (pyproject.toml, tests/test_cli.py, tests/test_api.py, tests/test_retrieval_contracts.py, tests/test_schema_and_chunking.py, tests/test_openai_schema.py, tests/test_eval_fixture.py)

uv run pytest

The tests cover API response shape and error mapping, CLI hybrid output and eval summaries, external source API sync wiring, multilingual answer prompt assembly, SQL bound-parameter compilation, hybrid fallback behavior, vector stale-hit handling, source schema validation, chunk stability, OpenAI strict schema normalization, and fixture presence. (tests/test_api.py, tests/test_cli.py, tests/test_source_api.py, tests/test_answer_prompt.py, tests/test_retrieval_contracts.py, tests/test_schema_and_chunking.py, tests/test_openai_schema.py, tests/test_eval_fixture.py)

Deployment Summary

The repository includes Docker Compose services for local Postgres and Qdrant, but no production deployment manifest was found in the inspected files. Needs verification before production deployment. (docker-compose.yml, pyproject.toml, src/stayquery/cli.py, src/stayquery/api.py)

The API can be served as stayquery.api:app by an ASGI server, and the CLI wraps this with stayquery serve. (src/stayquery/api.py, src/stayquery/cli.py, pyproject.toml)

Automated CI/CD was not defined in the inspected repository files. Needs verification if CI/CD exists outside this checkout. (pyproject.toml, docker-compose.yml, alembic.ini, tests/test_cli.py, tests/test_api.py)

More Documentation

• Production readiness gaps: GAPS.md
• Architecture overview: docs/architecture/overview.md
• Module map: docs/architecture/modules.md
• Data flow: docs/architecture/data-flow.md
• ADR template: docs/architecture/decisions/ADR-0001-template.md
• Public API: docs/api/public-api.md
• Internal API: docs/api/internal-api.md
• Deployment: docs/operations/deployment.md
• Observability: docs/operations/observability.md
• Runbooks: docs/operations/runbooks.md
• Testing notes: docs/development/testing.md
• Local setup: docs/development/local-setup.md
• Contribution guide: docs/development/contribution-guide.md
• Configuration and secrets: docs/operations/config.md
• Authentication and security notes: docs/security/auth.md
• Permissions: docs/security/permissions.md
• Threat model: docs/security/threat-model.md
• Changelog: CHANGELOG.md