Skip to content

feat: add vector store updates, content, and expiration - #282

Draft
franciscojavierarceo wants to merge 4 commits into
codex/files-apifrom
codex/vector-store-lifecycle
Draft

feat: add vector store updates, content, and expiration#282
franciscojavierarceo wants to merge 4 commits into
codex/files-apifrom
codex/vector-store-lifecycle

Conversation

@franciscojavierarceo

@franciscojavierarceo franciscojavierarceo commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Synchronize this layer with its updated parent after main's AgentPipeline refactor (#274), preserving the immediate-base review boundary.

Stack layer 5 of 7: #34 foundation → #278 pgvector/CI → #280 contextual retrieval/reranking/configuration → #281 streaming Files → #282 store lifecycle/content → #283 durable batches → #284 streaming/SDK compatibility.

Base: codex/files-api (#281). Next: #283. Review against the immediate base and land from the bottom of the stack.

Add Vector Stores updates, file attribute updates, parsed original content, status-filtered pagination, and expiration to the in-tree file-search APIs. Nullable patches distinguish omitted fields from explicit null. Expired stores retain metadata and stop serving attachments/search while independent uploaded files remain available.

Serialize activity, policy updates, publication, and cleanup in SQLite/PostgreSQL. Recheck expiry after database lock waits and model calls to prevent late requests from reviving expired stores. Store original parsed content separately from overlapping/contextual chunks, and publish resolved attachment chunking in the OpenAI response shape.

Test Plan

Layer-specific implementation verification:

  • All-feature focused core suites: 34 service, 17 model, 6 tool, and 5 lifecycle tests passed; 13 HTTP and 7 OpenAPI tests passed.
  • Real PostgreSQL 17/pgvector 0.8.6: 13 integration and 4 schema tests passed, including contended-lock/model expiry and cleanup versus policy-extension races.
  • Legacy parsed-content recovery regression covers a 900,000-word attachment originally ingested with 4096/0 chunking. A PostgreSQL round trip verifies omitted/value/null store updates.
  • Core file-search units (32) and SQLite schema tests (6) passed.
  • Strict OpenAI Python SDK 3.13.0 store updates/expiry and attachment attributes/content tests passed against the rebuilt server and local model fixtures.
  • All-feature changed-crate/all-target clippy, formatting, and all pre-commit hooks passed.

After synchronization with #274: This updated layer passed locked all-feature workspace compilation, diff checks, and all-files pre-commit hooks. The combined stack in #284 passed 1,389 workspace tests, 31 PostgreSQL/pgvector tests, and 17 maintained SDK cases, plus the Rust 1.88 all-feature check and all-target workspace clippy. The pipeline integration review is complete. GitHub CI is rerunning for the updated heads; previous green checks apply to the prior revisions.

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant