feat: stream Files API uploads and support expiration

[franciscojavierarceo]

Summary

Synchronize this layer with its updated parent after main's AgentPipeline refactor (#274), preserving the immediate-base review boundary.

Stack layer 4 of 7: #34 foundation → #278 pgvector/CI → #280 contextual retrieval/reranking/configuration → #281 streaming Files → #282 store lifecycle/content → #283 durable batches → #284 streaming/SDK compatibility.

Base: codex/file-search-retrieval (#280). Next: #282. Review against the immediate base and land from the bottom of the stack.

Streams Files API uploads and downloads through bounded buffers, supporting files up to 512 MiB without buffering the whole payload in memory. Multipart metadata is validated before publication; failed or disconnected uploads retain the existing cleanup and durability guarantees.

Adds the documented upload purposes, explicit file expiration and the batch-purpose default, purpose-filtered pagination up to 10000, empty-file support and the correct delete response object. Expired files immediately disappear from reads and retrieval. SQL deletion records durable blob-cleanup intent so later cleanup can recover interrupted filesystem deletion.

Test Plan

Layer-specific implementation verification:

• Final all-feature covering run: 33 service and 16 model tests passed, including PDF ingestion; 11 HTTP and 6 OpenAPI tests passed. Three framing-adapter and three publication/storage unit tests passed.
• Real PostgreSQL/pgvector runs cover persistence, index/migration preservation, filtering, schema readiness/privileges and expiration cleanup replay. A new lock-wait regression observes publication blocked on the store, lets the source expire, and verifies rollback after the lock is released.
• Actual 512 MiB streaming upload/download and 512 MiB + 1 rejection pass. TCP regressions reject an unfinished oversized header/preamble early; byte-preservation and eager-parser-drain tests verify the multipart adapter's bounds.
• Three strict OpenAI SDK 3.13.0 Files cases pass against the rebuilt server. The separately checked raw SDK evals roundtrip preserves its exact wire value.
• All-target, all-feature clippy for both changed crates and all applicable pre-commit hooks pass, including formatting. Independent task review and scoped regression review cover the layer.

The pinned OpenAI 3.13.0 SDK accepts purpose="evals" on upload but omits it from its response enum. The server preserves the correct evals wire value; direct HTTP coverage tests it, while the strict SDK expiration/filter/download scenario uses user_data to avoid that upstream schema inconsistency.

After synchronization with #274: This updated layer passed locked all-feature workspace compilation, diff checks, and all-files pre-commit hooks. The combined stack in #284 passed 1,389 workspace tests, 31 PostgreSQL/pgvector tests, and 17 maintained SDK cases, plus the Rust 1.88 all-feature check and all-target workspace clippy. The pipeline integration review is complete. GitHub CI is rerunning for the updated heads; previous green checks apply to the prior revisions.