feat: complete file-search streaming and API compatibility

[franciscojavierarceo]

Summary

Integrate with main's AgentPipeline refactor (#274). Capture the initiating public tool choice on the request-owned pipeline, carry response metadata through per-round translation snapshots, and emit grounded citation events through the existing stream delivery and sequence numbering. Preserve the post-loop continuation settings used by persistence.

Complete the Files/Vector Stores/file-search API stack with grounded response.output_text.annotation.added events, consistent citation indexes and sequence numbers, required public tools, tool_choice, and parallel_tool_calls response fields, and Unicode character limits for metadata, attributes, and filters. Preserve the executor's existing continuation settings and stored item history.

Preserve attachment identity through exact and pgvector retrieval, ranking, reranking, and deduplication. If an attachment is removed, replaced, or changes attributes while a model call is pending, the final search check validates the original chunk generation and current matching attributes. A valid duplicate retains its own current attributes and score. This applies to direct search and Responses file search.

Add a maintained OpenAI Python SDK 3.13.0 contract suite against the built server and local model fixtures, run the complete PostgreSQL integration set in CI, and document the compatibility matrix, ordered migrations, and worker shutdown bounds. Pin the new optional PDF dependency to AES 0.9.2 for the verified Rust 1.88 all-feature build.

Stack layer 7 of 7: #34 foundation → #278 pgvector/CI → #280 contextual retrieval/reranking/configuration → #281 streaming Files → #282 store lifecycle/content → #283 durable batches → this PR. Base: codex/vector-store-batches. Review against the immediate base and land from the bottom of the stack.

Compatibility covers the documented, exercised API contracts. Native GET /v1/responses/{id} and full stored response snapshots remain unavailable. SDK 3.13.0 accepts documented evals requests but omits that purpose from its response enum; that case has raw-wire coverage, while other SDK cases use strict validation. Hosted OpenAI ranking-model equivalence, guide-only filename filters, and exact hosted citation-index semantics are not claimed. See docs/api/file-search.md for the full matrix and resource limits.

Test Plan

• Locked all-feature workspace: 1,389 passed, 0 failed. The ordinary run ignores 31 PostgreSQL tests and one documentation example.
• All 31 PostgreSQL tests passed serially against PostgreSQL 17 / pgvector 0.8.6, including the ten-scenario exact/pgvector attachment mutation matrix and existing expiration, lease, cancellation, and lock-fencing regressions.
• Rebuilt all-feature server: 17 maintained SDK cases passed, including Files round trips, lifecycle/Unicode contracts, search/ranker/rewrite options, partial batch failure/cancellation, streamed citations and continuation, orderly shutdown/restart, and recovery after a real 30-second lease expires.
• Attachment provenance regressions failed before the fix and pass afterward on SQLite and PostgreSQL; a ranking regression excludes below-threshold duplicate origins. The covering service/model/Responses-tool suites passed 63 tests.
• cargo +1.88.0 check --workspace --all-features --locked passed on the final source. All-feature tests and all-target workspace clippy passed on Rust 1.98. The existing dependency graph's mismatch with the declared Rust 1.85 policy is disclosed; this PR does not claim a 1.85 build.
• Formatting, diff checks, and all-files pre-commit hooks passed. Optional PDF extraction tests are included in the workspace run.

Independent layer reviews, the complete stack review, and the scoped AgentPipeline integration review are complete. Integration findings are addressed and re-reviewed, including the fixture migration for request-authoritative tool choice. All local gates above passed after synchronization with #274. GitHub CI is rerunning for updated head 33effed; previous green checks apply to the prior revisions.