You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Make RestCatalog safe to share across threads #1091
Currently, RestCatalog doesn't document whether one instance can be shared between threads. Query engines usually share one catalog across concurrent queries (for example Spark within a session, Trino, and DuckDB), and iceberg-rust requires catalogs to be Send + Sync (code).
Without that guarantee, an embedder has to serialize every catalog call behind its own lock. That costs throughput, and one hung request then blocks every other caller. In https://github.com/smaheshwar-pltr/duckdb-proto-iceberg, we observed that making a catalog-heavy test about 4x slower.
RestCatalog is close to thread-safe already. After Make(), the only state it changes is the default catalog that AsCatalog() caches in a weak_ptr. It reads and writes that cache without a lock, so concurrent AsCatalog() calls race (rest_catalog.cc#L565-L573).
Note: sharing a catalog also depends on #1089, because every request goes through one HttpClient.
Activity