Skip to content

Conversation

Tmonster
Copy link
Collaborator

This is a quality of life improvement.

Basically, when you build iceberg now and try to run release or debug, there is an attempt to load the avro extension from ~.duckdb/extensions/hash/osx_arm64/avro.duckdb_extension, and there is no guarantee it is always there. This is caused because first iceberg is loaded, which in turn tries to load avro using the extension loader. Since avro is not yet loaded, the default location is tried which is the one I just mentioned.

To fix this, we move the avro extension to be before the iceberg config, which means it will be loaded before iceberg, so when iceberg tries to load avro, we don't look in the ~.duckdb location.

@Tmonster Tmonster requested a review from carlopi September 10, 2025 15:29
Copy link
Contributor

@carlopi carlopi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is OK as a local fix, but has high chance of breaking elsewhere / in the future.

We should really think of a proper solid solution for this.

Problem is:
Statically loaded extensions are loaded in "some order", fetched from a CMake variable, that gets appended to in some order.
If the order happens to be wrong (say, iceberg that has a hard dependency on avro and parquet is not loaded last between these 3), then LoadInternal for iceberg fails and make static binaries with these extensions bundled in non functional.

This is not great, and solution here is only local, but moving forward while we find a more comprehensive solution.

@Tmonster Tmonster merged commit 4b96717 into duckdb:main Sep 11, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants