Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty source loading fails with delta table format on remote filesystems for not-yet-existing tables #1685

Closed
jorritsandbrink opened this issue Aug 13, 2024 · 0 comments · Fixed by #1686
Assignees
Labels
bug Something isn't working

Comments

@jorritsandbrink
Copy link
Collaborator

dlt version

0.5.3

Describe the problem

Empty source loading (e.g. when using dlt.mark.materialize_table_schema()) fails with delta table format on remote filesystems for not-yet-existing tables.

Reason is that storage_options are not passed to DeltaTable.create.

Does not fail for local filesystem because it does not need storage_options.

Expected behavior

No response

Steps to reproduce

Run test_delta_table_empty_source with remote filesystem (e.g. add AZ_BUCKET to bucket_subset).

Operating system

Windows

Runtime environment

Local

Python version

3.8

dlt data source

No response

dlt destination

No response

Other deployment details

No response

Additional information

No response

@jorritsandbrink jorritsandbrink added the bug Something isn't working label Aug 13, 2024
@jorritsandbrink jorritsandbrink self-assigned this Aug 13, 2024
@jorritsandbrink jorritsandbrink moved this from Todo to In Progress in dlt core library Aug 13, 2024
@jorritsandbrink jorritsandbrink linked a pull request Aug 13, 2024 that will close this issue
@github-project-automation github-project-automation bot moved this from In Progress to Done in dlt core library Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant