Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support putIfAbsent #8380

Open
ion-elgreco opened this issue Nov 17, 2024 · 2 comments · May be fixed by #8428
Open

Support putIfAbsent #8380

ion-elgreco opened this issue Nov 17, 2024 · 2 comments · May be fixed by #8428
Assignees
@ItamarYuran
Labels
area/gateway Changes to the gateway area/integrations contributor feature-request good first issue Good for newcomers P2

Comments

@ion-elgreco
Copy link

Atomic writes are not possible currently due to this missing put request with a header.

AWS has native support for this now. MinIO, Cloudflare R2 and other S3 APIs also have this, and we can support atomic writes on most S3 APIs, except LakeFS. There is no safe way to write concurrently to a single branch
@arielshaqed
Copy link
Contributor

Hi @ion-elgreco !

Talk about synchronicity, I was just digging into this yesterday... I assume this is for a Delta log-writer? Can I already use Delta with put-if-absent?

Also: does Delta need it for put-object, for complete-multipart-upload, or for both?

@arielshaqed arielshaqed added good first issue Good for newcomers area/gateway Changes to the gateway area/integrations labels Nov 18, 2024
@ion-elgreco
Copy link
Author

ion-elgreco commented Nov 18, 2024 •
edited
Loading

@arielshaqed Yes this for delta-rs (deltalake), see the PR here I did a while ago: delta-io/delta-rs#2813

You can write delta tables with put-if-absent by simply passing this to the storage_options {"conditional_put":"etag"}.

At the moment its just for put-object. We don't leverage multipart-uploads yet

@talSofer talSofer added the P2 label Nov 21, 2024
@ItamarYuran ItamarYuran linked a pull request Dec 17, 2024 that will close this issue
support putifabsent #8428
Open
@ItamarYuran ItamarYuran linked a pull request Dec 17, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ItamarYuran ItamarYuran

Labels
area/gateway Changes to the gateway area/integrations contributor feature-request good first issue Good for newcomers P2
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

support putifabsent treeverse/lakeFS
4 participants
@arielshaqed @talSofer @ion-elgreco @ItamarYuran