Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE!] SNOW-1176557 Support max lob size to 128MB in client SDK #877

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

sfc-gh-ggeng
Copy link
Contributor

SNOW-1176557 Support max lob size increase to 128MB in client SDK

This PR implement the support for max lob size increase to 128MB in client SDK. We shall not merge the PR until the Server side change and fully rollout and stable. This PR also adds tests and demonstrates that the integration with the server side would work.

Other than increase the default value of the max size, there are a few addition changes to support this:

  • Since the max lob size has increase significantly, we also need to add the support for customer setting MAX_ALLOWED_ROW_SIZE_IN_BYTES
  • We also need to explicitly set the setStreamReadConstraints' max length to make sure that we can serialize/verify the customer input data without error.

We also added a new large lob size example, to show that we could ingest into a four large-lob columns with row size over 400MB in total.

@sfc-gh-ggeng sfc-gh-ggeng requested review from sfc-gh-tzhang and a team as code owners October 29, 2024 21:57
@@ -230,6 +230,13 @@ private void setParameterMap(
props,
false /* enforceDefault */);

this.checkAndUpdate(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sfc-gh-tzhang do you know why in the past we have the parameter here, but the customer can not set it? Is there any explicit reason or just missing implementation? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant