Skip to content

Conversation

@nsivabalan
Copy link
Contributor

Describe the issue this Pull Request addresses

We added streaming writes to metadata table in 1.1.0. Not all partitions in mdt supports streaming writes as of now. So, fixing writes to leverage streaming writes to metadata table only when eligible metadata partitions are enabled.

Summary and Changelog

  • Leverage streaming writes to metadata table in spark only when either of RLI (global or non global) or secondary index is enabled.

Impact

  • Avoids the new dag if not really required. Down the line, when we move col stats and other partitions to streaming flow, probably we will get rid of this optimization.

Risk Level

low

Documentation Update

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@hudi-bot
Copy link
Collaborator

hudi-bot commented Dec 4, 2025

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S PR with lines of changes in (10, 100]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants