Skip to content

Conversation

@PavithranRick
Copy link
Contributor

Describe the issue this Pull Request addresses

Consolidated multiple Hudi Spark SQL procedures into unified, comprehensive procedures that support both completed and pending instants with partition-specific metadata.

Summary and Changelog

  • Unified Show procedures, including:
    • ShowCommitsProcedure
    • ShowClusteringProcedure
    • ShowCompactionProcedure
    • ShowCleansProcedure
    • ShowRollbacksProcedure
    • ShowSavepointProcedure
  • All unified procedures now support both completed and pending instants.
  • Added partition-specific and, where applicable, file-specific metadata to the unified procedures.

Impact

Public API Changes:

  • Unified Show procedures now replace multiple legacy procedures.
  • Enhanced schemas with additional metadata (partition-level and file-level where applicable).

Risk Level

low

Documentation Update

  • Hudi website documentation for new unified procedures
  • Migration guide from legacy to new procedures
  • Updated procedure reference with enhanced schemas
  • Deprecation notices and migration examples

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

vamshikrishnakyatham and others added 30 commits August 25, 2025 14:00
…-basePath-and-the-generic-filter-option-for-all-the-Spark-Procedures
…ith partition metadata and supporting basepath and filters for the same
…-basePath-and-the-generic-filter-option-for-all-the-Spark-Procedures
…ith partition metadata and supporting basepath and filters for the same
…s and supporting basepath and filters for the same
…ns and supporting basepath and filters for the same
…ns and supporting basepath and filters for the same
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
…-for-plans-along-with-partition-metadata-in-all-the-Spark-action-Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
…h partition metadata, start and end instant times, generic filters flag options in all the Spark unified action Procedures
@github-actions github-actions bot added the size:XL PR with lines of changes > 1000 label Dec 2, 2025
@hudi-bot
Copy link
Collaborator

hudi-bot commented Dec 3, 2025

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Comment on lines +42 to +43
ProcedureParameter.optional(2, "startTime", DataTypes.StringType, ""),
ProcedureParameter.optional(3, "endTime", DataTypes.StringType, ""),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that we're following snake case in other procedures? we should do the same here to maintain consistency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL PR with lines of changes > 1000

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants