Datalake 0.15.0 071124#12
Draft
remeajayi2022 wants to merge 421 commits intodatalakefrom datalake-0.15.0-071124
+62,739-24,070
Commits
This pull request is big! We're only showing the most recent 250 commits
Commits on May 14, 2024
Revert "[HUDI-6438] Config parameter 'MAKE_NEW_COLUMNS_NULLABLE' to allow for marking a newly created column as nullable." (apache#10782)
[HUDI-7447] Fix not bootstrap when subTask restart when OPCoordinator handle CheckPointComplete not finished (apache#10767)
- committed
- committed
- committed
- committed
[HUDI-7413] Fix schema exception types and error messages thrown with schema exceptions (apache#10677)
[HUDI-7418] Create a common method for filtering in S3 and GCS sources and add tests for filtering out extensions (apache#10724)
- committed
- committed
- committed
[HUDI-6043] Metadata Table should use default values for Compaction preserveCommitMetadata field (apache#8393)
[HUDI-7495] Bump mysql-connector-java from 8.0.22 to 8.0.28 in /hudi-platform-service/hudi-metaserver/hudi-metaserver-server (apache#7674)
[HUDI-7496] Bump mybatis from 3.4.6 to 3.5.6 in /hudi-platform-service/hudi-metaserver/hudi-metaserver-server (apache#7673)
- authored
[MINOR] Fix and enable test TestHoodieDeltaStreamer.testJdbcSourceIncrementalFetchInContinuousMode (apache#10867)
[HUDI-7382] Get partitions from active timeline instead of listing when building clustering plan (apache#10621)
[HUDI-7492] Fix the incorrect keygenerator specification for multi partition or multi primary key tables creation (apache#10840)
[HUDI-7530] Refactoring of handleUpdateInternal in CommitActionExecutors and HoodieTables (apache#10908)
[MINOR] Restore the setMaxParallelism setting for HoodieTableSource.produceDataStream (apache#10925)
- committed
- committed
[HUDI-6317] Streaming read should skip compaction and clustering instants to avoid duplicates (apache#8884)
[MINOR} When M3 metrics reporter type is used HoodieMetricsConfig should create default values for HoodieMetricsM3Config (apache#10936)
[HUDI-7486] Classify schema exceptions when converting from avro to spark row representation (apache#10778)
[HUDI-7571] Add api to get exception details in HoodieMetadataTableValidator with ignoreFailed mode (apache#10960)
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4&size=32){kind=avatar}
Commits on May 15, 2024
- committed
- committed
[MINOR] Fix BUG: HoodieLogFormatWriter: unable to close output stream for log file HoodieLogFile{xxx} (apache#10989)
- committed
- committed
- committed
- committed
[HUDI-7626] Propagate UserGroupInformation from the main thread to the new thread of timeline service threadpool (apache#11039)
- committed
- committed
- committed
- committed
- committed
- committed
[HUDI-7640] Uses UUID as temporary file suffix for HoodieStorage.createImmutableFileInPath (apache#11052)
[MINOR] Added configurations of Hudi table, file-based SQL source, Hudi error table, and timestamp key generator to configuration listing (apache#11057)
[HUDI-7608] Fix Flink table creation configuration not taking effect when writing to Spark (apache#11005)
[MINOR] Fix incorrect catch of ClassCastException using HoodieSparkKeyGeneratorFactory (apache#11062)
- committed
- committed
- committed
- committed
- committed
- committed
- committed
- committed
[HUDI-7674] Fix Hudi CLI Command "metadata validate-files" to use file listing to validate (apache#11100)
- committed
- committed
- committed
- committed
- committed
- committed
[HUDI-7576] Improve efficiency of getRelativePartitionPath, reduce computation of partitionPath in AbstractTableFileSystemView (apache#11001)
- committed
- committed
[HUDI-7699] Support STS external ids and configurable session names in the AWS StsAssumeRoleCredentialsProvider (apache#11134)
- committed
- committed
- committed
[HUDI-7726] Restructure TableSchemaResolver to separate Hadoop logic and use BaseFileUtils (apache#11185)
- committed
[HUDI-7673] Fixing false positive validation failure for RLI with MDT validation tool (apache#11098)
[HUDI-7739] Shudown asyncDetectorExecutor in AsyncTimelineServerBasedDetectionStrategy (apache#11182)
[HUDI-7508] Avoid collecting records in HoodieStreamerUtils.createHoodieRecords and JsonKafkaSource mapPartitions (apache#10872)
- committed
- committed
- committed
[HUDI-7712] Fixing RLI initialization to account for file slices instead of just base files while initializing (apache#11153)
- committed
- committed
[HUDI-7532] Include only compaction instants for lastCompaction in getDeltaCommitsSinceLatestCompaction (apache#10915)
- authored
- authored
- authored
Commits on May 16, 2024
- authored
- authored
- committed
[HUDI-7766] Adding staging jar deployment command for Spark 3.5 and Scala 2.13 profile (apache#11234)
committed- committed
- committed
Commits on May 24, 2024
Commits on May 25, 2024
[HUDI-7785] Keep public APIs in utilities module the same as before HoodieStorage abstraction (apache#11280)
committed- committed
- committed
- committed
- committed
- committed
Commits on May 26, 2024
- committed
- committed
- committed
- committed
Commits on May 27, 2024
- committed
- committed
- committed
- committed
- committed
- committed
- committed
Commits on May 29, 2024
Commits on May 30, 2024
- committed
- committed
