Skip to content

Conversation

@dantengsky
Copy link
Member

@dantengsky dantengsky commented Dec 18, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

enhance table function fuse_encoding:

  • supports storage format Parquet
  • add new columns block_location and storage_format
  • the 2nd arg table_name and the 3rd arg column_name are optional
mysql> select * from fuse_encoding('default') limit 1;
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| table_name | storage_format | block_location                                         | column_name | column_type      | validity_size | compressed_size | uncompressed_size | level_one                | level_two |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| t0         | Parquet        | 1/5771/_b/h019b253321d570419b4a5c40bf3ac01b_v2.parquet | c           | INT NULL (INT32) |          NULL |            4774 |              7987 | plain,rle,rle_dictionary | zstd      |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
1 row in set (0.140 sec)
Read 112 rows, 22.33 KiB in 0.084 sec., 1.33 thousand rows/sec., 264.90 KiB/sec.

mysql> select * from fuse_encoding('default', 't1') limit 1;
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| table_name | storage_format | block_location                                         | column_name | column_type      | validity_size | compressed_size | uncompressed_size | level_one                | level_two |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| t1         | Parquet        | 1/5778/_b/h019b253321d47043aa0f6c5de4527a65_v2.parquet | c           | INT NULL (INT32) |          NULL |            4781 |              7987 | plain,rle,rle_dictionary | zstd      |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
1 row in set (0.048 sec)
Read 1 rows, 228.00 B in 0.025 sec., 39.56 rows/sec., 8.81 KiB/sec.

mysql> select * from fuse_encoding('default', 't1', 'c') limit 1;
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| table_name | storage_format | block_location                                         | column_name | column_type      | validity_size | compressed_size | uncompressed_size | level_one                | level_two |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
| t1         | Parquet        | 1/5778/_b/h019b253321d47043aa0f6c5de4527a65_v2.parquet | c           | INT NULL (INT32) |          NULL |            4781 |              7987 | plain,rle,rle_dictionary | zstd      |
+------------+----------------+--------------------------------------------------------+-------------+------------------+---------------+-----------------+-------------------+--------------------------+-----------+
1 row in set (0.056 sec)
Read 1 rows, 228.00 B in 0.033 sec., 30.09 rows/sec., 6.70 KiB/sec.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Dec 18, 2025
@dantengsky dantengsky force-pushed the feat/enhence-fuse-encoding branch from d2aba1a to 92c9236 Compare December 19, 2025 01:47
# Conflicts:
#	src/query/storages/fuse/src/io/read/meta/mod.rs
#	src/query/storages/fuse/src/table_functions/fuse_encoding.rs
@dantengsky dantengsky force-pushed the feat/enhence-fuse-encoding branch from 92c9236 to 378382d Compare December 19, 2025 03:53
@dantengsky dantengsky added the ci-cloud Build docker image for cloud test label Dec 19, 2025
@dantengsky dantengsky force-pushed the feat/enhence-fuse-encoding branch from 3284712 to 5240c18 Compare December 22, 2025 01:46
@dantengsky
Copy link
Member Author

@codex review

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. You're on a roll.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dantengsky dantengsky marked this pull request as ready for review December 22, 2025 07:10
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dantengsky dantengsky added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 22, 2025
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-19127-a093b8f-1766394796

note: this image tag is only available for internal use.

@dantengsky dantengsky added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 22, 2025
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-19127-4243adb-1766420222

note: this image tag is only available for internal use.

@dantengsky dantengsky merged commit 5d54da7 into databendlabs:main Dec 23, 2025
95 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants