Release v0.8.1

v0.8.1

Release date: May 30, 2024

This is a patch release, containing some important bug fix for flow's continuous aggregating in #4018

It's highly recommended to upgrade to this version if you're using v0.8.0.

🚀 Features

• feat: avoid some cloning when mirror requests to flownode by @fengjiachun in #4068
• feat: use cache in compaction by @evenyag in #3982
• feat: support compression on gRPC server by @shuiyisong in #3961
• feat: Adds RegionScanner trait by @evenyag in #3948
• feat: make create view procedure simple as others by @fengjiachun in #4001
• feat: respect time range when building parquet reader by @v0y4g3r in #3947
• feat: manual compaction by @v0y4g3r in #3988
• feat: add fallback logic for vmagent sending wrong content type by @sunng87 in #4009
• feat: Add TLS support for gRPC service by @realtaobo in #3957
• feat: remove one clone on constructing partition by @waynexia in #4028
• feat: add RawEntryReader and OneshotWalEntryReader trait by @WenyXu in #4027
• feat: round-robin selector by @waynexia in #4024
• feat: implement the LogStoreRawEntryReader and RawEntryReaderFilter by @WenyXu in #4030
• feat: support table level comment by @irenjj in #4042
• feat(opertor): check if a database is in use before dropping it by @etolbakov in #4035
• feat: change EXPIRE WHEN to EXPIRE AFTER by @waynexia in #4002
• feat: enable tcp keepalive for http server by @MichaelScofield in #4019
• feat: invoke flush_table and compact_table in fuzz tests by @WenyXu in #4045
• feat: open region in background by @WenyXu in #4052
• feat: Implement SHOW STATUS by @LYZJU2019 in #4050
• feat: Implements row group level parallel unordered scanner by @evenyag in #3992

🐛 Bug Fixes

• fix(flow): correctness bugs by @discord9 in #4018

• fix: try to fix broken CI by @WenyXu in #3998

• fix: move log_version() into build() of App to fix no log version on bootstrap by @zyy17 in #4004

• fix: try to fix unstable fuzz test by @WenyXu in #4003

• fix: can't print log because the tracing guard is dropped by @zyy17 in #4005

• fix(fuzz): sort inserted rows with primary keys and time index by @CookiePieWw in #4008

• fix: notifies all workers once a region is flushed by @evenyag in #4016

• fix(fuzz-tests): avoid to drop in-use database by @WenyXu in #4049

• fix(metric-engine): missing catchup implementation by @WenyXu in #4048

• fix: set local or session time_zone not work by @killme2008 in #4064

• fix: avoid acquiring lock during reading stats by @WenyXu in #4070

🚜 Refactor

• refactor: make the command entry cleaner by @zyy17 in #3981
• refactor: replace Expr with datafusion::Expr by @realtaobo in #3995
• refactor: remove unused log config by @v0y4g3r in #4021
• refactor(fuzz-tests): generate ts value separately by @WenyXu in #4056
• refactor: move Database to client crate behind testing feature by @tisonkun in #4059
• refactor(log_store): remove associated type Namespace and Entry in LogStore by @WenyXu in #4038

📚 Documentation

• docs: add toc for config docs by @zyy17 in #3974
• docs: add v0.8.0 TSBS report by @evenyag in #3983

⚙️ Miscellaneous Tasks

• ci: report CI failures with creating issues by @tisonkun in #3976
• chore: change binary array type from LargeBinaryArray to BinaryArray by @etolbakov in #3924
• chore: pin cargo-ndk to 3.5.4 by @WenyXu in #3979
• ci: check-status for nightly-ci by @tisonkun in #3984
• ci: fixup strings in check ci status by @tisonkun in #3987
• ci: add 'contents: write' permission by @zyy17 in #3989
• chore: remove a dbg! forget to remove by @discord9 in #3990
• chore(ci): add fuzz tests for distributed mode by @WenyXu in #3967
• ci: change the image name of nightly build by @zyy17 in #3994
• chore(ci): export kind logs by @WenyXu in #3996
• chore: add ttl to write_cache by @shuiyisong in #4010
• chore: log error for detail by @fengjiachun in #4011
• chore(ci): add more replicas by @WenyXu in #4015
• ci: skip notification for manual releases by @tisonkun in #4033
• chore: add logs for setting the region to writable by @WenyXu in #4044
• chore: add LAST_SENT_HEARTBEAT_ELAPSED metric by @WenyXu in #4062
• chore: bump to v0.8.1 by @discord9 in #4055

Build

• build(deps): upgrade promql-parser to 0.4 by @tisonkun in #4047
• build(deps): merge tower deps to workspace by @tisonkun in #4036
• build(deps): upgrade opendal to 0.46 by @tisonkun in #4037

New Contributors

• @LYZJU2019 made their first contribution in #4050

All Contributors

We would like to thank the following contributors from the GreptimeDB commun...

Release v0.9.0-nightly-20240527

What's Changed

• docs: add toc for config docs by @zyy17 in #3974
• ci: report CI failures with creating issues by @tisonkun in #3976
• chore: change binary array type from LargeBinaryArray to BinaryArray by @etolbakov in #3924
• docs: add v0.8.0 TSBS report by @evenyag in #3983
• chore: pin cargo-ndk to 3.5.4 by @WenyXu in #3979
• ci: check-status for nightly-ci by @tisonkun in #3984
• feat: use cache in compaction by @evenyag in #3982
• refactor: make the command entry cleaner by @zyy17 in #3981
• ci: fixup strings in check ci status by @tisonkun in #3987
• ci: add 'contents: write' permission by @zyy17 in #3989
• chore: remove a dbg! forget to remove by @discord9 in #3990
• feat: support compression on gRPC server by @shuiyisong in #3961
• feat: Adds RegionScanner trait by @evenyag in #3948
• chore(ci): add fuzz tests for distributed mode by @WenyXu in #3967
• ci: change the image name of nightly build by @zyy17 in #3994
• refactor: replace Expr with datafusion::Expr by @realtaobo in #3995
• fix: try to fix broken CI by @WenyXu in #3998
• feat: make create view procedure simple as others by @fengjiachun in #4001
• fix: move log_version() into build() of App to fix no log version on bootstrap by @zyy17 in #4004
• chore(ci): export kind logs by @WenyXu in #3996
• fix: try to fix unstable fuzz test by @WenyXu in #4003
• feat: respect time range when building parquet reader by @v0y4g3r in #3947
• fix: can't print log because the tracing guard is dropped by @zyy17 in #4005
• fix(fuzz): sort inserted rows with primary keys and time index by @CookiePieWw in #4008
• chore: add ttl to write_cache by @shuiyisong in #4010
• feat: manual compaction by @v0y4g3r in #3988
• chore: log error for detail by @fengjiachun in #4011
• feat: add fallback logic for vmagent sending wrong content type by @sunng87 in #4009
• chore(ci): add more replicas by @WenyXu in #4015
• refactor: remove unused log config by @v0y4g3r in #4021
• fix: notifies all workers once a region is flushed by @evenyag in #4016
• feat: Add TLS support for gRPC service by @realtaobo in #3957
• feat: remove one clone on constructing partition by @waynexia in #4028
• feat: add RawEntryReader and OneshotWalEntryReader trait by @WenyXu in #4027
• feat: round-robin selector by @waynexia in #4024
• ci: skip notification for manual releases by @tisonkun in #4033
• feat: implement the LogStoreRawEntryReader and RawEntryReaderFilter by @WenyXu in #4030

Full Changelog: v0.8.0...v0.9.0-nightly-20240527

Release v0.8.0

v0.8.0

Release date: May 18, 2024

👍 Highlights

• Flow Engine: which brings continuous aggregation capabilities.
• Column Type Modification: allows users to effortlessly alter the data type of columns within a table without the hassle of rebuilding the table or manually migrating data.
• Cluster Management Information Table: allows querying for information about the cluster. This functionality aids administrators in monitoring and managing the health status of the database cluster, facilitating prompt issue detection and resolution.
• Append-only Tables: Users can now create tables in Append-only mode by setting the append mode during table creation.
• DROP DATABASE: enables swift deletion of all tables and resources under a database instance.
• New Table Partitioning Methods and Syntax: considering the future need for frequent partition changes such as automatic partitioning or repartitioning, we're developing a new partitioning syntax.
• EXPLAIN ANALYZE <QUERY>: analyze and optimize query statements swiftly in distributed mode.

🚀 Features

• feat(cli): export metric physical tables first by @WenyXu in #3949
• feat(cli): prevent exporting physical table data by @WenyXu in #3978
• feat(drop_table): support to rollback table metadata by @WenyXu in #3692
• feat(flow): add types for every plan enum variant by @discord9 in #3938
• feat(flow): avg func rewrite to sum/count by @discord9 in #3955
• feat(flow): expire arrange according to time_index type by @discord9 in #3956
• feat(flow): flow node manager by @discord9 in #3954
• feat(flow): flow worker by @discord9 in #3934
• feat(flow): mirror insert req to flow node by @discord9 in #3858
• feat(flow): query table schema&refactor by @discord9 in #3943
• feat(flow): render map&related tests by @discord9 in #3581
• feat(flow): render reduce by @discord9 in #3769
• feat(flow): render source/sink by @discord9 in #3903
• feat(flow): transform substrait SELECT&WHERE&GROUP BY to Flow Plan by @discord9 in #3690
• feat(flow): tumble window func by @discord9 in #3968
• feat(fuzz): add alter logical table target by @CookiePieWw in #3818
• feat(fuzz): add create database target by @CookiePieWw in #3675
• feat(fuzz): add create logical table target by @CookiePieWw in #3756
• feat(fuzz): add insert logical table target by @CookiePieWw in #3842
• feat(fuzz): add validator for inserted rows by @CookiePieWw in #3932
• feat(fuzz): enable create-if-not-exists option by @CookiePieWw in #3732
• feat(fuzz): support to create metric table by @WenyXu in #3617
• feat(procedure): support to rollback by @WenyXu in #3625
• feat(promql): parameterize lookback by @etolbakov in #3630
• feat: Parquet reader builder supports building multiple ranges to read by @evenyag in #3841
• feat: add ChangeColumnType for AlterKind by @KKould in #3757
• feat: add filter_deleted option to avoid removing deletion markers by @v0y4g3r in #3707
• feat: add append mode to table options by @evenyag in #3624
• feat: add checksum for checkpoint data by @tisonkun in #3651
• feat: add metasrv start time to node info by @fengjiachun in #3883
• feat: add preserve arg to sqlness runner by @waynexia in #3724
• feat: add strict mode to validate protocol strings by @irenjj in #3638
• feat: add tinytext, mediumtext and longtext data types by @CookiePieWw in #3731
• feat: add validate method to CreateExpr by @waynexia in #3772
• feat: adds information_schema cluster_info table by @killme2008 in #3832
• feat: check partition rule by @waynexia in #3711
• feat: create database with options by @tizee in #3751
• feat: create view by @killme2008 in #3807
• feat: define CreateFlowTask and DropFlowTask by @WenyXu in #3801
• feat: get metasrv clusterinfo by @fengjiachun in #3696
• feat: ignore internal columns in SHOW CREATE TABLE by @WenyXu in #3950
• feat: impl Display for Statement by @irenjj in #3744
• feat: impl parser and operator for CREATE FLOW by @waynexia in #3806
• feat: impl show collation and show charset statements by @killme2008 in #3753
• feat: impl table_constraints table for information_schema by @killme2008 in #3698
• feat: implement FlowTaskMetadataManager by @WenyXu in #3766
• feat: implement CacheContainer & TableFlownodeSetCache by @WenyXu in #3885
• feat: implement drop flow parser by @WenyXu in #3888
• feat: implement drop flow procedure by @WenyXu in #3877
• feat: implement the CompositeTableRoute by @WenyXu in #3918
• feat: implement the CreateFlowProcedure by @WenyXu in #3810
• feat: implement the FrontendInvoker by @WenyXu in #3824
• feat: improve error message for typo in IF NOT EXISTS by @tisonkun in #3817
• feat: introduce TableNameCache & TableInfoCache & TableRouteCache by @WenyXu in #3895
• feat: introduce TableRouteCache to PartitionRuleManager by @WenyXu in #3922
• feat: introduce the CacheRegistry by @WenyXu in #3896
• feat: limit total rows copied in COPY TABLE FROM with LIMIT segment by @irenjj in #3910
• feat: limit total rows copied in COPY TABLE FROM stmt by @irenjj in #3819
• feat: limiting the size of query results to Dashboard by @realtaobo in #3901
• feat: metaclient builder options with role by @fengjiachun in #3909
• feat: metasrvs nodeinfo by @fengjiachun in #3873
• feat: migrate orc-rs to datafusion-orc by @realtaobo in #3923
• feat: pass QueryContext to FlowRequestHeader by @WenyXu in #3878
• feat: physical optimizer RemoveDuplicate to remove duplicate exec plans by @waynexia in #3839
• feat: remove timeout in the channel between frontend and datanode by @waynexia in #3962
• feat: skip read-only region when trying to flush on region full by @waynexia in #3966
• feat: support PromQL scalar by @Taylor-lagrange in #3693
• feat: support ALTER TABLE ... MODIFY COLUMN ... ... by @KKould in #3796
• feat: support any precision in PromQL by @Taylor-lagrange in #3933
• feat: support different types for CompatReader by @KKould in #3745
• feat: support distributed EXPLAIN ANALYZE by @waynexia in #3908
• feat: support evaluate expr in range query param by @Taylor-lagrange in #3823
• feat: support invalidate ...

Release v0.8.0-nightly-20240517

What's Changed

• feat(procedure): support to rollback by @WenyXu in #3625
• feat(flow): render map&related tests by @discord9 in #3581
• fix(readme): fix link of Ingester-js by @DiamondMofeng in #3668
• feat: add append mode to table options by @evenyag in #3624
• ci: setup-protoc always with token by @tisonkun in #3674
• chore: improve contributor click in git-cliff by @tisonkun in #3672
• feat(fuzz): add create database target by @CookiePieWw in #3675
• fix: DeserializedValueWithBytes::from_inner misusing by @WenyXu in #3676
• chore: remove global_ttl config by @v0y4g3r in #3673
• refactor: pass http method to metasrv http handler by @tisonkun in #3667
• refactor(flow): func spec api&use Error not EvalError in mfp by @discord9 in #3657
• refactor: bundle the lightweight axum test client by @tisonkun in #3669
• chore: add a fix-clippy Makefile target by @tisonkun in #3677
• chore: unify name metasrv by @fengjiachun in #3671
• feat: treat all number types as field candidates by @waynexia in #3670
• feat(fuzz): support to create metric table by @WenyXu in #3617
• feat: add checksum for checkpoint data by @tisonkun in #3651
• refactor: refactor alter table procedure by @WenyXu in #3678
• chore: create database api change in protobuf by @CookiePieWw in #3682
• feat: try get pk values from cache when applying predicate to parquet by @waynexia in #3286
• chore: remove TODO that has been done by @tisonkun in #3683
• chore(flow): more comments&lint by @discord9 in #3680
• refactor: drop table procedure by @WenyXu in #3688
• ci: add CODEOWNERS file by @tisonkun in #3691
• fix: cli export "create table" with quoted names by @dimbtp in #3684
• refactor: check error type before logging by @sunng87 in #3697
• feat(flow): transform substrait SELECT&WHERE&GROUP BY to Flow Plan by @discord9 in #3690
• feat: impl table_constraints table for information_schema by @killme2008 in #3698
• refactor: Move manifest manager lock to MitoRegion by @evenyag in #3689
• feat: add strict mode to validate protocol strings by @irenjj in #3638
• feat: get metasrv clusterinfo by @fengjiachun in #3696
• chore: cleanup tech debt in #3696 by @fengjiachun in #3705
• feat(promql): parameterize lookback by @etolbakov in #3630
• refactor: use toml2docs to generate config docs by @zyy17 in #3704
• test: Add tests for KvBackend trait implement by @realtaobo in #3700
• feat: add filter_deleted option to avoid removing deletion markers by @v0y4g3r in #3707
• refactor: remove redundant try_flush invocations by @tisonkun in #3706
• docs: update memtable config example by @evenyag in #3712
• feat: check partition rule by @waynexia in #3711
• fix(alter_table): ignore request outdated error by @WenyXu in #3715
• feat(drop_table): support to rollback table metadata by @WenyXu in #3692
• ci: use greptimedb-ci-tester account by @zyy17 in #3719
• chore(mito): set null value data size to i64 by @etolbakov in #3722
• chore: cleanup todos that should be panic by @tisonkun in #3720
• fix: remove ttl option from metadata region by @waynexia in #3726
• refactor: merge RegionHandleResult into RegionHandleResponse by @tisonkun in #3721
• docs: add style guide by @waynexia in #3730
• feat(fuzz): enable create-if-not-exists option by @CookiePieWw in #3732
• feat: add tinytext, mediumtext and longtext data types by @CookiePieWw in #3731
• feat: support invalidate schema name key cache by @realtaobo in #3725
• refactor(meta): Ensure all moving values remain unchanged between two transactions by @WenyXu in #3727
• chore: remove TableIdProvider by @fengjiachun in #3733
• refactor: avoid unnecessary cloning by @WenyXu in #3734
• build(deps): update datafusion to latest and arrow to 51.0 by @MichaelScofield in #3661
• refactor: avoid unnecessary alloc by using unwrap_or_else by @etolbakov in #3742
• build: update toolchain to nightly-2024-04-18 by @waynexia in #3740
• chore: update greptime-proto to main by @MichaelScofield in #3743
• feat: support PromQL scalar by @Taylor-lagrange in #3693
• feat: impl show collation and show charset statements by @killme2008 in #3753
• build(deps): bump rustls from 0.22.3 to 0.22.4 by @dependabot in #3764
• feat: add preserve arg to sqlness runner by @waynexia in #3724
• refactor(flow): split render.rs and other minor changes by @discord9 in #3750
• ci: temporary disable compatibility test by @waynexia in #3767
• ci: secrets.GITHUB_TOKEN not exist on self-hosted runner by @tisonkun in #3765
• feat: update dashboard to v0.5.0 by @ZonaHex in #3768
• docs: add how-to-write-fuzz-tests.md by @WenyXu in #3763
• fix: set is_time_index properly on updating physical table's schema by @waynexia in #3770
• fix: the dropping_regions guards should be dropped on procedure done by @MichaelScofield in #3771
• fix: operating region guards should be dropped when procedure is done by @MichaelScofield in #3775
• fix: consider both db param and extended db header in Prometheus HTTP API by @waynexia in #3776
• fix: do not remove deletion markers when window time range overlaps by @v0y4g3r in #3773
• chore: bump jobserver by @v0y4g3r in #3778
• fix: promql scalar when input empty batch by @Taylor-lagrange in #3779
• build: only build amd64 image for CentOS by @evenyag in #3784
• refactor: move the version string to common by @MichaelScofield in #3783
• feat: write manifests in background tasks by @evenyag in #3709
• feat: add ChangeColumnType for AlterKind by @KKould in #3757
• refactor: simplify the PaginationStream by @WenyXu in #3787
• feat: support different types for CompatReader by @KKould in #3745
• chore: always obsolete wal to avoid discontinuous log entries by @v0y4g3r in #3789
• feat: impl Display for Statement by @irenjj in #3744
• refactor(flow): refine comments and code by @zhongzc in #3785
• feat: add validate method to CreateExpr by @waynexia in #3772
• fix: post process result on query full column name of prom labels API by @waynexia in #3793
• fix: fix fuzz test ci by @WenyXu in #3795
• fix!: use the right pr...

Release v0.7.2

v0.7.2

Release date: April 08, 2024

Breaking changes

• refactor!: Renames the new memtable to PartitionTreeMemtable by @evenyag in #3547
• fix!: columns table in information_schema misses some columns by @killme2008 in #3639

🚀 Features

• feat: Partition memtables by time if compaction window is provided by @evenyag in #3501
• feat: acquire all locks in procedure by @WenyXu in #3514
• feat: implement the drop database parser by @WenyXu in #3521
• feat(mito): Checks whether a region should flush periodically by @evenyag in #3459
• feat(metasrv): implement maintenance by @tisonkun in #3527
• feat: update dashboard to v0.4.9 by @ZonaHex in #3531
• feat: support per table memtable options by @evenyag in #3524
• feat(flow): shared in-memory state for dataflow operator by @discord9 in #3508
• feat: update pgwire to 0.20 for improved performance by @sunng87 in #3538
• feat: support multi params in promql range function macro by @Taylor-lagrange in #3464
• feat: support append-only mode in time-series memtable by @v0y4g3r in #3540
• feat: Able to pretty print sql query result in http output by @YCCDSZXH in #3539
• feat: return new added columns in region server's extension response by @waynexia in #3533
• feat: update physical table's schema on creating logical table by @waynexia in #3570
• feat: implement the drop database procedure by @WenyXu in #3541
• feat: batch alter logical tables by @fengjiachun in #3569
• feat: support time range in copy table by @v0y4g3r in #3583
• feat: update physical table schema on alter logical tables by @fengjiachun in #3585
• feat(auth): watch file user provider by @tisonkun in #3566
• feat: Support printing postgresql's bytea data type in its "hex" and "escape" format by @J0HN50N133 in #3567
• feat: Implement append mode for a region by @evenyag in #3558
• feat: create regions persist true by @fengjiachun in #3590
• feat: remove support for logical tables in the create table procedure by @fengjiachun in #3592
• feat: adds metric engine to information_schema engines table by @killme2008 in #3599
• feat: support 2+2 and /status/buildinfo by @waynexia in #3604
• feat(tql): add initial support for start,stop,step as sql functions by @etolbakov in #3507
• feat: Implement an unordered scanner for append mode by @evenyag in #3598
• feat: allow cross-schema query in promql by @sunng87 in #3545
• feat: let alter table procedure can only alter physical table by @fengjiachun in #3613
• feat(function): add timestamp epoch integer support for to_timezone by @etolbakov in #3620
• feat: impl show index and show columns by @killme2008 in #3577
• feat: group requests by peer by @fengjiachun in #3619
• feat: Support outputting various date styles for postgresql by @J0HN50N133 in #3602
• feat: reject invalid timestamp ranges in copy statement by @v0y4g3r in #3623
• feat(procedure): auto split large value to multiple values by @WenyXu in #3605
• feat(procedure): enable auto split large value by @WenyXu in #3628
• feat: introduce wal benchmarker by @niebayes in #3446
• feat: adding victoriametrics remote write by @sunng87 in #3641
• feat: Add timers to more mito methods by @evenyag in #3659
• feat: cluster information by @fengjiachun in #3631
• feat: update dashboard to v0.4.10 by @ZonaHex in #3663

🐛 Bug Fixes

• fix: clone data instead of moving it - homemade future is dangerous by @waynexia in #3542
• fix: performance degradation caused by config change by @v0y4g3r in #3556
• fix(flow): Arrange get range with batch unaligned by @discord9 in #3552
• fix: set http response chartset to utf-8 when using table format by @xxxuuu in #3571
• fix: update pk_cache in compat reader by @waynexia in #3576
• fix: incorrect version info in by @waynexia in #3586
• fix: canonicalize catalog and schema names by @killme2008 in #3600
• fix: adjust status code to http error code map by @waynexia in #3601
• fix: run purge jobs in another scheduler by @evenyag in #3621
• fix: mistakely removes compaction inputs on failure by @v0y4g3r in #3635
• fix: move object store read/write timer into inner by @dimbtp in #3627
• fix: construct correct pk list with pre-existing pk by @waynexia in #3614

🚜 Refactor

• refactor: remove removed-prefixed keys by @WenyXu in #3535
• refactor: introduce the DropTableExecutor by @WenyXu in #3534
• refactor: refactor CacheInvalidator by @WenyXu in #3550
• refactor: handle error for http format by @tisonkun in #3548
• refactor: remove redudant PromStoreProtocolHandler::write by @v0y4g3r in #3553
• refactor: reduce one clone by carefully pass ready boundary by @tisonkun in #3543
• re...

Release v0.8.0-nightly-20240401

What's Changed

• feat: acquire all locks in procedure by @WenyXu in #3514
• docs: revise README file by @tisonkun in #3522
• feat: implement the drop database parser by @WenyXu in #3521
• feat(mito): Checks whether a region should flush periodically by @evenyag in #3459
• ci: unassign issues stale 14 days ago by @tisonkun in #3529
• docs: readme style and project status by @tisonkun in #3528
• docs: improve fn comments by @tisonkun in #3526
• feat(metasrv): implement maintenance by @tisonkun in #3527
• refactor: remove removed-prefixed keys by @WenyXu in #3535
• feat: http header with metrics by @shuiyisong in #3536
• chore: avoid unnecessary cloning by @fengjiachun in #3537
• feat: update dashboard to v0.4.9 by @ZonaHex in #3531
• refactor: introduce the DropTableExecutor by @WenyXu in #3534
• feat: support per table memtable options by @evenyag in #3524
• feat(flow): shared in-memory state for dataflow operator by @discord9 in #3508
• feat: update pgwire to 0.20 for improved performance by @sunng87 in #3538
• fix: clone data instead of moving it - homemade future is dangerous by @waynexia in #3542
• feat: support multi params in promql range function macro by @Taylor-lagrange in #3464
• feat: support append-only mode in time-series memtable by @v0y4g3r in #3540
• feat: Able to pretty print sql query result in http output by @YCCDSZXH in #3539
• chore: fix comment in fetch-dashboard-assets.sh by @tisonkun in #3546
• refactor!: Renames the new memtable to PartitionTreeMemtable by @evenyag in #3547
• refactor: refactor CacheInvalidator by @WenyXu in #3550
• refactor: handle error for http format by @tisonkun in #3548
• fix: performance degradation caused by config change by @v0y4g3r in #3556
• refactor: remove redudant PromStoreProtocolHandler::write by @v0y4g3r in #3553
• ci: use a PAT to list all writers by @tisonkun in #3559
• fix(flow): Arrange get range with batch unaligned by @discord9 in #3552
• refactor: reduce one clone by carefully pass ready boundary by @tisonkun in #3543
• chore: avoid confusing TryFrom by @tisonkun in #3565
• test: add a parameter type mismatch test case to sql integration test by @xxxuuu in #3568
• feat: return new added columns in region server's extension response by @waynexia in #3533
• fix: set http response chartset to utf-8 when using table format by @xxxuuu in #3571
• feat: update physical table's schema on creating logical table by @waynexia in #3570
• chore: retain original headers by @tisonkun in #3572
• feat: implement the drop database procedure by @WenyXu in #3541
• ci: use single commit on the deployment branch by @evenyag in #3580
• refactor: try upgrade regex-automata by @tisonkun in #3575
• feat: batch alter logical tables by @fengjiachun in #3569
• chore: limit OpenDAL's feature gates by @waynexia in #3584
• fix: update pk_cache in compat reader by @waynexia in #3576
• fix: incorrect version info in by @waynexia in #3586
• feat: support time range in copy table by @v0y4g3r in #3583
• build(deps): remove some unused dependencies by @dimbtp in #3582
• feat: update physical table schema on alter logical tables by @fengjiachun in #3585
• chore: Delete CODE_OF_CONDUCT.md by @tisonkun in #3578
• feat(auth): watch file user provider by @tisonkun in #3566
• feat: Support printing postgresql's bytea data type in its "hex" and "escape" format by @J0HN50N133 in #3567
• chore: add back core dependency by @shuiyisong in #3588
• feat: Implement append mode for a region by @evenyag in #3558
• feat: create regions persist true by @fengjiachun in #3590
• refactor: refactor drop table executor by @WenyXu in #3589
• test: add tests for drop databases by @WenyXu in #3594
• feat: remove support for logical tables in the create table procedure by @fengjiachun in #3592
• chore: do not reply for broadcast msg by @fengjiachun in #3595
• feat: adds metric engine to information_schema engines table by @killme2008 in #3599
• fix: canonicalize catalog and schema names by @killme2008 in #3600
• test: add more integration test for kafka wal by @niebayes in #3190
• fix: adjust status code to http error code map by @waynexia in #3601
• refactor(table): remove unused table requests by @zhongzc in #3603
• feat: support 2+2 and /status/buildinfo by @waynexia in #3604
• feat(tql): add initial support for start,stop,step as sql functions by @etolbakov in #3507
• feat: Implement an unordered scanner for append mode by @evenyag in #3598
• refactor: cache invalidator by @fengjiachun in #3611
• feat: allow cross-schema query in promql by @sunng87 in #3545
• ci: ignore type in sqlness sql and result files by @waynexia in #3616
• feat: let alter table procedure can only alter physical table by @fengjiachun in #3613
• feat(function): add timestamp epoch integer support for to_timezone by @etolbakov in #3620
• feat: impl show index and show columns by @killme2008 in #3577

New Contributors

• @YCCDSZXH made their first contribution in #3539
• @xxxuuu made their first contribution in #3568

Full Changelog: v0.7.1...v0.8.0-nightly-20240401

Release v0.8.0-nightly-20240325

What's Changed

• feat: acquire all locks in procedure by @WenyXu in #3514
• docs: revise README file by @tisonkun in #3522
• feat: implement the drop database parser by @WenyXu in #3521
• feat(mito): Checks whether a region should flush periodically by @evenyag in #3459
• ci: unassign issues stale 14 days ago by @tisonkun in #3529
• docs: readme style and project status by @tisonkun in #3528
• docs: improve fn comments by @tisonkun in #3526
• feat(metasrv): implement maintenance by @tisonkun in #3527
• refactor: remove removed-prefixed keys by @WenyXu in #3535
• feat: http header with metrics by @shuiyisong in #3536
• chore: avoid unnecessary cloning by @fengjiachun in #3537
• feat: update dashboard to v0.4.9 by @ZonaHex in #3531
• refactor: introduce the DropTableExecutor by @WenyXu in #3534
• feat: support per table memtable options by @evenyag in #3524
• feat(flow): shared in-memory state for dataflow operator by @discord9 in #3508
• feat: update pgwire to 0.20 for improved performance by @sunng87 in #3538
• fix: clone data instead of moving it - homemade future is dangerous by @waynexia in #3542
• feat: support multi params in promql range function macro by @Taylor-lagrange in #3464
• feat: support append-only mode in time-series memtable by @v0y4g3r in #3540
• feat: Able to pretty print sql query result in http output by @YCCDSZXH in #3539
• chore: fix comment in fetch-dashboard-assets.sh by @tisonkun in #3546
• refactor!: Renames the new memtable to PartitionTreeMemtable by @evenyag in #3547
• refactor: refactor CacheInvalidator by @WenyXu in #3550
• refactor: handle error for http format by @tisonkun in #3548
• fix: performance degradation caused by config change by @v0y4g3r in #3556
• refactor: remove redudant PromStoreProtocolHandler::write by @v0y4g3r in #3553
• ci: use a PAT to list all writers by @tisonkun in #3559
• fix(flow): Arrange get range with batch unaligned by @discord9 in #3552
• refactor: reduce one clone by carefully pass ready boundary by @tisonkun in #3543
• chore: avoid confusing TryFrom by @tisonkun in #3565
• test: add a parameter type mismatch test case to sql integration test by @xxxuuu in #3568
• feat: return new added columns in region server's extension response by @waynexia in #3533

New Contributors

• @YCCDSZXH made their first contribution in #3539
• @xxxuuu made their first contribution in #3568

Full Changelog: v0.7.1...v0.8.0-nightly-20240325

Release v0.8.0-nightly-20240318

What's Changed

• feat: acquire all locks in procedure by @WenyXu in #3514
• docs: revise README file by @tisonkun in #3522
• feat: implement the drop database parser by @WenyXu in #3521
• feat(mito): Checks whether a region should flush periodically by @evenyag in #3459

Full Changelog: v0.7.1...v0.8.0-nightly-20240318

Release v0.7.1

What's Changed

This release contains an important bug fix while decoding the Prometheus remote write proto #3505.

It's highly recommended to upgrade to this version if you're using v0.7.

Features

• feat: impl some "set"s to adapt to some client apps by @MichaelScofield in #3443
• feat: update dashboard to v0.4.8 by @ZonaHex in #3450
• feat: max-txn-ops option by @fengjiachun in #3458
• feat(influxdb): add db query param support for v2 write api by @etolbakov in #3445
• feat: support first_value/last_value in range query by @Taylor-lagrange in #3448
• feat: clamp function by @waynexia in #3465
• feat(fuzz): validate columns by @WenyXu in #3485
• feat: to_timezone function by @tisonkun in #3470
• feat(flow): accumlator for aggr func by @discord9 in #3396
• feat(flow): plan def by @discord9 in #3490
• feat: improve prom write requests decode performance by @v0y4g3r in #3478
• feat(fuzz): add insert target by @zhongzc in #3499
• feat(fuzz): add alter table target by @WenyXu in #3503
• feat: support decode gzip if influxdb write specify it by @tisonkun in #3494
• feat: Partition memtables by time if compaction window is provided by @evenyag in #3501

Fixes

• fix: add support for influxdb basic auth by @shuiyisong in #3437
• fix: fix incorrect COM_STMT_PREPARE reply by @WenyXu in #3463
• fix: impl RecordBatchStream method explicitly by @shuiyisong in #3482
• fix: make max-txn-ops limit valid by @fengjiachun in #3481
• fix: fix f64 has no sufficient precision during parsing by @WenyXu in #3483
• fix: freeze data buffer in shard by @evenyag in #3468
• fix: allow passing extra table options by @evenyag in #3484
• fix(common-time): allow building nanos timestamp from parts split from i64::MIN by @zhongzc in #3493
• fix: adjust fill behavior of range query by @Taylor-lagrange in #3489
• fix: decoding prometheus remote write proto doesn't reset the value by @waynexia in #3505
• fix: correctly generate sequences when the value is pre-existed by @MichaelScofield in #3502

Breaking Changes

• fix!: remove error message from http header to avoid panic by @sunng87 in #3506

Others

• build(deps): upgrade opendal to 0.45.1 by @tisonkun in #3432
• chore: add bin opt to build cmd by @WenyXu in #3440
• chore: specify binary name by @WenyXu in #3449
• test: add fuzz test for create table by @WenyXu in #3441
• refactor: separate the quote char and value by @WenyXu in #3455
• perf: Reduce decode overhead during pruning keys in the memtable by @evenyag in #3415
• chore: remove repetitive words by @gcmutator in #3469
• refactor: introduce new Output with OutputMeta by @shuiyisong in #3466
• refactor: make http api returns non-200 status code by @crwen in #3473
• ci: use another mirror for etcd image by @waynexia in #3486
• perf: more benchmarks for memtables by @evenyag in #3491
• refactor: validate constraints eagerly by @tisonkun in #3472
• ci: attempt to setup docker cache for etcd by @sunng87 in #3488
• chore: bump version to v0.7.1 by @evenyag in #3510
• docs: add v0.7 TSBS benchmark result by @evenyag in #3512
• ci: add bin options to reduce build burden by @WenyXu in #3518

New Contributors

• @gcmutator made their first contribution in #3469
• @crwen made their first contribution in #3473

Full Changelog: v0.7.0...v0.7.1

Release v0.7.0

What's Changed

Highlights

• Metric Engine: handles a large number of small tables, making it particularly suitable for cloud-native monitoring.
• Region Migration: enhances the user experience and simplifies region migrations with straightforward SQL commands.
• Inverted Index: dramatically improves the efficiency of locating data segments relevant to user queries, significantly reducing the IO operations needed for scanning data files and thus accelerating the query process.

Features:

• feat: add modulo function by @etolbakov in #3147
• feat: support HTTP&gRPC&pg set timezone by @Taylor-lagrange in #3125
• feat(mito): enable inverted index by @zhongzc in #3158
• feat: upgrade pgwire to 0.19 by @sunng87 in #3157
• feat: let tables API return a stream by @fengjiachun in #3170
• feat: add tests-fuzz crate by @WenyXu in #3173
• feat: tables stream with CatalogManager by @fengjiachun in #3180
• feat: adds date_format function by @killme2008 in #3167
• feat: auto config cache size according to memory size by @QuenKar in #3165
• feat: precise filter for mito parquet reader by @waynexia in #3178
• feat: adds parse options for SQL parser by @killme2008 in #3193
• feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator by @WenyXu in #3182
• feat: make procedure able to return output by @WenyXu in #3201
• feat: copy database from by @v0y4g3r in #3164
• feat: introduce information schema provider cache by @WenyXu in #3208
• feat: lazy initialize vector builder on write by @waynexia in #3210
• feat: make query be aware of timezone setting by @killme2008 in #3175
• feat: add create alter table expr translator by @WenyXu in #3203
• feat: read column and region info from state cache by @waynexia in #3222
• feat: enable concurrent write by @WenyXu in #3214
• feat: add Arrow IPC output format for http rest api by @sunng87 in #3177
• feat: change Range Query’s default align behavior aware of timezone by @killme2008 in #3219
• feat: update dashboard to v0.4.7 by @ZonaHex in #3229
• feat: http sql api return schema on empty resultset by @sunng87 in #3237
• feat: add pg create alter table expr translator by @WenyXu in #3206
• feat: read metadata from write cache by @QuenKar in #3224
• feat: batch create ddl by @fengjiachun in #3194
• feat: add insert/select generator & translator by @WenyXu in #3240
• feat: return request outdated error on handling alter by @waynexia in #3239
• feat: don't map semantic type in metric engine by @waynexia in #3243
• feat: create tables in batch on prom write by @fengjiachun in #3246
• feat: Only allow inserts and deletes operations to be executed in parallel by @fengjiachun in #3257
• feat: basic types for Dataflow Framework by @discord9 in #3186
• feat: initial configuration for grafana dashboard by @waynexia in #3263
• feat: support fraction part in timestamp by @discord9 in #3272
• feat: use simple filter to prune memtable by @waynexia in #3269
• feat: Basic Definitions for Expression&Functions for Dataflow by @discord9 in #3267
• feat: support cache for batch_get in CachedMetaKvBackend by @fengys1996 in #3277
• feat: put all filter exprs in a filter plan separately by @waynexia in #3288
• feat: administration functions by @killme2008 in #3236
• feat(mito): adjust seg size of inverted index to finer granularity instead of row group level by @zhongzc in #3289
• feat(mito): add options to ignore building index for specific column ids by @zhongzc in #3295
• feat: organize tracing on query path by @waynexia in #3310
• feat: impl partitions and region_peers information schema by @killme2008 in #3278
• feat: batch get physical table routes by @fengjiachun in #3319
• feat: Defines structs in the merge tree memtable by @evenyag in #3326
• feat(metric-engine): set index options for data region by @zhongzc in #3330
• feat: data buffer and related structs by @v0y4g3r in #3329
• feat: Implement KeyDictBuilder for the merge tree memtable by @evenyag in #3334
• feat: replace pk index with pk_weight during freeze by @v0y4g3r in #3343
• feat: merge tree data parts by @v0y4g3r in #3346
• feat: Defines more structs and methods for a partitioned merge tree by @evenyag in #3348
• feat(flow): impl ScalarExpr&Scalar Function by @discord9 in #3283
• feat: impl migrate_region and procedure_state SQL function by @killme2008 in #3325
• feat: add isnull function by @KKould in #3360
• feat: skip filling NULL for put and delete requests by @waynexia in #3364
• feat: impl merge reader for DataParts by @v0y4g3r in #3361
• feat: Implement write and fork for the new memtable by @evenyag in #3357
• feat: distinguish between different read paths by @v0y4g3r in #3369
• feat: Add freeze and fork method to the memtable by @evenyag in #3374
• feat: merge tree dedup reader by @v0y4g3r in #3375
• feat: Implement iter for the new memtable by @evenyag in #3373
• feat: Implement dedup for the new memtable and expose the config by @evenyag in #3377
• feat: change how region id maps to region worker by @waynexia in #3384
• feat: make tls certificates/keys reloadable (part 1) by @sunng87 in #3335
• feat(grafana): enable shared tooltip, add raft engine throughput by @waynexia in #3387
• feat: Implement partition eviction and only add value size to write buffer size by @evenyag in #3393
• feat: enable zstd compression and encodings in merge tree data part by @v0y4g3r in #3380
• feat(flow): impl for MapFilterProject by @discord9 in #3359
• feat: flush or compact table and region functions by @killme2008 in #3363
• feat: Use a partition level map to look up pk index by @evenyag in #3400
• feat(index): measure memory usage in global instead of single-column and add metrics by @zhongzc in #3383
• feat: enable ArrowFlight compression by @tisonkun in #3403
• feat: zero copy on split rows by @fengjiachun in #3407
• feat: Support automatic DNS lookup for kafka bootstrap servers by @J0HN50N133 in #3379
• feat: add configuration for tls watch option by @sunng87 in #3395
• feat: employ sparse key encoding for shard lookup by @v0y4g3r in #3410
• feat: support Create Table ... Like by @KKould in #3372
• feat: tableref cache by @fengjiachun in #3420
• feat: add verbose support for tql explain/an...