Skip to content

Latest commit

 

History

History
516 lines (323 loc) · 48.5 KB

CHANGELOG.MD

File metadata and controls

516 lines (323 loc) · 48.5 KB
Raw

v3.5.4

New features

  • Support for dynamic adjustment of full synchronization speed limit parameters rsync-timeout-ms and throttle-bytes-per-second#2633@cheniujh

  • Pika disk I/O speed limit parameters support OnlyRead, OnlyWrite, ReadAndWrite, with the default being OnlyWrite#2599@vacheli

  • Display the results of info key space 1 in info all and show it on the monitoring interface #2603@XiaoLiang2333

Improvement

  • Added Go tests for the slotsmigrate#2576@chejinge

  • Optimization of INFO command execution time, reducing disk check frequency #2554 @wangshao1

  • Added Redis tcl tests for five basic data types commands #2527@Mixficsol

Bugfix

  • Fixed an issue where using Pika Exporter could result in uneven slots distribution#2651@chejinge

  • Fixed an issue where the Codis dashboard could not correctly update the master instance status#2650@vacheli

  • Fixed a master-slave synchronization anomaly caused by Redis transaction binlog parsing failure#2642@chejinge

  • Fixed an issue where starting Pika Exporter without parameters caused startup failure#2640@Polaris3003

  • Fixed an issue where using Pika Operator to start a Codis-proxy cluster caused a panic#2632@chejinge

  • Fixed an issue where the cp command failed during automated tests of binaries compiled in CI #2614@cheniujh

  • Fixed an issue where an uninitialized variable caused cache startup failure#2613@chejinge

  • Fixed the abnormal function of dynamically modifying parameters of userpass and userblacklist#2600@chejinge

  • Fix the problem of inconsistent scard sscan results#2596@chejinge

  • Fix the problem that when max-rsync-parallel-num is greater than 4, slave will coredump during master-slave replication#2595@chejinge

  • Adjust the number of thread pool threads that are not commonly used to avoid performance loss due to idle running#2590 @chejinge

  • Fix the problem of Pika transaction edge test case not passing#2586 @chejinge

  • change cache-model to cache-mode#2585@chejinge

  • Fix the problem of info all deadlock after using info keyspace#2584 @chejinge

  • Solve the problem of incompatibility between 353 and 352 extreme scenarios caused by modifying the dictionary order of zsetscorekeycomparatorimpl#2583 @wangshao1

  • Fix compact deadlock problem#2581 @chejinge

  • Slotmigrate add go test#2576@chejinge

  • Update pika version used by Pika Operater#2572@chejinge

  • Fix the problem of abnormal blockcache value after config rewrite#2561@chejinge

  • Fixed the problem of incorrect value after slotmigrate config rewrite#2548@chejinge

  • Fix the problem that spop may cause inconsistency between master and slave data#2541@chenbt-hz

  • Fix the problem of out of bounds in CloseFd(it->second[i])#2539@chejinge

  • Fix the potential deadlocks in Flushall and FlushDB, and remove the FlushSubDB interface#2533@Mixficsol

  • Add a parameter to control whether to clean up data files generated by TCL tests, preventing obsolete data from occupying disk space#2507@Mixficsol

v3.5.3

New features

  • Pika supports ACL#2013 @lqxhub

  • Automatically resume service when Codis dashboard coroutine panics#2349@chengyu-l

  • During the full replication process, the slave node of the pika service does not receive read traffic requests.#2197 @tedli

  • Pika cache adds bimap data type.#2253 @chejinge

  • Delete the remaining Slots in Sharing mode. There is only DB under Pika, and there are multiple DBs under one Pika.#2251 @Mixficsol

  • Pika exporter exposes cache-related data collection indicators.#2318 @dingxiaoshuai

  • Pika supports separation of fast and slow commands.#2162 @dingxiaoshuai

  • After pika executes bgsave, retain the unix timepoint.#2167 @hero-heng

  • Pika supports dynamic configuration of the disable_auto_compations parameter.#2257 @hero-heng

  • Pika supports Redis Stream.#1955 @KKorpse

  • Pika supports large key analysis tools#2195 @sjcsjc123

  • Pika supports dynamic adjustment of Pika cache parameters#2253 @chejinge

  • Updated Pika benchmark tool to support more interface stress tests.#2222@wangshao1

  • Pika Operator supports automatic expansion of pika clusters.#2121@machinly

  • Add the CompactRange command to support compacting keys within a certain range.#2163@u6th9d

  • Add small time cost compaction policy.#2172@u6th9d

  • Upgrade RocksDB version to v8.7.3.#2157@JasirVoriya

  • Pika distributed cluster Codis proxy adds new observable indicators.#2199@dingxiaoshuai

  • Pika distributed cluster supports automatic failover.#2386@chengyu-l

  • Pika supports redis rename-command function.#2455@Mixficsol

  • Optimize codis slot migration speed and support dynamic modification of migration thread and speed.#2486 @chejinge

  • Pika supports dynamic adjustment of the max-conn-rbuf-size parameter.#2434 @HappyUncle

  • Pika-operator supports namespace and can deploy different clusters under different namespaces.#2480 @Y-Rookie

  • Pika-operator supports monitoring indicator collection and automatically launches Pika-experter.#2451 @chengyu-l

  • ACL forward compatible with userblacklist.#2459 @dingxiaoshuai

  • Enriched Pika TCL test set .#2497 @Mixficsol

  • Enriched Pika Gotest test set.#2502 @Mixficsol

Bugfix

  • Fixed an issue where Pika would accidentally delete dump files during full replication from the node.#2377@wangshao1

  • Fixed the processing logic after the slave node receives an abnormal response packet from the master during the master-slave replication process.#2319@wangshao1

  • Call disable compaction when pika executes the shutdown command to improve the process exit speed. #2345 @panlei-coder

  • Fix the problem of inaccurate Codis-dashboard Redis Memory value.#2337 @Mixficsol

  • The INFO command is time-consuming and optimized to reduce the frequency of disk checks. #2197 @chejinge

  • Fixed the issue where rsync deletes temporary files with incorrect paths and fails to delete them, causing rocksdb to fail to open.#2186@wangshao1

  • Fixed the problem that the compact, bgsave, and info keyspace commands did not specify the db name, resulting in some coredump commands.#2194@u6th9d

  • Codis dashboard uses info replication instead of info command to search master ip to reduce the performance impact on Pika. #2198 @chenbt-hz

  • Fix Pika cache to use edge cases to solve the problem of cache and DB data inconsistency in some scenarios.#2225 @chejinge

  • Fixed the issue where Segmentation fault would be reported when the dump folder is empty.#2265 @chenbt-hz

  • Fixed the problem that some command caches did not take effect due to flag calculation errors.#2217 @lqxhub

  • Fixed the problem that in master-slave replication mode, after the master instance flushdb, the slave instance cannot be accessed due to deadlock.#2249@ForestLH

  • Fixed the issue where some commands did not judge the return value of RocksDB.#2187@callme-taota

  • Fixed the problem that some command caches did not take effect due to flag calculation errors.#2217 @lqxhub

  • Fixed the problem that in master-slave replication mode, after the master instance flushdb, the slave instance cannot be accessed due to deadlock.#2249@ForestLH

  • Fixed the issue where some commands did not judge the return value of RocksDB.#2187@callme-taota

  • Fix the problem of info keyspace returning wrong results.#2369@Mixficsol

  • Standard function return value and initial value.#2176@Mixficsol

  • Fixed the problem of inaccurate network monitoring indicator statistics.#2234@chengyu-l

  • Fixed an issue where some parameters in configuration file loading were abnormal.#2218@jettcc

  • Fix Codis dashboard cpu used 100%.#2393@chengyu-l

  • Fix the problem of abnormal display of master and slave roles in Codis fe of pika.#2387@chengyu-l

  • Fix the problem of data inconsistency after migrating data.#2485@chejinge

  • Fix dbsize calculation error problem.#2494@chejinge

  • Fixed the issue of inaccurate display of the Codis-dashboard interface after scaling up or down or starting and stopping pods.#2475@chengyu-l

  • Fix the problem of repeated locking of DB layer.#2372 @Mixficsol

  • Fixed the problem of data loss caused by failure to perform full copy.#2439@wangshao1

  • Fixed the problem that during the master-slave replication process, the master instance did not correctly respond to the slave's synchronization request when executing bgsave.#2437@wangshao1

  • During the full copy process, add data synchronization status to clarify the data synchronization progress.#2430@baixin01

  • Fixed the issue where the slave database did not lock the key of the operation when applying binlog, resulting in data inconsistency.#2409 @chejinge

  • Fix the problem of master instance coredump during codis slot migration process.#2415 @chejinge

  • Fixed the problem of deleting the dump file being used during the master-slave replication process.#2377@wangshao1

  • Fixed the problem of rsync response error from slave instance during master-slave replication process.#2319@wangshao1

  • Fixed the problem that in master-slave replication mode, after the master instance flushdb, the slave instance cannot be accessed due to deadlock.#2372 @Mixficsol

v3.5.2

New features

Bugfix

  • Fixed coredump issue when using SETRANGE command in Pika. #2125 @chejinge

  • Fixed full replication issue caused by deleting Clearreplicationid from binlog. #2136 @Mixficsol

  • Modified lock granularity to improve binlog writing performance in Pika. #2129 @wangshao1

  • Added redis-copy traffic replication tool. #2044 @Mixficsol

  • Fixed potential data overflow issue in complex data types member variables. #2016 @u6th9d

  • Fixed incorrect return value issue in decr command. #2092 @dingxiaoshuai123

  • Fixed issue where SETRANGE and SETBIT commands did not retain the original key's expiration time. #2095 @u6th9d

v3.5.1

New features

  • slow log Adds statistics on queue waiting time #1997 @wangshao1

  • ReplicationID is used for primary/secondary replication #1951 @Mixficsol

  • WAL uses the disablewal command to support dynamic shutdown #2015 @Mixficsol

  • The number of threads flushed and the number of threads compaction dynamically adjust into one #2014 @Tianpingan

  • The RocksDB version was upgraded to v8.3.3 #1999 @dingxiaoshuai123

  • Added documentation for configuring Pika in Macos environment #2003 @klboke

  • Added the ability to periodically print the length of the work queue to quickly locate problems when the queue is blocked #1978 @Tianpingan

  • Added an indicator to detect the entire cluster using a pika_exporter #1953 @chenbt-hz

  • Realize automatic registration of Pika service on K8s environment, and automatically register at startup, so as to realize self-organization of cluster #1931 @machinly

Bugfix

  • Reduces unnecessary log printing by the exporter, reducing CPU utilization #1945 @Mixficsol

  • The sentinel mechanism has been upgraded to carry out logical deletion of primary nodes that fall offline #1949 @Mixficsol

  • Adjust the rate_limit parameter to fix the situation that RPS is 0 during pressure measurement #2009 @chejinge

  • Fixed the logical determination of empty path when traversing data files in INFODATA command #1996 @Mixficsol

  • Improved the description of some missing parts of the document #1962 @baerwang

  • Use make-j to improve build speed #1933 @xiezheng-XD

  • Fixed an issue where large burrs appeared on the Codis line #2016 @chejinge

  • Fixed an issue where tools could not be compiled in Macos #2011 @A2ureStone

v3.5.0

Major features

1.Removal of Rsync

  1. Remove Rsync.
  2. Implement breakpoint resuming, rate limitation, and file verification.
  3. When synchronizing master-slave in Pika, perform the master run_id verification.

2.Compatibility with More Redis Commands

  1. Support the UNLINK command.
  2. Support the INFO COMMANDSTATS command.
  3. Support the HELLO and SETNAME commands.
  4. Support the BLPOP and BRPOP commands.
  5. Added Pika original command DISKRECOVERY

3.RocksDB Version Upgrade and Tiered Compression

  1. Upgraded RocksDB version to v8.1.1.
  2. Implemented tiered compression for RocksDB.
  3. Added the RocksDB cache configuration option "num-shard-bits" to read from the configuration file.

4.Support for BlobDB:

  • In v3.5.0, we added support for BlobDB KV separation.

    #1456

    @lqxhub

  1. With this update, Pika now supports the separation of KV (Key-Value) data using BlobDB.

5.Codis Cluster Mode based on Codis

  1. Introducing Codis into Pika.
  2. Introducing Codis CI into the project.
  3. Supporting the command for Codis slot migration.
  4. Adding the "slotmigrate" status indicator for whether it is in the process of reloading.

6.Observability

  1. Added Pika observability system pika_exporter.
  2. Introduced monitoring metrics for network I/O traffic.
  3. Added metrics for tracking command execution durations.
  4. Introduced "estimate_pending_compaction_bytes" metric to analyze fragmentation.
  5. Added RocksDB metrics.

7.Cloud-Native Deployment

  • In v3.5.0, we introduced an MVP version of pika-operator, which focuses on enabling the quick deployment of a single-instance Pika service on Kubernetes.

    #1243 #1330 #1347

    @machinly

  1. Implemented fast deployment of a single-instance Pika service on Kubernetes.
  2. Implemented Pika deployment in the MiniKube environment.
  3. Added End-to-End (E2E) tests to the pika-operator.

8.Cross-platform compilation

  • In v3.5.0, Pika supports cross-platform compilation and can be compiled and used on MacOS, CentOS, and Ubuntu platforms.

    #1372 #1488

    @loveyacper @Mixficsol

  1. Supports MacOS platform

9.Multi-platform CI, Go integration tests, TCL unit tests, PythonE2E tests, CTest unit tests

  1. Added CI on CentOS environment to ensure compatibility on this platform.
  2. Added CI on MacOS environment to validate Pika's behavior on MacOS.
  3. Introduced an End-to-End (E2E) testing framework to comprehensively test Pika's functionality from end to end.
  4. Integrated CMake build environment in Github CI Workflow to facilitate the compilation process.
  5. Implemented a "populate" method in TCL scripts to simulate Redis debug populate method for test data population.
  6. Introduced a script to enable easy execution of unit tests.
  7. Added CMake files in Blackwidow and included unit tests for Blackwidow.
  8. Utilized CTest for conducting unit tests to ensure code correctness.
  9. Ported Redis testing scripts to Pika to leverage existing tests and verify Pika's compatibility with Redis.
  10. Included new tasks in the CI pipeline to verify successful compilation of Pika on CentOS systems.

Features

  • Added automatic rate limiting for "compact" operation to reduce its impact on the latency of upper-layer data read and write operations. #1374 @wanghenshui
  • Introduced the "aof_to_pika" toolset for data conversion. #1340 @Axlgrep
  • Implemented the printing of the Pika logo. #1787 @Mixficsol
  • Optimized certain code portions using clang-tidy. #1701 #1730 @longfar-ncy
  • Refactored C++98-style code to C++11-style following C++ coding guidelines. #1684 @chejinge
  • Introduced support for the "snappy" library. #1216 @kernelai
  • Added CLA (Contributor License Agreement) file. #1260 @kernelai
  • Extended memory-related configuration options to support units in K, M, or G. #1307 @lqxhub
  • Added libunwind as a dependency library. #1316 @kernelai
  • Included a compilation script for Pika. #1356 @lqxhub
  • Added "rate-limiter-bandwidth" option to pika.conf for rate limiting. #1272 @wanghenshui
  • Enhanced the "info" command by adding "redis_version" information. #1403 @wanghenshui
  • Provided CMake support for aof_to_pika tool. #1436 @A2ureStone
  • Introduced clang-formatted shell scripts. #1448 @lqxhub
  • Added a configuration option to disable the compilation of command docs. #1642 @tedli
  • Provided CMake support for benchmark_client, binlog_sender, manifest_generator, rdb_to_pika, txt_to_pika, pika_to_txt, and pika_port. #1451 @A2ureStone

Improvement

Bugfixes

Notice