Skip to content

Latest commit

 

History

History
413 lines (340 loc) · 40.7 KB

CHANGELOG.md

File metadata and controls

413 lines (340 loc) · 40.7 KB

Release v2.0.0 - 2020/04/10

Feature

  • Multi-Zone replication & Create volume on a specified zone. #407 #416
  • Support token authentication for readwrite-mount & readonly-mount of fuse client. #435
  • A command line tool for cluster operations. #441
  • Implemented user security and authorization system to improve resource access control. #441
  • Implemented extend attributes (xattr) for metadata and posix-compatible file system interface (mountable client). #441

Enhancement

Object storage related

  • Support folder operations in S3 APIs. #450
  • Reduce blocking under concurrency. #458
  • Implemented more Amazon S3-compatible object storage interfaces to improve compatibility. #441

Master related

  • Make replicas of the data partition a specific option when creating volume. #377
  • If meta node reaches threshold,set meta partition status to readonly. #411
  • Add cluster status API. #457

MetaNode related

  • Checks only file type instead of the whole mode. #381

Fuse related

  • Make followerRead a client specific option. #382
  • Support command line argument for fuse client. #418
  • Introduce disable dentry-cache to client option. #453
  • Filter target meta partitions in batch iget. #472

Others

  • Yum tool for deploying ChubaoFS cluster. #385

Bug fix

  • Fix the signature algorithm issues. #369 #476
  • Avoid inode unlink due to net error. #402
  • A map structure locked during serialization. #413
  • Wait for data sync in close syscall. #419
  • Fix empty result on list objects. #433
  • Set lookup valid duration for newly created file. #437
  • Fix iget error due to metapartition split. #446
  • Fix mount fail when volume is full. #453
  • Fix offline strategy for raft peers of data partition and meta partition. #467

Document

  • Add guide for running ChubaoFS by yum tools. #386
  • Update FUSE client mount options. #439
  • Add documentation for client token. #449

Release v1.5.1 - 2020/01/19

Enhancement

  • Support building docker image that contains both cfs-server and cfs-client. #353

Bug fix

  • Only one replica of meta partition can be taken offline at the same time. #345
  • Check if server port is open before raft leader change. #348
  • Solved several issue in signature algorithm version 2. #357
  • Solved the issue related to copying files across folder through object storage interface. #361

Refactoring

  • Update dashboard configuration of grafana. #347
  • Unified the configuration of master address and listening port in documentation. #362

Document

  • Added benchmark data and guidelines for deploying ChubaoFS cluster with Helm in README file. #350

Release v1.5.0 - 2020/01/08

Feature

  • Add a general Authentication & Authorization framework for ChubaoFS. commit
  • Object storage interface. Add ObjectNode to provide S3-compatible APIs. commit

Enhancement

  • Check disk path size in run-docker script. commit
  • Support building ChubaoFS docker image that contains cfs-server and cfs-client. commit
  • Add go test in docker-compose. commit
  • Add authorization to master api getVol. commit
  • Support building under the Darwin(Apple MacOS) and Microsoft Windows operating system environment. commit

Bug fix

  • Set DataNode disk size in docker script to be compatible with lower version of df. commit
  • The reservedSpace parameter invalid when gt 30GB. commit
  • The mtime of parent inode does not change when create or delete a dentry. commit
  • MetaNode opResponseLoadPartition removes duplicate locks. commit
  • MetaNode leak memory on DeleteMetaPartition operator. commit

Refactoring

  • Change configuration file of Master daemon. commit
  • Remove unnecessary function in raft store. commit
  • Change metaNode loadSnapshotSign command about inodeCount and dentryCount. commit
  • Rename rack to cell. commit
  • Improve meta partition replicas verification by MaxInode and DentryCnt. commit
  • Change parameter name in MetaNode configuration file. commit
  • Using current applyID to replace snapshot applyID in MetaNode LoadMetaPartition response action. commit
  • Delete data partition and meta partition synchronously. commit

Document

  • Add design and user guide document for AuthNode. commit
  • Update configuration file sample in document. commit
  • Add design and user guide document for object subsystem. commit
  • Add IO, small file and metadata performance benchmark data to document. commit

Release v1.4.0 - 2019/11/13

Feature

  • Datanode : support read from follower and if packet is tinyExtent,then do write it once commit
  • Vol add followerRead field to support reading data from foll owner commit
  • Support read from raft follower instead of just leader commit
  • Support to modify whether vol supports reading data from a replica commit
  • Introduce read and write iops rate limit commit
  • Add metrics commit

Enhancement

  • If vol has been marked deleted,data partitions, meta partition information reported by heartbeat will no longer be accepted commit
  • Use static ip for meta and data nodes commit
  • Improve debug environment using docker commit
  • Support custom meta node reserved memory commit
  • Data partition and meta partition must have three replicas except reducing replicas to 2 commit
  • Adjust demo config parameters commit
  • Update grafana dashbord for disk error metric commit

Bug fix

  • OpFollowerRead if read eof,return error commit
  • Get follower read option in init commit
  • Stream traverse process never gets triggered in some situation commit
  • Check LoadConfigFile before starting daemon commit
  • Return error from function LoadConfigFile to the caller commit
  • ExtentStorage engine :autoComputeCrc compute crc error commit
  • Clean up async delete process of metanode commit
  • Set default port to non-system reserved port commit

Refactoring

  • Leader change not warning on raft commit
  • Remove go module files for now commit
  • Clean up response of get all inodes info commit
  • Master, DataNode and MetaNode Fix dp or mp offline process commit
  • Use AddNodeWithPort replace AddNode,and delete AddNode API commit
  • Delete reserved space on DataNode config file commit
  • Refine labels of the disk error metric commit
  • Optimize auto compute crc commit

Document

Release v1.3.0 - 2019/09/12

Feature

  • Introduce writecache mount option. commit
  • Introduce keepcache mount option. commit
  • Add admin API for get all meta parititons under vol commit
  • Support for truncating raft log. commit
  • Dynamiclly reduce the num of replicas for vol. commit
  • The specified number of replica num is supported when creating vol. commit
  • Feature: daemonize server commit
  • Support log module change loglevel on line. commit

Enhancement

  • Extent_store LoadTinyDeleteFileOffset return s.baseTinyDeleteOffset. commit
  • Enable async read by default. commit
  • Improve log message details for clientv2. commit
  • Compatible with string when get bool config. commit
  • Add performance tracepoint for clientv2. commit
  • Align out message buffer size with max read size. commit
  • For splitting meta partition,updating meta partition and creating new meta partition are persisted within a single transaction. commit
  • If metanode used memory is full,then the partition must set to readonly. commit
  • Set report time to now after creating data partition. commit
  • Set writeDeadLineTime to one minute,avoid metanode gc reset conn which snapshot used as much as possible. commit
  • Add raft monitor. commit
  • If the creation of a data partition fails, the successfully created replica is deleted. commit
  • If the creation of a meta partition fails, the successfully created replica is deleted. commit
  • Add unit test case. commit
  • Passing create data partition type to datanode. commit
  • If create dp is normal,must start Raft else backend start raft. commit
  • The tickInterval and electionTick support reading from a configuration file commit

Bugfix

  • Fix: add del vol step after ltptest in travis-ci test script. commit
  • Clientv2 file handle memory leak. commit
  • Redirect stderr to an output file in daemon. commit
  • Exclude data partition only when connection refused. commit
  • When delete DataParittion,the forwardToLeader mayBe painc. commit
  • Metanode load error mayme shield. commit
  • Truncate raft corrupt data. commit
  • When meta node memory usage arrive threshold, split meta partition occurred dead lock. commit
  • SplitMetaPartition race lock with updateViewCache. commit
  • After the vol is created and before the heartbeat report, the status of the data partition is set to read only after the check dp operation is performed. commit
  • When disk error,the raft cannot start on new data server first. commit
  • OpDecommissionDataPartition delete dataPartition on new server. commit
  • Datanode may be painc. commit
  • Datanode auto compute crc. commit
  • DataNode: when dataPartition load,if applyId ==0 ,then start Raft. commit
  • The reported data partition usage decreased, and the statistical usage did not decrease accordingly. commit
  • Docker metanode.cfg add totalMem parameter. commit
  • Datanode deadlock on deletePartition. commit
  • DataNode may be painc. commit
  • Exclude dir inode in the orphan list. commit
  • Evict inode cache after successful deletion. commit
  • The actual reduction in the number of replicas exceeds the expected reduction in the number of replicas. commit
  • Compatible with old heartbeat mode, old heartbeat mode does not report volname. commit
  • Metanode mistakenly delete empty dir inode. commit
  • Treat ddelete not exist error as successful. commit
  • Fuse directIO read size can exceeds buffer size. commit
  • Fix Datanode retain RaftLog commit
  • Fix: Datanode: when tinyExtentRepair auto repair,it has been commit
  • Fix: Storage :when write tinyExtent,if offset!=e.datasize,return error commit
  • The buf is not reset and the old data is repeatedly written to the ump warning log, resulting in the overall performance degradation of the cluster commit
  • DataPartitionMap occurred error which is concurrent map iteration and map write commit
  • Client gets stale file size if streamer is auto evicted. commit
  • Update export init for consul register commit
  • Err is shadowed in server main function commit
  • If master only create DataPartitionCnt is 10,then cannnot mount commit
  • Update export init for consul register commit
  • DataPartition disk error ,not recvoery raft log on new datanode commit
  • Datanode register hang bug commit

Refactoring

  • Sdk When creating a datapartition, select the datapartition retry strategy. commit
  • Refactoring SDK: when write datapartition num greater 10 ,then trust master. commit
  • When disk error,the datapartition recover only recover avali data on tinyExtent. commit
  • Decommission Meta or Data Partition must sync response to master. commit
  • Synchronized decommission the data partition. commit
  • Datanode api /partition add raftStatus. commit
  • StartRaftLoggingSchedule not use goroutine. commit
  • Metanode must config totalMem. commit
  • Sync tinyDeleteExtent time change to 1 days. commit
  • Change partitionId varliable to partitionID. commit
  • Datanode delete SnapshotFile Pool. commit
  • Exporter add ump. commit
  • DataNode: compatible old dataPartition Meta info. commit
  • Add ltptest log. commit
  • Docker metanode config.json change totalMem to 6GB. commit
  • Datanode create dataPartition select disk function change. commit
  • Datanode start must start StartRaftLoggingSchedule func. commit
  • Master create vol min default DataPartition set to 10. commit
  • Docker: run ltptest print errorinfo. commit
  • When load dp,if dp status is normal,then start raft ,else wait snapshot has recover. commit
  • Datanode delete unused func. commit
  • If not config warnLogDir,then donnot write umplog to disk. commit
  • DataNode: if not config raftDir,then not start server. commit
  • Add log on metanode delete extent. commit
  • Doc :delete warnLogDir config. commit
  • Keep mount point in error stataus when client is killed commit
  • Log checkroration checkTime change to 1 second commit
  • Metanode change deleteDentry or deleteInode api to log.LogDebugf commit
  • Increase the judgment condition of disk error,add syscall.EROFS commit
  • Sync code from git.jd.com/chubaofs/chubaofs commit

Release v1.2.1 - 2019/07/19

Enhancement

  • When datanode or metanode start,must compare partition with master and gofmt project. commit
  • Datanode add backend check disk status func. commit

Bugfix

  • Update rocksdb build depends. commit
  • Metanode panic when deleting meta partition. commit
  • When raft member become leader,must apply from appliyID to commitID. commit

Release v1.2.0 - 2019/07/16

Feature

  • Clientv2: add dentry cache. commit
  • Vendor: introduce jacobsa daemonize package. commit
  • Client: start client as a daemon. commit
  • Clientv2: daemonize client.commit

Enhancement:

  • Update docker helper script. commit
  • Data sdk: use ip address instead of partition id to rule out unavailable data partitions. commit
  • Add monitor add prometheus, grafana in docker helper script.commit
  • Add GOPATH check in build.sh. commit
  • Integrated rocksdb compilation. commit
  • Update makefile. commit
  • Docs: start client as a daemon. commit
  • Update docs: add make build in readme and docs. commit
  • Use one thread to send and recive from follower on primary backup replication protocol .commit
  • Change random write raft serialize not use json. commit

Bug fix:

Change/Refactoring

  • Add cfs-base dockerfile. commit
  • Specification code and metanode config file. commit
  • Add go mod. commit
  • Docs: update according to recent changes. commit
  • If warnLogDir is not specified in the config files, server or client will return an error. commit
  • Rename RestSize to ReservedSize. commit
  • Print more detailed error information when server starts failed. commit
  • Remove docker client security opt. commit
  • Add disk config detailed description. commit
  • By default create 10 data partitions and 3 meta partitions when creating volume. commit
  • Doc: add clarification that resource manager is AKA master. commit
  • Normalize exporter cluster name. commit
  • Set the value of rack for data node to default. commit
  • Refine decommission disk. commit
  • Change document about metanode config. commit
  • Doc: update docker helper section. commit
  • Update quick-start-guide.rst. commit
  • Change dataPartition IntervalToUpdateReplica to 600 seconds. commit
  • Add third-party directory. commit
  • Write data partition decommission url message to log. commit
  • Go fmt project. commit
  • Data sdk: check amount of writable data partitions when mount. commit
  • Delete third-party directory. commit
  • Update docs: remove server and client. commit
  • Pass the hosts to data node when creating the data partition. commit
  • Warn log message unicode encoding is converted to utf8. commit
  • Ignore generated build files. commit
  • Build: update build.sh and Makefile. commit
  • Change packet attr func and ReadTinyDelete impl log. commit
  • Add log when read tinyDeleteFile failed. commit
  • Add log when streamRead and ExtentRepairRead. commit
  • Remove unused source code. commit
  • Build: update build.sh. commit
  • Update README.md. commit
  • Build: remove client unused depends. commit

Release v1.1.1 - 2019/06/11

Feature

  • Client: enable support to fifo and socket file types. commit
  • Clientv2: use jacobsa fuse package instead of bazil. commit
  • Docker: introduce docker compose to create a local testing ChubaoFS cluster. commit

Bugfix

  • Meta: update atime in inode get operation. [metanode] commit
  • Fix: potential panic if send returns nil resp with no error. [client] commit
  • Fix: raft election takes a long time and timeout; issue a panic if raft server is down. [raft] commit
  • Fix: potential deadlock if applyHandler. [master] commit
  • Fix: put vol to cache after it is persistent. [master] commit
  • Fix: partition is nil when apply remove raft node. [datanode] commit
  • Fix: metanode painc. [metanode] commit
  • Fix: panic when pprof does not start. commit

Enhancement

  • Sdk: retry if mount failed in case master is unavailable temporarily. commit
  • Build: add verbose build info. commit
  • Master: introduce data partition over-provision. commit
  • Master: reserve writable data partition amount according to capacity instead of a const. commit
  • Monitor: Use UMP performance monitor if exporter is not enabled. commit

Release v1.1.0 - 2019/05/07

Change / Refactoring

  • Rename the repository from cfs to chubaofs.
  • Use own errors module instead of juju errors due to license incompatibility.
  • Metanode: Change not raft leader error to tryOtherAddr error.

Bugfix

  • Master: Partition recovered but the status not changed.
  • Datanode: Report to client with proto.OptryAgain resultcode when datapartition does not exsit.
  • Raft: A member must apply playback from old apply id to commit id after elected as leader.
  • Metanode: generate identical inode number under extreme conditions. commit

Enhancement

  • Master: Add ump warn packet.
  • Master: Remove redundant calling of loadMetaData method.
  • Master: Reload meta data after leader changed.
  • Master: make dataPartition disk Path persistent. commit
  • Master: Volume creation supports specifying the amount of meta partitions. commit
  • Metanode: Add totalMem in configFile.
  • Datanode: Change default disk reserved space.
  • Datanode: Add volname in heartbeat report.
  • Raft: Use raft.ErrNotLeader instead of ErrNotLeader.
  • Client: Create a dummy node instance if inode does not exist.
  • Client: Add UMP monitor alarms for read/write/fsync errors.
  • Client: Suppress some error messages. commit
  • Log: Automatically create subdirectory under the log directory.

Release v1.0.0 - 2019/04/02

The initial release version.