reference/security: add encryption-at-rest doc #2410
Conversation
Signed-off-by: Yi Wu <[email protected]>
| * TiKV currently does not exclude encryption keys and user data from core dump. It is highly adviced to disable core dump for TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. | ||
| * TiKV keeps track of encryption key and method used for each data files, using absolute path of the files. As a result, once encryption is turned on for a TiKV node, user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. | ||
|
|
||
| ## BR S3 server-side encryption |
There was a problem hiding this comment.
TODO: copy this section to BR document.
There was a problem hiding this comment.
@overvenus do you think this is needed, and if so, which file should I modify?
ran-huang
changed the title
Signed-off-by: Yi Wu <[email protected]>
Signed-off-by: Yi Wu <[email protected]>
|
|
||
| The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. User would need to create a CMK through AWS KMS, and then provide the CMK key id to TiKV in config file. The TiKV process would need access to the KMS CMK while it is running, which can be done by using [IAM](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. | ||
|
|
||
| Alternatively, if using custom key is desired, supplying master key via file is also supported. The file need to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persistenting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs. |
There was a problem hiding this comment.
| Alternatively, if using custom key is desired, supplying master key via file is also supported. The file need to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persistenting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs. | |
| Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. |
There was a problem hiding this comment.
Why do we require a newline? It seems like we could chomp new lines if they exist?
There was a problem hiding this comment.
I would like to make the file format very strict to make sure user aware of the required key length. I would like to do without the newline, but text editors (at least vim) tend to add a extra newline at line end, which is not a bad idea to indicate end of the key.
There was a problem hiding this comment.
If I had to choose one, I would probably not have a newline. I generally agree that being strict about inputs is a much better approach, but chomping newlines is the one thin I don't see a problem with. I don't think it will cause any confusion about the file format and they can potentially avoid some deployment delays.
There was a problem hiding this comment.
we can make the newline optional, but it is too late for 4.0 GA branch cut.
There was a problem hiding this comment.
file key are not for production use anyway.
|
I think we should mention the caveats section in the introduction |
Signed-off-by: Yi Wu <[email protected]>
Signed-off-by: Yi Wu <[email protected]>
Signed-off-by: Yi Wu <[email protected]>
|
|
||
| The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. | ||
|
|
||
| * When deploying a TiDB cluster, the majority of a user's data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However there is a small amount of user data stored in PD nodes as metadata (e.g secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. We recommend using storage level encryption (e.g. file system encryption) to help protect sensitive data stored in PD. |
There was a problem hiding this comment.
Is this the only source of data leakage? Or it is the biggest one?
There was a problem hiding this comment.
We currently believe the secondary index is the only case, but not confirmed.
Signed-off-by: Yi Wu <[email protected]>
Signed-off-by: Yi Wu <[email protected]>
|
/run-cherry-picker |
Signed-off-by: Yi Wu <[email protected]>
|
cherry pick to release-4.0 in PR #2635 |
Signed-off-by: Yi Wu <[email protected]>
yikeke
added
size/large
yikeke
added
translation/done
Signed-off-by: Yi Wu [email protected]
What is changed, added or deleted? (Required)
Add user document for TiKV Encryption-At-Rest (also refer to as Transparent Data Encryption, or TDE), and BR server-side encryption when backup to S3.
Which TiDB version(s) do your changes apply to? (Required)
If you select two or more versions from above, to trigger the bot to cherry-pick this PR to your desired release version branch(es), you must add corresponding labels such as needs-cherry-pick-4.0, needs-cherry-pick-3.1, needs-cherry-pick-3.0, and needs-cherry-pick-2.1.
What is the related PR or file link(s)?
TiKV encryption-at-rest design docs: https://docs.google.com/document/d/14u0YKHXxjE0a9wAd3tsu0Hqjmknjr0bYU_EMzZSbzkA/edit?usp=sharing