plug-able encryption

[clach04]

PyTombo and original Tombo do not support alternative encryption systems/file-formats. Also see clach04/tombo#1

Puren Tonbo as of 2023-12-30 supports:

• WinZip AE-2 and AE-1 AES-256 ZIP - needs some docs, sample/demo is decent for concept
  • ZipCrypto read ONLY support present, write support will NOT be implemented, https://blog.devolutions.net/2020/08/why-you-should-never-use-zipcrypto/ has a nice short explanation.
• chi - Tombo Blowfish - needs docs (and copy/paste warning from chi_io)
• VimCrypt vim encrypted files - for decryption only PKZIP (ZipCrypto), Blowfish, Blowfish2
• OpenPGP - gpg / pgp OpenPGP - gpg / pgp encryption support  #30 - for passphrase symmetric decryption only
• OpenSSL encrypted aes-256-cbc (with pbkdf2) - OpenSSL enc / dec support #111 -
  • GCM mode will never be supported by OpenSSL command line tool in enc mode, https://github.com/openssl/openssl/blob/14d3bb06c9c11b3e13c64611913757c27bc057f2/doc/man1/openssl-enc.pod.in#L268 Why AEAD is not supported in command enc? openssl/openssl#12220 (comment) BUT PT could (but then would not have comparability with existing tools). Also check out cms https://www.openssl.org/docs/man1.1.1/man1/cms.html
• ccrypt - https://ccrypt.sourceforge.net/ ccrypt encryption support #44 based on the Rijndael block cipher, a version of which is also used in the Advanced Encryption Standard (AES, see http://www.nist.gov/aes).
• rot13/rot-13 - Substitution cipher add rot-13 / rot13 support - mainly as a demo but also to ensure all tests paths can be run even if there are no crypto libraries/modules/extensions available #157 - do not use with sensitive data. Implemented as a demo and for testing code paths. Potentially useful for the rare times rot13 is encountered in the wild (usually for spoilers rather than real secrets). NOTE password ignored
• rot47/rot-47 similar to rot13 do NOT use. rot47 support #158
• zlib / gz (and maybe Z) - Compressed gZip files via zlib / Z / gzip compressed files #159 - allows compression and decompression

Crypto Support is static, currently there is no plugin/discovery mechanism see a0ad483 for an example of what is involved in adding new format support. See https://github.com/tibonihoo/yapsy

https://soatok.blog/2020/07/12/comparison-of-symmetric-encryption-methods/

• https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html
• https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html
• https://cryptobook.nakov.com/symmetric-key-ciphers/popular-symmetric-algorithms

Ideas

The main aim of different encryption/file formats is compatibility to avoid lock-in and promote ease of transportation of data. The list below are existing formats. This project has no intent to create a new format/encryption-algorithm! Wish list, implementations in; Python (3 and 2), Javascript (for browser, ideally web crypto API), C (C++), and Java to allow cross-platform exchanges (e.g. with native Android), see clach04/tombo#1

• ✅ VimCrypt vim encrypted files - https://vim.fandom.com/wiki/Encryption
  • https://github.com/nlitsme/vimdecrypt
• ✅ OpenPGP - gpg / pgp
  • https://gitlab.com/sequoia-pgp/sequoia
  • https://wiki.python.org/moin/GnuPrivacyGuard
  • https://github.com/vsajip/python-gnupg
  • https://github.com/isislovecruft/python-gnupg
  • https://blog.patternsinthevoid.net/pretty-bad-protocolpeople.html
  • gpgme python, apt install python3-gpg, http://files.au.adversary.org/crypto/gpgme-python-howto.html#gpgme-python-install
  • https://pypi.org/project/pyassuan/#data
• openssl encrypted - OpenSSL enc / dec support #111 use https://github.com/clach04/openssl_enc_compat/
  • https://antofthy.gitlab.io/info/crypto/openssl.txt some history and command line stdin tips
  • https://www.openssl.org/docs/man1.1.1/man1/enc.html
  • https://sleeplessbeastie.eu/2021/05/12/how-to-encrypt-or-decrypt-files-using-openssl-utility/
  • https://github.com/MoserMichael/vimcrypt2
  • Example; openssl aes-128-cbc -in in_file -out out_file.aes128, considered less secure without pbkdf2
  • openssl enc -aes-256-gcm -salt -pbkdf2 -iter 100000 -in my_file.jpg -out my_file.jpg.enc
  • http://justsolve.archiveteam.org/wiki/OpenSSL_salted_format
  • keep out kpt header https://antofthy.gitlab.io/software/keepout.sh.txt
  • https://github.com/patc888/encrypt_java_openssl
  • https://www.google.com/search?client=firefox-b-1-m&sca_esv=593448590&sxsrf=AM9HkKkyUBoYunxAn3uHy_tsWwonnd_-gg%3A1703441747657&q=openssl+enc+file+python+&oq=openssl+enc+file+python+&aqs=heirloom-srp..
  • https://stackoverflow.com/questions/53359064/how-to-decrypt-a-file-in-python-which-was-encrypted-using-openssl
  • https://stackoverflow.com/a/16761459 python
  • https://stackoverflow.com/a/20457519 python padding fix
  • https://stackoverflow.com/a/61165770 pbkdf and js link
• Valv (v1) [FEATURE REQUEST] - Documented steps to decrypt photos on another device Arctosoft/Valv-Android#33 (comment) - https://github.com/Arctosoft/Valv-Android/blob/master/ENCRYPTION.md
• ✅ ccrypt - https://ccrypt.sourceforge.net/ ccrypt encryption support #44
  • https://www.reddit.com/r/crypto/comments/of5jy8/comment/h4b1o17/?utm_source=share&utm_medium=web2x&context=3 - also see comment ccrypt encryption support #44 (comment)
• vimCrypt4 xchacha20 / VimCrypt~05!xchacha20v2 from Vim 8.2.3022+
• obsidian plugin encryption; Melt Encrypt, Inline Encrypter and Global Markdown Encryption.
  • https://github.com/shlemiel/globaloe
• saltpack - https://saltpack.org/
• Secret Space Encryptor https://paranoiaworks.mobi/sse/formats_specifications.html - multiple formats and options (could support a subset), File Encryptor claims to use a HMAC https://paranoiaworks.mobi/sse/file_encryption_specifications.html
  • txt https://pteo.paranoiaworks.mobi/
• Markor / jenc - AES-GCM https://gitlab.com/opensource21/jpencconverter https://github.com/opensource21/jpencconverter
  • https://gitlab.com/opensource21/jpencconverter/-/blob/master/src/main/java/de/stanetz/jpencconverter/cryption/JavaPasswordbasedCryption.java#L229
• RNCryptor AES-256-CBC (pbkdf2 10000 iterations) https://github.com/RNCryptor/RNCryptor-Spec/blob/master/RNCryptor-Spec-v3.md
  • https://github.com/RNCryptor/RNCryptor
• https://docs.cossacklabs.com/themis/languages/python/features/#seal-mode AES-256-GCM, py, js, c, java
• AGE https://github.com/FiloSottile/age using passphrases age passphrase encryption #49
• DARE https://github.com/minio/sio - see how https://github.com/drakkan/sftpgo/blob/main/docs/dare.md uses sio for keygen
• scrypt https://github.com/Tarsnap/scrypt/blob/master/FORMAT
• rclone crypto_secretbox_easy() and kdf scrypt salsa208 sha256_ll. - The salt is stored verbatim at the beginning of the obscured password. This static key is shared between all versions of rclone. If you reconfigure rclone with the same passwords/passphrases elsewhere it will be compatible, but the obscured version will be different due to the different salt.
  • https://github.com/lithium0003/crypt_rclone C code (no rclone required), only needs libsodium https://github.com/jedisct1/libsodium
• sample/examples for example Substitution ciphers
  • XOR - as dummy ciphers (with zero dependencies) for UI testing (testing handled by add rot-13 / rot13 support - mainly as a demo but also to ensure all tests paths can be run even if there are no crypto libraries/modules/extensions available #157)
  • memfrob - XOR 42 (0x2a) https://en.wikipedia.org/wiki/ROT13#Variants https://www.gnu.org/software/libc/manual/html_node/Obfuscating-Data.html
  • Vigenere / Vigenère cipher
    • https://en.wikipedia.org/wiki/Vigen%C3%A8re_cipher
    • https://gist.github.com/dssstr/aedbb5e9f2185f366c6d6b50fad3e4a4?permalink_comment_id=4035102#gistcomment-4035102
• https://github.com/garyexplains/oceantoo same as above, idea would be built-in test support
  • https://blog.dan.drown.org/cryptanalysis-of-oceantoo/
• encpipe https://github.com/jedisct1/encpipe
  • https://github.com/jedisct1/libhydrogen
• monocypher https://monocypher.org/ C and many languages poly1305
  • https://github.com/jetperch/pymonocypher - cython wrapper
  • https://github.com/eugene-eeo/monocypher-py - cffi wrapper
• Git-crypt https://github.com/AGWA/git-crypt/pull/200/files
• miscreant - Miscreant: Advanced symmetric encryption library which provides the AES-SIV (RFC 5297), AES-PMAC-SIV, and STREAM constructions. These algorithms are easy-to-use (or rather, hard-to-misuse) and support encryption of individual messages or message streams. AES-SIV provides nonce-reuse misuse-resistance (NRMR): accidentally reusing a nonce with this construction is not a security catastrophe, unlike it is with more popular AES encryption modes like AES-GCM.
  • https://github.com/miscreant/miscreant.py
• https://en.m.wikipedia.org/wiki/AES-GCM-SIV siv safer than gcm_siv if nonce reused
  • https://crypto.stackexchange.com/questions/42982/safety-of-random-nonce-with-aes-gcm
• AES (256) GCM or CCM

  • https://soatok.blog/2020/05/13/why-aes-gcm-sucks/

  • https://stackoverflow.com/questions/36117046/is-the-fernet-cryptography-module-safe-and-can-i-do-aes-encryption-with-that-mo

    import secrets
    from cryptography.hazmat.primitives.ciphers.aead import AESGCM
    
    # Generate a random secret key (AES256 needs 32 bytes)
    key = secrets.token_bytes(32)
    
    # Encrypt a message
    nonce = secrets.token_bytes(12)  # GCM mode needs 12 fresh bytes every time
    ciphertext = nonce + AESGCM(key).encrypt(nonce, b"Message", b"")
    
    # Decrypt (raises InvalidTag if using wrong key or corrupted ciphertext)
    msg = AESGCM(key).decrypt(ciphertext[:12], ciphertext[12:], b"")
    

  • Pure python libs

    • https://github.com/boppreh/aes
    • https://github.com/Z-40/MicroAES

  • https://github.com/gchq/CyberChef compat AES (html5/js)

  • https://github.com/PrivateBin/PrivateBin compat AES GCM (html5/js)

  • https://github.com/bradyjoslin/webcrypto-example JavaScript AES AES-GCM-256 (PBKDF2)

  • Stanford Javascript Crypto Library (SJCL) has JavaScript and python implementations, see https://www.cryptedblog.com/blog_encryption

  • QOwnNotes built-in encryption AES-256 - https://www.qownnotes.org/getting-started/overview.html - AES-256/CBC/PKCS7? NOT recommended, ascii armored (using external tools) but OwnNotes can't search it's own format

    • https://github.com/pbek/QOwnNotes/blob/c1e9447fa845fe7b665e61b9be2b85d6abfa811a/src/libraries/botan/botanwrapper.cpp#L45 weak kdf

  • Trillium AES https://github.com/zadam/trilium/wiki/Protected-notes - AES-128-CBC

• AES Crypt file format https://www.aescrypt.com/ - problematic
  • https://www.aescrypt.com/download/ - js, java, c, py
  • https://github.com/marcobellaccini/pyAesCrypt
  • https://www.aescrypt.com/java_aes_crypt.html
  • https://github.com/paulej/AESCrypt
  • https://www.reddit.com/r/privacytoolsIO/comments/b7riov/aes_crypt_security_audit_1_serious_issue_found/
• enchive https://github.com/skeeto/enchive
• https://github.com/HACKERALERT/Picocrypt - (go) XChaCha20, Argon2id
  • https://picocrypt.pages.dev/ mini JavaScript implementation
  • https://github.com/HACKERALERT/Picocrypt/tree/fd14deb1da8b91d449e1442fa0d4cd8b529242c9/src python
• https://github.com/sh-dv/hat.sh XChaCha20-Poly1305 - for symmetric encryption. Argon2id - for password-based key derivation.
• https://github.com/jeanpaulrichter/nppcrypt NotePadPlusPlus (Notepad++) encryption plugin
• spawning arbitrary command line tools via a pipe?
  • see ccrypt encryption support #44
  • https://sourceforge.net/p/gjots2/code/HEAD/tree/trunk/gjots/lib/file.py#l271 not generic but has support for three external tools hard coded.
• https://www.fourmilab.ch/javascrypt/ JavaScrypt is AES
• Jasypt / Bouncy Castle compatible encryption - Jasypt / Bouncy Castle compatible encryption #160
• archive and compressed formats withOUT encryption support new file formats, for example compression without password #81
  • ✅ gz/zlib - zlib / Z / gzip compressed files #159
• CryptoTE serpent containers (multiple files in same archive/file) https://panthema.net/2009/cryptote/ - https://github.com/bingmann/cryptote
• https://github.com/WrightSW/DeadboltEdit blowfish
• easy to use for formats/API
  • NaCL / libsodium secretbox / crypto_secretbox() (XSalsa20 and Poly1305 to encrypt and authenticate messages), python https://pynacl.readthedocs.io/en/latest/secret/ and js implementations. NOTE TweetNaCl provides no functionality for performing password-based key derivation. password as a secret, generally recommended practice is to use a password-based key derivation function (argon2, scrypt, bcrypt; PBKDF2 only if you really have to). No container/tools other than step - https://libsodium.gitbook.io/doc/secret-key_cryptography/secretbox
  • fernet (AES) AES-128-CBC + HMAC-SHA-256 with a random IV https://github.com/fernet/spec (includes test reference values), both python https://github.com/pyca/cryptography and JavaScript implementations available. https://www.comparitech.com/blog/information-security/what-is-fernet/ - NOTE spec does not cover salt storage/transmission :-(
    • Decrypt existing Fernet with pycryptodome Legrandin/pycryptodome#691 (comment)
  • https://github.com/samtools/hts-specs/blob/master/crypt4gh.tex
  • PEA format (Pack Encrypt Authenticate) https://peazip.github.io/pea-file-format.html compare with AE-2
    • https://peazip.github.io/pea-archiving-utility.html
  • https://github.com/andrewcooke/simple-crypt - has hmac, uses AES, easy to use and has format support. Could be extended to use a newer KDF (format has version support)
  • https://github.com/Ayrx/python-aead - has hmac
  • https://github.com/mervick/aes-everywhere - supposedly compatible with many languages
  • https://github.com/mpetersen/aes-example - Interoperable AES encryption with Java and JavaScript (CryptoJS)

key derivation functions

• argon2
• bcrypt
• Scrypt
  • https://words.filippo.io/the-scrypt-parameters/
• PBKDF2HMAC

Not under consideration at this time

• bcrypt - bcrypt encrypts and decrypts files using the blowfish algorithm
• Kryptor - https://www.reddit.com/r/crypto/comments/m0gc2z/comment/gqn1vgp/ - https://www.reddit.com/r/crypto/comments/m0gc2z/comment/gqnxy97/?utm_source=share&utm_medium=web2x&context=3

Short List

XChaCha20-Poly1305, AES-GCM-SIV,

• https://github.com/andrewcooke/simple-crypt
• Age
• Fernet