PocketLzma is a cross-platform singleheader LZMA compression/decompression library for C++11. To use it, all you need is one
pocketlzma.hpp file, and you are good to go! The library is able to read data from both files and memory!
PocketLzma is designed to be a mix of a modern yet portable C++ library, only utilizing C++11 to make sure it can be used in projects where the latest versions of C++ are not available or otherwise not allowed to use.
LZMA stands for Lempel–Ziv–Markov chain Algorithm, and is an algorithm used for lossless data compression. The algorithm has been developed by Igor Pavlov since the late 90s. The C implementation made by Igor Pavlov is in fact what provides the core functionalty required for PocketLzma to work (with a few modifications to make it cross-platform and single-header compatible).
There is a Doxygen generated documentation of PocketLzma that can be found HERE
PocketLzma is first of all designed to be easy to use! If you want to see a fully working code example, take a look at the pocketlzma_program.cpp file in the root folder of this project. Otherwise: Follow along for some short examples.
First an important note! Since PocketLzma internally uses Igor Pavlov's C code, you are required to #define POCKETLZMA_LZMA_C_DEFINE ONCE (and only once) before including pocketlzma.hpp. The reason for this is to make sure the implementations of the C-code are only included once. In other words: If you include the pocketlzma.hpp file several places in your project, the other places must not use the #define
PocketLzma has a very simple API for file communication. However, you are free to use something else if you want. PocketLzma doesn't care how you got your data. Example:
std::string path = "./yourFile.txt";
//When you are 100% sure your loading will never fail, you can use this
std::vector<uint8_t> data1 = plz::File::FromFile(path);
//However - I recommend to use this overload instead:
std::vector<uint8_t> data2;
plz::FileStatus fileStatus = plz::File::FromFile(path, data2);
//If something went wrong
if(fileStatus.status() != plz::FileStatus::Code::Ok)
{
plz::FileStatus::Code statusCode = fileStatus.status(); //PocketLzma status code. Will be useful if errors not causing exceptions happen.
//You may or may not have some error information here
int code = fileStatus.code(); //Code returned from the OS in cases where an exception is thrown
std::string msg = fileStatus.message(); //Message from the OS in cases where an exception is thrown
std::string exception = fileStatus.exception(); //Exception message from the OS in cases where an exception is thrown
std::string category = fileStatus.category(); //Error category defined bythe OS in cases where an exception is thrown
}
//You can use memory data directly in PocketLzma, but you can use this if you want to transform them into a byte vector.
std::vector<uint8_t> memoryData;
plz::File::FromMemory(memfiles::_JSON_TEST_OK_HEADER_LZMA, memfiles::_JSON_TEST_OK_HEADER_LZMA_SIZE, memoryData);
//Finally, you can write to files like this
std::string writePath = "./yourOutputFile.txt";
plz::FileStatus fileWriteStatus = plz::File::ToFile(writePath, data2);
#define POCKETLZMA_LZMA_C_DEFINE
#include "pocketlzma.hpp"
int main()
{
std::string path = "./../../content/to_compress/from/json_test.json";
std::vector<uint8_t> data;
std::vector<uint8_t> compressedData;
plz::FileStatus fileStatus = plz::File::FromFile(path, data);
if(fileStatus.status() == plz::FileStatus::Code::Ok)
{
plz::PocketLzma p;
/*!
* Possibilities:
* Default
* Fastest
* Fast
* GoodCompression
* BestCompression
*/
p.usePreset(plz::Preset::GoodCompression); //Default is used when preset is not set.
plz::StatusCode status = p.compress(data, compressedData);
if(status == plz::StatusCode::Ok)
{
std::string outputPath = "./../../content/to_compress/to/j.lzma";
plz::FileStatus writeStatus = plz::File::ToFile(outputPath, compressedData);
if(writeStatus.status() == plz::FileStatus::Code::Ok)
{
//Process completed successfully!
}
}
}
return 0;
}If you are familiar with LZMA, you can use your own compression settings rather than using presets. Making it possible to tune every parameter to get your own balance of speed and compression ratio.
#define POCKETLZMA_LZMA_C_DEFINE
#include "pocketlzma.hpp"
int main()
{
std::string path = "./../../content/to_compress/from/json_test.json";
std::vector<uint8_t> data;
std::vector<uint8_t> compressedData;
plz::FileStatus fileStatus = plz::File::FromFile(path, data);
if(fileStatus.status() == plz::FileStatus::Code::Ok)
{
plz::Settings settings;
// These are actual default values used when choosing the Default preset,
// but shows the parameters that can be tuned
settings.level = 5;
settings.dictionarySize = 1 << 24;
settings.literalContextBits = 3;
settings.literalPositionBits = 0;
settings.positionBits = 2;
settings.fastBytes = 32;
plz::PocketLzma p {settings}; //You can alternatively use: p.setSettings(settings);
plz::StatusCode status = p.compress(data, compressedData);
if(status == plz::StatusCode::Ok)
{
std::string outputPath = "./../../content/to_compress/to/j.lzma";
plz::FileStatus writeStatus = plz::File::ToFile(outputPath, compressedData);
if(writeStatus.status() == plz::FileStatus::Code::Ok)
{
//Process completed successfully!
}
}
}
return 0;
}#define POCKETLZMA_LZMA_C_DEFINE
#include "pocketlzma.hpp"
int main()
{
std::string path = "./../../content/to_compress/to/j.lzma";
std::vector<uint8_t> data;
std::vector<uint8_t> decompressedData;
plz::FileStatus fileStatus = plz::File::FromFile(path, data);
if(fileStatus.status() == plz::FileStatus::Code::Ok)
{
//No settings / presets are used during decompression!
plz::PocketLzma p;
plz::StatusCode status = p.decompress(data, decompressedData);
if(status == plz::StatusCode::Ok)
{
std::string outputPath = "./../../content/to_compress/from/j.json";
plz::FileStatus writeStatus = plz::File::ToFile(outputPath, decompressedData);
if(writeStatus.status() == plz::FileStatus::Code::Ok)
{
//Process completed successfully!
}
}
}
return 0;
}You can use data directly from memory (both for compression and decompression), if you please. If you need a program to generate in-memory files, you can use my f2src program to do that job for you.
plz::PocketLzma p;
//Alternative 1
std::vector<uint8_t> decompressed;
plz::StatusCode status = p.decompress(memfiles::_JSON_TEST_OK_HEADER_LZMA,
memfiles::_JSON_TEST_OK_HEADER_LZMA_SIZE,
decompressed);
...
//Alternative 2
std::vector<uint8_t> bytes = plz::File::FromMemory(memfiles::_JSON_TEST_OK_HEADER_LZMA,
memfiles::_JSON_TEST_OK_HEADER_LZMA_SIZE);
status = p.decompress(bytes, decompressed);
...
//Alternative 3
std::vector<uint8_t> bytes;
plz::File::FromMemory(memfiles::_JSON_TEST_OK_HEADER_LZMA,
memfiles::_JSON_TEST_OK_HEADER_LZMA_SIZE,
bytes);
status = p.decompress(bytes, decompressed);- CPU: Intel Core i7-6700 @ 8x 4GHz
- GPU: GeForce GTX 1080
- RAM: 15969MiB
- OS: Linux (Manjaro 20.2 Nibia) - Kernel: x86_64 Linux 4.14.209-1-MANJARO
| Preset | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 70230 bytes | 3364 bytes | 2.25587 ms | 1.99016 ms | 3.17568 ms |
| Fast | 70230 bytes | 3283 bytes | 2.29653 ms | 2.02276 ms | 3.54542 ms |
| Default | 70230 bytes | 2693 bytes | 20.7517 ms | 19.3443 ms | 25.8784 ms |
| GoodCompression | 70230 bytes | 2485 bytes | 30.6193 ms | 28.5697 ms | 38.3485 ms |
| BestCompression | 70230 bytes | 2451 bytes | 45.8248 ms | 43.4147 ms | 57.934 ms |
| Preset (when compressed) | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 3364 bytes | 70230 bytes | 0.502977 ms | 0.497585 ms | 0.536352 ms |
| Fast | 3283 bytes | 70230 bytes | 0.492965 ms | 0.488002 ms | 0.50396 ms |
| Default | 2693 bytes | 70230 bytes | 0.42965 ms | 0.423914 ms | 0.448674 ms |
| GoodCompression | 2485 bytes | 70230 bytes | 0.405111 ms | 0.402765 ms | 0.419498 ms |
| BestCompression | 2451 bytes | 70230 bytes | 0.401403 ms | 0.400253 ms | 0.406537 ms |
| Preset | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 4145823 bytes | 702789 bytes | 253.102 ms | 232.259 ms | 330.66 ms |
| Fast | 4145823 bytes | 677754 bytes | 355.459 ms | 321.867 ms | 455.138 ms |
| Default | 4145823 bytes | 572742 bytes | 1889.8 ms | 1852.6 ms | 1993.94 ms |
| GoodCompression | 4145823 bytes | 521168 bytes | 2888.82 ms | 2879.99 ms | 2915.07 ms |
| BestCompression | 4145823 bytes | 520358 bytes | 3140.49 ms | 3107.47 ms | 3168.36 ms |
| Preset (when compressed) | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 702789 bytes | 4145823 bytes | 85.5723 ms | 84.871 ms | 87.2708 ms |
| Fast | 677754 bytes | 4145823 bytes | 81.7042 ms | 81.4525 ms | 82.052 ms |
| Default | 572742 bytes | 4145823 bytes | 78.0439 ms | 77.7491 ms | 78.7667 ms |
| GoodCompression | 521168 bytes | 4145823 bytes | 74.8225 ms | 74.1976 ms | 76.2949 ms |
| BestCompression | 520358 bytes | 4145823 bytes | 74.3353 ms | 74.1313 ms | 74.5187 ms |
- CPU: AMD Ryzen 9 5900X 12-Core @ 24x 3.7GHz
- GPU: NVIDIA GeForce RTX 3070
- RAM: 64216MiB
- OS: Manjaro 22.0.0 Sikaris - Kernel: x86_64 Linux 6.1.1-1-MANJARO
| Preset | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 70230 bytes | 3364 bytes | 0.541419 ms | 0.534969 ms | 0.570288 ms |
| Fast | 70230 bytes | 3283 bytes | 0.552374 ms | 0.548396 ms | 0.559747 ms |
| Default | 70230 bytes | 2693 bytes | 6.0338 ms | 5.98933 ms | 6.20445 ms |
| GoodCompression | 70230 bytes | 2485 bytes | 9.04424 ms | 8.9897 ms | 9.08903 ms |
| BestCompression | 70230 bytes | 2451 bytes | 13.1645 ms | 13.1083 ms | 13.2443 ms |
| Preset (when compressed) | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 3364 bytes | 70230 bytes | 0.182297 ms | 0.179499 ms | 0.199928 ms |
| Fast | 3283 bytes | 70230 bytes | 0.176741 ms | 0.17497 ms | 0.187393 ms |
| Default | 2693 bytes | 70230 bytes | 0.154774 ms | 0.153528 ms | 0.165992 ms |
| GoodCompression | 2485 bytes | 70230 bytes | 0.144384 ms | 0.142778 ms | 0.149791 ms |
| BestCompression | 2451 bytes | 70230 bytes | 0.14039 ms | 0.13909 ms | 0.146114 ms |
| Preset | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 4145823 bytes | 702789 bytes | 83.8579 ms | 82.4795 ms | 87.221 ms |
| Fast | 4145823 bytes | 677754 bytes | 125.257 ms | 115.714 ms | 146.505 ms |
| Default | 4145823 bytes | 572742 bytes | 813.151 ms | 720.423 ms | 972.822 ms |
| GoodCompression | 4145823 bytes | 521168 bytes | 1237.8 ms | 1158.11 ms | 1471.32 ms |
| BestCompression | 4145823 bytes | 520358 bytes | 1313.7 ms | 1241.61 ms | 1510.28 ms |
| Preset (when compressed) | Size before | Size after | Average time | Min. time | Max. time |
|---|---|---|---|---|---|
| Fastest | 702789 bytes | 4145823 bytes | 34.8025 ms | 34.5862 ms | 35.515 ms |
| Fast | 677754 bytes | 4145823 bytes | 32.8216 ms | 32.7728 ms | 32.9863 ms |
| Default | 572742 bytes | 4145823 bytes | 31.0607 ms | 30.862 ms | 31.3336 ms |
| GoodCompression | 521168 bytes | 4145823 bytes | 29.4726 ms | 29.4448 ms | 29.51 ms |
| BestCompression | 520358 bytes | 4145823 bytes | 29.5008 ms | 29.468 ms | 29.6048 ms |
All credits goes to Igor Pavlov, the genius behind the LZMA compression algorithm. He has distributed all his work under Public Domain for anyone to use. PocketLzma uses parts of Igor Pavlov's LZMA related C code in LZMA SDK v19.00.
While PocketLzma goes under the still very permissive BSD-2-Clause License I've created an optional cross-platform amalgamated lzma_c.hpp, which contains a slightly altered version of Igor Pavlov's LZMA implementation for C, and is released under the same Public Domain license to honor Igor's work. This can be found in the extras folder, but keep in mind that this file is not supported in any way and may not co-exist with PocketLzma.