1![Rust](https://github.com/PSeitz/lz4_flex/workflows/Rust/badge.svg)
2[![Docs](https://docs.rs/lz4_flex/badge.svg)](https://docs.rs/crate/lz4_flex/)
3[![Crates.io](https://img.shields.io/crates/v/lz4_flex.svg)](https://crates.io/crates/lz4_flex)
4
5# lz4_flex
6
7![lz4_flex_logo](https://raw.githubusercontent.com/PSeitz/lz4_flex/master/logo.jpg)
8
9Fastest LZ4 implementation in Rust. Originally based on [redox-os' lz4 compression](https://crates.io/crates/lz4-compress), but now a complete rewrite.
10The results in the table are from a benchmark in this project (66Kb JSON, 10MB dickens) with the block format.
11
12AMD Ryzen 7 5900HX, rustc 1.69.0 (84c898d65 2023-04-16), Manjaro, CPU Boost Disabled, CPU Governor: Performance
13
1466Kb JSON
15|    Compressor        | Compression | Decompression | Ratio		 |
16|----------------------|-------------|---------------|---------------|
17| lz4_flex unsafe w. unchecked_decode  | 1615 MiB/s   | 5973 MiB/s    | 0.2284   	 |
18| lz4_flex unsafe | 1615 MiB/s   | 5512 MiB/s    | 0.2284   	 |
19| lz4_flex safe        | 1272 MiB/s   | 4540 MiB/s    | 0.2284   	 |
20| lzzz (lz4 1.9.3)     | 1469 MiB/s   | 5313 MiB/s    | 0.2283   	 |
21| lz4_fear             | 662 MiB/s   | 939 MiB/s     | 0.2283	     |
22| snap                 | 1452 MiB/s   | 1649 MiB/s     | 0.2242      |
23
2410 Mb dickens
25|    Compressor        | Compression | Decompression | Ratio		 |
26|----------------------|-------------|---------------|---------------|
27| lz4_flex unsafe w. unchecked_decode       | 347 MiB/s   | 3168 MiB/s    |  0.6372  	 |
28| lz4_flex unsafe      | 347 MiB/s   | 2734 MiB/s    |  0.6372  	 |
29| lz4_flex safe        | 259 MiB/s   | 2338 MiB/s    | 0.6372 |
30| lzzz (lz4 1.9.3)     | 324 MiB/s | 2759 MiB/s    | 0.6372 |
31| lz4_fear             | 201 MiB/s   | 370 MiB/s     | 0.6372 |
32| snap                 | 286 MiB/s   | 679 MiB/s     | 0.6276 |
33
34## Features
35- Very good logo
36- LZ4 Block format
37- LZ4 Frame format (thanks @arthurprs)
38- High performance
39- 1,5s clean release build time
40- Feature flags to configure safe/unsafe code usage
41- no-std support with block format (thanks @coolreader18)
42- 32-bit support
43
44## Usage:
45Compression and decompression uses no usafe via the default feature flags "safe-encode" and "safe-decode". If you need more performance you can disable them (e.g. with no-default-features).
46
47Safe:
48```
49lz4_flex = { version = "0.11" }
50```
51
52Performance:
53```
54lz4_flex = { version = "0.11", default-features = false }
55```
56
57### Block Format
58The block format is only valid for smaller data chunks as as block is de/compressed in memory.
59For larger data use the frame format, which consists of multiple blocks.
60
61```rust
62use lz4_flex::block::{compress_prepend_size, decompress_size_prepended};
63
64fn main(){
65    let input: &[u8] = b"Hello people, what's up?";
66    let compressed = compress_prepend_size(input);
67    let uncompressed = decompress_size_prepended(&compressed).unwrap();
68    assert_eq!(input, uncompressed);
69}
70```
71
72
73## no_std support
74
75no_std support is currently only for the block format, since the frame format uses `std::io::Write`, which is not available in core.
76
77## Benchmarks
78The benchmark is run with criterion, the test files are in the benches folder.
79
80Currently 4 implementations are compared, this one, [lz-fear](https://github.com/main--/rust-lz-fear), the [c version via rust bindings](https://crates.io/crates/lzzzz) and [snappy](https://github.com/burntsushi/rust-snappy).
81The lz4-flex version is tested with the feature flags safe-decode and safe-encode switched on and off.
82
83- lz4_cpp: https://crates.io/crates/lzzzz
84- lz-fear: https://github.com/main--/rust-lz-fear
85- snap: https://github.com/burntsushi/rust-snappy
86
87Tested on AMD Ryzen 7 5900HX, rustc 1.69.0 (84c898d65 2023-04-16), Manjaro, CPU Boost Disabled, CPU 3GHZ
88
89### Results v0.11.0 02-06-2023 (safe-decode and safe-encode off)
90`cargo bench --no-default-features`
91
92![Compress](./compress_bench.svg)
93
94![Decompress](./decompress_bench.svg)
95
96### Results v0.11.0 02-06-2023 (safe-decode and safe-encode on)
97`cargo bench`
98
99![Compress](./compress_bench_safe.svg)
100
101![Decompress](./decompress_bench_safe.svg)
102
103## Miri
104
105[Miri](https://github.com/rust-lang/miri) can be used to find issues related to incorrect unsafe usage:
106
107`MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-disable-stacked-borrows" cargo +nightly miri test --no-default-features --features frame`
108
109## Fuzzer
110This fuzz target generates corrupted data for the decompressor.
111`cargo +nightly fuzz run fuzz_decomp_corrupt_block` and `cargo +nightly fuzz run fuzz_decomp_corrupt_frame`
112
113This fuzz target asserts that a compression and decompression rountrip returns the original input.
114`cargo +nightly fuzz run fuzz_roundtrip` and `cargo +nightly fuzz run fuzz_roundtrip_frame`
115
116This fuzz target asserts compression with cpp and decompression with lz4_flex returns the original input.
117`cargo +nightly fuzz run fuzz_roundtrip_cpp_compress`
118
119## Bindings in other languages
120 - Node.js: [lz4-napi](https://github.com/antoniomuso/lz4-napi)
121 - Wasm: [lz4-wasm](https://github.com/PSeitz/lz4-wasm)
122
123## TODO
124- High compression
125
126## Migrate from v0.10 to v0.11.1
127To migrate, just remove the `checked-decode` feature flag if you used it.
128
129
130