Zstd

perf: rebalanced compression levels, to better match the intended speed/level curve, by senhuang42
perf: faster huffman decoder, using x64 assembly, by terrelln
perf: slightly faster high speed modes (strategies fast & dfast), by felixhandte
perf: improved binary size and faster compilation times, by terrelln
perf: new row64 mode, used notably in level 12, by senhuang42
perf: faster mid-level compression speed in presence of highly repetitive patterns, by senhuang42
perf: minor compression ratio improvements for small data at high levels, by cyan4973
perf: reduced stack usage (mostly useful for Linux Kernel), by terrelln
perf: faster compression speed on incompressible data, by bindhvo
perf: on-demand reduced ZSTD_DCtx state size, using build macro ZSTD_DECODER_INTERNAL_BUFFER, at a small cost of performance, by bindhvo
build: allows hiding static symbols in the dynamic library, using build macro, by skitt
build: support for m68k (Motorola 68000's), by cyan4973
build: improved AIX support, by Helflym
build: improved meson unofficial build, by eli-schwartz
cli : custom memory limit when training dictionary (2925), by embg
cli : report advanced parameters information when compressing in very verbose mode (`-vv`), by Svetlitski-FB

api: Various functions promoted from experimental to stable API: (2579-2581, senhuang42)
`ZSTD_defaultCLevel()`
`ZSTD_getDictID_fromCDict()`
api: Several experimental functions have been deprecated and will emit a compiler warning (2582, senhuang42)
`ZSTD_compress_advanced()`
`ZSTD_compress_usingCDict_advanced()`
`ZSTD_compressBegin_advanced()`
`ZSTD_compressBegin_usingCDict_advanced()`
`ZSTD_initCStream_srcSize()`
`ZSTD_initCStream_usingDict()`
`ZSTD_initCStream_usingCDict()`
`ZSTD_initCStream_advanced()`
`ZSTD_initCStream_usingCDict_advanced()`
`ZSTD_resetCStream()`
api: ZSTDMT_NBWORKERS_MAX reduced to 64 for 32-bit environments (Cyan4973)
perf: Significant speed improvements for middle compression levels (2494, senhuang42 terrelln)
perf: Block splitter to improve compression ratio, enabled by default for high compression levels (2447, senhuang42)
perf: Decompression loop refactor, speed improvements on `clang` and for `--long` modes (2614 2630, Cyan4973)
perf: Reduced stack usage during compression and decompression entropy stage (2522 2524, terrelln)
bug: Improve setting permissions of created files (2525, felixhandte)
bug: Fix large dictionary non-determinism (2607, terrelln)
bug: Fix non-determinism test failures on Linux i686 (2606, terrelln)
bug: Fix various dedicated dictionary search bugs (2540 2586, senhuang42 felixhandte)
bug: Ensure `ZSTD_estimateCCtxSize*() `monotonically increases with compression level (2538, senhuang42)
bug: Fix --patch-from mode parameter bound bug with small files (2637, occivink)
bug: Fix UBSAN error in decompression (2625, terrelln)
bug: Fix superblock compression divide by zero bug (2592, senhuang42)
bug: Make the number of physical CPU cores detection more robust (2517, PaulBone)
doc: Improve `zdict.h` dictionary training API documentation (2622, terrelln)
doc: Note that public `ZSTD_free*()` functions accept NULL pointers (2521, animalize)
doc: Add style guide docs for open source contributors (2626, Cyan4973)
tests: Better regression test coverage for different dictionary modes (2559, senhuang42)
tests: Better test coverage of index reduction (2603, terrelln)
tests: OSS-Fuzz coverage for seekable format (2617, senhuang42)
tests: Test coverage for ZSTD threadpool API (2604, senhuang42)
build: Dynamic library built multithreaded by default (2584, senhuang42)
build: Move `zstd_errors.h` and `zdict.h` to `lib/` root (2597, terrelln)
build: Allow `ZSTDMT_JOBSIZE_MIN` to be configured at compile-time, reduce default to 512KB (2611, Cyan4973)
build: Single file library build script moved to `build/` directory (2618, felixhandte)
build: `ZBUFF_*()` is no longer built by default (2583, senhuang42)
build: Fixed Meson build (2548, SupervisedThinking kloczek)
build: Fix excessive compiler warnings with clang-cl and CMake (2600, nickhutchinson)
build: Detect presence of `md5` on Darwin (2609, felixhandte)
build: Avoid SIGBUS on armv6 (2633, bmwiedmann)
cli: `--progress` flag added to always display progress bar (2595, senhuang42)
cli: Allow reading from block devices with `--force` (2613, felixhandte)
cli: Fix CLI filesize display bug (2550, Cyan4973)
cli: Fix windows CLI `--filelist` end-of-line bug (2620, Cyan4973)
contrib: Various fixes for linux kernel patch (2539, terrelln)
contrib: Seekable format - Decompression hanging edge case fix (2516, senhuang42)
contrib: Seekable format - New seek table-only API (2113 2518, mdittmer Cyan4973)
contrib: Seekable format - Fix seek table descriptor check when loading (2534, foxeng)
contrib: Seekable format - Decompression fix for large offsets, (2594, azat)
misc: Automatically published release tarballs available on Github (2535, felixhandte)

bug: Use `umask()` to Constrain Created File Permissions (2495, felixhandte)
bug: Make Simple Single-Pass Functions Ignore Advanced Parameters (2498, terrelln)
api: Add (De)Compression Tracing Functionality (2482, terrelln)
api: Support References to Multiple DDicts (2446, senhuang42)
api: Add Function to Generate Skippable Frame (2439, senhuang42)
perf: New Algorithms for the Long Distance Matcher (2483, mpu)
perf: Performance Improvements for Long Distance Matcher (2464, mpu)
perf: Don't Shrink Window Log when Streaming with a Dictionary (2451, terrelln)
cli: Fix `--output-dir-mirror` rejection of `..` -containing paths (2512, felixhandte)
cli: Allow Input From Console When `-f`/`--force` is Passed (2466, felixhandte)
cli: Improve Help Message (2500, senhuang42)
tests: Remove Flaky Tests (2455, 2486, 2445, Cyan4973)
tests: Correctly Invoke md5 Utility on NetBSD (2492, niacat)
tests: Avoid Using `stat -c` on NetBSD (2513, felixhandte)
build: Zstd CLI Can Now be Linked to Dynamic `libzstd` (2457, 2454 Cyan4973)
build: Hide and Avoid Using Static-Only Symbols (2501, 2504, skitt)
build: CMake: Enable Only C for lib/ and programs/ Projects (2498, concatime)
build: CMake: Use `configure_file()` to Create the `.pc` File (2462, lazka)
build: Fix Fuzzer Compiler Detection & Update UBSAN Flags (2503, terrelln)
build: Add Guards for `_LARGEFILE_SOURCE` and `_LARGEFILE64_SOURCE` (2444, indygreg)
build: Improve `zlibwrapper` Makefile (2437, Cyan4973)
contrib: Add `recover_directory` Program (2473, terrelln)
doc: Change License Year to 2021 (2452 & 2465, terrelln & senhuang42)
doc: Fix Typos (2459, ThomasWaldmann)

hotfix: wrong alignment of an internal buffer

perf: stronger --long mode at high compression levels, by senhuang42
perf: stronger --patch-from at high compression levels, thanks to --long improvements
perf: faster dictionary compression at medium compression levels, by felixhandte
perf: small speed & memory usage improvements for ZSTD_compress2(), by terrelln
perf: improved fast compression speeds with Visual Studio, by animalize
cli : Set nb of threads with environment variable ZSTD_NBTHREADS, by senhuang42
cli : accept decompressing files with *.zstd suffix
cli : provide a condensed summary by default when processing multiple files
cli : fix : stdin input no longer confused as user prompt
cli : improve accuracy of several error messages
api : new sequence ingestion API, by senhuang42
api : shared thread pool: control total nb of threads used by multiple compression jobs, by marxin
api : new ZSTD_getDictID_fromCDict(), by LuAPi
api : zlibWrapper only uses public API, and is compatible with dynamic library, by terrelln
api : fix : multithreaded compression has predictable output even in special cases (see 2327) (issue not accessible from cli)
api : fix : dictionary compression correctly respects dictionary compression level (see 2303) (issue not accessible from cli)
build: fix cmake script when using path with spaces, by terrelln
build: improved compile-time detection of aarch64/neon platforms, by bsdimp
build: Fix building on AIX 5.1, by likema
build: compile paramgrill with cmake on Windows, requested by mirh
doc : clarify repcode updates in format specification, by felixhandte

fix : Always return dstSize_tooSmall when that is the case
fix : Fix ZSTD_initCStream_advanced() with static allocation and no dictionary
perf: Improve small block decompression speed by 20%+, by terrelln
perf: Reduce compression stack usage by 1 KB, by terrelln
perf: Improve decompression speed by improving ZSTD_wildcopy, by helloguo (2252, 2256)
perf: Improve histogram construction, by cyan4973 (2253)
cli : Add --output-dir-mirror option, by xxie24 (2219)
cli : Warn when (de)compressing multiple files into a single output, by senhuang42 (2279)
cli : Improved progress bar and status summary when (de)compressing multiple files, by senhuang42 (2283)
cli : Call stat less often, by felixhandte (2262)
cli : Allow --patch-from XXX and --filelist XXX in addition to --patch-from=XXX and --filelist=XXX, by cyan4973 (2250)
cli : Allow --patch-from to compress stdin with --stream-size, by bimbashrestha (2206)
api : Do not install zbuff.h, since it has long been deprecated, by cyan4973 (2166).
api : Fix ZSTD_CCtx_setParameter() with ZSTD_c_compressionLevel to make 0 mean default level, by i-do-cpp (2291)
api : Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX, by marxin (2228).
build: Install pkg-config file with CMake and MinGW, by tonytheodore (2183)
build: Install DLL with CMake on Windows, by BioDataAnalysis (2221)
build: Fix DLL install location with CMake, by xantares and bimbashrestha (2186)
build: Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions
build: Add ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics
build: Add STATIC_BMI2 macro for compile time detection of BMI2 on MSVC, by Niadb (2258)
build: Fix -Wcomma warnings, by cwoffenden
build: Remove distutils requirement for meson build, by neheb (2197)
build: Fix cli compilation with uclibc
build: Fix cli compilation without st_mtime, by ffontaine (2246)
build: Fix shadowing warnings in library
build: Fix single file library compilation with Enscripten, by yoshihitoh (2227)
misc: Improve single file library and include dictBuilder, by cwoffenden
misc: Allow compression dictionaries with missing symbols
misc: Add freestanding translation script in contrib/freestanding_lib
misc: Collect all of zstd's libc dependencies into zstd_deps.h
doc : Add ZSTD_versionString() to manual, by animalize
doc : Fix documentation for ZSTD_CCtxParams_setParameter(), by felixhandte (2270)