• Klaus Post's avatar
    compress/flate: add pure huffman deflater · 42ad1dc0
    Klaus Post authored
    Add a "HuffmanOnly" compression level, where the input is
    only entropy encoded.
    
    The output is fully inflate compatible. Typical compression
    is reduction is about 50% of typical level 1 compression, however
    the compression time is very stable, and does not vary as much as
    nearly as much level 1 compression (or Snappy).
    
    This mode is useful for:
     * HTTP compression in a CPU limited environment.
     * Entropy encoding Snappy compressed data, for archiving, etc.
     * Compression where compression time needs to be predictable.
     * Fast network transfer.
    
    Snappy "usually" performs inbetween this and level 1 compression-wise,
    but at the same speed as "Huffman", so this is not a replacement,
    but a good supplement for Snappy, since it usually can compress
    Snappy output further.
    
    This is implemented as level -2, since this would be too much of a
    compression reduction to replace level 1.
    
    >go test -bench=Encode -cpu=1
    BenchmarkEncodeDigitsHuffman1e4            30000             52334 ns/op         191.08 MB/s
    BenchmarkEncodeDigitsHuffman1e5             3000            518343 ns/op         192.92 MB/s
    BenchmarkEncodeDigitsHuffman1e6              300           5356884 ns/op         186.68 MB/s
    BenchmarkEncodeDigitsSpeed1e4               5000            324214 ns/op          30.84 MB/s
    BenchmarkEncodeDigitsSpeed1e5                500           3952614 ns/op          25.30 MB/s
    BenchmarkEncodeDigitsSpeed1e6                 30          40760350 ns/op          24.53 MB/s
    BenchmarkEncodeDigitsDefault1e4             5000            387056 ns/op          25.84 MB/s
    BenchmarkEncodeDigitsDefault1e5              300           5950614 ns/op          16.80 MB/s
    BenchmarkEncodeDigitsDefault1e6               20          63842195 ns/op          15.66 MB/s
    BenchmarkEncodeDigitsCompress1e4            5000            391859 ns/op          25.52 MB/s
    BenchmarkEncodeDigitsCompress1e5             300           5707112 ns/op          17.52 MB/s
    BenchmarkEncodeDigitsCompress1e6              20          59839465 ns/op          16.71 MB/s
    BenchmarkEncodeTwainHuffman1e4             20000             73498 ns/op         136.06 MB/s
    BenchmarkEncodeTwainHuffman1e5              2000            595892 ns/op         167.82 MB/s
    BenchmarkEncodeTwainHuffman1e6               200           6059016 ns/op         165.04 MB/s
    BenchmarkEncodeTwainSpeed1e4                5000            321212 ns/op          31.13 MB/s
    BenchmarkEncodeTwainSpeed1e5                 500           2823873 ns/op          35.41 MB/s
    BenchmarkEncodeTwainSpeed1e6                  50          27237864 ns/op          36.71 MB/s
    BenchmarkEncodeTwainDefault1e4              3000            454634 ns/op          22.00 MB/s
    BenchmarkEncodeTwainDefault1e5               200           6859537 ns/op          14.58 MB/s
    BenchmarkEncodeTwainDefault1e6                20          71547405 ns/op          13.98 MB/s
    BenchmarkEncodeTwainCompress1e4             3000            462307 ns/op          21.63 MB/s
    BenchmarkEncodeTwainCompress1e5              200           7534992 ns/op          13.27 MB/s
    BenchmarkEncodeTwainCompress1e6               20          80353365 ns/op          12.45 MB/s
    PASS
    ok      compress/flate  55.333s
    
    Change-Id: I8e12ad13220e50d4cf7ddba6f292333efad61b0c
    Reviewed-on: https://go-review.googlesource.com/20982Reviewed-by: 's avatarJoe Tsai <joetsai@digital-static.net>
    Reviewed-by: 's avatarNigel Tao <nigeltao@golang.org>
    42ad1dc0
deflate_test.go 13.2 KB