This post will introduce the concept of deduplication and a Go package I have written to help you implement it. It is inspired by the work on the zpaq journaling archiver done by Matt Mahoney. If you are looking for a pure commandline tool, I can also recommend SuperRep by Bulat Ziganshin. This will however provide you…
This is the final chapter in our trilogy of exploring deflate performance. In this I will give a detailed description of how you can get close to constant time compression for gzip/zip and standard deflate running at approximately 150MB/s per core.
In our previous performance test, we looked at high throughput compression, suited for log files, map-reduce data, etc. This post will deal with webserver performance, where we will investigate which gzip is best suited for a Go webserver. Update (18 Sept 2015): Level 1 compression has been updated to a faster version, so the difference…
As a continuation of my release of a Optimized gzip/zip packages. I have done some informal benchmarks to get an impression of performance characteristics of the various libraries. I have compiled the results in a spreadsheet you can view yourself. Update (18 Sept 2015): Level 1 compression has been updated to a faster version, so the difference to…
I have worked on optimizing the standard library deflate function, and I am happy to announce revised gzip/zip packages, that on x64 is about 30-50% faster with slightly improved compression.