"Exam" by Alberto G.

Travis/AppVeyor CI Script for Go

Continous Integration tests are an important part of maintaining package stability. Since Travis has recently made it possible to run tests on OSX and with AppVeyor having Windows support, we now can create free CI tests for all the major desktop platforms.

"Benson Triplets" by Christopher Allison Photography

Fast Stream Deduplication in Go

This post will introduce the concept of deduplication and a Go package I have written to help you implement it. It is inspired by the work on the zpaq journaling archiver done by Matt Mahoney. If you are looking for a pure commandline tool, I can also recommend SuperRep by Bulat Ziganshin. This will however provide you…

"ahead only" (cc-by-nd) by Terry Freedman

An Async Read-ahead Package for Go

After looking at some rare poor performance in rclone, I have created a package for read-ahead functionality, which I have implemented a few times in different contexts. This gave me the opportunity to create a solid implementation with solid tests to make sure it works with good performance. For the impatient here is the package:…

"Racing down the runway" by Philipp Lücke (cc-by-nc)

The case of the subtle race condition

Yesterday I got a strange race condition reported after I wrote some additional tests for pgzip. At first I couldn’t really work out why I was getting the race reported, and I suspected that the race detector was reporting wrong line numbers. The race reported was boiled down like this:

"Back where I belong" by Carl Jones

Long Windows paths (UNC paths) in Go

Lately I have been working a bit with various backup/file tools written in Go, mainly rclone and restic. I noticed that both of them has issues when dealing with long Windows paths, which is quite unfortunate for backup utilities, so I figured that might be a good thing to cover on my blog. This issue…

"Golang" by Renée French (http://reneefrench.blogspot.com)

Intrinsics in Go

I have written a Discussion Piece about how it could be possible to implement assembler intrinsics in Go based on an experiment I have been working on the past week. The goal of the experiment was to see if intrinsics are feasible within the Go language without breaking the language. It is my impression that…

On the fast lane by taufuuu

Fast Linear Time Gzip/Zip Compression

This is the final chapter in our trilogy of exploring deflate performance. In this I will give a detailed description of how you can get close to constant time compression for gzip/zip and standard deflate running at approximately 150MB/s per core.

"Colorful windows" by Dino Quinzani

How to make your Go Applications easy to port

Go is really well suited for cross-platform work. My main development environment is Windows, but I always deploy to Linux-based systems. Therefore I naturally try to avoid things that will get me into trouble.   My attitude towards cross-platform support is that if you consider yourself a serious developer, your code should at least compile on…

Colors! by Andrew Hart

Gzip Performance for Go Webservers

In our previous performance test, we looked at high throughput compression, suited for log files, map-reduce data, etc. This post will deal with webserver performance, where we will investigate which gzip is best suited for a Go webserver. Update (18 Sept 2015): Level 1 compression has been updated to a faster version, so the difference…

Go gzip/deflate benchmarks

As a continuation of my release of a Optimized gzip/zip packages. I have done some informal benchmarks to get an impression of performance characteristics of the various libraries. I have compiled the results in a spreadsheet you can view yourself. Update (18 Sept 2015): Level 1 compression has been updated to a faster version, so the difference to…

deferred dreams by torbakhopper

defer fun()

After programming Go for more than two years my favorite Go keyword is defer. I might go into detail about this in a future post, but this is one of the nice things I found. About a year ago I came across a funny little Go-ism when I was looking through some code. It was…

"Golang" by Renée French (http://reneefrench.blogspot.com)

Adding Unsupported Instructions in Golang Assembler

For an experimental project, I wanted to use an SSE 4.1 instruction (MPSADBW) for some Go assembler. I found out that this instruction wasn’t supported by the Go assembler in the current 1.4.2 version. The proposed solution is to add this as raw opcodes in the assembler. The instruction (in Go assembler syntax) is: MPSADBW…

Big Password Dictionary for Validation

Continuing my previous posts [1][2] on checking passwords against dictionaries, I have completed a dictionary for the extremely paranoid. It is a compilation of  the ‘Crackstation‘, ‘g0tmi1k‘ and ‘WPA-PSK WORDLIST 3 Final’ dictionaries. The passwords are all in lower-case, Unicode NFKD normalized, unique and sorted. I also dropped all passwords shorter than 8 characters and more…

Papa bloggt! 255/365 by Dennis Skley https://flic.kr/p/oTX6b9

Protect Your Passwords Against Dictionary Attacks

As a followup to my previous blog post: “Password Requirements Done Better“, I have finalized a package for dictionary password validation for Go. If you have a site where users have passwords, and you don’t already do dictionary checks, I would strongly recommend you look at this. The package can be used to protect your…

Blazingly fast Reed-Solomon Coding

Last week Backblaze published a blog post on their usage of Reed-Solomon Erasure coding to maintain data available even if some of their storage servers goes offline. They also released a java library with the code they use for creating the reed-solomon sets. I have always been fascinated by Reed-Solomon Erasure coding, but so far I…

Bass by Keoni Cabral https://flic.kr/p/92gVEU

XAudio2 Dolby Digital Live patcher

For many years a bug has remained in XAudio2 library, that will make XAudio 2 report stereo channels to games using the API, even though output is 5.1. This will make sound cards with DDL encoding output stereo instead of nice surround-sound. I found that this was still an issue, even with a fully updated…