Lately I have been working a bit with various backup/file tools written in Go, mainly rclone and restic. I noticed that both of them has issues when dealing with long Windows paths, which is quite unfortunate for backup utilities, so I figured that might be a good thing to cover on my blog.

This issue is the same for Go, as most other programming languages, and it doesn’t help you. It is a pity, because the issue could very likely have been made transparent by Go by translating path types behind the scenes. Unfortunately this means that it is now up to you, the developer to handle this. However, if it is any consolation, dot net users have the same issue.

What is a “UNC” path?

Way back in the old days Windows paths were limited to 260 bytes. More than a decade ago it was however decided that this was too small a limit to use, so Microsoft decided to allow for a total path length of 32 kilobytes.

However, to preserve backwards compatibility, they decided that they wouldn’t suddenly start sending backs paths that were longer than 260 bytes back to programs, since that would likely cause nasty overflow bugs and similar issues. So they created a “workaround”, where if you prefixed a path with `\\?\` you were allowed to use the “long” path, ie. up to 32k.

This gives us these overall path types:

c:\dir\file.txt

.\dir\file.txt
dir\file.txt
..\dir\file.txt
\dir\file.txt

\\server\share\dir\file.txt

\\?\c:\dir\file.txt

\\?\UNC\server\share\dir\file.txt

Technically using UNC paths also bypasses parsing of paths, so relative paths, as well as paths containing forward slashes ‘/’ will not work. So if you convert to UNC paths you must convert you path to an absolute path. Luckily in Go, we have filepath.Abs() to help us with that. To convert forward slashes to backslashes, you can use strings.Replace().

Converting to a UNC path

This small code snippet will convert any absolute path to a UNC path. It will work for traditional paths, network paths and it will pass through UNC paths.

import "strings"
import "regexp"

// Pattern to match a windows absolute path: "c:\" and similar
var isAbsWinDrive = regexp.MustCompile(`^[a-zA-Z]\:\\`)

// UncPath converts an absolute Windows path
// to a UNC long path.
func UncPath(s string) string {
	// UNC can NOT use "/", so convert all to "\"
	s = strings.Replace(s, `/`, `\`, -1)

	// If prefix is "\\", we already have a UNC path or server.
	if strings.HasPrefix(s, `\\`) {

		// If already long path, just keep it
		if strings.HasPrefix(s, `\\?\`) {
			return s
		}

		// Trim "\\" from path and add UNC prefix.
		return "\\\\?\UNC\\" + strings.TrimPrefix(s, `\\`)
	}

	if isAbsWinDrive.MatchString(s) {
		return `\\?\` + s
	}

	return s
}

So with this small function you can convert all your paths to UNC paths before opening them. If you would like, you can add an s = filepath.Abs(s) line in the beginning.

Also, be aware of this currently existing bug in os: MkdirAll fails when creating top-level UNC paths, that could affect your end users as well. If you know any more similar tricky issues when using UNC paths, please share them in the comments below!

go, golang, long path, long paths, path, paths, portability, porting, unc, windows