Flattening Go Dependencies With Glide

Now that Go packages can have a vendor/ directory where dependent packages can be stored (via the GO15VENDOREXPERIMENT) we’re starting to see some new types of dependent package issues popping up. Here I’ll cover how you can end up with more than one instance of a package, how it can cause a problem, and how we provide a fix in Glide.

Problem

When using the vendor/ directory each package can have its own dependencies. The go tool will look in the current packages vendor/ directory. If the package isn’t there it will look up the directory tree for parents with a vendor/ directory containing the dependency. Once the vendor/ directories are exhausted it will look in the $GOPATH and then the $GOROOT.

This opens the door for multiple dependencies of your application to have a shared dependency that they’ve put in their vendor/ directory. Two problems can come from this:

If you have something like a database driver you’ll see errors in your application. The package in each of its locations is seen as a different package.
If you have a large dependency, such as Docker, you can end up with package duplication leading to larger than needed binaries and other duplication issues.

We’ve already seen both of these cases. They aren’t theoretical but rather have already been experienced.

Solution

The solution is to have single version of a dependency for the entire application that all dependencies can share. In the database driver example that would mean a single instance of the package. For the Docker example that would mean a single checkout of the Docker codebase for all dependencies to share.

In Glide we accomplished this by adding a flatten property to the glide.yaml file. This is available to top level applications. It can be set to true globally or for a specific dependency.

When flatten is true Glide will only put the dependency in the applications top level vendor/ directory. All the dependencies will automatically use it from there. If there is a version mismatch you’ll be notified.

By default, running glide update with this set is a non-destructive operation. If a dependency of your application has a dependency stored in it’s vendor/ folder it won’t be deleted. If you want to delete the package from the dependencies vendor/ folder use the --delete-flatten flag.

How To Get It

This feature is currently on master and will be in the upcoming 0.6.0 release. To use it today, before the official release, you can install glide with: