Glide 0.5 and Go Vendor Support

With the release of Go 1.5 comes experimental support for vendor packages. We found this setup to be so useful we pivoted Glide, the Go package manager, to focus on supporting this new vendor setup. In this post I’ll walk through how it works.

Turning On The Vendor Support

The vendor support in Go 1.5 is opt-in. To enable it you need to set the environment variable GO15VENDOREXPERIMENT to 1. For example,

export GO15VENDOREXPERIMENT=1

Once this is set the go tool will will look for vendor packages in the vendor directory for a project. The directory structure within the vendor directory mirrors that of $GOPATH/src and comes first when the go tool searches for packages.

The structure looks like:

- myProject (Your project)
  |
  |-- main.go (Your main go code can live here)
  |
  |-- mySubpackage (You can create your own subpackages, too)
  |    |
  |    |-- foo.go
  |
  |-- vendor
       |-- github.com
            |
            |-- Masterminds
                  |
                  |-- ... etc.

Beyond this basic support for a vendor directory the go tool provides little help. That means it doesn’t provide the tooling to manage packages in this directory. That’s where Glide comes in.

The Basics of Glide

Once you have the glide tool installed your project needs two things.

glide.yaml

First, there is the glide.yaml file where packages and their versions are documented. You can either create this by hand or use the glide init function to create one for you. The format looks like:

package: github.com/Masterminds/glide
import:
  - package: github.com/kylelemons/go-gypsy
    subpackages:
      - yaml
  - package: github.com/Masterminds/cookoo
    ref:     master
    repo:    git@github.com:Masterminds/cookoo.git
    vcs:     git
    subpackages:
      - .
  - package: github.com/Masterminds/vcs
  - package: github.com/codegangsta/cli

The package top level property is the name of your project. Under import is a list of the packages to pull into the vendor directory and manage.

If the package name, using the package key, is the only element the latest commit on the default branch will be be used. This is the same version go get retrieves if it were to do any management of the vendor directory.

The vcs, repo, and ref keys for an import are where things get interesting. This allows you to set version control system, repo, and reference. For example, a package name could be github.com/Masterminds/cookoo. But, the repo could be git@github.com/mattfarina/cookoo where I keep a fork. The right import path will be used along with my repo of choice. No import rewriting required.

The ref key allows you to set any reference supported by the underlying VCS. That includes branches, tags, and individual commit ids.

glide up

Once you have a glide.yaml file defining your packages you can use the glide up command (aliased to update and install). This will fetch the packages, put them in the vendor directory, and set the referenced version.

glide up, by default, supports recursively fetching dependencies. So, if one of your dependencies has a list of dependencies it will fetch those as well.

While we often don’t vendor packages by checking them into our VCS we know others do. To support those who do vendor packages from vendors, when recursively installing updates Glide skips packages that have an existing vendor directory. If the vendor directory exists we assume that package is vendoring packages.

Godep, GPM, and Imports

There are a lot of projects being managed by Godep and GPM and they have done a great job up to this point. With Go 1.5 we wanted the ability import from these systems to the new vendor setup.

If you add the --import flag to glide up or glide get (we’ll talk more about that in a minute) it will automatically import Godep and GPM dependencies into each projects vendor directory and create a glide.yaml file for the project populated with the packages and their versions. This works recursively with glide up (if you don’t opt out of the recursion).

If you want to convert a project, Glide has the glide import command that can import from these systems to a glide.yaml file.

Adding Packages with glide get

The go get command is handy to add packages to your GOPATH. But, it doesn’t support adding packages to a vendor directory. So, we added glide get [package]. Using this command will add the package to your vendor directory and glide.yaml file.

Scripting with Glide

While the vendor directory is special, once you opt-in, not every tool realizes this. For example, running go test ./... will test your package and every package in your vendor directory.

To support scripting we’ve created the glide novendor and glide name. The glide name command simply returns the name of the package. The more interesting command is novendor. It will give you all the directories except the vendor one. So, to test your package and not the packages in vendor you could use the command:

$ go test $(glide novendor)

Dive In, Find Bugs, and Help Us Improve

This is just the quick rundown of Glide 0.5. We hope you like it and we know it would benefit from continued improvement. Please dive in, take it for a spin, find bugs and file them, and help us improve Glide.