Vendoring has been a recent hot topic in the Go community. This is partly because the upcoming 1.5 release will have some helper functionality for vendoring built into the toolchain. There sometimes appears to be the notion that the way to handle package dependencies is to always vendor. But, is that really the case?
&tldr; To be a mature ecosystem you need to support projects that do and don’t vendor.
What is vendoring?
Package vendoring is commonly referred to as the case where dependent packages are stored in the same place as your project. That usually means you dependencies are checked into your source management system, such as Git. This is the common case and how this post approaches the topic.
Those Who Don’t Vendor
I’m going to start with the story of those who don’t vendor because it is the one less talked about in the Go discussions I’ve been in. Many projects and even ecosystems regularly don’t vendor projects and the tooling supports this. I’ll illustrate with two examples.
First, look at node.js developers. To manage dependencies everyone uses npm. In a package.json file the packages and their versions are specified. Any developer can get the project, run npm install and have a reliable environment with all the dependencies installed.
Tools, like Travis CI, have extra support to help with this. By default Travis CI will run npm install for you on node_js projects. Travis CI also provides caching functionality you can tap into. This can be used to cache dependencies so you don’t have to go out to the Internet to get them each time.
Second, look at the OpenStack project. This is a cloud platform. It’s a project with many sub-projects and those use shared projects to interoperate with each other. It’s complex because of the nature of the space. It has its own CI/CD system and keeps copies of the dependencies close to the execution environment with mirrors. Imagine vendoring all the cross dependecies?
These kinds of situations are common and other ecosystems enable them to happen.
Not vendoring is common or default for many of the most popular languages whether you look at the TIOBE index or one of the reports from Red Monk.
But, what about Go projects? There are toolchains to support those who don’t vendor. This includes Buildpacks for Cloud Foundry and Heroku, package managers, and more.
Many Go projects are not vendoring today and many will continue to not vendor dependencies. Especially if the ecosystem makes it easier.
Why Not Vendor?
Why wouldn’t someone want to vendor their dependencies? Here are a few of the reasons I’ve compiled:
- Large VCS repository size.
- VCS history littered with vendored package updates.
- A goal of one repository per project. Keep the separation clean.
Those Who Vendor Packages
It’s widely talked about how Google uses vendoring. Vendoring isn’t uncommon and some external tools, such as godep, help you manage different dependencies for different applications.
Why would someone want to vendor? Here are a few reasons:
- Anyone who grabs the repo has everything needed to build the application.
- CI/CD systems can run more quickly when it doesn’t have to get any dependencies.
- The entire application and it’s dependencies are in one place.
A small comparison
Both methods have worked to solve some common problems. Whether you vendor or not you can have the dependencies locally to minimize going out to the Internet to get them.
If you fork or otherwise alter a project dependency that can be a pain to maintain in a large project with lots of changes. I know from experience.
Note, in Glide we’ve solved the forked project problem without rewriting the import path.
On the flip side, projects that don’t vendor don’t have to deal with project management tools to install dependencies. It’s a small time save you get at the cost or repository size.
One of the advantages of not vendoring means you really should have versioned packages spedified to get reliable builds. In doing that you have clear documentation on your dependencies for security and bug handling.
What Go Needs
The Go community simply needs to support those who vendor and those that don’t as first class citizens. Supporting one over the other or specifying a best practice may go too far here. This is about project management and application management rather than the nuances of the language itself.
Or, that’s my 2 cents from my years of experience building small and large complex applications while working with many different types of developers.