Golang Vendor Dependencies

We use Node.js extensively in our production apps. When we started with golang, the major discussion point amongst us was, for a beginner how golang dependency management is supposed to work.

Golang Dependency Management == Vendoring

Vendoring was the only method for golang dependency management. In this, you save a local copy of the dependent libraries that your application shall use. In Node.js world, dependency is managed using packages.json.

This resulted in big discussion between nodejs and golang folks - why in the world are we storing all the github libraries in our git repo. In nodejs, every time you want to run the executable, you fetch all the latest or specific packages at that point in time. So your git always contained only the sources that you had coded. Golang did not have such an option of fetching a particular package (this has changed now, which I will address later). Eventually the golang guys won the discussion (Hurray..), and we started saving all the dependencies locally.

Go 1.4 and Godep

As we started using golang, it was at version 1.4, and there were no recommendation from the go community. We ended up using godep, which was a pretty good tool. The drawback was, we had to use godep prefixed with all the go commands, such as godep go run and godep go build

Go 1.5 and above

With Go 1.5, the community driven requests for vendoring support resulted in a experimental flag GO15VENDOREXPERIMENT. This experiment was successful and from 1.6 it was included by default. With this, there will exist a vendor folder, which the go toolchain will look for imported libraries and then will search in $GOPATH. For the curious, you can look into the official design document.

.
└── src
    ├── server
        └── main.go
        └── vendor
            └── github.com
                └── sirupsen
                    └── logrus
                        ├── logrus.go
                        ├── logger.go
                        ├── ..
                        ├── ..
                        ├── ..
                        └── writer.go

From the above tree, the logrus library is now vendored and this is taken for both go run and go build.

How to vendor

If you want to manually vendor the sources, it is pretty simple to get started. You just copy the sources into the vendor folder:

go get -u github.com/sirupsen/logrus
mkdir -p $GOPATH/src/server/vendor/github.com/sirupsen/logrus
cp -r $GOPATH/src/github.com/sirupsen/logrus/ $GOPATH/src/server/vendor/github.com/sirupsen/logrus

Said that, only the newbies will vendor by manually copying. Every developer is an expert. And we use tools that the go community has provided. We choose govendor one of the many dependency management tools that are available.

Govendor

It is very simple to get started with govendor.

1. govendor init
2. govendor add +external -> This will add all dependencies from GOPATH into vendor
3. govendor fetch <package> -> Add dependencies
4. govendor lis -> List dependencies
5. govendor update -> Update dependencies

There are many more useful commands that govendor supports. Read them here

Also there are many other package management tools that you can choose from.

Vendoring what we need

With the new vendor support, Golang now supports a combination that we find very useful. Vendor only what breaks

With Go 1.4, we were forced either to vendor all or get the latest packages only. With the new vendor support, you can vendor only the packages that you think will break your code upon pulling the latest repo.

One very good example is the mgo - mongo golang driver. We do not vendor this, as we prefer to get all the bugfixes and latest changes. An example for vendoring is - if the original author of a library breaks the backward compatibility then freezing that package makes more sense, so that your code is not broken.

Of course, when Golang started, the recommended method from the community is to vendor all the libraries, so that you have reproducible builds and are not dependent on third party libraries availability. Doing so would avoid risks such as kik, left-pad, and npm.

Vendoring what you need is just my 2c. Decide on what works best for your projects.

Happy Vendoring!

Try Atatus Monitoring with free 14 day trial – no credit card required. Questions? - get in touch with us.