Packages as layers, not groups
Four years ago, I wrote an article called Standard Package Layout that tried to address one of the most difficult topics for even advanced Go developers: package layout. However, most developers still struggle with organizing their code into a directory structure that will grow gracefully with their application.
Nearly all programming languages have a mechanism for grouping related functionality together. Ruby has gems, Java has packages. Those languages don't have a standard convention for grouping code because, honestly, it doesn't matter. It all comes down to personal preference.
However, developers that transition to Go are surprised by how often their package organization comes back to bite them. Why are Go packages so different from other languages? It's because they're not groups—they're layers.
Understanding cyclic dependencies
The primary difference between Go packages and grouping in other languages is that Go doesn't allow for circular dependencies. Package A can depend on package B, but then package B cannot depend back on package A.
This restriction causes issues for developers later on when they need to have both packages share common code. There are typically two solutions: either combine both packages into a single package or introduce a third package.
However, splitting out into more and more packages only pushes the problem down the road. Eventually, you end up with a large mess of packages and no real structure.
Borrowing from the standard library
One of the most useful tips when programming Go is to look to the standard library when you need guidance. No code is perfect, but the Go standard library encapsulates many of the ideals of the creators of the language.
For example, the
net/http package builds on top of the abstractions of the
net package, which, in turn, builds on the abstractions of the
io layer below it. This package structure works well because it would be nonsensical to imagine the
net package needing to somehow depend on
While this works well in the standard library but can be difficult to translate to application development.
Applying layers to application development
We'll be looking at an example application called WTF Dial, so you can read the introductory post to understand more about it.
In this application, we have two logical layers:
- An SQLite database
- An HTTP server
We create a package for each of these—
http . Many people will balk at naming a package the same name as a standard library package. That's a valid criticism and you could name it
wtfhttp instead, however, our HTTP package fully encapsulates the
net/http package so we never use them both in the same file. I find that prefixing every package is tedious and ugly, so I don't do it.
The naive approach
One way to structure our application would be to have our data types (e.g.,
Dial) and our functionality (e.g.,
http package could depend directly on it:
This is not a bad approach, and it works for simple applications. We end up with a few issues though. First, our data types are named
sqlite.Dial. That sounds odd as our data types belong to our application—not SQLite.
Second, our HTTP layer can only serve data from SQLite now. What happens if we need to add a caching layer in between? Or how do we support other types of data storage such as Postgres or even storing as JSON on disk?
Finally, we need to run an SQLite database for every HTTP test since there's no abstraction layer to mock it out. I generally support doing end-to-end testing as much as you can, but there are valid use cases for introducing unit tests in your higher layers. This is especially true once you introduce cloud services that you wouldn't want to run on every test invocation.
Isolating your business domain
The first thing we can change is moving our business domain to its own package. This can also be called the "application domain". It's the data types specific to your application—e.g.,
Dial in the case of WTF Dial.
I use the root package (
wtf) for this purpose as it's already conveniently named after my application, and it's the first place new developers look when they open the code base. Our types are now named more appropriately as
You can see an example of this with the
In this code, there is no reference to any implementation details—just primitive types &
time.Time. JSON tags are added for convenience.
Remove dependencies by abstracting services
Our application structure is looking better, but it's still odd that HTTP depends on SQLite. Our HTTP server wants to fetch data from an underlying data storage—it doesn't specifically care if it's SQLite or not.
To fix this, we'll create interfaces for the services in our business domain. These services are typically Create/Read/Update/Delete (CRUD) but can extend to other operations.
Now our domain package (
wtf) specifies not just the data structures but also the interface contracts for how our layers can communicate with one another. This flattens our package hierarchy so that all packages now depend on the domain package. This lets us break direct dependencies between packages and introduce alternate implementations such as a
Breaking the dependency between packages allows us flexibility in how we use our code. For our application binary,
wtfd, we still want
http to depend on
wtf/main.go) but for our tests we can change
http to depend on our new
mock package (see
This may be overkill for our small web application, WTF Dial, but it becomes increasingly important as we grow our codebase.
Packages are a powerful tool in Go but are the source of endless frustration if you view them as groups instead of layers. After understanding the logical layers of your application, you can extract data types & interface contracts for your business domain and move them into your root package to serve as a common domain language for all subpackages. Defining this domain language is essential to growing your application over time.
Have question or comment? Please open a thread on the WTF Dial GitHub Discussion board.