Code Engineered

How We Did Logging Differently In Hypper

When we started working on Hypper we knew there would be an SDK and a client in the codebase. From the beginning, we wanted the core business logic to be easily accessible for other applications to use.

This meant we needed to have logging that worked for a CLI application and when the SDK was pulled into an app. All of this written in Go due to the need to pull in some outside libraries only available in Go.

The problems began to show up when I discovered that logrus, the popular logging library that we regularly go to, was in maintenance mode. The readme specifically says,

Logrus is in maintenance-mode. We will not be introducing new features. It’s simply too hard to do in a way that won’t break many people’s projects, which is the last thing you want from your Logging library (again…).

This does not mean Logrus is dead. Logrus will continue to be maintained for security, (backwards compatible) bug fixes, and performance (where we are limited by the interface).

Did I really want to start a new project with a logging library that’s in maintenance mode? Logrus recommends some other logging packages to use instead. Should I start with one of those? What about the apps I hope Hypper integrates with? How do I bridge the old and new logging along with logging that works well for a CLI?

An Interface and Separation of Concerns

As I reviewed logging packages, I discovered that many of the Go packages implemented the same features. Going beyond Go, the same features generally showed up in other languages as well.

It also occurred to me that there should be a separation of concerns between libraries and applications. A package, SDK, or library shouldn’t dictate the logging implementation of an application. Yet, this happens all the time with Go. It part of the reason that Kubernetes imports so many logging libraries.

This current logging problems with Go applications are due to a lack of an interface in the Go standard library that’s sufficient. Many languages now have a logging API as part of the standard library or through an agreed to external standard. Go has a simple package in the standard library that’s been ineffective.

So, it was time to create an interface along with some implementations.

github.com/Masterminds/log-go

log-go provides a simple interface for logging that works with zero configuration or with your logging library of choice. Hypper provided the first usage to make sure it works.

In a Go package it’s simple to log. You do something like…

import(
    "github.com/Masterminds/log-go"
)

log.Info("Send Some Info")

There are logging levels for Fatal, Panic, Error, Warn, Info, Debug, and Trace. Formatting and messages with fields are also supported for each of the levels.

This setup is just the simplest usage. log-go package provides an interface that can be used on structs or passed around. You can get fancy.

The main application can then configure the logger. The default logger writes to stdout. This can be overridden and any logger that has a wrapper to make it conform to the interface will work.

A simple example would be…

import(
    "github.com/Masterminds/log-go"
    "github.com/Masterminds/log-go/impl/logrus"
)

log.Current = logrus.NewStandard()

This creates a new logrus logger and uses it application wide. If you want to use a custom configured logrus logger you can use logrus.New instead. This function has an argument for a logrus logger and returns a wrapper that implements the interface.

While log-go has some reference implementations, the whole thing is designed to make the separation possible and for people to write their own implementations.

Working With Other Loggers

Hypper pulls in outside packages and some of them have their own logging setup. We ran into this quickly and the solution we came up with was to make the logger an io.Writer. We designed it this way so that you could make the io.Writer work with the logging level of your choice.

For example…

import(
    "io"
    "github.com/Masterminds/log-go"
    logio "github.com/Masterminds/log-go/io"
)

func main() {
    w := logio.NewCurrentWriter(log.InfoLevel)
    io.WriteString(w, "foo")
}

The one thing to realize is that the logger isn’t a general purpose io.Writer. Because it works on logs it adds a new line at the end of the message if one does not already exist. At least this is the case for the existing API implementations. If someone wanted it to act differently all they would need to do is write their own implementation.

Setting Logging Level

You might be wondering how you set the logging level that’s actively reported. This depends on the implementation. Each logger, such as logrus or the CLI logger (which supports colors), handles setting the level through their native means.

Conclusion

A logging interface solved our logging decision problem on Hypper. It’s allowing us to clean up the use of logging libraries in our app. This might work for you, too.

You can learn more about log-go on GitHub. You can see how we used it with Hypper by looking at the source, also on GitHub.