If you’ve spent much time working with a public cloud you may have used tags or labels to organize your infrastructure. AWS and Azure let you tag resources (I’m reminded of blog tags) while Google Cloud calls their tags “labels”. Kubernetes has a similar set of features called labels and annotations. But, what Kubernetes offers has a little more depth and nuance than simple tags. This nuance is worth exploring.
What’s The Difference?
If you’re used to a single “tagging” format that you can cram everything into you’re out of luck in Kubernetes. It’s not quite like the container image annotation or label schema formats. There are two ways to associate data with different uses and restrictions.
Labels
Labels are key/value pairs where the names must be 63 or fewer characters and the values are limited to 253 characters. This shorter length can limit the use of things like URLs as values because a URL can be thousands of characters long.
What’s special about labels is that they are designed to be used for querying. They can be used to identify a set of objects and query for them. For example, this is useful if you want to query for all the objects used by an application or a part of an application.
Annotations
Like labels, annotations are key/value pairs. Where labels have length limits, annotations can be quite large. It’s no issue to put a URL into one of them. But, you can’t query or select objects based on annotations. The data is available and you can do interesting things with it (more on that in a moment) but you can’t query based on it.
Namespace Separator (and extra capabilities with it)
AWS uses a : separator for namespaces in tags. The OCI spec uses a . for the same thing, which kind of reminds me of Java in some ways. Kubernetes notes that names can have an optional prefix where the prefix and name are separated by a /. This is a method of namespacing.
The prefix must be a DNS subdomain and can’t be longer than 253 characters.
When a prefix is omitted the key is presumed to be private to the user.
How you use this namespaced prefix is important for labels, since you use them to query, but isn’t limited to them.
Properties But As Annotations
In addition to associating general metadata with an object, annotations are an interesting way to add on functionality to your applications. While they can’t be queried, annotations can be read by other tooling. This is where things get interesting.
Properties on Kubernetes objects will live on forever. As long as that API version is supported they will be there. But, some things might need to be worked out on a trial basis or only pertain to some configuration for a type of thing. How can that data be recorded? In an annotation.
A simple example can be seen in the nginx ingress controller. When using nginx for ingress there are additional options that can be passed in as annotations. For example, configuration around forcing SSL.
The Gist
As a general rule:
- If you want to query for objects based on the data put it in a label.
- If having the data available for other tooling or general information is important, put it in an annotation.