Standards: Or, How to Program Engineers

I’ve been working in the internet standards space for well over a decade at this point, and have authored and/or edited several of them myself along with contributing to many others. Many years ago, I had a realization about how standards function in the world that has helped me immensely in the long run. It really comes down to two parts:

1. Standards specifications are like computer programs, specific instructions intended to be executed by an implementing engineer.
2. Engineers are the worst and most unreliable runtime platform ever imagined.

These aren’t just documentation

At first, a technical standard seems like it’s a form of documentation: you are writing down what goes into a protocol or system, why it’s there, and what every bit means. While that’s true on the surface, it ignores an important point: you are writing a standard because you want different people, who aren’t talking to each other, to implement things in a way that the different parts work together.

The whole point of a standard is interoperability. Defining what counts as interoperability is a completely other topic, but the goal of a standard is to make sure different implementations interoperate in the way the standard intends.

This means that a standard isn’t just about describing an idea, as much as the document’s authors might love the idea and their solution to it. It’s about giving explicit instructions to someone who needs to implement it. Is something optional? There needs to be guidance on when and how to choose which option. Is something not an option? Be clear that there’s only one way you expect it to be done. This means that another implementation can make the right assumptions on what’s allowed and what’s not.

If it can go wrong, it will

Even with the best standards, people are fallible and they’re going to get things wrong. This isn’t necessarily malicious. In fact, the vast majority of implementors want to follow the spec and do what it says. But more than that, they are implementing a standard because it does something that they need done but don’t want to invent themselves. Engineers are great at coming up with creative solutions to problems, but they’re even better at copying someone else’s solution to a problem they don’t want to care about anymore. And that is where standards come in.

Your average engineer will read a standard in the following order:

1. The title. Well, most of the title. They probably searched a couple keywords and if those are in the title, we’re done here.
2. The examples. These are far and away the most important portion of the standard, because implementors are going to either copy these directly or tweak their code until it looks, to their eyes, like the examples.
3. Any bulleted lists around the examples. After all, these probably show additional options that might not be in the examples. Developers will look at these if the examples don’t do exactly what they want.
4. Any actual text. This stage is only reached when things are starting to go wrong. By this point they’ve probably already added a bunch of stuff to the protocol and are trying to figure out why that’s breaking something.

Your beautiful specification is full of details that simply don’t matter to most developers unless their system breaks. If they can get to the happy path without doing error checking, they will. If they can hack together something or pull on a half-understood library module to do some bit they don’t care about, they will.

Really, it’s amazing the shortcuts people will take. I have seen people write to XML-based specs using regular expressions instead of an XML parser. I have seen people use string concatenation (with no escaping) to build all kinds of data structures. I’ve even seen a giant multi-level IF statement used to parse a data object, which broke as soon as someone turned on pretty printing on the server.

And lest you think I’m any better by my years of experience, I’ve done this kind of thing myself, too. I was even part of an early implementation of something that accidentally created a DDoS condition by following the draft specification exactly as it had been written. There was an optional field saying how many seconds the client needed to wait before trying again. Since it was optional, our server didn’t send it. The client we were testing against took that missing optional field and filled in a value of zero seconds to wait before trying again, and so it sent out its next message immediately. This one little client quickly overwhelmed our test server to the point it corrupted our low-volume test database (which wasn’t set up to scale at all). In addition to fixing the client implementation, we were able to add guidance to the specification in question before publication so that future developers wouldn’t make that same mistake.

Remember who is implementing

The final thing about a standard is that most engineers will never read it, at all. Most developers are going to approach your standard through library or vendor documentation, or webpages that describe in plain language what’s going on with the protocol. The list above is only for the kinds of folks who need to go back to the spec itself, which includes the authors of those libraries and the engineering teams at those vendors that everybody else uses.

I have heard it argued that because of this, a specification doesn’t need to be understandable by a layperson, since libraries will take care of the complicated portions for most people. To me, that is a sign that something is likely too complex to be implemented correctly, even by library implementors. The truth is that there will eventually be someone, on some platform, who wants to code to this spec but doesn’t have a set of tools on their platform. Somebody, somewhere, needs to write that library. These are the people who should be able to get all of the information they need from your document. If you expect people to have HTTP, tell them that. If you assume there’s a full JSON parser available, state that. That way, a developer approaching your standard will know which tools they need in order to build it correctly.

How a specification author can help

I have found it to be an incredibly helpful exercise to treat the instructions within a specification as code from the perspective of a naive implementor, someone who doesn’t have my background or goals when approaching the spec text. Take the spec text and do what it actually says. Not what you meant for it to say, not what you wanted to be able to do with it, but just what it says. It can be enlightening to see the gaps and assumptions that get made here.

Think in terms of code paths. Every single decision point needs to have clear branching context to allow the implementor to make the right decision for their context. If you made something optional, it’s optional for a reason, and a decision needs to be made when you hit that branching point. If you have a field that has a name, make sure you write down what name it is. If something has an internal format, make sure you write down that format and the rules for consuming and creating it.

Formal testing and analysis can help this, and we’ve seen fantastic results in the wild from making conformance testing freely available. However, it’s not a sure thing — I’ve also seen cases where the conformance suite and the specification disagree, and the community goes with whatever the conformance suite accepts. Remember, standards bodies do not have enforcement arms, and developers are going to do whatever they have to to solve their problems and move on.

It’s tempting for a specification to be loosely defined so that a lot of different implementations can claim compliance. This type of lip-service to compliance is particularly dangerous in the process of writing standards, and unfortunately it’s very common. I have seen far too many so-called “standards” get published where there is zero chance of code re-use or interoperability between different systems because everything is just far too loosely defined. Usually this happens because some company has a product that they want to stamp with the standard-compliance label, and they don’t want to change that product. If there’s anything that they can do to protect their proprietary magic, they will, but a good standards author and working group can actively guard against this kind of attack by always asking what a naive developer would do with the optional spaces.

In the end, you should never forget the specific part of specification — remember why you’re writing it in the first place, and who you’re writing it for. This code has to run, and it’s up to you to write something that will run consistently and correctly on the worst execution platform possible.