Developer productivity is a product of program understanding, a developer's (1) mental model, and (2) access to information for refining that model.
Former Defense Secretary Donald Rumsfeld's famous intuition is actually roughly correct for understanding program understanding:
There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.
Because software development involves constant decision making, it's also important to consider whether information is discoverable — that we can, when required, make "known unknowns" become "known knowns". This yields three dimensions:
- Awareness: does your (the developer's) mental model consider the information?
- Presence: is the information present with the software (i.e., known)?
- Computability: can you compute and update the information automatically (i.e., knowable)?
Across these dimensions, there are six situations that occur during software development:
| Level | Situation | Aware? | Present? | Computable? |
|---|---|---|---|---|
| 0 | Unknown Unknowns | No | No | Maybe |
| 1 | Uncomputable Information | Yes | No | No |
| 2 | Stale / Misplaced Information | Yes | Maybe | No |
| 3 | Developer-Supplied Information | Yes | Yes | No |
| 4 | Uncomputed Information | Yes | Maybe | Yes |
| 5 | Perfect Information | Yes | Yes | Yes |
In general, to increase the quality of information available,
you must supply additional information in the right form. The
information you supply signals intent, and can serve as
machine-checked documentation (with the right tools).
The rest of this post describes the six situations, providing some intuition behind their causes. It starts with Level 0, the most dangerous level.
Level 0: Unknown Unknowns. Properties of which the developer is unaware. These typically occur when you have not anticipated a use case, or do not understand a platform well enough to be aware of potential issues. Examples:
- Thread-safety
- The (unintuitive) behavior of Javascript's operators
- Argument aliasing
This is the worst situation to be in. If and when your software crashes, your only hope is to re-create and observe the effects of the behavior. Because the behavior is not part of your mental model, you won't know how to observe it directly.
You eliminate "unknown unknowns" by expanding your awareness in two ways: (1) experience, and (2) a broad exposure to software engineering topics and frameworks. For most types of software, your mental model doesn't need to be complete! You just need to be aware of potential "gotchas" so that you can direct your development and debugging.
Level 1: Uncomputable Information. Properties of which the developer is aware, but does not have an easy way of deciding automatically. Examples:
- The correctness of a sorting algorithm implementation
- Program termination
- How clients use your library
Level 2: Stale / Misplaced Information. Information that either might be out-of-date or is not present with the development artifact. Examples:
- Design decisions made by a developer who was fired
- Documentation for an old version of an API
- Requirements captured in an email thread
If you have no way of telling which information is accurate, you're back at "Level 1: Uncomputable Information". If you have no way of telling if the information is complete, you're back at "Level 0: Unkown Unknowns".
Level 3: Developer-Supplied Information. Information that is up-to-date, but the developer has no easy way of knowing how a change might affect that information. Examples:
- A written proof of an algorithm
- Class documentation
- Renaming a field in a dynamically-typed program
If you don't update this information when making a change, you'll quickly find yourself at "Level 2: Stale Information". By instead capturing information in a machine-readable form, you can instead approach "Level 4: Uncomputed Information".
Level 4: Uncomputed Information. Information that can be computed automatically, but is not yet available for the current version of the program. Examples:
- Non-incremental compilation (e.g., compiling a C++ program)
- A long-running test suite
- A long-running static analysis
When a failure occurs, you might be back at "Level 3: Developer-Supplied Information" or "Level 2: Stale / Misplaced Information". For example, if a test fails, you need to be able to determine (1) if the failure is valid, and (2) if so, how to modify the program and/or test suite.
For many types of information, you can effectively bring yourself to "Level 5: Perfect Information" by applying the Pareto principle. In the case of a long-running test suite, for example, you might prioritize the execution order of the test suite.
Level 5: Perfect Information. Information that is up-to-date and is updated automatically whenever the developer makes a change. Examples:
- Incremental compilation with type checking
- Continuous testing
- Speculative analyses that compute the effects of a change before you even make the change
Remember that to increase the quality of information available, you must supply additional information. For example, typed languages require type annotations; more-detailed type analyses (e.g., the Checker Framework) require yet more (and more precise) annotations.
Summary
Program understanding depends on information awareness, presence, and computability. The next time you hit a wall programming or debugging, take a step back and ask: "where is the information failure?" By answering this question, you can set yourself and your team up to be more productive.
This post is based on an early framing of my PhD research. As part of my research, I designed the Cupid framework for giving developers immediate insight into their project and team.