Tuesday, April 7, 2009

Never Too Many Asserts

My friend Keisuke says an assert is like a circuit breaker. It does no harm to be in the circuit when things are operating normally. But when the voltage goes out of bounds, your program stops.

We all accept that a defect costs more to fix the later it is discovered. If you have already checked in, it affects your coworkers. If you have already shipped you will have to go through a new release or public patch. The tenets of extreme programming are that if something is good, taking it to an extreme is probably better. So if we catch a defect in an assert while we are actually developing nothing could be better. If our coworkers "protect" their modules with asserts, we can be more confident that our use of their stuff is legitimate. And we can go faster, and have a more reliable system.

I like to use a number of different kinds of asserts:
  • External Interface (ASSERT_EI): validates that the parameters to a method that is expected to be used by other major modules or by the customer are legal and in range. But they should not be used to validate user input. This is a super-critical assert. The more of these the better.
  • Internal Interface (ASSERT_II): validates the parameters to methods that exist for good code structure and are intended for use only within the current module. Private methods would use these. They are for defensive programming, and reminding yourself what your assumptions were when you designed this method. They can provide a kind of documentation. These are less important than ASSERT_EI, but are great for debugging old code that you don't remember very well or that you didn't write. Again, the hope is that you can go faster. What if you took a short cut, and haven't finished a method for one use case. You can leave an assert such that if you accidentally use the method that way, you will obviously be reminded of the missing work, as opposed to having the method silently do nothing, or fail with some bizzare side effect.
  • Internal Consistency (ASSERT_IC): validates that the state of a class remains consistent as it is manipulated. It is an invariant check. The design of your module makes assumptions about its data structures (e.g. an data item is in a list only once). Assert this periodically. I like to add a SanityCheck method to almost every class I build. It executes all the invariant checks I can think of (and yes, it can be pretty slow). It is especially useful to sprinkle around if you are currently tracking down a bug. It makes sense to verify your invariants going into and out of a method. Centralizing those invariants in a SanityCheck function can be pretty useful.
  • External Consistency (ASSERT_EC): I don't use this very much, since nicely modular systems should not have tight interdependencies. For example, your module may be dependent on the configuration file being parsed before it is initialized. An ASSERT_EC can check (and document) that assumption.
I've seen game programmers take a couple of positions against asserts:
  • They slow down the execution of debug builds so much that the app becomes unusable. OK. It really is too slow. There should be easy ways to disable the asserts especially in heavily executed bits of code. In fact, one should think twice about putting asserts inside tight loops. Also, your assert implementation should be able to runtime skip the predicate based on runtime configuration files, and do it fairly efficiently (e.g. check a global boolean). Disabling per module would be a good start. Providing a level-of-detail argument to the asserts should be possible.
  • I keep having to skip some assert in someone else's code that I don't understand, and that interferes with my workflow. Everyone should be careful not to use asserts to remind us of work that needs doing. Some other mechanism should be used. Or wrap those checks in a per-developer ifdef. On the other hand, if a developer is hitting asserts they don't understand, maybe they are breaking code they don't understand. Using either comments or change-control software, a developer should find the auther of the assert, and learn about that bit of the system, or get them to change the assert if it is now obviated.
  • I changed one little thing and all these asserts starting going off. This one is pretty funny. Given that the asserts are there for a reason, there is a pretty good chance that the one little change had much further reaching implications than expected. For example: someone now wants to handle a member being NULL. But ASSERT_IC's start going off that the member shouldn't be NULL. The thing is, if the rest of the class was built assuming that member can never be NULL, it could easily derefernce the pointer without checking. An argument that "it should have checked for NULL" doesn't fly. The assumption was built in.
Adopting these "extreme" uses of asserts might even take the place of unit tests for that class.

2 comments:

  1. You had me until the last sentence. Lets not even think about asserts replacing Unit Tests. There are way too many permiations of outcomes for an assert to eplace a unit test. Used properly asserts can help unit tests but not replace.

    ReplyDelete
  2. I meant “fuse” instead of “circuit breaker”. It’s still the same idea, but fuses seem better analogy, as there are many little fuses all over as opposed to having one central circuit breaker . The analogy was taken from this old DDJ article: An Exception or Bug?

    The Design by Contract (DbC) is one of my favorite software engineering philosophies/principals/techniques. Unlike some newer languages such as “D”, C++ doesn’t have built-in support for DbC, so we use asserts to implement DbC (pre-conditions, post-conditions, invariants, and etc.) along the lines of asserts Darrin mentioned.

    I like the idea of having Internal Consistency (ASSERT_IC) checkers. I implemented a similar mechanism for the Telecom equipment I worked on some time ago. Running consistency checker periodically gives us reassurance that the system is running well for systems like Telecom devices that aren’t very visual and that need to continue to run almost 24/7. This may well be applicable to something like online game servers as well.

    David, my take on the last sentence is this: a unit test (in a very basic sense) does two things; drives an execution (gives input to the thing under test) and validates the output/side-efforts. By implementing DbC or asserts Darrin mentioned, you practically got the second half of what a unit test does embedded and covered. You still need the first half and that’s why you still need unit tests, but I think Darrin’s point is that it’s okay and often good to have the validation part (i.e., asserts) co-exist with the class under test, as opposed to it being in a unit test module.

    ReplyDelete