Games from Within

I've been following the discussion on the evils of assert started by Len Holgate. Poor old assert was getting beaten up from every side, so I'm going to have to step forward and defend it.

Yes, I do find assert useful. Yes, I'm doing full-out test-driven development. Yes, I'm a hardcore C++ programmer.

Before we even get started, let me recap what I see as the main reasons to use assert.

• Detect a program error or bug right away. The sooner the better. There's nothing more frustrating than trying to track down bugs that occurred several function calls away, especially if they've unwound the stack and started a new series of calls. Even worse are errors that propagate and don't become apparent until several frames later.

  Example: Creating a new character from a spwan point during gameplay should always work, so asserting on the pointer returned by the factory is a good idea.

• Document a decision, limitation, or assumption of particular functions. Nothing like calling a function and getting an assert right away to realize that's not the way it was intended to be used.

  Example: A function that takes a pointer that can never be NULL and we don't want to take a reference (maybe because the object is going to store and keep ownership of the pointer).

• Catch bad situations that will soon lead to trouble.

  Example: Asserting that the matrix passed to a rotate function is orthonormal. Everything will work fine for a little while if it isn't, but soon things will degenerate if errors keep accumulating.

One key point about the use of assert is that it's a message from the programmers to the programmers and nobody else. The idea is that whatever caused that assert is a top priority and should be fixed right away. An end user should never see an assert message because a) they don't care and b) it means their program is toast.

This distinction between programming errors and "user" errors is particularly important in game development. User errors are anything that a user of the tools or game could cause just by using it. In our case that includes the content creators (designers and artists). The last thing we want to do is stop everything just because they typed the wrong file, or tried to load an old version of a model. Those errors need to be caught and dealt with. They are expected.

Programming errors are the ones that can only be caused by something done by a programmer in the code. As an example, in game development we constantly have functions that look like this: void Something::Update(float deltaTimeInSec). Unless you have a simulation system that can go both forwards and backwards, trying to call that function with negative seconds is nonsensical. An assert there tells the programmer that something either went horribly wrong, or they're trying to use Update in a way it wasn't intended. Clearly, no user of the program will ever be able to call Update with a negative time value just by using the program. So in this case, I do want to assert and have that fixed right away.

With that out of the way, let's look at Len's five major complaints about assert. Let's take them one at the time.

1. The check will go away in a release build.

From my point of view, an assert is a way to tell a programmer (probably even myself in the future) that something has gone completely wrong. If an assert ever goes off, it's a bug in the code and a programmer needs to fix it.

So what if asserts go away in release builds? We're writing "shrink-wrapped" software, so that's a good thing because there will be no programmers around with their handy debugger to fix the problem. If a user ever manages to come up with a way to cause the code to get in a bad state, then all we can do is pray. Especially in the case of console games, trying to capture traces and exception reports is not going to help anybody.

2. The assert is covering poor design.

I think this is really the crux of the discussion. Is assert really just a crutch for "poor design"? If you define good design as creating bullet-proof code that will work under any circumstance then it's probably right. But you know, my job is not to write bullet-proof code. My job is to write whatever needs to be written to satisfy our customer stories every two weeks, and eventually ship a game with it.

This is very much in the agile development mindset. I'm not going to try to bomb-proof a piece of code for all foreseeable uses in the future. I'll make sure it does what it needs to do now and leave it at that. Under this view, asserts are post-it notes saying "This code doesn't even handle this situation. You either screwed up, or you better update the code to deal with it."

Let's go back to the Update() function. What are our options to make it work without assert?

• Implement Update() to go backwards in time as well as forwards. That will easily more than double development effort for something that will have no effect in the final product (unless you're implementing a mechanic like Sands of Time, although I suspect that not even that game had a negative-time update).

• Make it physically impossible to pass negative numbers into the Update function. We could create a delta time class that cannot be manipulated in any other way than to have positive values. While this is clearly possible, I feel it's a tradeoff between simplicity and "bullet-proofness." I'll take simplicity any day of the week for most situations.

• Have the update function check for negative values and return a failing value. That would be fine except that now everywhere we call Update we need to check for return values. Did you notice that Update(), the way it was declared, returned no values at all? That's a function that is supposed to work and never fail. What if an Update() function fails? What do we do next? Quietly move on to the next one, or try to fix the system in some way? I'd rather slap the programmer with an assert and have him fix it right away than quietly continue to work while things are not working as expected.

So I see it more as a tradeoff between simplicity, time, and robustness. There's no single correct answer, and it will depend on your particular situation. If you're writing flight-control software, to hell with simplicity and time constraints, and go for 100% robustness and throw a few redundant systems for good measure in case of cosmic rays interfering with the hardware. In my case, turnaround time and simplicity trump absolute robustness.

Also, I admit this might not work in every environment. It works for us because we're the only users of our own code. When something isn't good enough, we make it good enough. That's very different from writing a generic library for thousands of users to apply in their production code.

3. The assert makes it very hard to write a unit test for the failure condition.

I don't understand where this comes from. Maybe with a crappy unit-test framework it's a problem, but the unit-test framework I use deals with assertions just like any other exception, and fails the current test case. If your unit test framework barfs when your code tries to access an invalid pointer, you really need to look into getting a better one (I keep meaning to put up my version one of these days).

You should be able to hook up assert to anything you want (by using a custom assert or any other way you want). So the test runner first hooks up a special assert handler that throws a particular exception. Not only does this let us fail a test if an exception is triggered, but it even allows us to write tests to check that asserts are thrown. For example, I could write something like this:

TEST (UpdateWithNegativeTimeAsserts)
{
    Whatever stuff;
    CHECK_ASSERT(stuff.Update(-0.1f));
}

4. If the object is running without a user interface, then the assert's dialog box may cause issues.

What dialog box? Maybe I've been doing game development too long, but every single project I've worked on has had a custom assert macro that gives us a lot more control than the standard one. Seriously, are people calling the standard assert() everywhere? No wonder they have issues.

Game Programming Gems 1 had a great introduction to using custom asserts by Steve Rabin with lots of great ideas (here's an implementation based on his article). At the very least, you should be able to have full control of what you want the assert to do. A lot of the time that will simply be _asm int 3, but many other times you want to throw an exception, write out a log, send out an email, update a database, or even put up a dialog box to allow the user to bypass it. As a side note, I actually hate the option of continuing after an assert and I would never allow it in my own implementation. Assert means dead. Kaput. Done.

A custom assert takes five minutes to set up and will pay back in gold-pressed latinum in no time.

5. The assert is really just an "exploding comment."

Yes, that's completely true. It's more like a programmer yelling in your ear "YOU JUST SCREWED UP!" But then again, that's one of the things that make unit tests so great. They're like comments that complain every time you change the behavior of the system in a way that violates what the tests are checking. Those are the best kind of "comments."

That takes care of Len's objections. Clearly, I don't see any of them being an issue other than the second one, which is a matter of tradeoffs. Having said all that, I do use asserts less than I did before I used TDD. Unit tests take care of some of the things I would have checked with asserts in the past (did this loop really iterate over all the members that needed to be updated?), but unit tests still can't protect against misuse of functions or simply unexpected situations.

So no, I'm not willing to give up on assert any time soon. It might be an ugly, rusty, oversized tool, but it still has a very definite spot in my toolbox.