wholesalealgorithms.com Blog

The Trouble with Templates

matt — Sat, 12 Jun 2010 03:45:26 +0000

Don’t Misunderstand

My feelings about templates are often misunderstood – particularly by those who have known me long enough to remember the days when I swore that C++ had no place in video games. That was when we had to fit the whole game into 2 megs of RAM, and we were doing things like squeezing the entire stack onto a 2 k scratch pad. Those days were a long time ago. Things have changed, and I have no desire to go back.

These days, C++ is the language of choice, and with that comes templates. Template are powerful tools, and it would be foolish to dismiss them. The opinions expressed here are not intended to discourage their use, but more to temper the exuberance with which many developers have embraced them.

The Good Stuff

Let me begin with some good reasons to consider templates as part of your software architecture.

The golden rule of programming is, “Don’t write the same code twice.” I probably don’t need to justify that statement, but much to my amazement, I have run across some disagreement on that point. The theory is simple. Every time you repeat the same piece of code, you introduce a new chance of an error. If you instead re-use a single function, any potential bugs in that function can be fixed in one place.

Templates allow us to carry this concept to the next level. Simple functions can only be re-used if they are operating on the same type of object. Templates allow us to abstract the algorithm being used to apply to a wide range of different methods. By re-using a template rather than reproducing the algorithm with a new type, we reduce the number of different places where bugs can occur. If a bug is discovered, the template can be corrected, and we can be assured that fix will be applied everywhere. In contrast, the cut & paste method will leave you pouring over your code looking for the same bug repeated elsewhere with no assurance that you’ll ever find them all.

Unbridled Enthusiasm

When programmers encounter their first template, their reaction is often one of distrust and skepticism. But as you begin to get more comfortable with them, such fears tend to melt away. Once the fear is gone, the promise that templates offer becomes very alluring, and it is easy to be taken in. Naturally, it is those who are the most comfortable with templates that tend to use them the most, and it goes without saying that the authors behind your typical STL implementation are typically the most enthusiastic fans.

Like with all things in life, moderation is the key to a pattern of reliable success, and such enthusiasm can carry with it a price. What price am I thinking of?

Debugging STHELL

I was hired onto one project primarily to do performace analysis and improvement, but when it came to debugging nasty crashes, I was available for that as well. This team had fully embraced the STL paradigm, and no small number of their performance issues resulted from that, but what I want to point out here has more to do with the extreme difficulty of debugging issues that manifest themselves within STL.

In one case, I was called over to help analyze a crash, and I was so impressed by what I saw on the callstack that I copied it down. Here is the top line from the callstack - completely unedited:

default.exe!_STL::_Rb_tree,
_STL::pair const ,
FXShader::ExpressionEvaluator *(__cdecl(FXShader::Effect&)>,
_STL::_Select1st<_STL::pair const ,FXShader::ExpressionEvaluator *
(__cdecl*)(FXShader::Effect &)> >,
_STL::less >,
_STL::allocator<_STL::pair<
TypedAssetId const,
FXShader::ExpressionEvaluator * (__cdecl*)(FXShader::Effect &)> > >::
insert_unique(_STL::_Rb_tree_iterator<_STL::pair<
TypedAssetId const ,
FXShader::ExpressionEvaluator * (__cdecl*)(FXShader::Effect &)>,
_STL::_Nonconst_traits<_STL::pair<
TypedAssetId const ,
FXShader::ExpressionEvaluator * (__cdecl*)(FXShader::Effect &)> > >
* __position=0xffffffff,
const _STL::pair const ,
FXShader::ExpressionEvaluator * (__cdecl*)(FXShader::Effect &)>
&__v={...})
line 476 + 0x8 bytes C++

That’s a single function call, and the crash occurred somewhere within that function. What on earth is that function? A casual glance reveals that it clearly involves an STL Red-Black tree, and it has something to do with TypedAssetId and FXShader. But that function signature is so out of control… this must be some really extreme case where someone is building up some serious template right? Let’s take a look at that same line of code in the original C++:

possibleUnitsMap[ unit.m_id ] = &unit;

That’s it. This innocent-looking line of C++ translates into that incredible rat’s nest of STL – or as I like to call it, STHELL. This is no exaggeration. The example above was pasted directly out of the debugger – completely unedited.

Most programmers would scarcely dare to delve into that rat’s nest any deeper, but when you have a game that crashes, you have few options. Somebody is going to have to study the situation, determine what has caused this crash, and use that information to formulate a solution.

We did find the cause, and we got the problem fixed, but the effort that was required was greatly increased by the complexity of the template implementation for what amounts to a simple map insertion.

Speaking of Maps

Here’s another example that just came up for me last week. I was trying to figure out why the geometry was not appearing on my object in a timely fashion. As I traced the problem, I ran across this simple line of code:

return m_assetMap.find(guid, assetID);

I fully expected to find my asset in the map, so when the search failed, my immediate reaction was to assume that I had not properly loaded it. After tracing that problem through, I found that I had indeed loaded the asset and its guid was inserted into the map. So why does the search fail?

At this point, you say to yourself, “well if your guid isn’t in the map, what is in the map?” At least, that’s what I said to myself. I had avoided doing this for days now, and there was simply nothing else for it. It was time to debug the STL map. Attempting to examine the map in the debugger is an exercise in futility. Here’s what that looks like in the watch window:

-m_assetMap
  -stlp_std::map,lib::CustomAllocator >
    -_M_t
      -stlp_std::priv::_Rb_tree_base,
       lib::CustomAllocator >
        -_M_header
           lib::CustomAllocator > >
        -_M_data
           _M_color false bool
          +_M_parent 0x14f89030
          +_M_left 0x03617d20
          +_M_right 0x14f89090
     _M_node_count 11
    +_M_key_compare {...}

To get anything useful out of this data, you’re going to have to step into the template functions until you get deep enough that the debugger will actually understand what it’s looking at. Tracing down into map<>::find() I eventually got into a place where the debugger could show me the strings being compared. Here’s the callstack at that point. (In this case, I have changed some namespaces to protect the innocent guilty.):

lib::operator<(const lib::utf8string & lhs_utf8={...},
const lib::utf8string & rhs_utf8={...})

stlp_std::less::operator()(const lib::utf8string & __x={...},
const lib::utf8string & __y={...})

stlp_std::priv::_Rb_tree,
stlp_std::pair,
stlp_std::priv::_Select1st >,
stlp_std::priv::_MapTraitsT >,
lib::CustomAllocator >::_M_find
(const lib::utf8string & __k={...})

stlp_std::priv::_Rb_tree,
stlp_std::pair,
stlp_std::priv::_Select1st >,
stlp_std::priv::_MapTraitsT >,
lib::CustomAllocator >::find
(const lib::utf8string & __k={...})

stlp_std::map,
lib::CustomAllocator >::find
(const lib::utf8string & __x={...})

lib::map >::
find(const lib::utf8string & i_key={...}, lib::utf8string & o_data={...})

FindAssetID(const lib::utf8string & urn={...}, lib::utf8string & physicalID={...})

As I wade through my map element by element, eventually I did come across the guid I was looking for. It seems the guid I was searching for was:

guid:5075987e-0000-0000-0000-000000000000

but the guid that had been inserted into the map was:

guid:5075987e-0000-0000-0000-000000000000:modelformat

Duh! Why hadn’t I seen that right away? /sarcasm off

Breakpoints in STHELL

As part of the typical debugging process, I like to go straight for breakpoints. During the course of the above example, I naively attempted to insert a breakpoint onto line 552 of _tree.h on the line that says:

if (!_M_key_compare(_S_key(__x), __k))

When I did that, Visual Studio became non-responsive. After several minutes, it was eventually clear that with that single keystroke, I had inadvertently inserted 183 breakpoints. The compiler will generate new instructions for every unique instance of the template it comes across. Apparently there are 183 different permutations of _tree currently in this codebase.

Needless to say, breakpoints in the STL of a project like this one aren’t going to be very useful tools, and that’s a terrible price to have to pay.

Summary

So what should we do about this? Should we swear off templates once and for all? Outlaw STL?

Well I wouldn’t want to throw the baby out with the bathwater. Templates are indeed powerful tools, and it would be foolish to dismiss them entirely.

What about STL?

STL is a handy suite of pre-built and pre-debugged containers. They can save you a lot of development time up-front and they typically save you debugging time down the road. If your application isn’t particularly speed-critical or sensitive to excessive memory fragmentation, STL is a great asset as long as you know what you’re doing. I advise people to avoid STL in console game engines, but I find I am seldom heeded on that. On the other hand, STL makes perfect sense for your tools.

Curb Your Enthusiasm

If you should ever find yourself in a position to reinvent an STL-like suite of templates, I urge you to remember those of us that will eventually have to decode the horrible mess the debugger is likely to make of your nested templates and specializations. Readable code is a win for everybody, and we always need to weigh that against any potential benefits we would hope to gain by creating yet another template.

That’s My Namespace. Don’t Wear it Out.

matt — Tue, 30 Mar 2010 17:08:16 +0000

Case In Point

The game crashes, so you open up the debugger. You look at the callstack to see that the crash occurred in a function called:

void DynamicButton::Activate()
{
    if (m_parent && m_parent->IsValid(this))        // crash occurs here
        m_parent->DoSomething();
}

Naturally you’re working with an optimized version, so the debugger’s idea of local variables is completely useless. According to the debugger, your this pointer is 0xffffffff… Yeah right. This is no problem for you – you aren’t afraid of a little disassembly now are you? It’s a really simple function, so you quickly deduce that your this pointer was stored off in r31 at the top of the function, and it is still valid at the time of the crash. So you go to the watch window and type:

(DynamicButton*)r31

That’ll do the trick certainly. But for some reason, the debugger won’t show you anything. It can’t seem to figure out what a DynamicButton is. How can this be? You’re looking at the source code right in front of you. Anyone can see that your this pointer is something called DynamicButton… or is it? Could it be your this pointer is actually something like this?

(GameEngine::Interface::UIShared::UIController::UIButton::DynamicButton*)r31

What the heck is going on here? The answer becomes clear only after you carefully scroll up through the source code and count up all the using commands sprinkled throughout.

Good Reasons to Use Namespaces

Don’t get me wrong, namespaces are a powerful and effective tool – especially for library development. Namespaces can be used to resolve naming conflicts for classes or variables. Naming conflicts don’t come up that often if you use reasonable names for things, but there are times when the inclusion of different library modules can introduce a naming conflict. Imagine you have a definition for class Quaternion in your math library. That’s pretty reasonable. In fact it is so reasonable, that it comes as no surprise that the author of your brand new animation library has a class called Quaternion too. Ideally, you’d both be sharing the same Quaternion, but sometimes that just isn’t possible. Not to worry. Just throw yours (or theirs or both) into a namespace and they no longer collide. Of course now your code doesn’t compile because you’ve referred to Quaternion everywhere, and it is now called MyNamespace::Quaternion. This is also easily solved. Just toss a little using namespace MyNamespace and all is well again.

This problem often comes up when you include libraries from different authors. Because of that, it is actually a good practice for library architects to enclose their stuff in a namespace.

Bad Reasons to Use Namespaces

At what point did it become good practice to create a new namespace for every layer of your hierarchy?

Namespaces prevent naming conflicts, but they do so at the cost of readability. When there is no chance of naming conflict, the namespace provides absolutely no benefit. Are we seriously worried that there might be another class called DynamicButton somewhere in the code? Ok I’ll give you that one. So Interface::DynamicButton still isn’t good enough? Ok maybe somebody has a namespace Interface there already? (We all know they don’t but what if they did?) I can see going all the way to:

GameEngine::Interface::DynamicButton

But this:

GameEngine::Interface::UIShared::UIController::UIButton::DynamicButton

…is just ridiculous.

Why Not Use as Many Namespaces as I Can Think Of?

It doesn’t hurt anything does it? Take my debugging example from above. That isn’t a made up story. That happened to me. The namespaces have been changed to protect the innocent guilty, but I am making no exaggeration.

Don’t Hide Your Code

Don’t forget the 80/20 rule. Most programmers spend 20% of their time writing code and 80% debugging code. Due to my particular specialty, I spend a great deal of time debugging other people’s code, and this gives me a unique perspective on things. The using statement hides the full name of the functions and variables, and that makes the detective work more difficult. Would you deliberately hide elements of your code from the other programmers on your team? (I suspect that some of you out there might actually answer “yes” to that. Why on earth would you do that? We’re trying to work together here, and that will go much easier if I can see what you’re doing.)

What about readability?

Longer variables and more words do not necessarily make your code more readable. Is “Moby Dick” more readable than “Horton Hears a Who”? Of course not. More words mean more reading not more readability. Once a single line of code no longer fits on the screen, it becomes dramatically more difficult to read. Consider our little debugging example. After discovering that there is in fact no such class as DynamicButton, we’ll probably want to take a look at the callstack window to see what it’s really called. What we see looks something like this:

TheGame.exe!GameEngine::Interface::UIShared::UIController::UIButton::DynamicButton::Activate() line 437

Or if your windows are arranged like mine, most of that will get cut off leaving you with something like this:

TheGame.exe!GameEngine::Interface::UIShared::UICont

Now you can clearly see the cost of all those namespaces. You’ve effectively made the names of your variables and functions so long that they don’t fit on the screen anymore. Obviously that isn’t going to work, so you’ve generously applied the using statement to allow you to shorten them to a more manageable size. Those using statements are your acknowledgement that you really didn’t need such long variable names – at least not in this part of the code.

And that brings us to the real problem…

Abusing the using

The using statement in C++ shows all the hallmarks of an “afterthought.”

“We need a way to avoid naming conflicts.” The namespace is born.

“Typing out the entire namespace all the time is a pain.” The using statement is tacked on.

The real problem here is that using statements are not scoped. For inexplicable reasons, the using statement is unlike virtually everything else in the C++ language. It becomes active as soon as it is encountered and stays active until the end of the file. There is no way to shut it off, and (at least at this point) completely opaque to debuggers.

This fact becomes painfully clear if you ever find yourself forced to put a using statement into a header file. (I say “forced” because you certainly know better than to ever do this on purpose.) Once you #include that header into your source file, that source file gets those using statements too. If the whole point of the namespace was to avoid naming conflicts, how are you going to do that now that all your source files automatically get using statements in them, and you have no way to turn them off?

Parting Thoughts

I’ll leave you with this little tale.

It just happened to me again.

void Viewport::UpdateCulling()
{
  DoTheOldStuff();
}

I’m testing something new, and I need a way to enable/disable it easily. So I go into the code like this:

bool gDoNewStuff = false;   //enable this to try the new way
void Viewport::UpdateCulling()
{
  if (gDoNewStuff)
    DoTheNewStuff();
  else
    DoTheOldStuff();
}

Simple right? Just change the value of gDoNewStuff in the debugger to change the behavior of the code and see the results in realtime. Only what’s this? The debugger says:

gDoNewStuff     CXX0030: Error: expression cannot be evaluated

What happened? Did the compiler outsmart me and optimize out my global variable? It looks like it did, but wait… could it be… could my global variable actually be called:

GameEngine::Renderer::Viewport::gDoNewStuff

Sad? …Yes.

Funny? …Maybe.

True? …Absolutely.

It’s just not worth it.

Asserts: Proper Care and Feeding

matt — Wed, 28 Feb 2007 21:50:49 +0000

Vocabulary

First off, we need to agree on a common vocabulary. Language is how we communicate, and it only works when we all agree to the same meaning for the words. Dictionary.com offers this as a defintion for assert: “to state with assurance, confidence, or force; state strongly or positively; affirm.” That’s good background information, but in the context of C++ we have a slightly more specific meaning. To avoid confusion when speaking to C++ programmers, we need look no further than <assert.h> for the definition we seek.

Simply put, assert is a macro which evaluates an expression and halts the program if the expression is false. The name of the file as well as the line number where the assert can be found are provided as output.

Yes you knew that already didn’t you? Of course you did. We all know what an assert is. Yet why do I keep hearing the phrase, “non-fatal assert?” We’ve agreed that an assert will halt the program. That’s what it does. So what would be the meaning of an assert that does not halt the program?

It is very reasonable to want to use some of the features of the assert macro for debugging purposes in a way that would not require the program to stop. Perhaps there is some sort of non-fatal glitch you want to monitor, but you don’t want to interfere with everyone else’s progress. You would like a macro that evaluates an expression and outputs the filename and line number if that expression is false but does not halt. This is a great idea. I would suggest that we refer to such conditions as warnings so that they are not confused with asserts.

Asserts are Not Break-Points

Where people start to really go wrong with their asserts is when they overuse them. Many years ago, I was working on a game with one other engineer – a perfectly competent one at that. One day a designers came to me because his game was crashing every time he tried to go to a particular area of the map – naturally the area where he was working that day. I tried it for myself and found that the crash he was experiencing was actually an assert. With only the 2 programmers on the team, it wasn’t too hard to find the right guy to ask about this. When I pointed it out to him, he replied, “Oh, that’s just there for debugging, you can go ahead and step over that.”

Seems he was looking for some sort of bug, and he was using asserts sprinkled all over to try to catch it. Clearly this bug wasn’t so severe that it was going to cause major problems without the assert because he had just told me to skip over the assert.

I should not have to say this, but this is not the correct way to use asserts. In this case, the programmer had managed to turn a minor bug into a progression-blocker that had already wasted multiple man-hours of designer time not to mention the time I had also wasted on it already.

Do all of your artists and designers run with debuggers open so they can “step-over” your asserts? Yeah mine don’t either. Do the other programmers on the team know which asserts are important and which are just your own personal experiment? How would they?

Do Not Allow Asserts to be Skipped

As I said, this example shows how the really catastrophic assert abuse gets started, but that’s only the beginning. What happens next goes something like this…

There are asserts in the code which are otherwise non-fatal (such as in the above example).
The rest of the team must not be blocked by innocuous asserts.
We must therefore set up a way for asserts to be skipped-over!

Brilliant! Oh, and you thought you were a genius for coming up with that one first right? Well congratulations, you’ve just broken asserts. Once you implement this lovely new feature, asserts will immediately start becoming more and more useless.

Don’t agree? Here’s how it goes. Once asserts can be skipped over, they are no longer a progression blocker. This gives you a license to put asserts in wherever you want to with no worries that it will prevent anyone from getting their work done. So what happens next is asserts start going in where warnings belong. The team will start using asserts to warn artists and designers about mistakes they seem to have made:

assert(pFrog->pLilyPad);  // the frog must have a lily pad!
assert(hitPoints > 0);    // monsters must have hit points!
// make sure your triangle is not degenerate!
assert(vertex[0].x != vertex[1].x || vertex[1].x != vertex[2].x
|| vertex[0].y != vertex[1].y || vertex[1].y != vertex[2].y
|| vertex[0].z != vertex[1].z || vertex[1].z != vertex[2].z);

Ok so I made up the lily pad bit, but the degenerate triangle is one I’ve actually run across before.

Still don’t see the problem? The designers get warned every time they create a frog and forget to give him a lily pad. Sounds great right? In practice, you don’t have a serious problem yet. Like hair in your shower drain, it takes a while for the clog to really build up.

The way it happens is so simple, it’s almost hard to believe it could become a problem. One day an artist is working happily along and he gets an assert because some frog doesn’t have a lily pad. “Not my problem,” he says to himself. He quietly hits the “ignore it” button and moves on. And then he ignores the next one, and the next until before long, there are so many of them that he no longer even pays any attention to them at all. Now he will silently click right past the assert that was intended for him to see and take no action to fix it.

There’s a certain critical mass of error messages that gets reached at some point, and once you hit it, all error messages become just another line of spam to be ignored as they scroll past glazed-over eyes. This critical mass is different for everybody, but the key factor to remember is that it is invariably lower for the end-user than it is for the programmer. For many non-technical people, I would guess this critical mass may be as low as 2 errors per run. In fact, if a non-technical person becomes accustomed to seeing 2 errors every run, I would hazard a guess that it would become very unlikely that they would even notice a 3rd error should it suddenly appear. Once you’ve added that “ignore it” button, that’s exactly what will happen – people will start to ignore your errors.

If you’re one of those people who still doesn’t see the problem, you’re probably thinking something like, “It’s not my fault if people ignore errors that are right in front of them. They need to pay attention to those.”

If you’re still thinking that way, then consider this: Just how bad could it get?

How Bad Could It Get?

Imagine the scenerio I described is allowed to go unchecked for a little while. Imagine a team of 100 (30 of whom are programmers) working together for 2 years under the philosophy that every conceivable error will have an assert on it. Now imagine you are dropped into the midst of that team. Perhaps you are a new hire, or maybe you are a temporary contractor brought on for some last-minute optimization. What might this project be like?

I’ll give you a real-world example. Without naming names, I have seen an actual video game project where booting into the first level of the game from a cold start results in 17 unique asserts with another 11 over the course of playing that first level. I tried to count the total number of occurances, but after getting 20 instances of the first assert, I gave up and punched the “ignore this assert” button. Hard to believe? This is a true story, and I’m not talking about early in development. This project was post-beta at the time I performed this test.

So when you find yourself dealing with this, what’s the first thing you do? That’s right – you figure out a way to disable asserts altogether. Only now you find yourself asking, “what do I do when I actually want an assert?” Well you’ll probably eventually break down and create a new assert macro. Maybe this time you’ll call it “crash” because the assert doesn’t crash anymore, and you want to be sure yours does. In fact this same project I’m talking about already has a crash() macro. I’ll give you one guess what that macro does. If you guessed “it crashes,” sorry, but thanks for playing. Once upon a time it crashed, but at some point, the brain-trust who decided that asserts should be skippable came along and “fixed” crash() too. On this project, crash() merely logs the event to a text file and happily moves on without a blip – completely unnoticed.

Improved Asserts

Don’t get me wrong, there are ways to legitimately improve the assert macro. Just remember not to alter the fundamentals we agreed on for the definition of “assert”.

Evaluate an expression
Halt the program if the expression is false
Provide the location of the assert

So what are some ways to improve the simple assert?

Warnings The first thing you should do is create that warning() macro so that people are not tempted to use asserts for non-fatal issues. I urge you for all of our sakes please do not call your warning macro assert_nonfatal. Don’t perpetuate the idea that asserts can in any way be non-fatal.
Messages The standard assert will echo back only the expression, the filename, and the line number. These are likely to mean absolutely nothing to the end user. Adding a meaningful string to the assert will help the team bring it to the correct programmer’s attention faster. I like to use a little macro assert_msg(expression, message) for that.
- Author These asserts must be addressed by programmers – you aren’t using asserts to remind designers to give their frogs lily pads right? So the only important part of the assert message is which programmer needs to know about it. I know of one company where they have created special assert macros with the initials of the programmer on it like assert_mrw(expression). When one of these goes off, it tells the victim which programmer to go talk to.
On-Screen Displays I mostly work on consoles, and some consoles don’t provide text output outside of the debugger. What that means is artists, designers, and even testers will get an assert and be unable to see the text. The assert is there to provide valuable information about the crash, so in these situations, it is worth the effort to print the relevant information on the display in addition to the text output before breaking.
Unconditional Asserts I see these all the time, and to be honest they bug me, but there’s technically nothing wrong with them so enjoy yourself. If you’d prefer to type assert_unconditional() instead of assert(0), well that’s your choice. You won’t catch me doing it.

Summary

Asserts are really very simple as long as we all agree on what they mean and use them correctly. If you maintain a strict difference between a warning and an assert, your project will run smoothly.

You may still be sitting there saying, “this will never happen to me because we won’t let it get that bad.” Maybe you’re right, but I think it comes down to a matter of philosophy. If you have a philosophy at your organization where it’s ok to misuse asserts, you should expect them to be misused.