A key element in the law of attraction is faith. If you repeatedly think about something that you would like to have (or have happen), and you have absolute faith in getting it, you will get it, the law says. This technique can be put to use most efficiently by repeatedly thinking about the desired thing as if it would already exist (thereby exercising absolute faith). For example (modern examples for this law seem to be notoriously monetary), instead of saying, “I have faith that I will have a million dollars”, one would rather say, “I have a million dollars”, ideally repeating this several times a day.

By stating the object of our desire as if it were already a reality, we create a strong tension between the current reality (not having whatever we desire) and the desired reality. Our persistent and absolute faith then compels us to take action that will resolve this tension, by moving us towards the reality that we desire. Depending on your believe system you might also say that this action is guided by god, the universe, your subconscious mind, the divine, your muse, inspiration, etc.

If you are a programmer, and you are still reading, congratulations! Because now, let’s think about Test Driven Development, where usually one goes about writing a test first, that is, before the code that would make it pass even exists. So by writing this test, we pretend that the code that will make it pass (or even compile) is already there. Our test talks about the desired object, the code, as if it were already written. In other words, we express absolute faith that the code will eventually be written. In fact, our failing test starts to attract the code that will make it pass.
Now, after we clearly stated what we want, we run our compiler and/or test framework and immediately get in touch with the current reality. The test will fail. Since we have a desire to make it pass, and absolute faith that we can make it pass, the tension between the current reality and our desired reality will compel us to write the code necessary to make the test pass. Every test becomes a mini-fulfillment of a clearly stated desire. That might just be the reason why writing failing tests and making them pass is so satisfying.

Rant Start

So we hear this year our AAA projects will cost 10 million dollars to make and next year they will cost 15 and in 5 years it will be a gazillion. Why? Well hmm, let’s see, the hardware is getting more complex, content creation becomes so much harder, photo realism, content pipelines, episodic content, oh my!

The real reason for exploding costs in game development is incompetent software developers. I am sorry to say. It’s not the big, all encompassing, unrelenting and unchangeable advancement of technology and refinement of consumer taste and expectations, neither is it those “risk averse” publishers who wont finance your creative idea. No, it is simply incompetence. Our skill in devising reliable and flexible software architectures is, to put it mildly, severely lacking. The advanced hardware and those pesky creative designers have created an environment that requires excellence, an environment we can’t possibly keep up with using our current skillz (sic).
Programmers are the least trusted individuals in the industry. Why? Because we are poor and misunderstood individuals? Because we work so hard and are tired all the time and therefore make mistakes every now and then? No. It is because we can’t code for shit. We have no clue about the simplest and oldest principles of software development. We consistently violate well established principles and practices of object oriented programming, while cranking out one class after the other in our “oh we started out with some classes but then it got out of hand uhuhuhu”-piece-of-shit-hacked-together system, which sometimes, but not always, through brute force, individual genius, or sheer luck finds it’s way onto the shelves. And then we go on complaining about those long hours “they” put us through, when in reality, it is our crap code that has us sit there, debugging deep into our utterly untested systems, looking at the same functionality in 10 places and treading to add another line to that one forbidden include file, that will send the time of our next build cycle through the roof. But hey, more time for coffee and bitching about management in the kitchen!

The designer walks up to us and asks us to do a simple thing that would enhance his game, and we tell him that it will take, oh, hmm, days or maybe a week, and we will not be sure how it will affect all the other systems and therefore we can’t say if the game will behave the same as before the change, look out for those side effects, testers! The designer looks at us funny, and we roll our eyes, hidden or otherwise, because how can she possibly understand the intricate goings on in our wonderful and complex software and oh how hard it is to do this cool (and sexy) programming thing but don’t worry, we got it! After all, we are your heroes. Right? Right?? And they keep throwing these change requests at us .. can’t they make up their mind! Do it right the first time, for fucks sake!
We add a feature to your game, but we just can’t tell you if we broke it or not! I mean What. The. Fuck! Dear designers, dear artists, dear producers, we have fooled you. We have stringed you along. We somehow convinced you that this is the way it goes, that software development just IS THAT WAY. No, sorry, we are full of shit. We lied to you. We lied to you because we wanted to hide our own lack of software development competency. And some of us are really good at hiding that behind shiny graphics and oh those mind blowing physics. And we usually do this by crunching our asses off and burning out in our twenties.

So, we need software architecture you say? You mean the thing where you write those small classes with only 3 members or something, uh, and you, like, test them, and, like, your classes have as little responsibility as possible, so you can test them in isolation, and then in the end you plug your well tested small classes together, yielding a well tested and reliable layer of functionality, ready to carry your next layer, which you can build with confidence and the courage to display excellence?
Are you, like, gay or something? Oh and .. you can’t use virtual functions because on the 360 this will cause cache misses and our performance uh uh uh uh. Well fuck cache misses. Tell your programmers to learn how to write tested and reusable software first, and then have them go and profile those 20% of the code that is exercised 80% of the execution time. They might find out how to get past the same 20% of problems they run into over and over again, using 80% of your budget.

So here we are, blaming Walmart and EA and those evil money wielding publishers in general, and oh the risk it is nowadays with all that money involved.
Or, rather, with all those incompetent developers involved. They build you software which will be hard to maintain and hard to change. They build you software of which they will not be able to tell you if it works the same tomorrow or 10 minutes later. They build you software that will drag them back to their desks for 80 hours a week, spending most of their time in their favorite tool, the debugger. They build you software full of non-commitments and guilty conscience. Producers, have a look at those C++ files in the elusive “Code” folder of your multimillion project, and look at the lines beginning with // or /*. You might get a feeling for how your developers feel about their work. Search for “fuck” and “shit” and “crap” or “god”, “OMG” and “sucks”. You will see comments throughout the code saying “oh .. this sucks but I put it here” or “shit! This is sooo bad but for now it is good enough. NEED TO CHANGE LATER”. Or, my favorite one, which I read personally in the source code of a Triple-A title: “I have no shame left”.
That is right. We, your developers, have no shame left. We walk around in your office, our penises tangling, scratching our balls, and providing you with a heap of shit that you are to use for your creative ideas and dreams, and that you are to market to a hopeful crowd of gamers, which is growing more and more frustrated. Please show us the way to the closet where we left our pants and send us back to programming school.

Rant End

The spike

In "Extreme Programming" the term "Spike Solution" basically means venturing out into the untested waters and "hacking" together a piece of code which roughly does what we want, in order to learn about the problem as well as the solution. So I did exactly that. I found a convenient place to implement my picking code in the update loop of the game. The code would shoot a ray from the mouse curser position into the world, and intersect it with the geometry, providing me with the intersection point (if any). As a visual indication, I would render a sphere of some size at the intersection point. If the mouse cursor appears to be always in the center of the sphere, I assume my code is correct.

After a few iterations over the code, and some grappling with high school trigonometry, I arrive at a satisfactory solution. Visually, everything looks correct, the data in the (occasionally invoked) debugger also looks good. Here is an actual screenshot, with a wireframe sphere indicating the point of intersection on the ceiling, close to the viewer:

Now that I have some code which does roughly what I want, it is time to put it under test. After all, I will get tired of staring at my sphere to test my code for correctness whenever I change it.

The failing test

To put my code under test, I first write a failing test. The code I have written so far essentially calculates the end-point for a ray which will be tested for intersections with the game geometry, the start-point being the location of the viewer. In the game, I am interested in that intersection point, to find out what is in the game world at this point, in order to operate on it in a certain manner. In the test, I am really only interested in the end-point of the ray - if this point is correct, I assume the rest of the operation will be fine. My goal here is to use data that a computer can verify (in my case, a vector consisting of three floating point numbers), as opposed to the visual cues I have gotten so far, which need the human eye to judge (approximate) correctness.

So now, having rid myself of any dependency on the game data, which is expensive and time consuming to load, my test simply has some inputs, and an expected output, and will run completely independent of the game.

My inputs are (for completeness sake):

• The screen coordinates of the mouse cursor
• The horizontal and the vertical field of view
• The view matrix
• The eye point (position of the viewer)

My output is another 3D point, the end-point of the ray which is shot in the view-direction, distorted accordingly using the field of view variables. So, I put the code I have "hacked" into the game into a class, and put this class under test. This test lives in a seperate project from the game project, and is run independently (and cheaply). The first version of my test looks like this:

As you can see, I set up some fantasy data, put it through a function in my new "Pickray" class, which is a copy of my spike code, and check if the result is equivalent to another fantasy 3D point (taking a small epsilon into account). Of course the test fails. This, in the words of Kent Beck, is progress. Because now I know exactly what to do next: I need to make this test pass. How do I make it pass? By feeding it the correct pairs of input data and expected output data. Where do I get this data? Why, by exercising the code that I have put into the game, the code which was visually verified to be correct.

Sampling data

So, back in the game code, I add some lines that print the inputs and the expected outputs I am interested in. Since I am really excited at this point, I don't bother much with formatting. The obvious idea of formatting the output so it becomes correct C++ code ready to be pasted directly into my test eludes me at this point. That's ok though, I remembered it early enough later on. At any rate, an early version of the output code looks like this:

This code dumps the current data whenever I move the mouse (and whenever a certain console variable is set to 1). The high precision in the output is necessary to avoid imprecise results in the test - so the more digits after the comma I can grab, the better. I fire up the game, move the cursor around a bit, and I get a lot of "real world" data to choose from. A few copy and paste operations later, and some reformatting, and the fantasy data in my test is now replaced with one set of this real world data. One version of my first passing test looks like this:

This looks pretty good already: I can get the game to output data which I can use in my test, testing the exact code that the game gives me output samples from. By now I am confident enough to replace the "loose" code in the game with the class I have since put under test. I have moved my tested code already to "production". Always a great feeling.

At this point I am ready to refactor my test for using multiple data samples. I add a little infrastructure in the test, enabling me to add an arbitrary amount of samples for testing. My test now looks something like this:

The class PickrayTester takes care of collecting samples and running the tests. Meanwhile, the code in the game that outputs my samples has been changed to produce correct C++ code, nameley a block of code starting with tester.addSample... which I can directly paste into my test, for every data sample I want, and for as much samples I want.

The final thing I want to do is to remove some of the remaining manual tedium. Also, my test starts looking a bit fat, depending on how many data samples I use. So, approaching the inevitable happy end, I change my sample output code in the game to directly write into a cpp file, which is in the same directory as my test. I call it PickraySamples.cpp. This file now solely contains a list of tester.addSample.. blocks. In my test, I replace the inlined sample data with an #include statement that includes the generated file:

At this point I declare my mission to be accomplished - I can now change my picking code with the safety net of an automated test. Obviously, there is more to picking up 3D objects than just shooting a ray into the world. But the automatically asserted correctness of this integral part is already a nice and solid foundation to build on.

I have previously used this framework to run unit tests via a simple text interface. Tests are easily defined using a simple set of macros and automatically enumerated. Now, whenever my modified version of Doom 3 starts up, it will do the same, it will enumerate tests wherever they have been implemented in the code, and I can operate those tests via the in-game console. Have a look:

Now, why did I do this? I will of course also have a seperate program with a lot of unit tests, as I develop my software test by test. However, I will want to write tests that will need the game running in some stage, and part of the beauty of all id Software engines is that they make it extremely easy to hook in any kind of custom code that is readily executable in the game at any time, via custom console commands, thus opening up an extensible command line interface to the running game. And so I not only have the environment in which to run my tests, it also hosts them and provides an interface to them. That, I think, is quite beautiful.

I am not working for Id Software, in case you were wondering. I am using the publicly available tools and source code for modifying the original game.

The "Enum Example"

Let's say you have an enum, and you would like to associate a string with each element, to provide a readable name. A common way to do this is as follows:

The output of this program is:

ID0 -> ID 0
ID1 -> ID 1
ID2 -> ID 2

Now, whenever you add an element to IDs you will have to remember to add an element to idStrings. The compiler wont tell you about your omission, and things could get nasty only very late at runtime. Even more so, if you have code that operates on all members of the enum, you have to add to this code as well, as you would have to in the above example, where we output all the string names on standard output.

X-Macros relieve you of all that. Let's rewrite our code like this:

Let's examine what is happening here. At (1) we define a macro which expands to a set of "X-Macros". In our case, the X-Macro has 2 Parameters. At this point in the code, they are not expanded. Note that this uses the fact that the preprocessor will not expand macros embedded in other macros, before the "parent" macro is expanded. Therefore it does not matter that, at this point, X is not defined to anything. At (2) the whole magic happens. The X-Macro is defined. It will expand to its first parameter (ID0, ID1, ID2) and a colon.
Now, inside the enum definition, we expand GENERATE_IDS. This will now also expand the current definition of our X-Macro, and, voila, an enum is generated. At (3) we do the same thing, only this time, we expand the X-Macro to something else, namely, it's second parameter and a colon, resulting in a list of strings generated by GENERATE_IDS, fitting nicely into our array definition. Note that the definition of X had to be undefined before we could define it again.
Now, whenever we want to add an enum, all we have to do is add a line to the definition of GENERATE_IDS, with the enum value, and the enum string name. There is no need to edit multiple places in the code any longer. The output of the new program will be exactly the same as the first one.

Beyond Enums

While the above example demonstrates a useful technique to avoid code duplication in a particular implementation, X-Macros can be put to use in many different scenarios. I would like to give one more example in this post. Let's say you want to have a class which has many different data members, and the frequency of adding, removing, or changing those members is currently very high. Here is how I would implement such a class using X-Macros:

In this more elaborate example, X-Macros are used to generate a considerable amount of code. All members of a class Properties are defined, as well as getters and setters for them. The code that initializes all members to their type-default is also generated within the constructor. Since I got tired of manually checking the correctness of the output, I also introduced a minimal testing "framework" in this example - assertEquals will report an error if it's two arguments are not equal. Before the program terminates, it will report any of these discrepancies.
Now adding a new member to Properties in the above code is simply a matter of adding a line to the generation macro at (1), thus completely eliminating the need to hunt around in the code for places that need to be adapted or expanded (or even reduced!).

By the way, X-Macros can of course also be used in tests. Testing "some" of the members (see (5)) is not satisfactory, neither is adding a new line in the test whenever a new member is defined (precisely what we want to avoid for the production code!). One way of automatically testing the correctness of the generated code is adding a test value as the third parameter to the X-Macro. The generation macro would then look like so:

Now of course all the places where X is expanded have to be adapted, to include the new parameter. In the code we have seen so far, this parameter is ignored. However, in the test, we can do something like this:

Now, (5) expands to code that first tests the default value of a member, then sets it's value to the test value given in the third parameter (tv) of our X-Macro, and then tests that the getter returns the same value. No new code has to be added here when a new member is defined.

A few words of warning

As always, there is a flipside of the coin. X-Macros degrade readability of cour code. When introducing an X-Macro, it is well worth spending some time figuring out how to keep the code that is eventually in an X-Macro as short as possible. Sometimes one jumps the gun and wraps too much code into it, while some static elements could still be extracted. Long macros make it hard to debug the code within them. On the other hand, do you see a reason to ever need to debug the Property class above, assuming that it's feature set will not change?

I have put X-Macros to great use in great code, and I encourage you to do the same, and to discover many more uses for this surprisingly simple yet effective technique.

You can download the code used in the examples here:
XMacros_Enum.cpp
XMacros_Properties.cpp

This insecurity, however, is natural. You are insecure because you don't know and you think you are supposed to know. But still, my friend, you DO NOT KNOW. And that is just how it is.

Insecurity also does not matter, in that it does not hinder or otherwise impede creativity and productivity. What does hinder and severely impede creativity is your superflous thoughts and interpretations of a small, natural bodily sensation.

So here is a suggestion, the Cyclic 4-Step Model of Creating:

(1) Explore the unknown, feel and accept your insecurity as part of the process

(2) Find a safe harbour

(3) Make sure you can come back to that safe harbour at any point in time easily

(4) Go to (1)

Can you see how tests can help you there? Can you see how agile development facilitates exactly this? Do you think I have forgotten the word "Goal" or do you think it is irrelevant for this process?

What are short assignments? Ms Lamotte skillfully describes what happens to her when she sits down to write something. We hear about mental illnesses pulling up chairs around her, trying to be quiet but their "weird coppery breath" will remind her of their presence, always. We hear about the thoughts that begin to flood her mind, that tell about how her well of creativity has run dry, how she comes to the conclusion that she can not do anything else and thus will starve, and how her breath gets shallow. Finally, she finds the one-inch picture frame on her desk, that she put there to remind her of the idea of short assignments. All she has to do for now is write down as much as she can see through a one-inch picture frame.

When I sit down to program, sometimes my breath starts getting shallow. I, too, have my selected audience of mental illnesses, only they are not even trying to be quiet, they talk to me about how I should not be here in this high profile company in this high profile business, and that I should go back where I came from. They also eloquently read a list of programmers that have achieved so much more, while still being younger than me. Sometimes they throw in pictures of said programmers with nice cars. During that period of mental terror, I robotically switch through source files, or my email, or some webpages, until, after about 2 minutes or so, I notice this little doodle I made yesterday before I left work, an UML-ish diagram, sketching out a set of classes and their relationships, for the next tier of a system I am developing. On there I spot a box, about one inch wide, with a class name written inside of it, along with a few public functions. I notice the class does not depend on anything. And I suddenly realize, again, that all I have to do for now is write only as many tests that will establish the functionality contained in this one-inch box.

A few months ago I read a few pretty much life-changing words in David Allen's "Ready for Anything". They go something like this: You can not do a project. You can only do the next small physical action in a long list of actions that will eventually produce the result. I suggest you read that again. This is huge. Is this not liberating? You can not do it. You can not write a complex software system. You do not have to resolve the huge tension that seems to exist between nothing and the end result of a big project. You can only resolve the tension between now and a few minutes later, while you look through your little picture frame, and write your test. For me, this is the mode of work that makes me most productive, focused and passionate. I make it my primary activity to ask myself the following question, over and over again: "What is the smallest thing I can do now to advance the given assignment a little further?". Or, more to the point: "What is the smallest thing I can test that will tell me a little bit more about the system I am developing".

When I have neglected to pose and answer this question for a certain amount of time, I am becoming less efficient, I am stressed, and irritated. There are a few things I watch out for, to notice this as soon as possible (because after all, it does happen):

I have changed too many files since the last checkin: More than, say 5 files checked out already make me a little nervous. It means I am working on something that is probably too big for my current short assignment.

I start to comment out tests: It is ok to comment out a test to work on a single other one that has to be fixed first. It means I went too fast, but there is a threshold. If a test stays commented out for a long time, I probably became obsessive with some other aspect of the system, and I am most likely pressing my frame against my eye very hard to spot the "dead angles" that unnecessarily bother me.

I have "ok to fail tests": I run my tests, and there is a list of tests that are "ok to fail" at the moment, and they "will be fixed as soon as...." This is not only tedious, as I have to check the failing tests and check if those are the "ok to fail" ones or "not ok to fail" ones. Again this indicates a burst of compulsive obsession with something I have spotted at the corner of my current 2-inch vision.

Luckily, my body faithfully reminds me of what I am doing. My neck gets tense, my stress level rises, my breath becomes shallow and I become a miserable person. Until an angel dutyfully and gently taps me on the shoulder and hands me my one-inch picture frame.

The beautiful thing when writing tests is that you usually don't have to look back. When I look through my one-inch picture frame, it helps to not have to look back. How else would I be able to focus? It helps to feel safe and not threatened. It helps to stand on a solid foundation. In my daily work, testing and test driven develpopment provides me with exactly that. If I remembered to look through my one-inch picture frame often enough, I have built up a system test by test, layer by layer, every layer as solid as the next. It is like building my own staircase, and I can safely and faithfully step on to the very next step that I have just built, and I know it will not break down. This bundles all my abilities and experience to a narrowly defined problem, and I can rise to excellence in ways I never thought possible, because I am simply not bothered with anything else. When you were last in the flow, how many thoughts were in your mind?

There is, of course, a caveat to all of this. In order to work in little tiny steps, you must disconnect emotionally from the outcome of your big project. You must also make peace with the fact that you are not doing anything special. So many people want to build that cool engine, or that cool AI or that cool system that does whatever, and they want do feel like they are doing exactly that. With attempting to do that, there comes recognition from your peers, overtime, stress, tension, broken relationships and probably an unhealthy diet. But it is special. All those things are making it special! Your suffering is your special invitation to the party. On the other hand, writing a little test that might test one or two functions of a new, say, string class does not sound all that glamorous. And it is not. It is nothing special. If you connect emotionally with your activity and with the outcome of your work, that is, if it means life and death for you if you write the 3D engine of your game over a year or two, or just a small test over 30 minutes, you will have trouble with what I have written here. The fact of the matter is, we all do nothing special. And it certainly does not make us special. All we do is little things, and we string them together, and sometimes something interesting appears. Everything else is a delusion.

For those of you who are now angry, or just delusioned, maybe this quote from "The 15 Seconds Principle" will comfort you:

You must be more interested in producing pleasurable and specific actions in present time and less concerned with past failures or future results. Your mission is just to live and create with relaxation, energy, focus and faith. In this masterful state, there is no past or future. There is just creating in the now.

I think the problem there was that my body and mind got into "day off mode". When I have a day off, I lazyly go about making breakfast, check news, read my blogs, all those "lazy day off things". Which is perfectly ok, when I DO have the day off, and it is fine to go into that mode. At work, this is a problem, and my emotional state quickly makes friends with all the guilt feelings of not getting anything done, or dramatically less. In this mode I am also much more prone to procrastination and goofing off.

So fairly recently I am doing this: I come into work, grab a water, and do 30 minutes of work as soon as my computer is started up. I look at my lists of tests to write, or I have left a failing test from yesterday, or even a not-compiling one, and I start to work. Two factors are important: Quickly find a point to pick up from yesterday, and timebox it to 30 minutes, so your mind can not really convince you that you will be working forever without even grabbing a coffee.

After this little routine I grab some breakfast, some coffee, and chat with people, but usually the coding part of my mind has started up sufficiently, and I can't wait to get back to the problem at hand. Remarkably, it is equally hard to snap out of THIS mode the whole day.