Testing Practice: Use your eyes, when necessary
Imagine the situation: You have a third party system, and you would like to query it for some information. You are not sure yet how this information will look like, and it will also change over the course of exploring and discovering said system in your test. Let’s also say, the output is a somewhat lengthy string, with lots of details that you are not even interested in. You just want to confirm that you got the right answer. It would be tedious to immediately use an automatic check for that, because everytime you change the way you interface with the system, it might give you a different string, and you have to adapt your test. So here is what I normally do: I use my eyes on the early results of my queries to the system, i.e. I print out the results on the console or whatever output device. I do that until I can visually confirm to the best of my knowledge, that the result is what I want, or that the result indicates correct operation of the system. Only then I take the result, and write some automatic checks for it. Maybe a simple string compare, or maybe I look for certain substrings. Don’t delete the code that displays the results, though. You might need it again, when things change, and you need your visual checks again. It helps to have a flag to turn verbosity on and off. It does not need to be fancy at all. Just comment out the line that turns verbosity on when you are done.
August 14th, 2006 at 10:21
[...] I recently implemented “picking” in the publicly available parts of the Doom 3 engine. Picking in 3D games (or applications) means that the mouse cursor position is mapped to a point in 3D space, that intersects with an imaginary ray sent from the current mouse position into the 3D world. It can be used, for example, to pick up objects in the world and drag them around. In this post I would like to describe how I used testing in implementing this technique, by first implementing a “spike” solution with visual testing, and then using data from this solution to implement a piece of code that is fully tested without manual intervention. This is an exercise in arriving from an unreliable testing method, that is, looking at the screen, to a reliable one, using data that a machine can process automatically. I briefly touched on the same topic here. [...]