Spreading TDD throughout your company

First, the pitch- CCP is looking to hire someone to come to Iceland and teach TDD with Python for a couple days in August. We’re looking for someone that’s taught TDD before. We already have people who have been doing TDD for a while now and to support it in the long run, but we want to get more people on board and grow some recent evangelists. If you’re interested, contact me rob.galanakis@gmail.com or rgalanakis@ccpgames.

Now to make this post not void of content, I just want to give a little bit of history, and insight I’ve gained while pushing TDD at CCP for the last two years.

I joined CCP two years ago and since before day 1 I’ve been doing TDD. Since then, I can’t say it’s caught on like wildfire (we have a legacy Python codebase of around 2 million lines), but we’re at a point now where it’s taken hold in enough areas, and proved itself both doable and useful, that we’re ready to start ‘pushing’ it on engineers, rather than just waiting for individuals to ‘pull’ it.

I started by focusing on a non-critical area of the code (internal tools) and wholesale replacing functionality rather than trying to retrofit legacy code because this was non-critical code I could generally afford to not replace all of its functionality, or have some things “broken”. A few months later, a different team working on a new area of the code embraced unit testing and TDD, so we had it working from two angles.

Eventually we needed to test more than pure Python. We needed to test code that’d only run in our game environment, or Autodesk Maya. So I developed some systems to run code in a remote environment from a host Python process, so we could still run our tests from PyCharm or whatever pure Python process, but under the hood it’d bootstrap one of these other processes, tell it to run the test code, and report the result to the original Python process.

This was a big ‘aha!’ moment that made TDD go from impractical to imminently possible. We conquered the biggest hurdle to automated tested we had. It was now possible to find and grow TDD evangelists working in game code, and start learning there. It is a much more difficult problem to test! But something more interesting happened. This helped crystallize a movement where we wouldn’t need this craziness. Unit tests started to become too important, so we did things like:

• Build versions of our binaries for various vanilla Python flavors, not just our proprietary executable environment.
• Created a new ‘home’ under which we put cohesive Python packages and expect good code (it’s hard to develop well in the face of 2 million lines of legacy).
• Extracted code out of our ‘monolith’ and these packages (or totally separate environments). Our ‘monolith’ actually used a proprietary importing mechanism which made it impossible to use outside of our game environment, and depending on all sorts of builtins.
• Adopt better tools. Since we started writing vanilla code, and using standard testing tools, IDEs like PyCharm had real benefit over Sublime or TextPad.
• We removed our proprietary importing mechanism and are removing as many things as possible that diverge our codebase from vanilla Python.

Certainly I’m not saying I or TDD were the only drivers of these improvements, and to a large degree our testing push was in the right place at the right time. And certainly it’s not been a short time since we started. But now we’re ready to push out TDD more aggressively, with official approval. Which is why we’re looking for a trainer to come in and help educate about 15 Python devs for a day or two.

The insight I’ve learned is that the common-sense tips people give about creating change are generally correct. Start with small wins, find early adopters and make sure they are happy, keep pushing steadily, overcome harder problems, change peoples’ minds with results, etc.