Agile project management versus agile development
I have a saying I like to use when discussing Scrum: Scrum is an Agile project management methodology, not an Agile development methodology. Scrum delivers tools for managing the project (planning, scheduling), but very little for how to develop (design, program, test) it. To do Agile properly, you really need both. This is why eXtreme Programming (XP) and Scrum “fit together” so nicely, with XP telling you how to build your product and Scrum usually taking a higher-level view (and the overlaps are usually the same practices).
The complimentary nature of project management and development methodologies is important to understand and embrace. It is also one reason I don’t believe you can implement Scrum effectively without a programming background. Ultimately this comes down to a very simple thing for me:
If you do not have comprehensive automated tests, you cannot be Agile.(1) I consider this a fundamental truth. Yet rarely do books about Scrum spend more than a few pages talking about how vital this is, rarely is it discussed enough in Scrum Master training courses, rarely do the project managers now running most Agile implementations seem to understand this.
Automated tests are the fundamental building blocks from which all other Agile practices flow. This also informs how I treat Agile as a whole: Agile project management should be free-flowing and not rigid, but Agile development should be rigorously adhered to. That is sure to be inflammatory so let me elaborate.
Rigorous Agile development
- If automated tests are fundamental, and Test Driven Development (TDD) is the only way to get comprehensive test coverage, that means TDD is not optional. You don’t need to use it everywhere, but TDD should be the rule, not the exception.
- In the same way, you cannot be Agile if one person ends up being a bottleneck, so collective code ownership is required. I think it’s less important whether this is done through pairing or code reviews (probably both), and more that collective code ownership is a rule and any exceptions are seen as a big problem. Automated tests are required to collectively own code, as it’s the only way people can reliably make changes.
- Continuous integration is equally important, and depends on automated tests to be reliable and problems to be quickly fixed (it relies on the other two practices). You must always have a high quality build available and the code people are writing should be quickly (once a day or more) and easily getting into that build.
These three practices I consider absolute.(2). Maybe you can add some, but you are not allowed to decide you want to exclude any of those three from your Agile implementation.(3) To do so invalidates the principles on which Agile rests. So it follows that you should not be allowed to have zero experience in them if you’re an Agile leader. You cannot be Agile without them, no matter how briefly Scrum literature covers these topics; I would bet most of the writers of that literature would agree.
Flexible Agile project management
Opposite the rigorous adherence to specific development practices is experimentation with general project management practices. In this area, things are much more about principles (primarily feedback and continuous improvement) and less about practices or processes. Your sprint should be as long as you need to get good feedback, which varies depending on project/team/technology. Your retrospectives should be run so you can continuously improve. Your planning should be so you can get more accurate and insightful. Ten solutions can work well at ten different workplaces. Even more, ten solutions can work well at the same workplace(4). Just make sure you are continually improving, and keep trying new things!
Having your cake and eating it too
This distinction between development and project management is how I navigate the rift between Agile nihilists and Agile purists. The former say with disdain, “Whatever works for you!” The latter chant fervently, “You cannot pick and choose!” It turns out they can both be right, but it’s important to understand how and why. The nihilists end up floating, never realizing the transformative power of Agile because they refuse to adhere to the three vital, but initially taxing, processes. The purists can drive change, but not transform, because they do not create new practices that fully embrace each unique situation.(5) Rather than trying for a happy medium between the poles, I find Agile is best done by being at the extremes simultaneously.
- I don’t know how to rebut arguments against this point, other than asking “have you worked on a project that was well-developed with TDD?” If not, I would try it out before you make excuses for a compromised form of Agile without comprehensive automated tests.
- More accurately, what they achieve I consider absolute. If you wanted to get rid of any of them, you’d need to replace them with something suitable. For example, if you didn’t want to use TDD, you’d need to demonstrate some other way to reliably build comprehensive automated tests. And actually I have great hopes for some automated alternative to TDD one day…
- Uncle Bob recently posted about how software projects need this sort of discipline, perhaps by having a “foreman”. I don’t agree on the solution, but I do agree that we do need to rigorously adhere to certain practices. http://blog.8thlight.com/uncle-bob/2014/02/21/WhereIsTheForeman.html
- This is probably attributable to the Hawthorn Effect: http://en.wikipedia.org/wiki/Hawthorne_effect
- To be fair, many purists (especially outside of programming) overlook the three vital development practices because they are so keen on implementing the easier Scrum project management tools that require less training and invasive changes.
Acknowledging that I’ve not worked on a project where TDD was rigorously applied I have to disagree slightly with “if you do not have comprehensive automated tests, you cannot be Agile”. It’s a bit prescriptive for reasonably woolly principles.
Isn’t it more “you cannot work as efficiently without comprehensive automated tests and the preference in Agile is for shorter iterations of working software so efficiency matters”.
I personally also think that development practices should also be subject to the same flexibility as project management practices. The danger for both development and project management is the misuse of flexibility and it does seem to me that your wish for rigidity is a response to flexibility being misused rather than a requirement for rigidity. In big teams with multiple sub-teams this is an issue of governance from the managers of production and technical disciplines.
Then it comes down to whether or not there is a good reason for not following certain development practices. In the case of the three you highlight the occasions where you wouldn’t want them as practices on any reasonable sized project are pretty rare.
I’m pretty sure there are some project management practices that probably shouldn’t be thrown out for the vast majority of projects as well. For example velocity tracking, developing in timeboxed iterations and maintaining a backlog.
I do think that teams should rigorously adhere to current practices in both project management or development otherwise you are going to make measuring their effectiveness pretty hard. Which in turn will make iterative improvements harder.
“Then it comes down to whether or not there is a good reason for not following certain development practices.” Are there any cases where you wouldn’t want to follow them? If not, why leave the wiggle room?
“For example velocity tracking, developing in timeboxed iterations and maintaining a backlog.” Well, I don’t disagree, but I think those are at a bit higher level and less concrete. The 3 practices I list are pretty easy to check up on: is your test suite comprehensive, can anyone work on any area, are people checking in once a day without breaking things, is new code frequently shown to customers (for some definition of customer)? Things like velocity are a lot harder to measure and use; in fact velocity measurement is probably used incorrectly more often than not. I could see ten alternatives to accomplish the goal of velocity measurement (which is what, BTW?), that I cannot when it comes to building automated tests.
I stand by my statement “if you do not have comprehensive automated tests, you cannot be Agile”. It isn’t just a matter of efficiency. It is a matter of culture, work style, and principles as well. I’m not saying without automated tests you cannot good code, that you can’t release rapidly, or any number of individual things. I’m saying that without comprehensive automated tests you cannot achieve the comprehensive benefits of being Agile.
TDD is a very important prerequisite for being able to deliver in a way your Agile management expect you to – “working software in short cycles”. If you can’t automatically test it and deploy it effortlessly, you waste a huge amount of time of your development cycle for maintenance. TDD is just a good practice to make sure that you test what you code. Nothing maddens me more in software projects like a practice to write tests AFTER production code is produced and manually tested.
Regarding foreman – well, there was a good idea somewhere in another discussion – a good development team needs at least one person who is passionate about the product. A good public example is Unity – once Aras Pranckevicius asked “is it normal that I review every single commit log message” – and believe me, they have quite a few of those per day:) It does not have to be a foreman – it has to be someone who brings things to attention on a constant basis and does not let your code base to rot.
I think most of my examples boil down to small, throwaway projects.
On the last paragraph I agree on almost all points but the principles are written in a graduated fashion. As such I think a project could be said to ‘be Agile’ if it is following the Agile principles and using them to drive culture and work style. I also think a project that could be described as ‘being Agile’ without comprehensive automated testing would in all likelihood be working towards it as the gold standard. Though in principle its use should be contingent and corrigible.
Two of those 3 points you make about Rigorous Agile development map to Alister’s Cockburn’s properties of Crystal Clear. The 2nd point maps somewhat.
See: http://alistair.cockburn.us/7+Properties+of+Highly+Successful+Projects+from+Crystal+Clear
The key thing is that it is more important to have these kind of invariant properties of a team & environment than it is to have a solid process to make success more likely in a software development project!