You know what really gets my goat? People who say things like this;
- “We don’t write that stuff down, we’re Agile“
- “We don’t do design, we’re Agile“
- “We don’t have to do <insert-name-of-thing-I-don’t-like-doing-here>, we’re Agile
The usual time these kinds of comments are made is generally around documentation. There are some dinosaurs senior managers who still insist that everything gets documented up front, with lots of heavy-duty analysis and design work written in enormous tombs of easily ignorable templates. Then there’s the everyone else who have convinced themselves that “being agile” (if you can stomach that phrase) means “not writing stuff down and hoping no one asks you any questions about it later”. So here’s my take on the kind of documentation that is useful to have in any development environment that is trying to follow the Agile Manifesto.
Let’s start by thinking about what kinds of stuff we should be writing down.
- High level project architecture
- High level architecture of track (sub-project)
- High level architecture of story
- Impact the story has on the rest of the track
- Story design and implementation notes
Off the top of my head, I can’t think of much else that doesn’t (more or less) fit into one of the above buckets(1). But it’s the composition of established document templates, when they’re written and by whom that I want to define.
So what I’m going to do is describe how I’ve experienced documentation in the Waterfall world. Point out the holes and then reveal a completely new (not new) and revolutionary (not revolutionary) way of writing good documentation that is meaningful and up-to-date (really.)
The Waterfall Documentation
The project kicks off with pages of requirements. A handful of people read and understand all the requirements. They define some high level use cases that are mapped back to every requirement so nothing gets missed. A thorough design and architecture document is written. System architectures get produced, design decisions are made, implementation decisions are made. Fully detailed class diagrams are drawn. This work is then handed to the developers who immediately understand all the original thinking behind the project. They diligently follow the class diagrams and produce working code that mimics them exactly. The whole system slots seamlessly together. We all go home happy.
Ha ha, just kidding.
What really happens is someone tries to read the requirements. But they can’t read them all because they’re boring and at vastly different levels of detail. No one really, fully, knows what’s going on. (No one thinks to ask the customer). But we have to write an architecture document so we take our best stab at the kind of things we think we’re going to have to do to get this work done.
We know we should write some class diagrams at some point, so someone has a go based on what they think the implementation for the whole system should look like. The work gets handed to a developer, who takes a gander at the class diagram and associated requirements. They have a slightly different understanding of what the system is trying to do, based on the badly defined requirements, and so do their best. But they find that what they need to write doesn’t match the class diagram very well – or worse, they have to write badly designed code in order to mash it into matching the class diagrams. But that’s by-the-by, because the class diagrams were incomplete anyway. But by this time, everyone is so busy at work they’re not going to update a class diagram that someone else wrote just to keep it up to date, especially when they know what no-one is ever going to read it. So now the class diagrams are both wrong and out of date.
Now time is running short. We need to get something together. Hacks are made, fudges happen. But at last, the Sunday before the project is supposed to be delivered, a successful build happens. It gets put into the test environment. Everyone holds their breath… and… it works!
There is much rejoicing.
Then the application bombs out, no-one is really sure why. More hacks and fudges are made. The code gets further and further from the original documentation, but it’s back on the server and (mostly) working. Okay, it’s a little slow but it works. The Business Consultants looks at the result…
“Oh. I didn’t think it’d look like that.”
Well that’s more of how Waterfall projects as a whole go, but the story is useful because we can use it to pick out the bits that involve documentation and talk about those. (The process side of things is a whole other blog…)
Days and weeks (or possibly even months) at the beginning of a project were wasted reading the requirements and trying to design the whole thing from scratch. As you can see from the example, by the end of the project those documents were far different from the reality of what the code looked like. That’s what makes it a pointless exercise.
Firstly, don’t get me wrong. There is an absolute need to have one, very high level Global Project architecture document. There may even be arguments to have one for every distinct track. But it should contain principles, not details. It contains guidance on service granularity, not details of the actual services required. It contains details on “who” and “where” but not “how” and “what”.
Secondly, there was an initial effort to decide what code had to be written, before anyone really understood the business requirements (class diagrams). So what’s the result of that? Someone spent time writing documents that were (at best) ignored, or (at worst) dictated a code design that was sub-standard once the problem was better understood.
So, a summary of the bad points;
- Time is wasted writing lots of documentation
- Documentation can adversly affect the solution implementation
- Documentation is out of date – no one can read it to understand what’s going on
Whether people see it or not, this is how a lot of (or at least, my) departments approach “agile” documentation. For all kinds of noble reasons there is an emphasis on “write-up-front”. The alternative method requires disipline and governance, but it makes for a more productive team and much better quality documentation.
The Incremental Documentation
Everytime anyone needs to do something or makes a decision it should be written down so others can follow your thought process. “Doing Agile” (and I presonally hate that phrase) means doing what makes sense. It makes sense to allow others to follow in your footsteps, it makes sense to record decisions so that they can be discussed and validated (and audited). So the question everyone needs to ask themselves is not “if” or “when” but “where”. Let’s talk about these questions.
IF – should I write this down
Except in very few circumstances, the answer is invariable yes. You should write it down. It might seem like an obvious decision to you, but it might not be to someone who doesn’t have the same facts as you. It may not even make sense to you when you come back to that problem next week. So yes, write it down. And write down enough so that it makes sense. There is no point in writing something down just to get a tick in some “did you write it down” box. What you write down should be detailed, but not verbose.
WHEN – when should I write this down
No one can be expected to know all the details at the beginning of a project, so why do we pretend we do? Write it down as it comes to you. Obviously there should be some effort to record things and attempt to plan and cope with them before they happen. ‘Impacts to other systems’ is a good example. But there is no point in busting a gut trying to come up with all the impacts on day zero of the project, because with the best will in the world you’ll not get them all. Instead, a process of immediate assessment, feedback and communication should exist such that when someone does something they’re thinking about what the impact on others is. They can then assess it, document it and plan around or for it.
WHERE – the only difficult question (which is actually quite easy)
Which leaves “where”. Where should you write it down. This is a really easy question to answer if only people would think. Lets say I come across A Thing. I need to write about A Thing. These are the steps I personally follow to decide where to write about A Thing.
- Does A Thing affect only the method I’m writing? Yes, then it goes in some inline or method-level comment. No, go to next step
- Does A Thing affect only my class? Yes, then it goes in some class level comment. No, then go to next step
- Does A Thing affect only my story? Yes, then it goes in your story specification. No, then go to next step
- Does A Thing affect only my team? Yes, then it goes in some high-level document for the team. No, then go to next step
- Does a Thing affect only a few other teams? Yes, then talk to the other teams to decide what to do and document it. No, then go to next step
- Does A Thing affect everything? Yes, then it goes in the project level High-level Arch doc
Again, do what makes sense. Whether “A Thing” is a new story that you have been assigned or a bug you’ve found or a massive show-stopper that could wreck the whole planet and make us slaves to invading hordes of shape-sifting alien lizards(2) – there’s a place to write it down. But only write it down when it comes to you!
And finally…
This whole blog was inspired (or not) by a question I was asked about some up-front design document which basically boiled down to “should I stick this detail in here”. The above steps can be applied in reverse also. Think of the steps as filters and put the detail at the lowest possible level.
There’s also a lot of high-level angst about documentation. Specifically about getting audited and being able to argue for decisions made based on documentation produced. Which is something everyone should all be worrying about. As the Owner of a piece of work you need to show that you’ve given it proper thought and showed due dilligence when carrying out your tasks. That’s why you write stuff down. The iterative documentation method I describe still satisfies the “can we show we thought about it?” argument. But the differences are these;
And that, in my opinion, make it better documentation.
1 A Service Catalogue might be a good idea, but you’ll get that for free if you write appropriate JavaDocs!
2 I, for one, welcome our new shape-shifting alien lizard overlords.
