What’s so special?
For a software development approach to justify having its own name, it needs to differentiate itself somehow. For agile development this special property is the fast delivery of business value: the goal of an agile software development approach is to deliver a subset of software features into production business operation, followed by another subset, and so on, in shorter timeframes than other approaches could achieve. In a nutshell: closely-spaced incremental production releases.
The higher level business objective, of the sponsor of the development, is to either:
- Gain business capability urgently, either to avoid missing short-lived opportunities in the market or to solve problems that have severe business impact or very tight timeframes (maybe externally imposed), or
- Deal with the uncertainty of new ways of operating the business, via the tactic of learning what IT features and business procedures are really required, by operating a software release in a live business setting.
A project that doesn’t deliver increments of software into production, but is iterative and uses techniques such as unit testing or continuous integration, is not particularly novel and hardly warrants a special name. These techniques have been around for years and have succeeded in spite of the hype of methodologies. Without agile development’s key value proposition of fast business value, the techniques may be beneficial, or they may be more trouble than they are worth. It depends on the context. There’s certainly no justification for conflating a specific technique and the achievement of business agility. Quickly delivering working software that fits into the business depends on a whole lot more than one technique or even several techniques.
But let’s look into the implications of delivering business value quickly, for which good reasons have been given in the two cases above.
Hello real world
In the first of these cases an agile development project takes advantage of the things the business learned while it operated with the partial solution provided by the first few releases. In developing subsequent releases of the system, the development project is influenced by real world feedback such as re-prioritisation and refinement of user requirements. This influence occurs earlier than it would in a non-incremental approach, and this is beneficial because more of the project is conducted with the improved understanding of requirements.
If the development is fast enough to justify being called something special, rather than just the usual good practice iterative development, then the speed is likely to be at the expense of full functionality and some of the qualities the system needs. Subsequent iterations will rework some of the software already developed, to add functionality and bring the quality up from bare minimum to a level that will sustain business operation. This rework occurs in all projects to some extent, but is kept to a minimum. This is because rework can easily degrade the integrity of the code, and it slows the convergence of design fragments into a coherent whole that reduces technical risk and enables productive development. Embracing necessary change is something that professional developers take in their stride; causing unnecessary change by skimping on analysis or imposing inappropriate techniques is simply reckless.
In the second case the dependency on business feedback is much greater – the business vitally depends on the development team incorporating the feedback to identify the features that are really required and the priority order that they are required in. Incorporating this feedback may involve reworking a substantial amount of software that has already been developed. Agile approaches justify this rework on the basis that the uncertainty of how to operate the business can only be resolved by gaining an understanding through using the features of new software. Apparently even highly competent and efficient business analysis wouldn’t enable the business people to envisage a future mode of operating.
Repeat what?
In each of the cases above, the extent of iteration is different. Iterate means repeat, but rarely are people clear about why they are repeating and what they are repeating. In fact, you perform a repeat examination of different parts of the problem space and the solution space:
- In the first case you repeat the activities of analysing business needs, deciding on a suitable set of behaviour (user “requirements”) for the system, designing and developing software. You repeat these for a different subset of business needs and the result is mainly additional software.
- In the second case, the expectation is that much of the behaviour already developed for the system will have to be revisited because the purpose of the software release was to confirm new ways of working, and change whatever doesn’t work.
Rethinking the system’s behaviour can lead to a large amount of development rework. Some people advocate that rework is not such a great penalty because a smart development team has techniques and tools to specifically deal with rework to minimise its effects. More about that later!
As with any approach, the agile approach is only valid in situations where the benefit outweighs the liabilities, i.e. where the type and size of the risks of an agile approach are more appropriate than the type and size of the risks of a different approach. Agile approaches are weighted towards addressing the risk that an IT system will not meet the business needs that exist at the time the system goes into production. The premise is that other approaches that are not fast enough and sufficiently engaged with the business will not produce production software soon enough, and the software that is produced will ‘miss the boat’. The software is designed to meet the business needs that existed way back when the project was initiated, but have since changed.
It is quite common for the perceived business needs to change substantially over a period of time, because the needs are quite unclear at the beginning or they are likely to change significantly due to the length of time that the business is exposed to a volatile business environment and rapid evolution in the business’s people, processes, information domain or technology. Other approaches attempt to address this risk in other ways, not always successfully. All the approaches address a mixture of risks that exist to a greater or lesser extent in all business contexts. In a given context, some approaches are a better fit than others.
Because an agile approach brings its own risks of failing to produce an adequate solution, due to excessive rework for example, or applying unsuitable tactics and techniques, an agile approach is justifiable only where the need to meet the business objectives is so great that the risk is worth taking. In other words the alternative approaches (or doing nothing) are actually more risky, or are even fatal for the business.
Do the means support the end?The tactics and techniques used to produce the solution must serve the goal of producing a series of production software releases that meet the business and IT development objectives over the whole life of the series and beyond. The choice of iteration length, the timing and depth of analysis and design work, the scope of each role on the project, the form of collaboration, the extent of developer testing and the amount of development rework must ensure that the project achieves the outcome of business-workable production software releases. Any tactics or techniques that do not contribute positively to the outcomes do not have a place in the development project, no matter what the methodologists say. For example:
- Making user stories tiny, just so they can meet the arbitrary project management constraint of two-week iterations, is not a justifiable tactic if it produces only non-production releases and it prevents the team from seeing enough of the big picture to create building blocks that require little rework.
- Just-in-time requirements investigation is not a justifiable tactic if it provides such a narrow view that it prevents the team from producing software that fits properly into the business at every production release.
Favourable outcomes will only be achieved if the various tactics complement each other in the context of the particular project. To follow methodology-prescribed practices out of context is counter-productive and harmful, to put it mildly.
What does it mean for software to “fit properly into the business at every production release“? If there is an old system to be replaced or supplemented, then any production release of a new IT system will only be acceptable if the business capability-enablers (people, process, information and IT systems) in the new configuration ‘work’. This means the business can either stop using the old system and start using the new system, or use both old and new systems, and the people have the required knowledge, skills and information to interact with the systems, the systems produce information that is accurate and sufficient, and the business procedure work-arounds and swivel-chair integration are sustainable.
As Jeff Patton said
As Jeff Patton said
"What you have is fine, but there’s not enough here for me to get my job done. I still have to do much of my work manually. I have to transfer information from my paperwork to the software and back again. Sometimes I can’t even figure out a paper workaround. I’ll wait for the next release to see if you get farther along"
Most businesses will endeavour to tolerate messy operating procedures for a short period of time, but if the new system is so functionally incomplete or lacking in quality characteristics such as data integrity and reliability that it is unsafe or grossly inefficient to use in business operations, then the approach that produced it is invalid and harmful.
So an agile development approach, defined as one which puts a subset of final features into production, is only valid if the business can successfully operate with a subset of the features, i.e. without having all the features that are scoped for the IT solution.
Agile’s home turf: greenfield business
The ‘it must work’ criterion is likely to be present immediately in many greenfield business areas, because there is little or no business practice to fit into and no IT system usage to disrupt. There is little to lose. However in existing business areas, fitting new business practices and IT system features into the business operations will be much more difficult to achieve.
Success by design
The fit between IT system features, information flows, system interfaces and business procedures will not just 'evolve'. To paraphrase Charles Darwin, evolution is OK if you have a few aeons and don’t mind a lot of casualties, weeded out by failure. But that’s hardly fast delivery of business value. The level of casualties amongst the business sponsors and developers won’t be sustainable either. Developing incremental business solutions involves business design and software design, lots of it.
You might not write up all the software design decisions, but they’re still design and some of them are architectural design. Although any developer can reorganise some code with a few clicks, rework of any significance requires refactoring across a significant portion of the code, according to some design goals, which of course requires a design. Refactoring safely doesn’t just depend on having the safety net of verification techniques such as unit tests and integration tests, it depends on having a design that you can reason about making sensible changes to.
Designing the business changes so that business operations benefit rather than suffer from the increments of software features will require some highly competent business analysis, to ensure that every production software release ‘fits’ into the people, process, information and IT system landscape.
If you have this level of competence with business analysis and software development and have good commitment from business people, then you’re cleared for take-off. With these capabilities and commitment you can use the iterative development approaches that have seen a lot of success over the years. You could even use the agile flavour of iterative development if that provides a better match to the project risk profile. If you don’t have these skills and the business commitment to engage at a detailed level, then neither approach will succeed. No amount of project management, process, tools or favourite techniques can support business agility unless there is a strong cause-and-effect relationship between business commitment, business design, software design, complementary techniques, team experience, team behaviour and software releases that fit the business.
Prepare the business to receive the software in increments. Do the discovery and analysis at the earliest opportunity at which a solid understanding can be achieved. Design the software so that it can be released into the prepared business in increments without much rework. Choose a development process that copes with some late discovery, employs techniques that are beneficial, and delivers the production releases. Project management practices and parameters such as iteration length and user story size come last in this chain of logic - they are there to serve not to rule.
Prepare the business to receive the software in increments. Do the discovery and analysis at the earliest opportunity at which a solid understanding can be achieved. Design the software so that it can be released into the prepared business in increments without much rework. Choose a development process that copes with some late discovery, employs techniques that are beneficial, and delivers the production releases. Project management practices and parameters such as iteration length and user story size come last in this chain of logic - they are there to serve not to rule.
Posts
