The delivery pipeline is an idea which resonates with our clients. The simplicity of the metaphor and the truths that it reveals make it a compelling message to share with others in the organisation.
The pipeline helps to visualise two key Lean principles; seeing the end-to-end value stream and ‘optimising the whole’ http://bit.ly/ImpLean It illustrates the emerging emphasis on Continuous delivery’ http://bit.ly/CDAgile
The overall organisational aim is to balance the pipeline, as any constrictions generate waste.
The pipeline helps with the key challenge of getting senior stakeholders to prioritise and budget the money, and time, to balance the pipeline end-to-end.
Instinctively, because developers create functionality then, if we want more functionality, we must need more developers. Whilst jarringly simplistic, this is a perspective held by too many senior stakeholders.
However, adding more developers or teams, immediately leads to an imbalance in the capacity of each section. More development teams develop more code, which we then try to force through the lower environments – where it sits, wastefully, in queues or trying to muscle its way past other projects waiting for a limited slot in a integration test environment.
In addition to the waste, the quality plummets due to the stress of trying to get work through this constriction. A situation that is often made worse when these downstream environments are shared with other projects / programmes.
Similar constraints usually exist upstream of the developers. The Product Owner role is new to most businesses and the level of on-going work they need to do is often underestimated. Beyond the responsibility for preparing stories every sprint, collaborating with the team on a daily basis, POs need to actively engage with stakeholders and users.
The diagram below illustrates the actual capacity of the pipeline, which can be quite an attention grabber. It’s troubling for senior stakeholders to realise they are paying for all that under-utilised development capacity.
Steps to a balanced pipeline
Step 1: Communicate the need of optimising the system end-to-end
Use the pipeline diagram to communicate the essence of the challenge. These issues are difficult for even IT people to see and understand, even though they live with the stress of these systemic shortcomings on a daily basis. Create a sense of urgency by providing estimates of how much this waste is costing, both in money and in delayed value to customers.
Step 2: Prioritise and target specific interventions at the points where they will have most impact
- Investment in the environments, automated testing and deployment tools to ensure frequent/continuous movement of software through the environments to live
- Improve the quality and frequency of collaboration across traditional organisational boundaries, e.g. between developers, service introduction, support and infrastructure.
- Improve ways of working upstream between Product Owners, stakeholders and the teams. A clear vision with stable portfolio backlog is more likely to create a steady flow of high priority work into the development teams.
- The teams themselves can improve quality and throughput by adopting Agile/XP technical practices and working more closely with both the product owners and the ‘downstream’ teams.
Step 3: Visualise the waste with a value stream map
Visualisation and quantifying of the waste is key to building a sense of urgency for change and then targeting interventions at the most wasteful sections of the pipeline.
Value stream mapping is the simplest and most effective way to identify where the biggest queues and worst bottlenecks are in the system. The best resources on value stream maps are Mary Poppendieck’s books, including Implementing Lean Software Development’ http://bit.ly/ImpLean
Real-time visualisation of queues is best done with Kanban techniques, which can highlight emerging issues in the system.
Step 4: Get senior stakeholder support and the budget and time
The level of investment needed in environments and tools are often new to a business, which may not be clear on the sizeable returns available. These returns are often in the form of reduced waste, which many senior stakeholders can’t see or evaluate – even though they can sense waste all around.
A senior director estimated that 50% of their software development expense was wasted. How have we ended up accepting, or not able to deal with, this level of waste?
A smoother flowing pipeline also reduces cycle time, meaning that profit-generating functionality is deployed sooner.
Whilst getting the budget is difficult enough, ensuring the work actually gets done, month after month, is even harder, given the prevailing culture in many businesses.
Businesses and managers are rewarded for delivering value to customers. Few businesses reward managers for improving the capability of their system to deliver value.
It’s common to recognise the sacrifice in fixing problems – rather than rewarding those who prevented the problems in the first place.
The ‘non-functional’ work needed to improve system capability is difficult to value and prioritise in any company-wide backlog of work. Those that understand the potential benefits of a balanced pipeline need to articulate that value and be prepared to be held accountable for delivering the promised benefits.
This cultural bias, towards delivery and not improving the means to deliver, continually sabotages attempts to balance the pipeline.
Step 5: Expect resistance and be in it for the long haul
The value stream maps highlight that many constrictions occur at existing boundaries within the organisation. The delay, hand-offs, lack of visibility and accountability at those boundaries are the biggest challenge to creating a balanced pipe.
These changes to what people do, how they work with others, changes to accountability and their daily work will generate resistance.
Specific action is the best way to ignite the change. People on both sides of the boundary typically suffer from its existence and are often keen to improve the situation.
And… it’s not just the pipeline… but the shape of what you put down it..
In addition to working on the pipe itself – the throughput will be significantly improved by looking at the material going through it. This is the subject of a follow-up blog, where we’ll look at batch size, variance and work-in-progress as key variables.
In the meantime, Mary Poppendieck already has some good insights on this at http://bit.ly/LeanPipe