AGILE IN ACTION: 'Infrastructure' isn't all it's cracked up to be

Monday, January 15, 2007

'Infrastructure' isn't all it's cracked up to be

At Google, Ken Schwaber talked about Scrum and described Infrastructure (or Core) Software and the pain that it generates for an agile team. Infrastructure Software provides functionality that most other software depends on and must therefore integrate with. Typical problems are:

It's fragile because so many things are dependent on it. Change something in the infrastructure and some external, dependent software breaks.
It's brittle because it wasn't developed with sufficient automated tests. Change something in the infrastructure and something else in the infrastructure breaks.
There are only a few people left who know the code and are willing to work on it.

So, you're working in an agile team, hustling along at a sustainable pace. You can develop new functionality quickly, that is until you have a dependency on the infrastructure. Bang! There goes your velocity. Your productivity is completely throttled back by the velocity of the infrastructure team/s (assuming they need to write some code to support your new functionality). And their velocity is much less than yours because of the problems above.

Ken asked, where Infrastructure Software comes from? He determined that it was a result of teams coming under pressure to get more done, and habitually cutting quality and writing crap code (rather than saying 'No! We can only get this much done in an iteration'). This has knock-on effects during subsequent iterations, as they stop paying constant attention to design and technical excellence and end up building legacy out of the box. Voila! Infrastructure Software.

Another infrastructure problem relates to physical organisation, e.g. when companies centralise specific skills, such as operations or database administration, apparently to improve efficiency. Unnecessary external dependencies such as these will also throttle an agile team's velocity. This is not an effective way of working together to get things done and deliver business value. Build whole, cross-functional teams that include their own operations and database administration skills.

Because centralised teams, such as operations, are disconnected from say, product teams, and because they probably support many product teams, they cannot be expected to possess extensive domain knowledge or a detailed familiarity with each product. Therefore, to make centralised teams viable, organisations allow them to enforce standardisation. And, standardisation kills innovation. Bummer!

Tags: agile, scrum, infrastructure software, organisational infrastructure

4 Comments:

At permalink, pdam said...: ? Standardisation doesn't kill innovation, it constrains it, sure - but doesn't automatically kill it ...
So why wouldn't it be possible to Agilize your enterprise architecture and centralised core infrastructure teams? Co-ordinating these teams with your development teams and ensuring you don't create a bottleneck in infrastructure (so as not to further constrain development) - - why not?
At permalink, sjb140470 said...: Standardisation doesn't have to kill innovation, but it usually does. In the pursuit of standardisation companies often forget about the importance of other factors such as innovation.

Even when all the teams are agile, dependencies introduce difficulties and overheads. Dependencies between teams do slow things down. When many teams are dependent on, say, a single core team that core team will feel the pressure, agile or not, and a queueing effect will inevitably materialise. It's very difficult keeping everything synchronised and flowing, although not impossible. I'm just not sure it's always a cost-effective way of getting things done.
At permalink, Anonymous said...: Hi Simon, have you ever tried to do scrum with infrastructure components? Does it make sense? how would that work?

From LMN thinker
At permalink, Simon Baker said...: Yes, I've used Scrum in large enterprise programmes where there are separate infrastructure teams and product teams. And it never works well. Communication and collaboration between teams is not something that occurs naturally. Separate teams introduce waste - handovers across dependencies, chasing people and things in other teams, extra process to manage the dependencies. The focus shifts from delivery to process.

I use a lean approach where everything is organised and delivered through product streams serviced by dedicated cross-functional teams. Code that is shared by different product teams doesn't reside with a dedicated infrastructure team, it simply moves between the product teams. For example, if product team A needs to add X to some common code then they do that as part of their Sprint work. Similarly, if product team B needs to add Y to the same common code then they do that as part of their Sprint work. The teams can work independently on the same common code simultaneously because they're truly test-driven, the automated unit, integration and acceptance test coverage is approaching 100%, developers are checking changes in at least once an hour and continuous integration is building the whole system and running all the tests on every check-in (with an automated build that takes less than 10 minutes). Common code is a shared codebase. And knowledge of the common code is shared throughout the teams, which reduces the risk of losing key people.

Here's the crunch: You can only achieve this if the product teams are uber-agile. Otherwise you're going to end up with spaghetti hell. This can't be achieved with half-arsed agility. Uber-agile means that the entire organisation has a culture that is deeply engrained with agile values and principles. Teams are cross-functional (no upfront creative work nor trailer-hitched QA) and empowered to self-organise. People are not silo'ed by role. People doing the work are making the decisions. Servant leaders facilitate teams; project managers do not control them. A single product owner manages each product stream. They request work based on business value prioritisation and collaborate with the team to provide immediate and valuable feedback. Everyone is collocated (no outsourcing or off-shoring) in informative workspaces.