Thursday, 23 January 2014

Why we moved from Scrum to Kanban

When I joined my current team, we were a full on Scrum shop: three roles (product owner, scrum master, team), three ceremonies (sprint planning, spring review, retrospective), three artifacts (etc), the whole shebang.  And, it mostly worked, in that things got done, software got shipped.
However, in our retrospectives we had a few recurring issues that kept coming up:

  • Our sprint planning meetings were horrible.  They went on forever (at least half a day), were exhausting, and everyone left them feeling like there must be a better way
  • Testing was always getting squeezed towards the end of the sprint   
  •  A reasonable proportion of stories (~30%) fell into the next sprint
When we delved into these, we developed some detailed explanations.


Sprint Planning Hell

The problem with our sprint planning meetings was that nobody wanted to commit to building the story until they understood how they would build it.  Consequently, each sprint planning became a design meeting, and often asked questions that the product owner couldn't answer, because at that point the story consisted of “As a user I want to A so that B”.
 
So we moved to asking the analysts on the team to develop a lightweight spec for the story before bringing it to Sprint Planning.

This resulted in the planning meeting becoming even more of a design session, which then often asked technical questions which couldn't be answered without some investigation.

So then we moved to having a ‘solution design’ session for each story before Sprint Planning, so that there was an approved technical design in place before we did planning poker on the story.

When was this analysis and design being done?  During the previous sprint, of course…

Squeezed Testing

You can’t test until there’s some code to test.  Of course, you can do test prep, you can write the acceptance criteria, you can set up test data, but… ultimately, if you’re a tester, you’re at the back end of the process.  And this is a problem, because any overrun by anyone ahead of you in the process affects you.  Was it ever thus.  

Consequently our testers were always under pressure – either they have to bust a gut to test the story before the end of the sprint, or they had to be the bad news bears and tell the Scrum Master (me) that the story wouldn't be ready and would have to be completed in the next sprint.

Stories Over-running

This issue is really a symptom of the previous two – with analysis and design being squeezed out of the front of the sprint, and testing out of the back, the sprint really just became the actual coding phase of the activity.  We even considered formally moving testing out of the sprint, but I drew the line at that one, and started looking for a better way.

Problems with Traditional Scrum

Based on the above, I thought Scrum had the following problems:

It assumes that everybody can do everything

I've never worked in a team like this.  At the very least, all the teams I've worked with have a split between analysts, developers and testers.  And within these, there will be people with specific skills – one story might need some serious database analysis, another some significant graphic design activity. Different people have different skills, and whatever process you use should recognise that.

It assumes that developing stories is a single task

Even within the (good) disciplines of a two week sprint, a story needs to go through a number of phases: 

  • Elaborate
  • Develop
  • Test
Inside these can be multiple other steps (design, style, review, etc).  These steps are sequential – you can’t style the UI until it has been designed, you can’t test the story until the code has been written.

The combination of these two problems, that building software has different sequential steps and these steps are typically done by different people, led us to look at Kanban.

Why Kanban Works For Us

This article doesn't have the space to describe the history or background of Kanban – the web is awash with such articles, and I don’t think I have a great deal to add to them.

The key thing Kanban brought to our small team was the concept of flow.  That a story flowed through a sequence of steps, from “As a User I want to A so that B” to some shipped working software.  This flow has a sequence, and some dependencies.  Anyone who’s worked in a team building software for more than ten minutes knows this, so let’s recognise it and make it explicit.

Our First Kanban Board

As is traditional, we got a whiteboard, some markers, some post-its, and constructed our board.  It consisted of six columns:

Next
The next story to be pulled from the backlog
Elaborate
Add detail to the story so that it can be coded, together with a technical design
Develop
Write the code
Test
Test the code
Review
Business review the implementation, to make sure it meets the original idea
Closed
Released!

Each story flows across the columns, and, importantly, is pulled from left to right by the following step.  So, developers pull elaborated stories only when they've finished the story they've been coding; that story, in turn, is now ready to be pulled into test.

WIP Limits

A key feature of Kanban is its ‘work in progress’ (WIP) limits.  This is the way of defining when a stage in the process flow is at its limit, and no new work should be pulled.  In the early days, we didn’t impose these, wanting to see if any natural limits emerged.  In practice they did, and once everyone realised that the essence is only to pull new work when you have capacity, we had few problems, and didn't need to enforce WIP limits.

Advantages of Kanban

Having been running Kanban for over a year now, we've found the following advantages:

Visualisation

The state of play for the whole development team is immediately obvious to everyone in the office – including all the people who don’t work on the software delivery team. We used to have the Kanban board in the open part of the office, where everyone could see it.  We now have a big screen TV showing the virtual board in our ALM software, but that’s a whole different blog post.

Of course, every article out there on Kanban promotes visualisation as a key benefit, but I just wanted to add my voice to the crowd.

Continuous vs Batch

To me, Scrum often felt like a ‘batch process’ model of software development – you do these things, heave them over the wall (into production) and then do the next things.  Everything had to fit the two-week sprint cycle.  With Kanban, I feel like we're now using a continuous-flow process – small stories scurry across the board quickly, and can be shipped when ready; larger stories take longer.  I think it’s no accident that this is a good fit with the ‘continuous delivery’ model of software delivery.

7 comments:

  1. Good article, I'm genuinely pleased the change is working for you.
    It is possible that your Scrum board could have been a kanban system as you've implemented it. All still working within the Scrum framework.
    What I see from this is the Scrum team wasn't resolving the issues arising.
    Planning hell and the other symptoms you write about are not uncommon. What is missing from the story is the continuous improvement aspect to address the problems.

    I come across more and more teams that move away from Scrum because it's hard to do well to a kanban system. This isn't a criticism, just an observation.
    Done well Scrum is great, done badly it's a pain in the arse.
    What's most interesting is that it needn't be an either/or choice, Scrum and Kanban can complement each other to great effect.

    ReplyDelete
    Replies
    1. I agree with your point that it doesn't have to be an either/ or choice - we still use some aspects of Scrum - we start the day with a 15 minutes stand-up, and we have a fortnightly retrospective too.

      We did *try* to resolve the issues - the changes I describe above to address planning hell came out of our retrospectives.

      I've had some good feedback from people saying that their experiences mirror ours, so I think it's more complicated than "Scrum is hard to do". We have some smart people in my team, and there were no obvious solutions to the issues I describe. If there are, I'd like to hear about them.

      Delete
  2. This is really common scenario, but switching to Kanban is throwing baby out with bathwater. All you do is hide the problems and dysfunctions instead of tackling them. Read the official Scrum guide, and you will see that all your problems are self imposed process issues that have very little to do with Scrum itself.
    This is not the best form for me to coach you, but you're making some obvious mistakes. Take planning meeting for example, as the name implies you use it to plan work, not design it. All you really need is to agree on what the problem is and rough idea how to tackle it. Figuring out technical details is part of doing the actual work that happens after planing is concluded. The solution you actually end up implementing can turn out to be completely different from what was discussed in the meeting.
    I do feel your pain though, I've been doing Scrum for 6 years and experienced a lot of the same problems at some point. Read the guide and try to reset your understanding of Scrum, it should help a lot. Feel free to message me if you want more advice.

    ReplyDelete
    Replies
    1. Difficult to message you when you post as 'Unknown'. Also, amazingly, I have read the Scrum guide, and I'm even certified. But how do you get a dev team to commit to delivering a 'rough idea' of an implementation?

      Delete
    2. Right, I falsely assumed you would see my email if I post with my google account, I updated my profile :) Make sure you read the latest version of the guide, it's a framework that evolved a *lot* over the last few years. To answer your question, there's no mention of committing to anything in the Scrum guide any more. It was something very easy to misinterpret and abuse, probably responsible for more failed Scrum implementation than anything else. I sadly see this perpetuated by most Scrum teams, to their own demise.

      Delete
    3. Ah, apologies for being spikey :-) Clearly I am out of date on the 'commit' thing, so I'll read the latest version.

      Delete
    4. We're going through a similar transition on the team I'm on, and I don't see the transition from Scrum to Kanban as throwing the baby out with the bathwater at all. We're doing most of the same things we were doing before moving from Scrum to Kanban. The visualization of work and our new planning process so far seem like good, incremental steps forward.

      It seems to me that in addressing what we saw as issues with our process, we could keep calling what we're doing "Scrum", but eventually others knowledgeable about Scrum would say that we're not doing Scrum. In the end, the label matters less than having a process that works for us.

      Delete