Idea Mingle to Bootstrap Open Space

Dear Junior

Some of the things I do on a regular basis is to facilitate workshops and events of different kinds - open space being one of them. I'd like to share with you a small trick-of-the-trade I have evolved while facilitating those: the Idea Mingle.

I really like the energy the open space format brings to the discussions. People are familiar with the format of participating in a discussion, and also shy or introvert people are given room to participate in a comfortable way.

However I have sometimes found that the initial part can be a little bit slow, the part where you pitch topics for discussion. This is seldom a problem with groups that are used to open space. Everybody involved understands what "level of ambition" is needed for posting a topic - acknowledging the fact that the really interesting part will be the discussion that unfolds around the topic. However, with groups unused to open space I have found that people can get shy. Posting a subject might seem pretentious: "what have I that everybody else should find interesting". So, when asked at point blanc "what do you think should be discussed", their mind goes blank.

To give the "pitching phase" of open space a soft start I do a "Idea Mingle" that gives people a chance to evolve their interests into discussion topics before it is time to post them. It is a "mingle party" with the purpose of distilling ideas for discussion, thus "Idea Mingle".

The instructions I give are simple: 

• Look around the room and lock eye contact with someone you have not yet spoken to today. 
• On my call, pair up
• Introduce yourself by saying "Hi, my name is …" to each other
• Continue by saying "I think it would be interesting to discuss something around …" and fill in a field. It does not need to be specific, it can be broad or vague, e g "something about testing".
• Have a short discussion digging a little bit into each others interests
• After two minutes I will break the discussion

The basic idea is that the one-on-one-format is a more comfortable format for talking about an idea than to do it in public. Speaking to a stranger is still an uncomfortable situation for a lot of people, me included. However, the short format and the very focused scope makes it managable - as opposed to the usual mingle setting "find a stranger, talk to him/her, and be nice and interesting for an undefined period of time". Just the thought gives me creeps.

Now, after the first one-on-one, all participants have bounced one of their interests on another participant. Almost certainly they have received acknowledgment that "it is an interesting field" and most have probably got feedback of the form "testing is interesting; I myself is interesting in how to use automation to make it easier".

OK, so people have now got a slightly refined idea about an interesting discussion. But, for most of them it is not yet ready to publish as a open-space-discussion-subject.

Allow me a slight detour. 

At a workshop class with Lyssa Adkins and Leslie Stein there was a small knowledge-sharing exercise. During that exercise I was given the task to explain "sprint retrospective" to a few of the other participants. I was given literally no time to prepare, and my presentation time was limited to five minutes (if I remember correctly). It was really awkward.

Immediately after the first group of listeners, I was sent a second group of a few class-mates, and had to explain to them. This time it was less awkward. A third group was sent to me, and this time I basically knew what I should say or not. The fourth time, the explanation went smooth. So, in four iteration a very awkward rambling refined into a pretty crisp micro-presentation.

Back to open space and Idea Mingle.

The next instruction for Idea Mingle is: Now, lock eye-contact with someone else. And, we do the same thing once again.

At the end of this second one-on-one the idea might have evolved to "automation in testing, and the trouble with databases".

And then we repeat the one-on-one a total of four times.

At this point of time, many of the participants have received acknowledgement that they are interested in a field others also are interested in. And, a lot of them have refined an initial rough idea into something that is a more specific topic, e g "How to involve DBAs in test automation" - a topic ready for discussion.

And we are ready to form a queue for pitching topics to discuss.

Also, everyone is a little bit frustrated over that each one-on-one was interrupted just when they were getting started. So, all participants are eager to start the open space discussions.

Yours

   Dan  

The Way Agile is Fast - Fast as in Orienteering

Dear Junior

It is often claimed that agile development is faster than the alternative approaches. However, "faster" used this way might be misunderstood. For example, we all know that running can be tiring, so running faster can be exhausting. 

Does agile development mean that we will exhaust the poor developers? And do so sprint after sprint? Does it mean we induce stress ulcer? Or will the developers have to cut down on testing, design, and quality to meet this faster pace of deliver?

Quite the opposite. Agile is faster in another way. It is not faster as in a long-distance runner being faster, rather in the way an orienteering runner might be faster - by making smart choices of route.

In agile we realise that developing systems is not running down a pre-routed track. It is more alike to make our way through wild terrain. When doing this it is crucial to early see if we are moving down a dead-end route, and then stop ourselves. To run down a dead-end route without getting anywhere is the most wasteful exhausting thing an orienteering runner can do.

In agile we try to adress this at as many levels as possible. To prevent the individual developer go down a dead-end without realising it, we have pair-programming or daily stand-ups as reality checks. At a team-level we have short sprints, or watch the kanban cycle-time, to prevent us from working on stuff in a bad direction for too long. At project- or initiative level we do impacts map and similar to ensure we do stories that matter. At the level of finance/funding and portfolio management we track effects to stop us from continue spending huge effort on stuff that give no or low effect. 

So, agile does not make us faster by making us run at a more exhausting pace. Agile makes us faster by giving us tools to take smart routes, preventing us from detours.

Yours

   Dan

ps This is of course related to how Domain Driven Design makes us faster, even in the short run. Using the orienteering metaphor, what you to is to slowly jog for some a short period of time while exploring the map for different routes.

Agile Projects should have Effect Targets

Dear Junior

Measuring projects and setting targets for them is a tricky business. And, it does not get easier when agile projects enter the scene. This is not really because agile projects are strange per se, but because they are different from non-agile projects.

Setting targets and measuring projects is also an important business. I have seen several agile initiatives fail. Most of them have failed because they did not succeed in setting goals for projects, and monitor their progress. And if you cannot do that, you quickly lose the confidence and support from upper management. Agile initiative terminated, or left to dwindle away. End of story.

However, it need not to be so. Agile projects can have measurable targets, and they can be monitored - you just have to do it right.

There are some bad news and some good news.

The Well-Established Chaos Rod for Measuring Success

The bad news is that agile projects cannot be measured using the de-facto established standard rod for project success.

The standard rod for measuring projects is "on-time and on-budget, with all features and functions as initially specified", as used by e g Standish Group in their much-to-cited Chaos Reports. Let us call this the "Chaos rod". Most organisations use some version of the Chaos rod for measuring their projects.

As we have discussed earlier it is pretty obvious that "all features implemented" is not a good way to measure "project success" - not for any project. Nevertheless, this is the standard rod.

Damned if you do, damned if you don't

Of course agile projects will fail if measured using the Chaos rod. The reason is simple - agile projects does not manage to keep their hands away from fiddling with the original specification. Agile projects remove stuff, change stuff, add stuff - agile projects are in a constant state of scope-creep.

This is no coincidence - it is how agile projects are designed.

Think of it this way: If we learn something during the run of the project - shall we let that insight effect the plan of what we intended to do? Or shall we ignore that insight, sticking to the original plan even though inferior? Of course we will adapt! Anything else would be ridiculous.

An agile project anticipates that such insights will emerge, so its processes are designed to harness and leverage upon those insights: demos, retrospectives, re-prioritisation of product backlog etc. These practises are all aimed at constantly refine and redefine the scope. But then, we have derived from the original specification "all features and functions as initially specified".

Thus, any agile project longer than a sprint will fail - by definition. That is, if you use the Chaos rod for measuring success.

To put it bluntly, if you measure an agile project using the Chaos rod, it will fail. Either the agile process will fail, or the project as defined will fail.

Either, the project adapts to its measuring rod and blindly follows the "functions as specified", throwing whatever insight gained aside. Then, "project" will succeed, but "agile" will fail.

Or,  the project will work according to its agile-minded processes, and then certainly the result of the project will not be "all features and functions as initially specified". Then, "agile" will succeed, but "project" will fail. I e project will fail as measured by Chaos rod.

Damned if you do, damned if you don't.

A New (or Old) Hope

The good news is that the Chaos rod is not the only way to measure projects. As we have discussed earlier, there has always been two ways to measure projects: feature list (Chaos rod) or measuring business effect, what we can call the Effect rod.

Let us take our earlier example with an on-line book store. They had an idea that recommendations of the type others-have-bought would increase their sales, so they set of some money for that projects.

Using the Chaos rod they would have created a feature list around others-have-bought recommendations. The project would later be evaluated on how well it implemented the list. But let us look at a better idea.

Using the Effect rod they might set a target that customers will by 0.35 more books per checkout on average. The Effect rod is used to state the value of a project. These 0.35 books can probably be converted to money, that makes the project worth the effort.

The project might start out with an idea that others-have-bought recommendations would cut the cake. After the first small stories, deployed to production and put into hands of customers, the team learns that some kind of category would be helpful. To facilitate this they implement a simplified "search similar" as one of their features.

Suddenly the number of books in the checkout carts raise to a level 0.4 above the pre-project baseline. And the level sustains, it was not just random noise - the change is statistically significant.

The project has fulfilled its target according to the Effect rod, and is declared a success.

Measuring using the Chaos rod "on time, budget, and specification" - the project would have been deemed a failure, because it did not deliver the initially specified others-have-bought.

So, agile projects can be measured. You just have to use a measurement that is well suited. And, to be honest - which is better anyway.

A sad side-note is that I know of several projects which have worked in an agile manner, and delivered enormous business benefit - but in the project report the project lead has had to apologise repeatedly for not meeting the project target as defined in the project specification - a feature list.

Granted, to measure projects on business effect is not a new idea in any way at all. The idea was certainly there before the Agile Manifesto was written around the turn-of-millennia. What is new is that this way of measuring is essential to provide rigour to agile projects.

For Agile to succeed at larger scale, we certainly need lots practises around agile-minded processes. Measuring agile projects on effects is certainly one of them.

The way to measure agile projects is to set targets for business effect.

Yours

   Dan

PS Interesting enough, there is a sub-category "agile projects" in the later Chaos Reports, but the rate of success is not 0%, it is around 40%. I wonder what is going on here.

PPS Within the agile community, the practice of projects is debated - at least in the sense of the common description "temporary organisation with limited time, resources, and ambition". So, it would probably be better to use some other term, e g talking about "initiatives". However, for convenience, I have kept the often-used and familiar term "project". Check out the twitter hashtag #NoProjects.

All Features Implemented does not equal Project Success

Dear Junior

Setting goals for agile projects is trickier that setting goals for waterfall projects. As I mentioned in an earlier letter, for waterfall projects it is possible to set the goal in the form of a feature list to be delivered. This approach has several drawbacks - it does not guide the thousands of micro-decisions that are made, it gives no sense of purpose to support the drive or inner motivation, and it gives no guidance for evaluating whether the money, time, and effort was well spent.

However, although all these problems, and although the approach is not advisable - it is still possible. The project management mindset and tools for waterfall projects are applicable for reporting project status or following up visavi a feature list. It is also possible to evaluate success by checking whether everything on the feature list is implemented.

Well, I still think it would be better to evaluate whether the project created some value.

Now, ridiculous as this seems it is still industry standard.

The CHAOS report 1995 from Standish Group might be one of the most cited reports in system development. According to its statistics only 16% of software projects succeed. In the 2012 report this has been updated to 39%. And they collect data from tens of thousands of projects so the figures should be pretty reliable. 

Now, this seems scary and low, but only until you check out their definition of success.

"Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified."

OK. Not a word about actually getting value for the effort. Remember the on-line bookstore where they wanted others-have-bought recommendations to increase the number of books customers. Now, imagine they implemented the others-have-bought recommendations, but the customers did not buy more books anyway. Is this project a success?

Well, you have spent lots of money building something, but made no money from it. To me it sounds like money, time, and effort down the drain - not as a success.

To Standish Group, it is considered a success.

On the flip side, imagine the project realising that a simplified "search similar" would increase the number of books each customer buys. Imagine further that this feature would be much easier to implement. So, the project decides to implement that feature instead. And the number of books sold per customer increases 0.4 on average.

Spending less money, time, and effort on something else than originally envisioned, but still getting the benefit - that sounds like success to me.

To Standish Group, it is considered a failure.

Thus, the figure 16% or 39% actually says nothing at all about the state of software projects.

So, Standish Group is probably filled with smart people. Why did they not evaluate software projects according to some meaningful metric instead? Most probably "generated business value as specified upon funding" would be more interesting. 

My guess is that too few projects actually stated an envisioned business effect, so analysing those projects would give so little data that it would not be possible to make any significant conclusions.

So, instead of measuring something that would actually matter ("fulfilled envisioned business effect") they just measured something that could be measured.

Now, from the second example, where the team implemented another feature than originally envisioned, it is also clear that this approach is an absolutely worthless way of evaluating agile projects.  

Yours

   Dan

PS The way to measure agile projects is to set a target for a business effect.

PS The CHAOS 1995 report has been republished openly available for academic purposes. It can be found at http://www.projectsmart.co.uk/docs/chaos-report.pdf. The 2013 version can be found at http://www.versionone.com/assets/img/files/CHAOSManifesto2013.pdf

Project Goals using Effects or Feature List

Dear Junior

There has always been two ways to set goals for a project, either you can define the goal of the project to be a list of feature to be implemented, or you can define some effect you want to see. Both of these have been around for a long time, and effect goals have always been superior. 

Setting goals as a feature list is actually pretty simple - you simply state "these features I want to see before we consider this project closed". An online bookstore could say that it wants search-by-author, categories, and others-have-bought recommendations.

Following up during the project will simply consists of checking how far on this check-list the development team has progressed.

Effect goals are a little bit harder to set. You have to figure out and express why you want to have work done, what purpose it fulfils, in what way it makes your world better. Said bookstore must then express that it hopes people each customer will by 0.35 more books per checkout on average, or that it will increase its customer base with 10 000 new customers.

Following up during the project using this approach will consist of measuring the business parameters and watch the values change. A tricky part is that the full impact of the project might not be seen until the development work is "finished" (whatever that means), or even not until some time thereafter.

To put things a little bit into context, it is not the "bookstore" that starts a project - it is always a person involved, in this case probably the product owner of the online store. And, this is the person who should explain why she is about to spend a lot of peoples' time and a lot of the organisation's money.

I have always found that measuring success on effect, or impact, is the more intellectually honourable approach. To be frank, setting goals as a feature list does really say "this much work I want to have done" or rephrased "this much money I want to spend", nothing about what value it should result in. And if you want to have a group of people to work to bring your ideas to come real, the least you can do is to explain to them the value you think it will bring. Setting goals on effect is really about that - bringing purpose to the work.

Now, these two ways of setting goals and measuring success have a fundamental impact on how to manage projects in an agile manner, but that will have to wait until a later letter.

Yours

   Dan

PS An agile project cannot be feasibly measured using feature lists; agile projects should be measured by setting a target for a business effect.

The Three Elements of Agile

Dear Junior

Not that I necessarily claim to be an old hip-hopper, but an interesting part of the hip hop culture is the concept of "the four elements". 

• Breaking - more commonly known as breakdance
• MC - more or less what rap is about
• DJ - you know disk-jockey
•  Graffiti - public painting

An interesting thing is that within the hip hop community it is commonly agreed that all four elements are needed. If you should take away any of them, the hip hop culture would not be whole, it needs all four of them to be complete.

Also, all the elements are on the same level. That MCing should be "higher" than graffiti, or vice verse, is an idea that would be refuted as ridiculous. All the elements are of equal importance and each indispensable. 

Obviously, the individual hip-hopper might not practice all of the elements. He or she will most probably immense in one of them. A few might practice two or three. Some might practice all four but are certainly not expected to master all of them.

Still there are a lot of respect between practitioners of the different elements: a talented MC easily gives public cred to a vicious graffiti painter. They are both needed and they would both feel poorer without the other. 

The four elements must be in balance for the hip-hop culture to thrive.

Well, that sounds all nice and cosy. But what does that have to do with agile?

I think the Agile community have a similar situation. Cutting the cake with a blunt knife I see three elements within the "agile culture": tech, process and organisation. OK, the knife is blunt so the cuts might not be perfectly clear, but I think it suffice to clarify my point.

Agile Tech consists of the technologies we have developed and use for the direct building of software. In this category we find frameworks as Hystrix for resilience; we find products as nosql-databases; we find tools as Chaos Monkey. Aside from the code we also have the close-to-code practices like Test-Driven Development, Domain-Driven Design, Continuous Integration, Build Pipelines etc. All these things enable us directly to write awesome software.

With "process" in Agile Process I simply mean "the way we work". We work in sprints á la Scrum, or WIP-limited continuous flow á la Kanban; we demo at regular intervals; we use information radiators and stand-ups to synchronise work; we make forecasts to synchronise our work with other departments; we run retrospectives after sprints and projects to have a structured learning etc; we set targets and measure progress visavi effect goals, e g using Impact Mapping.

Finally Agile organisation is about how we structure ourselves. Within teams we self-organise, sure, but this is what we do on a larger level as well. We must ensure that information propagate across the organisation in a sound way; we want the architecture to stay reasonably consistent, preferably without a command-and-control chief architect. We must also synchronise and prioritise initiatives across this organisation so we also find portfolio management and finance/funding practices like Beyond Budgeting. Last but not least we find Agile HR/Agile Talent Management as part of this element.

Each agile practitioner might not do all the elements. On the contrary, most will stay mostly within one element or have their foot-hold in one and cross over to the other. 

Comparing to the hip-hop culture I must claim that the agile community is not complete without all three elements. Doing only tech does not help us; doing only process does not help us; doing only org does not help us. Skipping any of them would hurt us.

Nevertheless I have during the history of agile I have seen competition between the element, sometimes even hostility between practitioners where one side have claimed to be the "true agile".

I think it is unnecessary and harmful.

Furthermore, I think it is unworthy of us, of a community praising "humans and interactions" over all things, a community that thinks "cross functional teams" are the superior model - should we not strive for a "cross functional community"?

In the same way as the hip-hoppers give cred across element borders I would love to see the kick-ass developer praise the agile-minded HR Director for "getting it"; I want the beyond-budgeting-inspired CFO to thank the engaged and engaging scrum master for the fantastic project retrospective; I want to see the impact-mapping business analyst to thank the devops for coming up with insightful real-time metrics about what customers actually do.

OK, I might be guilty of hippiesm here, but I want the three elements of agile to share love.

Yours

   Dan

PS One of the reasons I care so much is that deep in my heart I feel that Agile is different.
PPS There is a video in Swedish from the conference Agila Sverige 2014 where I do a blitz presentation on this subject and some related stuff.

Blitz Retrospective

Dear Junior

One of my favourite method tools is a retrospective format I call the "Blitz Retrospective". I have used it in a lot of situation where it has been impossible to do "large" retrospectives. For example, in some situations, the teams have been very "retrospective sceptic". In other, there has been a formally appointed "scrum master" who was not very open to suggestions, so suggesting changes to his retrospectives would not achieve much. In yet other, the sprints have been so long I have not had the patience to wait a few weeks. In all these situations it has been possible to suggest, and do, a "really quick retrospective".

The "Blitz Retrospective" only takes 25 minutes, and its focus is to find *some* change the team think would improve things. The format consists of three parts: the startup, collection of ideas, and vote. 


Startup


If it is the first time, the startup is spent explaining the format so everybody feels comfortable with what will happen. I clarify that the purpose is to find one single idea of improvement that we will try out next week.

A question that always come up on purpose is why we are limited to one idea, and what happens with the rest. Well, if the timespan is only a week, it would be foolish to focus on more that one thing - it would just result in none of them being changed. 


As for the rest of the ideas that come up - they will be there in the back of peoples minds and might change things implicitly - but they will not be actively in focus. If it is not the first time, I usually quickly rehearse the procedure, but I spend the time on evaluating the last retrospective's "winning idea" - more on that later.

The startup in full should not take more than five minutes. 


Collection of Ideas


Second section is the collection of ideas. To help the team members I split a whiteboard in three sections labeled "Continue/Increase", "Start/Try", and "Quit".

First section "Continue/Increase" is for things we already do and that support our work. Second section "Start/Try" is for ideas that we think we should benefit from, but we do not do it already, at least not in any significant amount. Last section "Quit" is for things team members think we do, but should stop doing as it does not serve us well, or even harm us.

Then team members are free to write down suggestions in any of these sections. Usually, I let them write on stickies, so that they can write a short statement on the sticky and explain it briefly when they post it on the board. If a short discussion emerges, then fine. However, if it seems to turn into a debate, or risk running long, I cut it short pointing to the purpose (find one idea) and the procedure (there will be a vote later). 


I also try to convince the team to keep the suggestions very concrete and limit them to thinks they directly can effect. For example, replacing the ventilation system of the building might really improve, but selecting such improvements is just asking for failure. In other retrospective formats such ideas are really valuable, but the purpose of Blitz Retrospective is basically to get acceptance for retrospectives - and then they must make a difference. We want things we actually can do, and that we can do within a week.

The collection of ideas could take ten to fifteen minutes. On occasions when time has not been an issue, I have let it run on longer - but usually most ideas are on the board after ten minutes. 


Grooming the List of Suggestions


When getting closer to the end of collection of ideas, I take a more active role, starting talking about the stickies in the "Quit" section. 


It is extremely valuable to get the "Quit" feedback, and I really encourage people to post such suggestions. However, I am a firm believer of that you should focus on "telling the good stories", because the stories you tell (and repeat) will become part of the "team lore" and shape the atmosphere of the team work. This is the basic idea behind the organisational philosophy "Appreciative Inquiry".


So, telling "bad stories" will basically make people feel bad - but not do much good in the long run. Telling "good stories" will culture a nice team atmosphere as well as enforcing the good habits.


Therefore I take on the role as the "positivist fanatic" and try to rephrase each sticky under "Quit" into "positive" ones. For example, if a quit-note says "stop coming late to meetings", I might suggest a start-note "meetings begin exactly on time - even if people are missing".


Before throwing away the "Quit" not I ask the original poster whether the new notes have captured the original purpose. Surprisingly often there is an additional aspect I had not understood. In the "begin on time" example, the poster might say: "It is not only about starting on time, it is also that every time someone arrive (late) there is a start of chit-chat and small talk that disrupts the meeting". To capture this there will be a second note start-note "keep meeting focus when people arrive mid-meeting".

I also ask if there are some suggestion that the poster think is a duplicate of some other - if so they can have it removed. Merging two similar suggestions to one of course gives them a better chance to "win".

Still when "grooming", the board is still open for new ideas, it is just for the team members to step up, post a sticky, and present the idea.

At the end of collection of ideas there often are a load of ideas and we only have a few minutes until we shall leave the room with one selected idea: time to vote.

Voting

For the voting, I simply rearrange the stickies and let each team member give three suggestions one vote each. The sticky with the most votes is the winner and is what the team should try for improvement the coming week.


As for the rest - things might improve just by having vented them, but they are not in active focus.

Evaluation


Next week I use the startup section for a quick evaluation. Here I want to separate two parts. 


One question is whether we did as we intended at all. Not trying isn't necessarily failure. Things change quickly from time to time and we might have had valid reasons not to do it. Even "did not have time" is a valid reason - and suggests that we should put aside time.


If the team did try the suggestion, the second question is whether it helped us or not. If it was helpful, we try keep doing it. If it was not helpful, then it is important to keep in mind that it was an experiment - and such should have positive and negative outcomes from time to time.


If the suggestion was tried and found helpful, we should find a way to formalize it. Technical stuff might go into a new check on the build server, working habits might go into the team's "Team Rules" or "Working Agreements" - whatever the name. The important part is that we do something that makes it plausible we will continue doing this good thing we have just found.

Getting Retrospectives Going, at All

I have found that running a few Blitz Retrospectives often results in an acceptance of having retrospectives at all. In combination with the not-very-scary format "ok, we can spare 25 min after Friday lunch", it is a good way to get retrospective started at all.

Of course, such a brief format misses many points - things a longer and deeper retrospective will catch. But the purpose of Blitz Retrospective is not to catch those - it is to win acceptance for having retrospectives at all an pave way for deeper retrospectives down the road.

I have found that the format works well for that purpose.

Yours

   Dan

ps Ester Derby and Diana Larsen have written a great book named Agile Retrospectives that is really helpful once you have gotten past that initial resistance, have established retrospectives, and want to elaborate them, specialise them, or just improve them in general.


pps Tobias Fors has made the point (blog post in Swedish) that feeling safe and secure is fundamental to engaging in a retrospective. He suggests the fundamental rule being "Everyone did their best given the conditions", and I have seen it helpful to start each retrospective with writing some similar statement on the whiteboard.

Joint Venture, Rather than Customer-Provider

Dear Junior

When a customer wants something from a provider, she places an order — and after a while there shows up a delivery. This works wonderfully well for a lot of situations. For example I needed a new laptop, so I went to the on-line Apple store. There I could configure how I wanted my MacBook (fast processor, no extra memory, and a solid state disk, please), and push the "order" button. About a week later there showed up a package at the office; enclosed was the computer - to specification. So the customer-provider system with order and delivery works in a lot of situations. Unfortunately, system development is not one of them.

The Customer-Provider Role-Play

Nevertheless, it is quite popular to let developers and business engage in a "role-play" where the business people are denoted the role of the Customer - often "because they pay". And as a contrast, the IT department take on the role as the Provider. I understand that these roles can come into play when business and IT actually are different companies.

However, it is really interesting to see this role play when both sides actually are departments within the same juridical entity. This makes the "they pay" argument kind of suspicious. In those situations we are not customer and provider. Rather we have chosen taken on those parts of the role-play, as a metaphor.

Hopefully we have made a careful analysis when deciding to use this metaphor. I think that in most situation, this is not the case.

Hopefully we have decided that this metaphor will help us to promote and guide valuable behaviour. I think that in most situations, it does not.

Customer Metaphor not Helpful

To me it seems there are two main reasons that the customer metaphor does not help us much.

First, the metaphor is so heavily loaded with associations and "wisdom" that our thoughts and behaviour is led astray. For example, we have all heard "the customer is always right". Unfortunately, this often creates a mental barrier to go and ask the business people.

On a similar tone, I have on several occasions heard programmers defend doing a bad job by "I only give the customer what they asked for" - when the proper thing to do would have been to stand up, walk away 25 meters and ask the business representative if the effect we have realised, but they probably not anticipated, really is intended or desirable.

Customer-Provider Misses Information Exchange Intensity

The second reason that the customer-provider metaphor does not help us is that I think it is fundamentally broken. If we study the information exchange in the metaphor, there is first one information transfer from the customer to the provider - i e the order. Then nothing happens, until delivery. Then there is a bulk of information going in the opposite direction - the delivery with all the details of the solution. In my experience, the information exchange of a highly successful system development initiative does not follow this pattern at all.

In my experience the key aspect of system development is knowledge generation. This journey is the joint endeavour where we combining the tactic ("able-to-do") domain knowledge of the domain experts with the expertise of creating mathematically precise logic systems of the programmers. In system development those two competences combine to create something new, that did not exist before. Together they create a precise understanding of a selected aspect of the problem at hand. In Domain-Driven Design terms, this distilled and precise domain expertise is captured in the domain model.

The information exchange when distilling the model is very intense, and is far from suitably described by one "order" followed later by one "delivery". I think the Domain-Driven Design focus on creating a shared, precise understanding of the problem domain very well captures the feeling of how much cooperation it takes.

Of course, we can "patch" the Customer-Provider by allowing (and encouraging) the provider to repeatedly ask the customer for clarifications and explanations. The other way around we can let the provider make delivery in parts - incrementally and iteratively show results en route. This is better, but I think it still misses a central point, the feeling of solving a problem together.

Customer-Provider Misses Mutual Initiative

Another way to view the problem with the customer-provider metaphor is to look at where the initiative lies at any given point of time. In the metaphor, the initiative is first at the customer. The customer will do whatever preparations needed to complete the order. During this process, the customer might interact with the provider, but the initiative to drive the process forward remains at the customer. This remains until the order is sent over.

After the order, the initiative goes to the provider, who will start working. At this point of time, the provider might reach out to the customer to discuss issues and ambiguities, however the initiative remains at the provider. 

Finally, upon delivery, the initiative is once again transferred to the customer, who will verify the delivery, accept it, or file deficiency reports.

Characteristic for the metaphor is that at any given point, it is clear who has the initiative. During this time, the other part can comfortable sit back and rest assured that it is "the other's" responsibility to pick up the phone.

System Development Requires Mutual Proactive Involvement

In system development, I want both sides to be equivalently engaged and involved in the work all the time. Creating a strict model a la Domain Driven Design is a creative and collaborative work that requires both sides (business and IT) to feel the responsibility to take the initiative whenever they are struck with an idea (often late at night) and ensure that every new insight or idea is brought into the common work. At no time can either side lean back and rest assured that "I am not responsible at the moment".

End of Customer-Provider

So, can we please put the Customer-Provider metaphor behind us? It does not really help us in guiding our ideas, thoughts, or behaviour in any direction that is well aligned with the agile values of modern system development. If we absolutely need a metaphor, we must look for another one.

On a side note, Swedish agile front figure Marcus Ahnve pushes the idea that it is time for us as a line of industry to stop working by metaphors. Instead of saying "system development is like construction" or "is like product manufacturing" we should start talking about system development on its own merits. I agree that it is about time - actually my grandfather could have been a programmer, and I could have been the father of some young developer in the profession today. On a similar tone, Eric Evans has also mused on the subject under the tittle "Software is (Not) Like That". I second both of these thinkers that we need to be careful with metaphors, and we would be better of without metaphors that with bad ones.

Joint Venture as a Better Metaphor

However, if we necessarily must metaphors at all, and if we insist on picking them from the business world, then I would suggest another one. I would suggest one where the information exchange is more intense, where both sides have the initiative all the time and where booth sides care about the result they create in common. I would suggest using Joint Venture as a metaphor instead.

In a joint venture two parties decide that they should cooperate over some common interest and work towards some goal that is out of reach for either of them. A good example is the cooperation between Sony and Ericsson to create a mobile phone that was also a nice piece of consumer electronics.

Sony and Ericsson did not solve this by Sony putting in an order to Ericsson to build a mobile phone according to their consumer-electronics specification. Neither did Ericsson put in an order the other way around. Instead they started a new company, SonyEricsson, where each part owed 50% of the stock of shares. In this shared company, both parties invested by sending some of their best engineers, and both parties committed to supporting the joint venture so that it would bear fruit. Which, by the way it did. I think this situation better resembles the interactions and caring that we experience in a well functioning system development initiative.

Bear in mind; we still use "joint venture" just as a metaphor. If someone wants to actually do business this way I will be really interested to sit by and watch. But I will probably not line up to invest. However, used as a metaphor, I think joint-venture would be more helpful that customer-provider.

And being helpful is all I ask from a metaphor.

So, at Monday morning, we might as well stop referring to the business people as our Customers, and instead refer to them as our Partner. It might actually make a difference.

Yours

   Dan


ps I did present a blitz presentation on this idea at the Agile Sverige 2010 conference at which the use of language and metaphors was a theme that emerged.

Why Release Planning Works

Dear Junior

A very efficient way of planning releases is to extrapolate the team's work within the limits of a "high velocity" and a "low velocity". Mike Cohn (one of the front figures of Scrum) makes a good work in showing this in a blog post entitled "Why There Should Not Be a 'Release Backlog'", where a team with velocity 33±4 can reliably be trusted to complete 165±20 story-points within the upcoming five sprints.

People that have tried this technique report that it works remarkably well – in fact it seldom fails. However, you should not use it to predict what you can finish within the next sprint or two, but for a release four-to-five or more sprints ahead the method seems to work fine.

This catches my attention. If somethings works remarkably well, it would be nice with some explanation; and that explanation should also explain why the short-term does not work. What is so magical about four-to-five sprints? If something works fine for seven sprints, it should work equally fine for three, should it not?

Actually, applying some not-to-advanced statistics and probability theory can explain both why the method works so well for five-sprint planning and why it is risky for two-sprint planning. Better up, it can even tighten up the estimates for longer horizons. E g the "next quarter" which roughly equals 6 two-week-sprints, the estimate is 20% tighter.

Before getting started we have to note that there is an inherent difference between guessing velocity and measuring velocity, especially if you want to do estimates based on statistics.

If you guess that velocity is in the range 29-37 you do not gain any extra information when guessing twice. Making a careful analysis of team composition and their environment is of course better than guessing out-of-thin-air, but from a statistical perspective it is equivalent to guessing – doing the same analysis twice gains no extra information. This is not said to talk down on guessing or estimating a velocity, before the first sprint it is the best we can do, it is said to point out the limitations.

So, if you guess (or calculate) a velocity of 33±4, all you can say about the completion within five sprints is that it will be in the range 145-185 story points.

Graphically, your view of the future will look like this:

A key difference when you measure velocity is that a second measurement gives extra information, even if it yields the same result. And this is why we can do better through statistical analysis. Note that I will take the liberty of being a little bit sloppy with terminology, I just want you to get the gut feeling across, not writing a thesis.

To start with, we have to model the sprint velocity as a probabilistic event. Looking at Mike Cohn's example (used with permission) we get the data-points 36, 28, 36, 38, 24, 35, 32, 35.

Looking at the data it seems fair to model it as a normal distribution. Actually, we could be wrong, it could be some other distribution. However, we will later see that even if we guess the wrong distribution, it will not matter in the end.

Pulling the data through standard formulas gives us an average of 33 and a standard deviation of 4.75. Using mathematical notation, we denote the sprint velocity with 'X'. The long-term expected average is denoted E[X], and the standard deviation is σ[X].

So, in statistical terms we are observing the outcome of the process X (giving one number after each sprint) where

E[X] = 33

σ[X] = 4.75

Let us pause for a moment and compare this standard deviation with the numbers that Mike Cohn got when calculating "high velocity" and "low velocity". He took the mean of "best three" and of "worst three" giving a range of 28-to-37, which correspond to ±4.5, pretty close to the 4.75 we got.

This is not a coincidence. Taking the average of three best and three worst out of eight, is an estimation of the standard deviation.

This actually explains why the Mike-Cohn-trick does not work very well for predicting what will be the outcome of the next sprint (note that nobody claimed it would work).

A normal distribution does only give 68% of its outcome within avg±1σ. So if we make a guess that next sprint will get velocity between 28.25 and 37.75 we will be wrong one-third of the times, which is not reliable. If we want e g 95% confidence (being right 19 times out of 20) we have to increase the interval to two standard deviation, in our example 33±9.5, i e 23-43. However an estimate where the higher bound is roughly double the lower bound is not impressive – you would probably not leave the management board meeting alive.

So how come the trick work for longer sequences of sprints? What we basically want to know is the average and standard deviation of

X1+X2+...+Xn

where each of X1...Xn is the outcome velocity of a sprint.

This is what Central Limit Theorem talk about. To no surprise the average

E[X1+X2+...+Xn] = E[X1]+E[X2]+...+E[Xn] = n*E[X]

so the expected long-term average of n sprints is just n times the average of each sprint.In out example, for five sprints we get

E[X1+X2+...+Xn] = 5*33 = 165

and no-one is surprised.

However, presenting the expected outcome to stakeholders is dangerous, as 'expected' only mean 'close to', not 'likely'. For example throwing tree six-side dices will have expected value 10.5, but you will never ever see an outcome that is 10.5, it might be 10, it might be 11, but never 10.5.

Adhering to the phrase "rather roughly right than precicly wrong" this is why I prefer to present intervals. Smaller intervals are more precise, but less probable to be correct, larger intervals are more probable to be correct, but less precise, and thus less useful. The trick is to find the balance.

I usually settle for an interval with the confidence 95%, i e I take the risk of being wrong one time in twenty. For a process that follows normal distribution (note: no-one said that sprint outcome do follow that distribution) the 95% interval is avg±2σ. We know the average (n*E[X]), and thus we need to know the standard deviation of the sum X1+...Xn. Here Central Limit Theorem come with a nice surprise. The standard deviation of a sum do not grow linear with the number of terms. Instead

σ[X1+X2+...+Xn] = √n*σ[X]

The intuition behind this is that if there are many sprints, then good and bad sprints tend to even out, but it takes consistently good luck to get high sums, and consistently bad luck to get low sums.

Put another way, if you think about each sprint as throwing a dice, then five sprints will be throwing five dices and counting the sum. The risk of five dices simultaneos showing low numbers is of course lower than the risk of one single dice showing a low number. Roughly, for throwing one six-side dice the risk of the lowest value (1) is one out of six; for throwing two dices the risk of the lowest sum (1+1) is one out of thirty-six (6*6); for throwing three dices the risk of the lowest sum is one out of two hundred (216 = 6*6*6). The extremes becoming relatively less and less usual accounts for the standard deviation to grow non-linear. If you do the math, it turns out it follows the square-root.

So, 95%-confidence interval will be given by

n*E[X]±2*√n*σ[X]

which we can calculate by putting in our observed average 33 and standard deviation 4.75.

Hold the horses! The 95%-confidence interval is based on that the random process is of normal distribution, and there is little evidence that the sprint outcomes are of that distribution. Do we not risk fooling outselves?

Here the Central Limit Thorem comes to our rescue. It roughly states that if you add a sum of independent outcomes (X1+X2+...+Xn), then the outcome-sum will be of normal distribution whatever the distribution of X. In practice this convergence is really fast. On the Wikipedia page there is a wonderful graphical example of how a really weird distribution become very near normal distribution already after adding three terms. So, for any practical purpose we can view the velocity over a few sprints as following a normal distribution.

Returning to our calculations we can now compute the 95% confidence interval for running five sprints.

n*E[X]±2*√n*σ[X] = 5*33 ± 2*√5*4.75 = 165 ± 21.2

or phrased another way (desperately trying to avoid mentioning the expected average) as the interval 144-186., which might even be acceptable.

Interesting enough this result (144-186) is very close to, and only slightly better than, the 140-185 that the Mike-Cohn trick gave for a five-sprint-prediction. Let us check why; the reason is simple.

The mike-cohn-trick was number of sprints times an estimated standard deviation.

5*σ

The statistical analysis was twice (to get 95%) the stdev for the sum, which was the stdev for an individual sprint times square root out of number of sprints.

2*σ*√5

As √5 is approx 2.24, the latter gives

2*σ*2.24 = 4.48*σ

which is a slightly tighter interval that 5*σ.

Actually, for four sprints, the methods coincide. Mike's method gives

4*σ

and statistical analysis gives

2*σ* √4 = 2*σ*2=4*σ.

Graphically, the two methods look like this:

We now see clearly both why the extrapolate method work for at least four-to-five sprints, and why it does not work (fail often) for fewer sprints. Note also how the lines intersect at x=4.

Now we can easily calculate a prediction for "next quarter". Following Mike Cohn's advise on running a one-week catch-your-breath sprint after six two-week sprints, 6*2+1 make up the 13 weeks of a quarter, which give you six "production-sprints" in each quarter. With a 33 point average, a quarter's worth of work will be around 200 points, and the 95%-confidence interval will be 2*√6*σ = 4.9*σ, which is 20% tighter than the "linear" 6*σ. With our standard deviation it gives 175-221 instead of 170-226; enough to make the difference of a few small stories, but far from the size of a sprint.

Of course for larger sprint number, the difference will be higher – 9 sprints will give a 50% better interval. However, as it is hard to keep circumstances constant for more that a quarter, so if you do two week sprints, the analysis does not become meaningful.

Just as a though experiment we can run the analysis on the 13 one-week sprints of a project. Imagine that you have observed the following velocitys during the last months:

7,8,3,10,9,5,11,8

Then standard formulas give

E[X] = 7.6

σ[X] = 2.6

and a next-quarter-planning of 13 one-week sprints give

n*E[X]±2*√n*σ[X] = 13*7.6 ± 2*√13*2.6 = 99 ± 19 = 80-118

which seems reasonable. Using "linear estimation" we get

n*E[X]±n*σ[X] = 13*7.6 ± 13*2.6 = 99 ± 34 = 65-133

which would basically be useless.

If we move in the other direction and imagine that we only have two sprints to a release. Then we know that Mike's trick does not work, and statistical analysis give an interval that is quite broad.

n*E[X]±2*√2*σ[X] = 3*33 ±2*√2*4.75 = 66±13 = 53-76

Well, even if the interval vary with 50%, you still have a lower limit that is not bad. You can still safely plan for 50 story points, and send the boxes for printing. The risk you run is less then 5%, actually just half of that (2.5% risk of underestimating, 2.5% risk of overestimating), and depending on circumstances and personality you might find that risk level acceptable.

If you want to reduce the risk further or plan more that those 50 points, there are always non-statistical methods like spending some effort breaking down even the remaining stories in yet smaller substories. Or the team can do some look-ahead tentative designs to gain confidence in their estimates, perhaps revising them. From an inspect-adapt point-of-view, this is of course guesswork but can be usable nevertheless. If you want to honour inspect-adapt, you better do this a few sprints beforehand, so you can observe a lower variation in sprint outcomes and act upon that observation.

What you do when breaking down stories in smaller substories is actually reducing the standard deviation of X, and can be analysed using similar statistical methods, but that is a topic in its own right, and have to wait.

In conclusion: Mike's trick work because his "low" and "high" velocities are roughly a quick method for calculating standard deviation, and for sprint sequences of at least four sprints the trick yields an interval that encompasses the 95% confidence interval. And, it is much simpler that doing the statistical calculations in full.

So, now we can go back to our daily life and continue to use Mike Cohn's trick, but at least we know why it works and why it has got the limitations it has.

I like to know why things works.

Yours

Dan

ps Of course it is a constant work keeping the backlog in shape to be able to do this kind of analysis. I try to spend just enough time on the lower priority items, just to keep them going, and focusing on what will probably be up for development the next few sprints. To do this I keep the backlog separated in the "usual part" and a short-list, for which I have some harder rules.