Most products and projects tend to sooner or later get a really long list of things to be done, the Product Backlog to use the Scrum terminology. Keeping the backlog in shape can be a really tedious job. This is certainly true for all places that I have been working on. We need something more agile.
The backlog can take many shapes - excel spreadsheets, Jira items, wiki pages or all of the above - please try to collect them in one place. In an ideal world all items on the list should be well-written, well-understood and reliably estimated (preferably using "points", "trees" or some other unit not based on physical time).
We cannot really stop the list from growing. The reason for the list growing long is simple, it is easier to generate ideas than to realise them. Either you inherit a log backlog or you will soon have one. So, we cannot avoid if from growing, and we cannot spend all our time waking through that list polishing each item. Doing so would be both frustrating and a waste of time.
However, there are some points why estimating stories is important. And these are the once that we should focus on, so we can get the most benefits without paying the full price. I see three main good effects from estimation.
The most obvious benefit is that when the product owner selects what should be developed next, she needs to know the cost associated with each item - to be able to get the most bang (business value) for the buck (development effort). However, estimation also forces the developers to understand what is requested - which is helpful so they do not have to spend time within the sprint to find out (of course, they will need to refine their understanding and iron out wrinkles together with the business people, but at least they get the scope right). But thirdly, and not much talked about: estimation also drives a discussion on architecture and design, if allowed. To estimate you must have a rough idea about how to design it, and if peoples' perception of that differs a lot they will give very different estimates. This is an excellent way to find out areas where the team needs some time to synch their ideas. Unfortunately, it happens that project leads do not realise the necessity of this.
To get the benefits of estimation without paying the price of maintaining 1000 stories (change requests, bugs and enhancement ideas), I use what I call a "Backlog Shortlist". The Backlog Shortlist is simply the upper-most (i e highest prioritized) part of the product backlog. These are the items that the product owner will keep in crisp condition and ensure that they are properly ordered according to importance. E g, when working with the BDD story format I usually insist that these should have at least two example scenarios, which also gives a good idea about how the story should be demoed. Outside the shortlist I am far more tolerant against sliminess and other non-crispiness.
At each planning occasion I want to walk through all the items on the shortlist, ensure that everybody understands them and re-estimate them. This re-estimation further improves the benefits of estimation I mentioned: product owner's awareness of development cost, scope understanding, and synchronized ideas about design within the team.
Firstly, the cost estimates benefit a lot from re-estimation. From time to time the team realises that a story has become much easier because of job already done and the estimates might drop significant. For example I have experienced that during one sprint we implemented a story that made us create some objects. At the next planning we realised that another story now became almost trivial when the object structure where in place, and the estimate dropped. I do not remember the precise numbers, but it was something like dropping from thirteen to two.
Secondly, repeating the estimation a few times when the story is just a hot candidate, but not yet to be developed, gives the product owner repeated opportunities to get feedback about the cost, and perhaps modify the story by simplifying it or breaking it into parts. It also keeps the developers up to date about what is hot, so that the selection for the next sprint does not come as a surprise. The already recognise the topic and are pretty familiar with it.
Thirdly, as long as the team has a common understanding of the architecture (should we use a domain modelled oo-structure, or dao us directly to the database) the re-estimation is fast. However, if the team have very different views on how a feature should be implemented, it is better to resolve those issues before it is selected for a sprint.
To get these benefits I try to size the shortlist to be double the size of what we can do in a sprint. In that way, each item will be re-discussed and re-estimated twice before being selected for development. At the same time, the time to do the re-estimation does not get too long and burdening.
In short: using a backlog shortlist is just a way to avoid premature exactness, or the other way around: to enforce exactness when it becomes productive but not before.
So, next time you try to get a backlog in shape: focus the effort on where it gives the best result - on the shortlist.
Yours
Dan