Two Types of Performance

Dear Junior

In architecture one of the most important tasks is to keep an eye on the non-functional (or quality) attributes of the system. Often this importance is enhanced by stakeholders holding up some non-functional requirement (NFR) saying "the system must really fulfill this". Unfortunately, these NFRs are often quite sloppily formulated and a key example is "performance". I have stopped counted the times I have heard "we must have high performance".

I think a baseline requirement for any NFR is that it should be specific. There should be no doubt about what quality we talk about. In my experience "performance" can mean at least two very different things. Instead of accepting requirements on performance I rather try to reformulate the NFR to use some other wording instead. I have found that the "performance" asked for often can be split into two different qualities: latency and throughput.

Latency or Response Time


With latency or response time I mean the time it takes for some job to pass through the system. A really simple case is the loading of a web page, where the job is to take a request and deliver a response. So we can get out our stop-watch and measure the time it takes from the moment we click "search" until the result-page shows up. This latency is probably in the range of 100 ms - 10 s. Of course, this response-time is crucial to keep the user happy.

But latency can also be an important property even without human interaction. In the context of cron-started batch jobs it might be the time from the input-file is read until the processing has committed to the database. The latency for this processing might have to be short enough so the result does not miss next batch downstream. E g it might be crucial that the salary-calculation is finished before the payment-batch is sent to the bank on salary day. 

In a less batch-oriented scenario the system might process data asynchronously, pulling it from one queue, processing it, and pushing it onto another queue. Then the latency will be the time it takes from data being pulled in until the corresponding data is pushed out at the other end.

All in all, the latency is seen from the perspective of one single request or transaction. Latency talks about how fast the system is from one traveller's point of view. Latency is about "fast" in the same way as an F1-car is fast, but will not carry a lot of load.

Throughput or Capacity


On the other hand, throughput or capacity talks about how much work the system can process. For example, a news information portal might have to handle a thousand simultaneous requests — because at nine o'clock coffee break a few thousand people might simultaneous surf to that site to check out the news.

Throughput is also important in the non-interactive scenario. Each salary-calculation might only take a few seconds, but how many will the system be able to process during those 10000 s between midnight (when it starts) and 02:45 when the bank-batch leaves. If we cannot process all 50 000 employees, some will complain. To meet the goal we need a throughput of five transactions per second.

In other words, where latency was the performance from the perspective of one client or transaction, then throughput is the performance seen from the perspective of the collective of all clients or transactions, how much load the system can carry. Here "load" in the same way as a bus will take a lot of load transporting lots of people at once, even if it is not fast.

Fastness vs Load Capacity

Both F1 cars and heavy-duty trucks are no doubt "high-performing" cars. But they are so in completely different ways. To have a F1 car showing up at the coal mine would be a misunderstanding that only could be matched by the truck at the race track.


So, I avoid talking about "performance" and risk misunderstanding. Instead I try to use "latency" and "response time" to talk about how fast things happen — while thinking about an F1 car.  And I use "throughput" and "capacity" to talk how much load the system can handle — while thinking about a bus full of people.

What is the latency for transporting yourself between Stockholm and Gothenburg using a F1 car or public-transport bus? What is the throughput of transporting a few thousand people from Stockholm to Gothenburg using an F1 car or public-transport bus?

Yours

    Dan


P s Now when we are moving to multicore, we will see increasing capacity but latency leveling out or getting worse. This in itself will be a reason to move to non-traditional architectures, where I think the event driven architectures (EDA) is a good candidate for saving the day. My presentation at the upcoming JavaZone will mainly revolve around this issue.

Two Observations on Overtime in Traditional Projects

Dear Junior
I have seen quite a few classically managed projects, and through friends and collegues I have been in contact with even more of them. I would just like to make two observation on the practice of working
overtime in different phases of such projects.

* It is very uncommon, actually I have neither experienced nor heard of, that project management orders overtime for requirements analysists so that they will deliver the full and finished requirement documentation on a specified date.

* It is very common, actually I have experienced it several times myself and heard of it numerous times, that project management orders overtime for programmers so that they will deliver the full and finished implementation on a specified date.

From these two observations it is probably possible to deduct lots of interesting conclusions. I will refrain from doing so here and leave it up to you as an interesting mind-game.

Yours

Dan

'As a' as stakeholder, not actor

Dear Junior

The story format ‘As a <> I want <> so that <>’ is quite well established within the agile sphere by now, and almost the standard format in Scrum. In a way it is a pity because even though I like the format, and use it myself extensively, almost exclusively, there is a risk that we are setting in the mould. I have even heard equivalents of “if you are not doing asaiwantsothat, you are not agile”. I think we should be doing more experimentation, we should be avant-garde, not dogma. Nevertheless, well established as it is, I think it is misunderstood.

Old habits die slowly, and so old mental models. One old mental model is the one of the “use-case”, where some actor performs some action, and the system reacts in some described way. The format ‘As a <> I want <> so that <>’ can be used to describe a use case by inserting the actor in the as-a-clause “As a(n) <actor> I want …”, where the I-want-clause often takes the form of “be able to <GUI-enabled feature>”. My point is that even if the format can be used in this way, it is not limited to that usage.

If we use the format to express enterprise value, we can express things like ‘As a head of accounting department, I want nightly synchronizations and comparisons with our bank accounts, so that we early can catch failed sales”. Barring that we know little about ‘synchronizations with bank’ or ‘failed sales’, we can see that this is not a use-case: the as-a is not the actor, it is the person feeling the pain of this functionality lacking, and based on this way of expressing the story, it will be much easier to get the Head of Accounting to sponsor and promote the story.

Opening up this door we see that there are multiple ways to address other difficult stories. Expressing quality characteristics (aka non-functional requirements) can be done in the same was “As a marketing manager, I want the system to be able to handle 100 000 new users registering the same evening, so that it does not break down when we run our successful advertising campaigns”. You can also express ‘tech stories’ “As a QA manager, I want the system to be put under continuous build with the tests run automatically, so that we early know if a quality problem arises”. You can even express ‘soft issues’ like training “As a Development Manager, I want all team members to get basic training on version control so that we get rid of all those merge-mistakes that we suffer “. In all these examples, we have a stakeholder expressing something that would give enterprise value to their aspect of the development effort, thus something a team could estimate in effort and a product owner could prioritize.

Of course, we can now notice that the ‘actor’ interpretation is just a special case where the stakeholder happens to be an end-user of the system.

When working with this format I have also found it helpful to drop the ‘a’ in ‘As a’. Phrasing it “As Head of …” makes it a little more succinct, and seems to give the stakeholder him/her-self a more direct bond when reading it. Phrasing it “As a Head of …” seems to be a little bit more ‘distant’ or ‘observing’ way, and does not (in my humble experience) to catch on in the same way.

So, obviously we are nowhere near the perfect format for “requirements”, we should not restrict our formats unnecessarily, we should look for new way of using those we have, and we should definitely do more experimentation.

Yours

Dan

ps Mike Cohn has written some related blogs and was kind enough to point me too them. Check out Non-functional Requirements as User Stories, Advantages of the “As a user, I want” user story template, and Writing User Stories for Back-end Systems that muse on adjacent themes.