Deep Learning as a Strategy for Information Age

Dear Junior

Last friday we had the pleasure of having CS students Sofie Lindblom and Anton Arbring as guests visiting the monthly competence day at Omegapoint. 

After this visit, Sofie have done me the honour of musing on the theme of our letters by writing an open letter "Dear Senior, Letter to a Senior Programmer". 

That post is so full with interesting topics it would take a day just to briefly discuss them. But those topics are also way too important to leave uncommented. So, to do something, let us pick one important thing and discuss it. I pick the topic about "what to learn".

Too much information out there - as always have been

Sofie writes:

But there is too much information out there to know where to start. I am not stupid, I did very well in all programming courses and is a fast learner. But I feel exhausted by the amount of information available.

To start somewhere, let us start with the vast amount of information, technology, frameworks, etc that are out there. Obviously there is no way to take in all of that. If we want to use metaphors, it does not suffice saying "drinking from the fire-hose", it is rather to try to gulp the Nile. 

Good part is that the situation is not new. Of course there are more information out there now compared to 15 years ago when I left university. But even then the amount of information available was too much for any individual to comprehend. And the situation is even older. The proverb "so many books, so little time" is not fresh-out-of-the-presses. 

For those leaving university today, there will be truckloads of technology you will be using at work that you did not learn in class. But that situation is not new either. When I left university, I had not used a relational database in any single class or lab. Still, most systems (not all), I have worked with professionally have included SQL databases of some sort. Actually, one of my first jobs was to teach a class on the Java database API JDBC. How did I manage?

The obvious solution is to replace "know everything" with "able to comprehend". We cannot know everything beforehand, but we need to be able to understand any technology we come across with just spending a reasonable amount of work.

Killing a meme

There is a meme around in this information age that basically goes "you do not need to know, you need to be able to find information". I want to kill that meme in the context of system development.

The meme might very well be true, with Wikipedia and the rest of the web at our fingertips we will be able to find data like "first historically recorded solar eclipse" (5th of May 1375 BC in Ugarit). Nevertheless true, it is worthless to us as software professionals. Because what we need is not data or information, but understanding.

Deep knowledge feed deep knowledge

Now, this is only my own meandering experience, but I have found it invaluable to know a few things really well. Deep knowledge has interesting side effects. Suddenly you see some pattern apply to a new domain. It seems like no domain of knowledge is an island. Even if facts do not carry across boarders, some structures of thinking and reasoning actually apply.

This is really vague, so let me throw some examples to clarify. When studying law I suddenly found that my studies of formal logic really helped me. I studied negotiation theory and found how it applies to finding a good architecture for a software system. I studied compiler technology and found it helpful when studying linguistics. Through my lifelong studies of math, I see wonderful aspects of beauty in the world every day. (OK, the last is a little bit off topic - but it makes my life richer, and that is worth something)

The strategy I try to apply myself is to study subjects in depth, to the level when I have to think hard about them. The specific knowledge might not be immediately applicable - I will probably not have any specific use of knowing e g how to count the number of ways to paint a cube using several colours. However, thinking hard has probably etched new traces in my brain - and those ways of thinking will probably pop up applicable in a new domain.

To fall back on metaphors again. As software developers we need to dig deep to understand a new technology. To get down to depth we are not very helped by having dug a meter deep over a large area. But if we have dug a few 20 m deep holes in other places, there is a good chance that we can dig a short tunnel at the 20 m level from the bottom of some other hole. 

How did I survive that first job-gig teaching an API that I had myself never used before? Well, having studied functional programming in depth (using e g ML) had made me comfortable with the ideas of abstract datatypes. So the idea of an API was not unfamiliar. Having studied linguistics I was very familiar with formal grammars of languages so SQL syntax was not strange. Having studied compiler technology I could understand the semantics of SQL. Having studied algebra and set theory I could easily pick up how SELECT and JOIN worked.

It took me a few days to read the JDBC API and specification combined with some small hacks to validate that I had got it right. And after those few days I did not only knew about JDBC, I actually understood it well enough to be able to teach it in a reasonable way. Not being an expert, but reasonably competent.

Without the deep knowledge in some very obscure subjects (linguistics, set theory, compiler technology etc) I would have been utterly lost. No skill in "searching information on the web" would have helped me the least.

Sofie writes

The more I learn, the more I realize how little I know. It creates contradictory feelings towards the field I love. To twist it even further the part I love the most is that you can never be fully learned and that there is never a ”right” answer.

I understand the frustration. But I am not sure I would like to have a field where there was a right answer, a proven best practice - many in our field dream of such. 

However, to me a large portion of the beauty of the field is that we are constantly pacing unchartered terrain. The challenge is to constantly search your tool-box for something that seems applicable, to adapt, to improvise, to search, to try, to fail, to back up, to learn, to grow, to try again, to discuss, to exchange ideas, to finally nail it.

This is nothing but my own personal experience, but if I should offer an advice to handle the world of information we have around us it would be the following:

Find things to learn that you find interesting and that challenge your intellect. Take the time, pain, and pleasure to learn a few of those things to depth. The deep thinking will etch your brain in ways that will help you enormously whenever you approach a new field. And enjoy the pleasure of deep understanding when it dawns on you.

Yours

  Dan

PS Should you come across Sofie and Anton, take the time to have a discussion with them. And do not stop at a chat about everyday things - they have really interesting ideas to delve into.

Complexity Drives Effort

Dear Junior

There has been a recent discussion on story point estimation (or sizing) that is so common in Scrum. Especially the discussion has been about how sizes should relate to complexity  and to effort needed to complete story. In a blog post Mike Cohn made a good example between effort and complexity by comparing "licking a thousand stamps" on one side, and "performing a simple brain surgery" on the other side. These where assumed to take approximately the same effort, but had drastically different complexity. Point was that they should have same amount of story points.

It is Effort that Makes a Difference

I think there is a lot of merit to the point Mike makes that it is amount of effort that is the usable parameter - we need it to make rough cost estimates and to predict what  functionality can plausibly be released when. Complexity is not usable in the same way.

However, even if "effort" is what we primary care about, to me complexity plays a very central role - more than I have seen mentioned in the discussion. To clarify I would like to dive into the thousand-stamps-example.

Story is Effect, not Task

To start with, the team should not get a story saying: "lick a thousand stamps"; because that prescribes a specific solution. A good story should only point out the desirable effect, and leave to the team to pick a good solution. I guess the story could instead have read: "ensure a thousand stamps get licked", or something along that line. The difference is subtle but important. In the first case, the team is given a task to perform. In the second case, the team is given a problem to be solved. 

Bulk Driving Effort

If you are to lick ten or a hundred stamps, you will certainly just do it. If you are to lick a thousand you will get mighty bored and tired - with ten seconds per stamp that amounts to three hours of work. However, it is still manageble, so you just get onto it. So here the effort will be driven by the bulk of the job - more stamps, more effort.

However, if you are to lick ten thousand stamps, or a million, then you are facing another magnitude of effort. To lick a million stamps would take 10 000 000 seconds, which is 70 weeks of full-time work - the better part of two years. Facing that task I would probably start building a stamp-licking machine.

Interesting enough this changes the game. The effort of building such a machine is independent of the amount of stamps to be licked. It is roughly as hard to build such a machine whether it will lick thousand or ten million stamps. So, bulk does no longer drive the effort.

Complexity Driving Effort

So, how hard is it to build a stamp-licking machine. Well, the actions involved is pretty straight forward, there are only one set of maneuvers, and no decisions to make - so, not-too-hard is a fair guess. 

And here comes the other half of the interesting change of the game. The effort is decided by how hard it is to build such a machine - i e the complexity of the task. So, bulk does not drive effort any longer, instead it is the complexity of the task that drives the effort.

Software Development Teams do not do Bulk

Going back to software domain, the equivalent of licking lots of stamps (10 seconds each) is probably to send a pre-formatted mail to a long list of email addresses found in an excel sheet ("new mail", CnP mail content from template, CnP email from next line in sheet, "send").

A team of developers given the story "send mail to top thousand customers" would certainly not open Outlook and start at top of list. Instead they would write some script that generates a mail to mailer daemon given the to-addr - effort 15 min. Then they would write another script that loops over a text-file with email addresses and sends same mail to all of them using first script - another 15 min. Last, they would export the mail addresses from the excel sheet to a file, feed it to the looping script, and off we go - another 10 min. OK, let us allow for extensive testing and coffee break but we are still around two hours.

Complexity Drives Effort

The important point here is not whether the script-solution would be faster or not. The important point is that the effort needed is not determined by the size of the batch. The effort needed is determined by how hard the problem is to solve - how much time must be spent discussing design, how tricky it is to get right in coding, how much testing is needed to feel confidence in the solution. In other words, its complexity.

The effort which we need to put in to solve a problem in software development is very much a function of the complexity of the problem. The complexity can be essential to the problem, i e if we are working in a knowledge-intensive domain. This is of course the call for Domain-Driven Design. The complexity can also be accidental, e g if we have an over-complex architecture or a codebase in bad shape. This is of course the call for software craftsmanship.

Disregarding the reason, overcoming said complexity will be what decides whether we need to spend a lot of effort or not. 

Sizing, or estimating, in story points should definitely reflect the amount of effort we think we need to put in. However, in most cases it is the underlying complexity - essential or accidental - that drives that effort.

Yours

   Dan

Ps On of the reasons why complexity of problem is so central is that software development to a large degree is a knowledge journey.

pps It is easy to be fooled into discussing what "story points really are", but from a pragmatic view that is uninteresting, unless it makes a difference. What matters is "what story points are useful for", i e the point of story points.

Joint Venture, Rather than Customer-Provider

Dear Junior

When a customer wants something from a provider, she places an order — and after a while there shows up a delivery. This works wonderfully well for a lot of situations. For example I needed a new laptop, so I went to the on-line Apple store. There I could configure how I wanted my MacBook (fast processor, no extra memory, and a solid state disk, please), and push the "order" button. About a week later there showed up a package at the office; enclosed was the computer - to specification. So the customer-provider system with order and delivery works in a lot of situations. Unfortunately, system development is not one of them.

The Customer-Provider Role-Play

Nevertheless, it is quite popular to let developers and business engage in a "role-play" where the business people are denoted the role of the Customer - often "because they pay". And as a contrast, the IT department take on the role as the Provider. I understand that these roles can come into play when business and IT actually are different companies.

However, it is really interesting to see this role play when both sides actually are departments within the same juridical entity. This makes the "they pay" argument kind of suspicious. In those situations we are not customer and provider. Rather we have chosen taken on those parts of the role-play, as a metaphor.

Hopefully we have made a careful analysis when deciding to use this metaphor. I think that in most situation, this is not the case.

Hopefully we have decided that this metaphor will help us to promote and guide valuable behaviour. I think that in most situations, it does not.

Customer Metaphor not Helpful

To me it seems there are two main reasons that the customer metaphor does not help us much.

First, the metaphor is so heavily loaded with associations and "wisdom" that our thoughts and behaviour is led astray. For example, we have all heard "the customer is always right". Unfortunately, this often creates a mental barrier to go and ask the business people.

On a similar tone, I have on several occasions heard programmers defend doing a bad job by "I only give the customer what they asked for" - when the proper thing to do would have been to stand up, walk away 25 meters and ask the business representative if the effect we have realised, but they probably not anticipated, really is intended or desirable.

Customer-Provider Misses Information Exchange Intensity

The second reason that the customer-provider metaphor does not help us is that I think it is fundamentally broken. If we study the information exchange in the metaphor, there is first one information transfer from the customer to the provider - i e the order. Then nothing happens, until delivery. Then there is a bulk of information going in the opposite direction - the delivery with all the details of the solution. In my experience, the information exchange of a highly successful system development initiative does not follow this pattern at all.

In my experience the key aspect of system development is knowledge generation. This journey is the joint endeavour where we combining the tactic ("able-to-do") domain knowledge of the domain experts with the expertise of creating mathematically precise logic systems of the programmers. In system development those two competences combine to create something new, that did not exist before. Together they create a precise understanding of a selected aspect of the problem at hand. In Domain-Driven Design terms, this distilled and precise domain expertise is captured in the domain model.

The information exchange when distilling the model is very intense, and is far from suitably described by one "order" followed later by one "delivery". I think the Domain-Driven Design focus on creating a shared, precise understanding of the problem domain very well captures the feeling of how much cooperation it takes.

Of course, we can "patch" the Customer-Provider by allowing (and encouraging) the provider to repeatedly ask the customer for clarifications and explanations. The other way around we can let the provider make delivery in parts - incrementally and iteratively show results en route. This is better, but I think it still misses a central point, the feeling of solving a problem together.

Customer-Provider Misses Mutual Initiative

Another way to view the problem with the customer-provider metaphor is to look at where the initiative lies at any given point of time. In the metaphor, the initiative is first at the customer. The customer will do whatever preparations needed to complete the order. During this process, the customer might interact with the provider, but the initiative to drive the process forward remains at the customer. This remains until the order is sent over.

After the order, the initiative goes to the provider, who will start working. At this point of time, the provider might reach out to the customer to discuss issues and ambiguities, however the initiative remains at the provider. 

Finally, upon delivery, the initiative is once again transferred to the customer, who will verify the delivery, accept it, or file deficiency reports.

Characteristic for the metaphor is that at any given point, it is clear who has the initiative. During this time, the other part can comfortable sit back and rest assured that it is "the other's" responsibility to pick up the phone.

System Development Requires Mutual Proactive Involvement

In system development, I want both sides to be equivalently engaged and involved in the work all the time. Creating a strict model a la Domain Driven Design is a creative and collaborative work that requires both sides (business and IT) to feel the responsibility to take the initiative whenever they are struck with an idea (often late at night) and ensure that every new insight or idea is brought into the common work. At no time can either side lean back and rest assured that "I am not responsible at the moment".

End of Customer-Provider

So, can we please put the Customer-Provider metaphor behind us? It does not really help us in guiding our ideas, thoughts, or behaviour in any direction that is well aligned with the agile values of modern system development. If we absolutely need a metaphor, we must look for another one.

On a side note, Swedish agile front figure Marcus Ahnve pushes the idea that it is time for us as a line of industry to stop working by metaphors. Instead of saying "system development is like construction" or "is like product manufacturing" we should start talking about system development on its own merits. I agree that it is about time - actually my grandfather could have been a programmer, and I could have been the father of some young developer in the profession today. On a similar tone, Eric Evans has also mused on the subject under the tittle "Software is (Not) Like That". I second both of these thinkers that we need to be careful with metaphors, and we would be better of without metaphors that with bad ones.

Joint Venture as a Better Metaphor

However, if we necessarily must metaphors at all, and if we insist on picking them from the business world, then I would suggest another one. I would suggest one where the information exchange is more intense, where both sides have the initiative all the time and where booth sides care about the result they create in common. I would suggest using Joint Venture as a metaphor instead.

In a joint venture two parties decide that they should cooperate over some common interest and work towards some goal that is out of reach for either of them. A good example is the cooperation between Sony and Ericsson to create a mobile phone that was also a nice piece of consumer electronics.

Sony and Ericsson did not solve this by Sony putting in an order to Ericsson to build a mobile phone according to their consumer-electronics specification. Neither did Ericsson put in an order the other way around. Instead they started a new company, SonyEricsson, where each part owed 50% of the stock of shares. In this shared company, both parties invested by sending some of their best engineers, and both parties committed to supporting the joint venture so that it would bear fruit. Which, by the way it did. I think this situation better resembles the interactions and caring that we experience in a well functioning system development initiative.

Bear in mind; we still use "joint venture" just as a metaphor. If someone wants to actually do business this way I will be really interested to sit by and watch. But I will probably not line up to invest. However, used as a metaphor, I think joint-venture would be more helpful that customer-provider.

And being helpful is all I ask from a metaphor.

So, at Monday morning, we might as well stop referring to the business people as our Customers, and instead refer to them as our Partner. It might actually make a difference.

Yours

   Dan


ps I did present a blitz presentation on this idea at the Agile Sverige 2010 conference at which the use of language and metaphors was a theme that emerged.

System Development is Knowledge Journey

Dear Junior

There are many complex things that need to be overcome to create a system, but in my experience there is one thing that overshadow them all: to gather and create enough knowledge to create a system that not only "works (kind of)", but actually works - i e help the enterprise to get things done in an unobtrusive and natural way. 

In most systems I have worked on this, this aspect is far more challenging than technical issues like getting quick enough response time for persistence, or capacity to support enough "simultaneous users" (whatever that is). In essence, to really understand the problem we are solving is the critical complexity to solve for success. Of course, this is basically what Domain Driven Design is about.

The goal is a really tough one, to create a system that will handle any case that comes in its way, without manual involvement. Well, some cases might be delegated for manual handling, but still - the decision to delegate a specific to a human must be done automatically.

I like to compare with the challenge of riding a bike. What we are trying to do is not so much to ride a bike. We are trying to construct a bike-riding robot.

If the goal is a tough one, the starting point is the weaker. The people around are business people, who know not much about system development; and system developers who not know much about the business at hand. So, there are no single person with enough competence to solve the problem. Hence, we need to talk. 

Even worse, not even if we gather the total competence of everybody involved will we have covered the competence we need. Again compare with bike riding. Get a pro BMX-rider and a robot engineer into a room; not even together they know how to build a bike-riding robot. Hence, it is not enough to talk. We have to depart onto a knowledge creating journey together.

I have mainly been working on information systems (in contrast to e g games or automation). In these systems, what we need to do is to create a model of the domain. Of course, not the entire domain, but a part of it that constitutes the essence of the tasks we want to support. But the model of that limited domain must be very precise. In fact it must be free from all the ambiguities and vagueness that is acceptable if we "just" work in the domain - it must be precise enough to construct a computer program.

To succeed , there are three types of knowledge that we need to improve, by learning or investigation. . One is well known and obvious, the second is less well known and often overlooked, and the third is almost never discussed explicitly.

First the programmers need to learn about the domain. This is pretty well understood and I have often heard claimed that "the programmers do not understand the business". This is totally correct, as most programmers have been working in other fields and thus do not know the ins and outs of this precise problem. Compare with the robot expert - she will not necessarily know much about bicycling. The good part is that in learning the basics and overview of the domain, they can learn a lot by talking to the domain experts. The bad part ... well, I will get back to that part.

Second, and often overlooked, is that the domain experts need to learn some modeling. This is basically for communication. When modeling the problem we need to get precise - dead to the point precise. Unfortunately domain experts are not used to this, and "works for most cases" or "more or less" are not phrases that helps very much when creating automatic systems. Compare with the BMX pro, they will probably need to learn some things about robots. Do not get me wrong - I do not want to make modelers out of domain experts. I just want them to understand a little about modeling. I guess I am after the same difference as between reading and writing. The good part is that there are expert modelers around from whom they can learn. The bad part ... well, I will get back to that.

The third and final part of the missing knowledge jigsaw puzzle pieces, is unfortunately the combination of the two "bad parts". This is the knowledge that neither part have when starting out. This is the knowledge that we cannot learn from someone else. This is the knowledge we have to create together. This knowledge is the precise modeling of our domain problem. 

Again comparing with the robot team: this is the knowledge that enable us to make a robot to make a left turn without falling flat to the ground. On one hand, the robot engineers knows a lot about fast-feedback-systems, but not about bike-turning. On the other hand, the BMX pros can make a bike under very harsh conditions, but they probably do this on an intuitive level without knowing the precise mechanical details. But the depth of knowledge needed is not at a "do"-level ("apply" in Blooms taxonomy), it is needed at "analyze"-level. And at that level, most people do not know how to ride a bike. Telling the rider to "follow the bike" might help a human, but will certainly not help when constructing a robot.

The tough journey is that the robot engineers and BMX pros have to figure out what exactly happens when you make a turn on a bike. They will probably film bike-riders in different situations, try to make some explanation and theory; from that theory build some simple prototype, test it, discard it, start over, make a new theory, fail, simplify, prototype, build, succeed, trying a harder case, fail, back down, redo, try again etc in iterative investigation to build some model of what happens when turing. Finally they might have made a model that holds for having the robot ride a bike through a simple slalom path on flat surface. From there on, they can try doing something a little more complicated, like driving over a small bump.

In the same way programmers and domain experts must interact to find a simple case to start with, and try to model it sufficiently well. Once they have managed that, they can expand to cover interesting cases the first model could not handle. While doing this they will probably have to revise, or trash, parts or all of the first model. But iteratively, and incrementally they will learn (or invent) new ways of thinking about the domain. These are ways of thinking that neither the domain experts nor the programmers had thought about before they started working together. 

This is very far from "lets ask the expert", as the domain experts do not have a "ready-baked" model inside their heads. It is also very far from "lets just build what we think they want", as the critical problem is not technical - it is about creating knowledge.

The bottom line is that the most critical activity is to make the domain experts and developers work in cooperation to create the model. To do this we need to mutually learn from each other. But also, and more important, we must create new knowledge as a joint effort, knowledge that did not exist when we started. If that knowledge journey fail, we do not end up with a system that actually works and is fit for purpose.

Facilitating that joint knowledge journey is the key factor to create systems.

Yours

   Dan

The Domain Model is not in the Head of the Expert

Dear Junior

Among programmers it is not uncommon to get into discussions on naming of variable, methods, classes etc. A pretty common way to resolve these discussions is to say “let’s ask the domain expert”. Even though I encourage communication across disciplines, there is a sub-tone that alarms me: the sub-tone of “checking the answer”, like if the proper domain model was in the head of the domain expert, and the task of the programmers is to uncover it and write it down in code. I think this surfaces two different misconceptions about domain modelling in general as understood by domain driven design.

The first misconception is the kind of expertise the domain expert has. He or she might very well be an expert stock trader, online campaign marketer, or transportation router – but that expertise is about performing tasks within the domain, not necessarily to think and reason about the domain. This is analogous to the difference between doing a left turn on a bike, or to explain how to make a left turn.

To put it another way, most domain experts are not consciously aware of their expertise. Of course, they probably are aware that they are experts – but they are not aware of what makes them experts – the same way most people don’t know why they can make a left turn. So, asking them to give a model will probably just give you a “first order approximation” or a naïve model that will hold for “typical” cases – but will probably break down when things are getting a little bit trickier. Key phrase for spotting this is: “Oh, well, of course, if the shipping organisation [whatever] is actually the ordering organisation [whatever], then there of course shall be no transit-through fee [obviously not …]”.

At one project I entered there where some severe confusion on the tech side of what went into a “notification” as opposed to an “announcement”. Different tech people had talked to the domain experts and this had resulted in inconsistent requirements. The approach to “ask the expert” had been used several times, but the confusion had only gotten worse. After a while I realised that the domain experts where not aware of the difference themselves – to them the two nouns where more or less synonyms and used interchangeably, because the precise meaning was obvious by context. They where perfectly able to handle any “notification” or “announcement” but could not verbalise a precise distinction between the two – that took some serious domain modelling to work out.

The domain experts are experts in the domain, not experts in modelling the domain. Domain modelling is a separate skill set, and that is why they need you to help them out. They simply cannot do it themselves. So, the proper way is not to go “ask the domain expert”, it is to go “work with the domain expert” to see what domain you can come up with in mutual cooperation.

Naturally there are exceptions. If a domain expert has spent considerable time and effort thinking about the domain, they might well have reached a knowledge depth that enables them to model the domain. To spot that kind of domain expert, keep your eyes open for signs that he or she has spend explicit efforts on verbalising the domain. The expert might have e g worked as an instructor in the field, given presentation for larger auditoriums such as at conferences, or might have written articles or books on the subject. If you spot such an expert; grab, hold on, and do not let go – this is a gold mine.

The second misconception in “model in expert’s head” is even more fundamental. Central to domain driven design is the attitude that models are evaluated by their usability (as in “fit for purpose”). A model could in a philosophical meaning be “true” or it might “cover a lot of the world” – but those properties are simply uninteresting. What counts is if the model can handle the cases you have at hand, and doing so in a simple way.

In other words, for the set of behaviour (“stories”, “use cases”) we have, there are an infinite number of models. Of those, none is “the right”. On the contrary, there might well be an innumerable number of models that all give the correct behaviour, and the way to discriminate between those is to find those that seem simple, in other words using Occam’s Razor.

It is logically possible that there might be several domain models that all fulfil the correct behaviour, and where none of them can be said to be simpler that another – they are all simple, but in different and incomparable ways. Then, any of these will do as a good domain model. So it is not logically necessary to search in the head of the expert. Instead let us investigate a lot of options (possible models) and select from those.

This also sets the tone of how I prefer to work with the domain experts. Instead of focusing on one model, I try to come up with several optional models that I see would fulfil the requirements, and then ask the domain expert for gut feeling. Best way to do this is a few people in a small room with a whiteboard with a lot of “generate and test”, i e find up a few models, ask for opinions, try them out on some scenarios, watch the reaction of the expert, throw away those models that where less promising, come up with some variants of those we kept, etc.

To know how long to continue this evolutionary modelling, Eric Evans has a rule of thumb I like: “keep modelling until you have created three really bad ideas”. As long as you just create good models you are probably not creative enough. It is the generation of bad ideas that prove that you have been pushing the borders.

In conclusion: I do not think there is a domain model in the head of the expert. Firstly, it is improbable because the domain experts are in general not aware of their expertise. Secondly they are not skilled modellers (their expertise is elsewhere), so the model you get by asking is often very naïve and incomplete. Thirdly, even if there would be a model there, that model has no advantage over any of the other possible models that fulfils the same purpose.

So, the way to communicate with domain experts to create a domain model is not to “ask them” it is to “work with them”.

Yours

Dan

ps When asking if someone "knows" something, if you then require them to understand the subject in depth, then most people do not know how to ride a bike.

Most People do not Know How to Ride a Bike

Dear Junior

Of course most people can be claimed to know how to ride a bike. The proof is simple: give them a bike and watch them without instructions mount onto it and take a ride without falling of – even in tricky conditions such as bumps or on slippery surface. However, the notion of “knowledge” is a little bit trickier than that, and a too “binary” notion of knowledge hurt us.

For example, take anyone of those that “know” how to ride a bike, and ask them to explain how to make a left turn. Chances are high, that you either will get no answer at all, or an answer that if followed will make you fall over - which in a pragmatic sense can be judged to be “wrong” or at least “incorrect” (did not cause the desired effect).

In the same way I can ask about myself: do I know “Java exceptions”, “EJB 3”, “agile system development” or “project management”? And when reading CVs I am regularly frustrated over just being given a long list of acronyms and frameworks; a list that tells me nothing about whether this person has been to a one-day class on Hibernate, or if she can wrinkle out modelling mistakes that will hurt performance.

For my own use I like to view knowledge not as binary (“know” vs “not know”), but rather as having levels of “depth of knowledge”. A main source of inspiration is Benjamin Bloom’s taxonomy. He defined six levels of “cognitive learning” (knowledge, comprehension, application, analysis, synthesis, and evaluation).

However, when applying this to system development, I usually collapse the three deepest levels (analysis, synthesis, and evaluation) to one, ending up with four levels. The reason for doing this is that I seldom see the need to distinguish between the deepest three, but often want to point out the difference between the first three (knowledge, comprehension, and analysis) in relation to the four latter stages. I guess a cynic might point out the choice of collapse as an observation of the state of our industry.

One thing I really like with Bloom’s taxonomy is its focus on utilisation of the knowledge. It quantifies by measured in the abilities to process and use information in a meaningful way, simply by telling what you can do at a given level. Note here “utilisation” and “measure” – two words that strongly align with my affection for pragmatism in software development.

With each level there are a couple of typical verbs attached – things a person is supposed to be able to do at that level. And this gives something that is measurable in a testable fashion.

So, the four levels I use with some typical verbs are

• Recognition (renamed: originally “knowledge”): recite, list, label, locate
• Comprehension: restate, give example, summarise
• Application: apply, solve, show
• Analysis (including Synthesis and Evaluation): compare, compose, invent, critique, evaluate, judge

Let me apply these levels to myself and some areas of knowledge just as examples.

So, do I know the Java memory model? Well, I follow along when my colleague Tommy talks about it, but I am not sure I could restate it without serious mistakes; so at recognition level yes, and perhaps at comprehension level, but not deeper.

Do I know Scrum, Java exceptions, or Domain Driven Design? Hey – how long do you want me to talk about it? I guess I could spend endless hours in discussion on those topics with other experts – so I would say analysis level knowledge.

Do I know JPA 2.0? Sure, I could solve a problem by using it, but I would probably fail when people start discussing trade-offs and what is happening “under the hood” in different implementation; in other words application level knowledge.

As a side note I would say the levels are logarithmical (like the Richter Scale), so that each level of depth takes ten times amount of thoughtful experience to reach, compared to the previous. You can learn a buzzword by listening to a “sales-level” presentation of one hour. To be able to restate it yourself, you probably need to spend a day (10 h). Before you can practice it without checking up details all the time, you probably need two-to-three weeks experience (100 h). And the level where you can compare and contrast different approaches will probably take half a year (1000 h). Of course it is the two latest levels that are most interesting and important to system developers.

Back to the bike riding: recognition would be “yeah – bikes are a kind of vehicle – I see one over there!”; comprehension would be “you get onto it, pedal to get speed, steer by pulling the handles, and break by pedalling backwards”; application would be: “let me show you – watch me!”.

And analysis level? Well, if you want to turn left you should not pull the left grip – that will cause you to tip over to the right due to the centrifugal force you expire when the bike starts turning left. In short – the bike will turn left, but you and your centre of gravity will continue forward; so you fall into the ground on the right hand side of the bike.

And, by the way: centrifugal forces do exist.

To actually turn left you first pull the right handle. The effect is that you start falling over to the left. After a split of a second you have fallen the appropriate amount so that your angle towards the ground will make gravity and centrifugal force balance for the turn you intended. Then, you pull the left handle to actually do the left turn. And as you now have the appropriate angle, forces balance out and you stay on top. People trying to explain this often express it as doing a small “contra-turn” in the opposite direction as a preparation before making the “real” turn.

If you want to see this in action, have a friend film you while just doing some cycling around on the backyard, and then watch it frame by frame. You will see that you intuitively do the right thing – but you probably had no idea you did.

Note that for most of us there is no need for analysis level knowledge on bicycling – we are perfectly happy to just be able to ride that bike. The analysis level knowledge of bike riding you find at people like motorcycle driving instructors or those that just have studied a little bit too much analytical mechanics.

The same goes with people in the system development field. Just doing stuff and gaining “experience” does not give analysis level knowledge. To there you also have to have an analytical process in place that constantly evaluates what you do, why you do it, and what the alternatives could be.

Culture that analytical process.

Most people do not know how to ride a bike – they simply do it.

Yours

Dan