Technical Details as Part of Domain Model

Dear Junior

A classical questions in Domain-Driven Design is whether technical details should be part of the domain model. The usual answer is "no, there should only be business concepts", but I would like to refine that a bit.

One of the drives of Domain Driven Design is obviously to drive development from a design based on the domain - hence the name. So, concepts as "database", "table", "network" and "connection" should typically not be part of the domain and its language.

This kind of technical details does not make sense per se in the domain - they add nothing do the direct understanding. However, databases and network and other technical details might have indirect effects that that we need to understand. The tricky part is to find appropriate abstractions for that understanding.

Let me take "buying a laptop" as an example. Recently I bought a new laptop from the Apple site. This is the "laptop to buy" model, and when doing this I filled out an order form, which is what constitutes the "laptop to buy" domain model.

I know that a laptop is a pretty complex thing, so the "laptop to buy" domain is a really rich one. Some of my hardware freak frieds could talk to lengths about it. As I am not much of a hardware freak myself, I prefer if the "laptop to buy" domain model is as simple as possible - without "technical details" that I am not interested in.

For example, the laptop I buy have a hard-drive, which is made by a manufacturer. I happen to know there are numerous subtle details that differ between hard-drive brands - but none that I care of. It is an uninteresting technical detail, and I do not want to see the choice of hard-drive brand in the domain of "laptop to buy".

Luckily for me, Apple agree, and there is no option for choosing hard-drive brand that makes the order form more complex and harder to understand.

Another technical detail is that the hard-drive have a lot of sectors and blocks, and the number of sectors and blocks differ from hard-drive to hard-drive. This is also something that I do not wish to care about - I just want to stuff my documents, photos, and film clips onto the laptop and it should just work. I do not want this to be part of the "laptop to buy" domain model either.

Unfortunately, Apple disagrees here. They think I need to be aware of that the hard-drive consists of storage cells, and that these actually are limited. Otherwise, I would live in the illusion that I could put things on my laptop indefinitely. That illusion would break down when there suddenly are "no space left on device" - an event which would leave me utterly confused.

In order to make me understand the nature of how it is to work with a hard-drive, I need to be aware of the finite number of sectors and blocks - although it is a "technical detail". But, unfortunately, it is a technical detail I need to be aware of.

Now comes the hunt for the appropriate abstraction to make the blocks and sectors understandable to non-hardware-technical me. "Blocks and sectors" does probably not work very well, but the concept of "size" does. I get that we are not talking about physical size, but some kind of analogy to the size of a suitcase. A small suitcase will become full fast, but you can put more stuff into a larger suitcase before it gets full. The size is still a "technical detail", but it is a detail that cannot be ignored, and it is put in a form I can understand.

Thus Apple has chosen to have options in the order form for how big a hard-drive I want, making it a part of the "laptop to buy" domain model. 

And I can agree with Apple.

When considering whether a concept should be in the domain or not, it is really not interesting whether it is a "technical detail" or not. The interesting discussion is whether this is a concept which the user (or other stakeholder) needs to be aware of. We cannot take away concepts that will make their understanding "break down". And for those technical details, we have to find an appropriate abstraction.

This reminds me of an Einstein quote: "Things should be made as simple as possible, but not simpler" which I think is an excellent way to put Occam's Razor.

So, technical details should not be part of the domain if they are irrelevant to the business. However, it might be that the technical detail is a concept that you need to understand to comprehend how things work. And in that case, it should be part of the domain, even if it stems it origin from the technical side.

Yours

   Dan

Joint Venture, Rather than Customer-Provider

Dear Junior

When a customer wants something from a provider, she places an order — and after a while there shows up a delivery. This works wonderfully well for a lot of situations. For example I needed a new laptop, so I went to the on-line Apple store. There I could configure how I wanted my MacBook (fast processor, no extra memory, and a solid state disk, please), and push the "order" button. About a week later there showed up a package at the office; enclosed was the computer - to specification. So the customer-provider system with order and delivery works in a lot of situations. Unfortunately, system development is not one of them.

The Customer-Provider Role-Play

Nevertheless, it is quite popular to let developers and business engage in a "role-play" where the business people are denoted the role of the Customer - often "because they pay". And as a contrast, the IT department take on the role as the Provider. I understand that these roles can come into play when business and IT actually are different companies.

However, it is really interesting to see this role play when both sides actually are departments within the same juridical entity. This makes the "they pay" argument kind of suspicious. In those situations we are not customer and provider. Rather we have chosen taken on those parts of the role-play, as a metaphor.

Hopefully we have made a careful analysis when deciding to use this metaphor. I think that in most situation, this is not the case.

Hopefully we have decided that this metaphor will help us to promote and guide valuable behaviour. I think that in most situations, it does not.

Customer Metaphor not Helpful

To me it seems there are two main reasons that the customer metaphor does not help us much.

First, the metaphor is so heavily loaded with associations and "wisdom" that our thoughts and behaviour is led astray. For example, we have all heard "the customer is always right". Unfortunately, this often creates a mental barrier to go and ask the business people.

On a similar tone, I have on several occasions heard programmers defend doing a bad job by "I only give the customer what they asked for" - when the proper thing to do would have been to stand up, walk away 25 meters and ask the business representative if the effect we have realised, but they probably not anticipated, really is intended or desirable.

Customer-Provider Misses Information Exchange Intensity

The second reason that the customer-provider metaphor does not help us is that I think it is fundamentally broken. If we study the information exchange in the metaphor, there is first one information transfer from the customer to the provider - i e the order. Then nothing happens, until delivery. Then there is a bulk of information going in the opposite direction - the delivery with all the details of the solution. In my experience, the information exchange of a highly successful system development initiative does not follow this pattern at all.

In my experience the key aspect of system development is knowledge generation. This journey is the joint endeavour where we combining the tactic ("able-to-do") domain knowledge of the domain experts with the expertise of creating mathematically precise logic systems of the programmers. In system development those two competences combine to create something new, that did not exist before. Together they create a precise understanding of a selected aspect of the problem at hand. In Domain-Driven Design terms, this distilled and precise domain expertise is captured in the domain model.

The information exchange when distilling the model is very intense, and is far from suitably described by one "order" followed later by one "delivery". I think the Domain-Driven Design focus on creating a shared, precise understanding of the problem domain very well captures the feeling of how much cooperation it takes.

Of course, we can "patch" the Customer-Provider by allowing (and encouraging) the provider to repeatedly ask the customer for clarifications and explanations. The other way around we can let the provider make delivery in parts - incrementally and iteratively show results en route. This is better, but I think it still misses a central point, the feeling of solving a problem together.

Customer-Provider Misses Mutual Initiative

Another way to view the problem with the customer-provider metaphor is to look at where the initiative lies at any given point of time. In the metaphor, the initiative is first at the customer. The customer will do whatever preparations needed to complete the order. During this process, the customer might interact with the provider, but the initiative to drive the process forward remains at the customer. This remains until the order is sent over.

After the order, the initiative goes to the provider, who will start working. At this point of time, the provider might reach out to the customer to discuss issues and ambiguities, however the initiative remains at the provider. 

Finally, upon delivery, the initiative is once again transferred to the customer, who will verify the delivery, accept it, or file deficiency reports.

Characteristic for the metaphor is that at any given point, it is clear who has the initiative. During this time, the other part can comfortable sit back and rest assured that it is "the other's" responsibility to pick up the phone.

System Development Requires Mutual Proactive Involvement

In system development, I want both sides to be equivalently engaged and involved in the work all the time. Creating a strict model a la Domain Driven Design is a creative and collaborative work that requires both sides (business and IT) to feel the responsibility to take the initiative whenever they are struck with an idea (often late at night) and ensure that every new insight or idea is brought into the common work. At no time can either side lean back and rest assured that "I am not responsible at the moment".

End of Customer-Provider

So, can we please put the Customer-Provider metaphor behind us? It does not really help us in guiding our ideas, thoughts, or behaviour in any direction that is well aligned with the agile values of modern system development. If we absolutely need a metaphor, we must look for another one.

On a side note, Swedish agile front figure Marcus Ahnve pushes the idea that it is time for us as a line of industry to stop working by metaphors. Instead of saying "system development is like construction" or "is like product manufacturing" we should start talking about system development on its own merits. I agree that it is about time - actually my grandfather could have been a programmer, and I could have been the father of some young developer in the profession today. On a similar tone, Eric Evans has also mused on the subject under the tittle "Software is (Not) Like That". I second both of these thinkers that we need to be careful with metaphors, and we would be better of without metaphors that with bad ones.

Joint Venture as a Better Metaphor

However, if we necessarily must metaphors at all, and if we insist on picking them from the business world, then I would suggest another one. I would suggest one where the information exchange is more intense, where both sides have the initiative all the time and where booth sides care about the result they create in common. I would suggest using Joint Venture as a metaphor instead.

In a joint venture two parties decide that they should cooperate over some common interest and work towards some goal that is out of reach for either of them. A good example is the cooperation between Sony and Ericsson to create a mobile phone that was also a nice piece of consumer electronics.

Sony and Ericsson did not solve this by Sony putting in an order to Ericsson to build a mobile phone according to their consumer-electronics specification. Neither did Ericsson put in an order the other way around. Instead they started a new company, SonyEricsson, where each part owed 50% of the stock of shares. In this shared company, both parties invested by sending some of their best engineers, and both parties committed to supporting the joint venture so that it would bear fruit. Which, by the way it did. I think this situation better resembles the interactions and caring that we experience in a well functioning system development initiative.

Bear in mind; we still use "joint venture" just as a metaphor. If someone wants to actually do business this way I will be really interested to sit by and watch. But I will probably not line up to invest. However, used as a metaphor, I think joint-venture would be more helpful that customer-provider.

And being helpful is all I ask from a metaphor.

So, at Monday morning, we might as well stop referring to the business people as our Customers, and instead refer to them as our Partner. It might actually make a difference.

Yours

   Dan


ps I did present a blitz presentation on this idea at the Agile Sverige 2010 conference at which the use of language and metaphors was a theme that emerged.

The Domain Model is not in the Head of the Expert

Dear Junior

Among programmers it is not uncommon to get into discussions on naming of variable, methods, classes etc. A pretty common way to resolve these discussions is to say “let’s ask the domain expert”. Even though I encourage communication across disciplines, there is a sub-tone that alarms me: the sub-tone of “checking the answer”, like if the proper domain model was in the head of the domain expert, and the task of the programmers is to uncover it and write it down in code. I think this surfaces two different misconceptions about domain modelling in general as understood by domain driven design.

The first misconception is the kind of expertise the domain expert has. He or she might very well be an expert stock trader, online campaign marketer, or transportation router – but that expertise is about performing tasks within the domain, not necessarily to think and reason about the domain. This is analogous to the difference between doing a left turn on a bike, or to explain how to make a left turn.

To put it another way, most domain experts are not consciously aware of their expertise. Of course, they probably are aware that they are experts – but they are not aware of what makes them experts – the same way most people don’t know why they can make a left turn. So, asking them to give a model will probably just give you a “first order approximation” or a naïve model that will hold for “typical” cases – but will probably break down when things are getting a little bit trickier. Key phrase for spotting this is: “Oh, well, of course, if the shipping organisation [whatever] is actually the ordering organisation [whatever], then there of course shall be no transit-through fee [obviously not …]”.

At one project I entered there where some severe confusion on the tech side of what went into a “notification” as opposed to an “announcement”. Different tech people had talked to the domain experts and this had resulted in inconsistent requirements. The approach to “ask the expert” had been used several times, but the confusion had only gotten worse. After a while I realised that the domain experts where not aware of the difference themselves – to them the two nouns where more or less synonyms and used interchangeably, because the precise meaning was obvious by context. They where perfectly able to handle any “notification” or “announcement” but could not verbalise a precise distinction between the two – that took some serious domain modelling to work out.

The domain experts are experts in the domain, not experts in modelling the domain. Domain modelling is a separate skill set, and that is why they need you to help them out. They simply cannot do it themselves. So, the proper way is not to go “ask the domain expert”, it is to go “work with the domain expert” to see what domain you can come up with in mutual cooperation.

Naturally there are exceptions. If a domain expert has spent considerable time and effort thinking about the domain, they might well have reached a knowledge depth that enables them to model the domain. To spot that kind of domain expert, keep your eyes open for signs that he or she has spend explicit efforts on verbalising the domain. The expert might have e g worked as an instructor in the field, given presentation for larger auditoriums such as at conferences, or might have written articles or books on the subject. If you spot such an expert; grab, hold on, and do not let go – this is a gold mine.

The second misconception in “model in expert’s head” is even more fundamental. Central to domain driven design is the attitude that models are evaluated by their usability (as in “fit for purpose”). A model could in a philosophical meaning be “true” or it might “cover a lot of the world” – but those properties are simply uninteresting. What counts is if the model can handle the cases you have at hand, and doing so in a simple way.

In other words, for the set of behaviour (“stories”, “use cases”) we have, there are an infinite number of models. Of those, none is “the right”. On the contrary, there might well be an innumerable number of models that all give the correct behaviour, and the way to discriminate between those is to find those that seem simple, in other words using Occam’s Razor.

It is logically possible that there might be several domain models that all fulfil the correct behaviour, and where none of them can be said to be simpler that another – they are all simple, but in different and incomparable ways. Then, any of these will do as a good domain model. So it is not logically necessary to search in the head of the expert. Instead let us investigate a lot of options (possible models) and select from those.

This also sets the tone of how I prefer to work with the domain experts. Instead of focusing on one model, I try to come up with several optional models that I see would fulfil the requirements, and then ask the domain expert for gut feeling. Best way to do this is a few people in a small room with a whiteboard with a lot of “generate and test”, i e find up a few models, ask for opinions, try them out on some scenarios, watch the reaction of the expert, throw away those models that where less promising, come up with some variants of those we kept, etc.

To know how long to continue this evolutionary modelling, Eric Evans has a rule of thumb I like: “keep modelling until you have created three really bad ideas”. As long as you just create good models you are probably not creative enough. It is the generation of bad ideas that prove that you have been pushing the borders.

In conclusion: I do not think there is a domain model in the head of the expert. Firstly, it is improbable because the domain experts are in general not aware of their expertise. Secondly they are not skilled modellers (their expertise is elsewhere), so the model you get by asking is often very naïve and incomplete. Thirdly, even if there would be a model there, that model has no advantage over any of the other possible models that fulfils the same purpose.

So, the way to communicate with domain experts to create a domain model is not to “ask them” it is to “work with them”.

Yours

Dan

ps When asking if someone "knows" something, if you then require them to understand the subject in depth, then most people do not know how to ride a bike.

Domain Driven Security and Making Stuff Explicit in the Model

Dear Junior

”But ’ OR 1=1 -- is not a valid username! That is just bad indata validation!”. Well, ‘ OR 1=1 -- might not look like the kind of username we had in mind, but invalid? Says who?

If we have a look at the code, the signature of the authentication method says:

Integer authenticate(String username, String passwordMD5)

Basically, in the code there is nothing saying that username is any special kind of data – it is just a string. And, as such, it can be any string – including ’ OR 1=1 --.

There might be conventions, even documented such, that a username should have certain structure – but the model represented in the code consider any string to valid to send into the method.

The Domain Driven Design take on this is that if you have more restriction in your intended model, then you should better put those restrictions in the code – explicitly.

So, let us take a small step in that direction – let us make Username an explicit part of the model. Later on we can elaborate that part of the model by making restrictions on usernames explicit, and even enforcing them. But let us not take too big a bite – for now we settle for shaping up the model.

If we think about it we can surly agree that username is a special kind of data, separate from amounts, order numbers, or phone numbers. It would simply not make sense to have a phone number “+4615210000” used as a username. This is analogous to the distinction between int and boolean. Under the hood they are both “just bits and bytes”, but we want to keep them distinct in our language so that we do not accidentally use an int as the condition in an if-statement, for example. In C many hard-to-find bugs have been caused by that specific mistake.

In static typed programming languages like Java, C# or ML, we use the type system with interface and classes to separate different kinds of data. However, if we audit the authentication code we will see that there is no representation of username on that level. The only place “username” show up is as the name of a String-typed variables and parameters. The knowledge “username is a specific kind of data with its own rules and restrictions” is not explicit in the code.

Enter class Username, which at this stage might be the simplest kind of value object.

public class Username {
  public final String username; // final making it immutable
  public Username(String username) { this.username = username; }
}

The important part here is of course that we now have a new type, which can be used by variables, fields, parameters, and returns to make the code explicitly talk about usernames.

The authentication method will change somewhat.

  /** Authenticates a user with a given password.
   * @param username
   * @param passwordMD5 hash of password
   * @return user id, or null if no matching account
   */
  Integer authenticate(Username username, String passwordMD5)
          throws SQLException {
    Connection con = accountDs.getConnection();
    Statement stmt = con.createStatement();
    String sqlSelect = "SELECT uid FROM Accounts";
    String usernameMatch = "username = '" + username.username + "'";
    String passwdHashMatch = "passwdHash = '" + passwordMD5 + "'";
    String sql = sqlSelect +
            " WHERE " + usernameMatch +
            " AND " + passwdHashMatch;
    ResultSet rs = stmt.executeQuery(sql);
    Integer result;
    if(rs.next()) { // found account with matching password
      result = rs.getInt("uid");
    } else { // no matching account
      result = null;
    }
    return result;
  }

So, whoever wants to call the authentication method with a username, must first create a Username object via the constructor.

public class LoginAction {
  void doit() throws SQLException {
    Username username = new Username(form.username);
    String passwordMD5 = form.password;
    accountService.authenticate(username, passwordMD5);
  }
}

Now the concept of username is explicit throughout the code, and actually talks the same language as the people working with it. In effect, we have made username a part of the ubiquitous language talking about the system.

Note that we are still not yet protected from bad usernames, that will be a later step - but at least we talk about usernames, not strings.

The distinction between username strings and usernames is subtle. This distinction might seem small, but I think it is essential – as the language form how we think. The moment the programmer start expressing herself in domain terms (creating a Username object), chances are higher that she will also question the indata parameter: Is this string really a username? Where did it come from? Has it been properly checked? No guarantee, but chances are higher.

We still have some way to cover before we have an API that is both easy to use correctly, and hard to use incorrectly – but at least we have taken a step in that direction. We still lack the constraints on usernames, and there is no enforcement at all.

However, if we can guide the programmers into thinking about the model a la DDD, and thus decrease the risk of severe application security flaws, then we have at least done something useful.

And it is usefulness that is the ambition of Domain Driven Security.

Yours

Dan

PS My colleague John Wilander just published a nice example on how they did with Swedish "person number" (roughly social security number) [Swedish]

Domain Models, Data Models, and How They Resemble Each Other

Dear Junior

In what ways domain models and data models should resemble each other is a really interesting topic. It is not uncommon for me to ask for what a “Foo” and a “Bar” really are and what their relationships are and upon that question being answered by the architect who with a smiling face show me the database schema, complete with join tables and everything. That was not what I had in mind.

So what is the difference between a database data model and a DDD domain model? Why do I care?

For me, it basically boils down to the purpose of the respective model. The purpose of the database data model is to provide the system with an efficient storage for persisting data, and for searching same data. The purpose of the domain model (a la Domain Driven Design) is to work as the focal point and a vehicle for communication about the system. In the database model you might have redundancy and pre-aggregated data for efficiency. In the domain model you try to make each concept and each relation as clear and unambiguous as possible.

True, in some places the database model is used as the domain model, and even the business side domain experts have learned to read the schemas. My experience is that even though this might work, for example by using ER-diagrams instead of the actual tables, it has its limitations. First, people tend to forget the distinction between the concepts per se, and their realisation in the database. So, if you want to re-conceptualise (which is good to try from time to time) you might be struck with counter arguments such as “but that will violate the constraints in the triggers”. Secondly, a database data model must by necessity cover every single piece of persistent data – however marginal its importance. To do strict modelling of a concept is a quite brain-expensive operation, thus it is simple waste of precious resources to do so on every minute detail. To get leverage out of your modelling efforts it usually suffices to concentrate on the most central parts. Thirdly, a database model will leave out data that is not persistent, or is extracted or imported from other sources - and those might well be very central to the conceptual understanding of the problem at hand.

For these reasons, I simply advice to do a separate domain model, even if you have a complete database model in place. They simply facilitate different kinds of discussions.

So, how much should these models resemble each other? Well, let us say that we have established that the domain model is the focal point of discussion. Further on, the domain model has grown out of a discussion where several types of input and considerations have been taken into account: how the customers of the product (e g end users) think, standards within the domain, GUI experts, and database experts. All those inputs are needed to give a model that fulfils its purpose – to enable us to build a good system. In that way the domain model is the result of a negotiation between several parties with the goal of meeting each party’s need as well as possible. So, as the database expert has his or her say in that discussion, there should be some level of resemblance.

But, how much should they resemble each other? Well, if the domain model is the common and central model, then each party will need to shrew it slightly to realise it into its context. The programmers might need to translate concepts to a different language, to make working code. The interface designers might need to lay components in the design in a slightly different way. And, the database designer might need to add an aggregation of data for efficiency, or add a join table to make ends meet. However, any unmotivated difference should be considered a technical debt and a target for a refactoring effort.

For the motivated adaptations, the burden of maintaining the respective mapping falls back on each party. If we need to reconceptualise and change the domain model, then each party will need to do both that change and the changes in their adaptations. So, in order to survive each party should keep the adaptations to the minimal motivated level.

Domain models a la Domain Driven Design and data models for database persistence should in a sound system resemble each other. But, they are not the same as they fulfil different purpose, and should not be confused with each other.

Yours

   Dan

PS When trying to keep down the semantic difference between the domain model and the data model, you probably want to refactor the database continuously - something that gets a lot easier if you version your database.
PPS A related question is to what extent technical details are allowed to be part of the model.
PPPS Domain Driven Design is in general a large and interesting field in and of itself.