Writing a book - Secure by Design

Dear Junior

I'm sorry for not writing to you for quite some time. But, you see, I have been working on a book together with my two colleagues and friends Daniel Deogun and Daniel Sawano. Late this summer we decided to try to write down our collective experiences and insights in design and security as a book. We are far from finished, but by now the first few chapters are released from the publisher Manning as part of their "Manning Early Access Program" (MEAP).

The book is named Secure by Design and our main message is how good design and good code can help developers avoid security problems. We have found that lots of security vulnerabilities are due to bugs that could have been avoided if the design of the code had been different. 

You as well as I know that it is a lot easier to think about "good design" when developing features, that it is to think "security" at the same time.

Of course not all good design lead to avoiding security vulnerabilities. So we have gathered our experience on what design-tricks that are most effective in giving security as a side-effect. It's probably no big surprise to you that Domain-Driven Security is an important part of this, but there are many other parts as well.

We still have a lot of material to cover. But, what we have started with things that are close to the code, such as building Domain Primitives, how to structure validation, immutability etc. I guess you will find several of these ideas familiar, as we have discussed them earlier - but there are also some new insights I have not yet had the time to write to you about. Well, not to mention all the ideas from Daniel and Daniel which you and I haven't had possibility to reflect on earlier.

In the later parts there will be more material on integration, security benefits from cloud thinking. We also want to share our ideas around architecture such as security concerns for microservice architecture or how to handle legacy codebases.

We will continue to write during the spring and early summer. Hopefully we will see the book in print sometime during the fall this year.

Yours

   Dan

PS The early-access was released last night; you can check the book at https://www.manning.com/books/secure-by-design

Make Concepts Explicit when Fixing Bugs

Dear Junior

To find implicit concepts and make them explicit is a very powerful way to improve code. But, sometimes it is hard to judge beforehand which concept is important enough to make explicit. We try to make good judgements, but sometimes we miss.

But, no reason to despair, we can take Time as our helpful aid. So, instead of everything perfect from start, we continuously perfect it whenever we find a reason. Two of the topmost reasons are either a bug appearing or further development of the code.

Let me for now fokus on the case of when a bug appears. If the code does not behave the way we intended it to do, then there is most often a place in the code that was to convoluted. In that convoluted state, subtle misunderstandings and simple mistakes could hide. The solution is to clarify the code until it is obvious whereof the mistake consists.

Things get clearer by examples, so let me clarify what I mean with a very minimal example, the story about a bug and its fix.

In real life, more or less a decade ago, I worked on a project where we did a batched import of records coming from an external source. The code looked roughly like this following in the class ImportCron.

public class ImportCron {

    private RecordImporter importer;

    private Logger logger;

    void runImport()  {

        List records = importer.listWaitingRecords();

        logger.log(Level.INFO, "Importing records: " + records.size());

        for ( Object rec : records) // imports each of the records in turn

   …

    }

Now, the import was run once a minute, but most of the time there where non records waiting.  Problem was the import log became filled with rows saying "Importing records: 0" so we had to grep out the non-zero lines each time we wanted to look at what was happening.

One of those days I got tired of this, fired up the editor, and wrapped the logging in an if statement to filter out all those zero-import writes.

    void runImport()  {

        List records = importer.listWaitingRecords();

        if (records.size() == 0)

            logger.log(Level.INFO, "Importing records: " + records.size());

        for ( Object rec : records) // imports each of the records in turn

   …

    }

A few days later we released during lunch, which was the best time for our users. The deploy went smooth, apps started up as expected, and soon it was time for the first import "tick".

Within a minute, import was run the first time, and the log line read:

<timestamp> Importing records: 0

While standing confused, a minute went by, and the log line read:

<timestamp> Importing records: 0

Another minute went by … and no log line.

While firing up my editor to check the code, another minute went by and a third log line appeared:

<timestamp> Importing records: 0

A colleague of mine looking in the database calmly reported "We just imported three records".

It only took me a split of a second to realise my mistake once I had the code in front of me.

    void runImport()  {

        List records = importer.listWaitingRecords();

        if (records.size() == 0)

            logger.log(Level.INFO, "Importing records: " + records.size());

        for ( Object rec : records) // imports each of the records in turn

    }

Obviously my spine reflex for coding "check for zero" made me write the boolean expression the wrong way around. By the way, did you capture that mistake at first glance in the code on my first description? Confirmation bias is a nasty thing.

Now, I must say this kind of bugs are pretty uncommon, bugs that are typos or simply mispunching the keys. Most bugs in my opinion are rather that pieces of code subtly misunderstand each other.

Well there are at least two ways of fixing this code. The obvious, and fastest would be to quickly change "==" to "!=". However, humbled by the mistake I had done in the first place, I realised that that kind of hasty coding was what got me in the trouble in the first place.

One of my coding mantras since long have been "Code should mean something, not just do something". So, a better way out would be to make the code more meaning-ful. From Eric Evans I learned the phrase "Make implicit concepts explicit", which says the same thing in this context, but gives a better guiding direction forward.

I took to the challenge of fixing the bug by finding what implicit concepts had been missed, and making them explicit until it was blatantly obvious that the code was wrong. An added benefit would be to be able to make a test proving the code was wrong.

Extracting the boolean condition to a method of its own would force me to spell out the meaning of that piece of code.

    void runImport()  {

        List records = importer.listWaitingRecords();

        if (containsRecords(records))

            logger.log(Level.INFO, "Importing records: " + records.size());

        for ( Object rec : records) // imports each of the records in turn

    }

    static boolean containsRecords(List records) {

        return records.size() == 0;

    }

Granted, this is more code than I started with - but I think the code is more "to the point" (phrased inspired by Rickard Öberg, another of the great programmers).

At least this made me able to write a test

    @Test

    public void shouldConsiderEmptyListNotContainingRecords() {

       Assert.assertFalse(ImportCron.containsRecords(Collections.EMPTY_LIST));

    }

Well, at least this is what the test would look today - at the time JUnit looked slightly different.

Anyway, now I have a failing test. Claiming "the empty list does not contain records" is simply false - as proven by the test. We safely update the code to fix the bug.

    static boolean containsRecords(List records) {

        return records.size() != 0;

    }

Upon which the test switched to green as expected.

During this "refactor -> put under test -> fix" something interesting has happened. The concept "import list contains records" that hitherto had been implicitly represented by the technical "records.size() != 0" has now been made explicit and given a name "containsRecords".

It might be claimed that the main benefit in this example was derived from the drive to put code under test before making a change. Undeniably, that is a point, but I think it only tells half the tale. 

Simply putting things under test can be done in a myriad of ways, and I have seen several very technology-based efforts. I do not think those efforts have paid off handsomely. Mostly the code get cut along technical lines to push in tests in the gaps. But, the code does not get more comprehensible over time.

Focusing on "make implicit concepts explicit" have been a more productive course for me when working with code.

Yours

   Dan

ps My team-mates did not force me to skip lunch to fix the bug. We went out together, and when we came back satisfied and rested, I sat down and fixed the code. We released the fix during lunch-time the day thereafter.

Make Implicit Concepts Explicit in Code

Dear Junior

In our letters on software development we have touched upon the idea several times, but I think it might be worth spelling it out - the idea to make implicit concepts explicit; something I see as one central message of Domain Driven Design applied to coding.

Let me pick that phrase apart for a moment. When coding we have a lot of concepts in mind, e g when writing code that handles people I might have a class namned Person. In this case the concept of person has an explicit representation in the code - i e an explicit concept.

A person might have a birth date, represented in code as a data-field Date dayofbirth in the Person class; another example of an explicit concept.

We might have business logic restrictions that are based on the age of the person, e g you have to be at least fifteen years to access some content. So there will be code calculating this.

    void contentRequest() {
        ... 
        boolean access = ((new Date().getTime() - customer.dayofbirth.getTime()) / msPerYear) >= 15;

Again we have a concept represented in code, this time age. However, this time the representation is not explicit - there is nothing in the code saying "age". Age is an implicit concept. 

There is nothing strange with having implicit concepts. In the short code snippet we have several implicit concepts: current point of time, point of time of customer's birth, and age-limit for content are just three obvious. This is not a problem. The programmer reading the code will intuitively re-construct the relevant concepts.

However, the design guideline from Domain Driven Design advice us to keep an eye open for important concepts - and if we find the represented implicitly, then make that implicit concept explicit. 

Let us look at the code snippet again.

        boolean access = ((new Date().getTime() - customer.dayofbirth.getTime()) / msPerYear) >= 15;

Of the implicit concepts dwelling in this like, which of them seem important enough to make explicit? 

The concept of "current point of time"? Probably not.

The concept of "milliseconds since birth"? Nah, not that either.

The concept of "milliseconds since birth expressed as years, rounded downwards"? Hey! I know this! We already have a word for it - we call it "age"! And we talk about it all the time!

Well, that sounds important enough. Let us start with giving that value a name. 

    void contentRequest() {
        ... 

        long age = (new Date().getTime() - customer.dayofbirth.getTime()) / msPerYear;

        boolean access = age >= 15;

By the way: this is where I really like the IntelliJ shortcuts "cmd-w" to widen the selection until the expression is selected, then "cmd-alt-v" for the refactoring to extract the selection as a variable - naming it "age" in the pop-up. It literally takes less than 10 seconds.

Now, we actually have made the implicit concept age into an explicit concept. Mission completed.

Well, not completely. We need to find the concept a good home where it can lead a good life. Right now it is stranded in the middle of some access computation. At least we can give it a method of its own.

Applying another of my favourite refactoring "Replace Temp with Query" yields this code.

    void contentRequest() {
        ... 
        boolean access = customerAge(customer) >= 15;

    private long customerAge(Person customer) {

        return (new Date().getTime() - customer.dayofbirth.getTime()) / msPerYear;

    }

Better, but can be improved to make the concept clearer. If we talk about the "age of the customer" it will vary from time to time, and it might cause confusion: the age when the customer requested access to the content, the age when content was first accessed, when it was last accessed, the age now? Better to clarify.

Of course we could rename the method to "customerAgeNow". However, I do not feel comfortable having my concepts depend implicitly on external things like the system clock. I prefer to make those dependencies explicit. So we make the time-in-question a parameter. 

    void contentRequest() {

        ... 

        boolean access = customerAgeAt(customer, new Date()) >= 15;

    private long customerAgeAt(Person customer, Date timepoint) {

        return (timepoint.getTime() - customer.dayofbirth.getTime()) / msPerYear;

    }

 

This design also improves testability drastically. The tests can now use explicit test-data, and need not rely on checking, or changing, the system clock.

Kudos once again to the lovely refactoring support of modern IDEs. Another less-than-10-seconds-refactoring: two cmd-W to select "new Date()", cmd-alt-P to extract as parameter, change name to "timepoint" in pop-up. Enter. Done.

By now it is pretty obvious that the method "customerAgeAt" suffers from feature envy. The most relevant data it operates upon is the Customer, but it resides somewhere else. We should consider moving it.

Should we move it, the Customer class would get a method "ageAt", taking the liberty to rename it slightly. That method certainly makes sense in this context, but does it so in other contexts? As we do  not have the rest of the codebase at hand, we will just have to pretend that "ageAt" makes sense in other context - and might actually be useful there as well.

This is really one of my favourite refactoring: move method. Again extremely smooth thanks to modern IDEs. Literally two keystrokes (F6, Enter - in IntelliJ) for the move, and a few for renaming. 

    void contentRequest() {

        // ...

        boolean access = customer.ageAt(new Date()) >= 15;

class Person {
    Date dayofbirth;
    // ...
    long ageAt(Date timepoint) {
        return (timepoint.getTime() - dayofbirth.getTime()) / TimeUtil.msPerYear;
    }
}

You might not believe me, but I remember the days of yore when you actually had to do all the involved steps manually; cut-n-paste the method body and header, change the list of parameters, update the code to use the fields of "this" instead of the argument, change all client calls (which might be several).

Now, we have finally ended up with a version I feel comfortable with. The concept "person" has now an conceptual attribute "age at" which has an explicit representation in code. In code we have "enhanced" our language of what we can talk about directly. So, when we discuss the age of a customer with business people, it is likely that we consistently mean the same thing - even if we for sure still can misunderstand each other.

As a side effect the code checking the access has become a lot clearer.

        boolean access = customer.ageAt(new Date()) >= 15;

This is a line of code that can be shown some person from the business side and explained by reading it out "access is granted if the customer's age at 'now' is at least fifteen years". Given that support, they can see by themselves - something that really builds trust. 

The time invested was not huge. Coding-wise all refactorings took less then a minute together. The time to slowly realise that "age" is an important concept in this particular domain is not counted. However, that insight will have to dawn on the programmer sooner or later, and when that happens, that extra minute is well invested.

To make concepts explicit does not only apply to code, it can be fruitfully applied to other areas as well, e g capturing and writing requirements/specifications. 

As we all stand on shoulders of gigants, I also must give credit where credit is due. The phrase "Make Implicit Concepts Explicit" I have picked up from Eric Evans, the thought leader of Domain Driven Design. 

In conclusion: to make implicit concepts explicit help us to over time close the gap between the code and the understanding of the business domain. It is not uncommon to in the process find bugs or crucial misunderstandings, that then can be adressed in a proactive manner instead of popping up as nasty surprises later on. 

Yours 

   Dan

The Three Elements of Agile

Dear Junior

Not that I necessarily claim to be an old hip-hopper, but an interesting part of the hip hop culture is the concept of "the four elements". 

• Breaking - more commonly known as breakdance
• MC - more or less what rap is about
• DJ - you know disk-jockey
•  Graffiti - public painting

An interesting thing is that within the hip hop community it is commonly agreed that all four elements are needed. If you should take away any of them, the hip hop culture would not be whole, it needs all four of them to be complete.

Also, all the elements are on the same level. That MCing should be "higher" than graffiti, or vice verse, is an idea that would be refuted as ridiculous. All the elements are of equal importance and each indispensable. 

Obviously, the individual hip-hopper might not practice all of the elements. He or she will most probably immense in one of them. A few might practice two or three. Some might practice all four but are certainly not expected to master all of them.

Still there are a lot of respect between practitioners of the different elements: a talented MC easily gives public cred to a vicious graffiti painter. They are both needed and they would both feel poorer without the other. 

The four elements must be in balance for the hip-hop culture to thrive.

Well, that sounds all nice and cosy. But what does that have to do with agile?

I think the Agile community have a similar situation. Cutting the cake with a blunt knife I see three elements within the "agile culture": tech, process and organisation. OK, the knife is blunt so the cuts might not be perfectly clear, but I think it suffice to clarify my point.

Agile Tech consists of the technologies we have developed and use for the direct building of software. In this category we find frameworks as Hystrix for resilience; we find products as nosql-databases; we find tools as Chaos Monkey. Aside from the code we also have the close-to-code practices like Test-Driven Development, Domain-Driven Design, Continuous Integration, Build Pipelines etc. All these things enable us directly to write awesome software.

With "process" in Agile Process I simply mean "the way we work". We work in sprints á la Scrum, or WIP-limited continuous flow á la Kanban; we demo at regular intervals; we use information radiators and stand-ups to synchronise work; we make forecasts to synchronise our work with other departments; we run retrospectives after sprints and projects to have a structured learning etc; we set targets and measure progress visavi effect goals, e g using Impact Mapping.

Finally Agile organisation is about how we structure ourselves. Within teams we self-organise, sure, but this is what we do on a larger level as well. We must ensure that information propagate across the organisation in a sound way; we want the architecture to stay reasonably consistent, preferably without a command-and-control chief architect. We must also synchronise and prioritise initiatives across this organisation so we also find portfolio management and finance/funding practices like Beyond Budgeting. Last but not least we find Agile HR/Agile Talent Management as part of this element.

Each agile practitioner might not do all the elements. On the contrary, most will stay mostly within one element or have their foot-hold in one and cross over to the other. 

Comparing to the hip-hop culture I must claim that the agile community is not complete without all three elements. Doing only tech does not help us; doing only process does not help us; doing only org does not help us. Skipping any of them would hurt us.

Nevertheless I have during the history of agile I have seen competition between the element, sometimes even hostility between practitioners where one side have claimed to be the "true agile".

I think it is unnecessary and harmful.

Furthermore, I think it is unworthy of us, of a community praising "humans and interactions" over all things, a community that thinks "cross functional teams" are the superior model - should we not strive for a "cross functional community"?

In the same way as the hip-hoppers give cred across element borders I would love to see the kick-ass developer praise the agile-minded HR Director for "getting it"; I want the beyond-budgeting-inspired CFO to thank the engaged and engaging scrum master for the fantastic project retrospective; I want to see the impact-mapping business analyst to thank the devops for coming up with insightful real-time metrics about what customers actually do.

OK, I might be guilty of hippiesm here, but I want the three elements of agile to share love.

Yours

   Dan

PS One of the reasons I care so much is that deep in my heart I feel that Agile is different.
PPS There is a video in Swedish from the conference Agila Sverige 2014 where I do a blitz presentation on this subject and some related stuff.

Domain Driven Design to Speed us Up by Opening Options

Dear Junior

At the DDD Summit a year ago there was an interesting discussion about how we motivate doing Domain Driven Design. We noted that one of the classical argument have been "proper domain modelling will make maintenance and further development easier", an argument which I have used myself. 

However, this is really a bogus argument; it more or less boils down to saying "spending a lot of time now will save us time later", which is a non-argument for anyone buying into the agile view of creating software. We need a better argument - If we are doing DDD it should be because it saves us time now.

The general agreement last year was that domain modelling actually do speed us up now, so doing domain modelling saves time in the short term, e g the stories at hand in the current sprint if you are doing scrum. However, we did not much discuss the mechanisms that make it so.

I have had this discussion in the back of my head during the last year, trying to keep my eyes open for how this could be the case - how doing more work actually could result in less work.

At the DDD Summit this year I had a really rewarding discussion about this with Steve Bohlen, sharing our experiences from the passed year. 

If you do not do domain modelling, you will look at the story to implement from a technical perspective. Doing so there is probably an "obvious" way to do it - "run that job, select the data from the database with a new where-clause, fetch it via XyzService, and push it to presentation". The approach will include activities A, B, C, D, and E - so write down those as five tasks, post them as stickies on the sprint board and start working. How could extra domain modelling save us time? Will it really make us work faster?

In order to make sense, let us think about "saving time" in the opposite direction. Efficiency can be seen as working fast. However, it can also be view as "maximising the work not done", which is how it is phrased in the Agile Manifesto as the principle of simplicity.

One of the important mechanisms of domain modelling (if done right) is that it opens up options. In a domain modelling session you challenge yourself and the team to think about the problem in different ways, and a good modelling session might well result in three different ways of thinking about the problem - and thus generating three different solution. 

Evaluating three different solutions might give you that first include activity A, C, E, and F. Second include B, F, and G. Third include A, B, D, F, H, and I. Now of these some might be worse than the "technically obvious solution". In this case third solution is probably worse; six activities instead of five. 

Finding solutions that are worse than the one you had (the "technical")? Surely that must be a true waste of time!

However, you do not only find solutions worse, you most probably find some that is better. And, you only have to implement one of them - the best one. In this case, the simplest solution is the second, only requiring three activities. 

It is a little bit like throwing dice trying to throw an as low number as possible. Throwing one die will give 1 to 6 equal probability, so your average low will be 3.5. If you throw two dice, each die will give 1 to 6 equal. However, for 6 to be the minimum you will have to throw [6, 6] which is 1 chance out of 36. For five to be minimum you have to throw [5, 5], [5, 6], or [6, 5] - 3 out of 36. For three it will be 7 out of 36. For one it is 13/36. Throwing one extra die heavily skews the odds for throwing a lower throw ("finding a simpler solution").

Back to software creation, the simplest solution found during the domain modelling is most probably easier that the solution that was obvious from the technical perspective. A small investment in opening options was well outweighed by finding a easier solution. So, we save time not by working faster, but by working smarter.

And just to be clear - we are talking about a small investment in domain modelling, like a few hours or a day. It is not about a "phase of the project" were you spend weeks in day-in-day-out workshops and discussions.

During the year my colleague Daniel Deogun had a wonderful example of this happening at our common client. At the sprint planning the team was convinced that they knew how to implement the story at hand. However, just to open up options, Daniel kidnapped the team for a two-hour domain modelling session. During that time they found new ways of looking at what they were building. So they ended up not touching the code they had envisioned to change. Instead they changed somewhere completely different and the amount of work shrank from being a dominating part of a sprint to be a few half-days of work for a few team members. And that insight was driven by asking questions about the domain - trying to phrase the answer in different ways.

So, doing more work to save time might seem very unintuitive. However thinking about it as finding options to "maximise work not done", I think it makes sense. In a way, what we are avoiding is the sadness of a really fast boat sailing at high speed in the wrong direction.

Yours

   Dan

PS BTW, Steve and I had more interesting discussion around this topic, but that will need to be the topic of another letter.

PSS Using the metaphor of orienteering what DDD modelling do is to spend a while to study the map, exploring different routes. As an orienteering runner this is definitely a smart move to make us faster.

DDS Value Object Presentation Video

Dear Junior

At the OWASP AppSec conference I had a presentation on value objects and Domain Driven Security. It turned out into a 35 minute code kata with refactorings, where I used value objects "DDD style" in two ways.

Firstly, I used them to create a design that makes indata validation come natural in the context of Injection Flaws. I used the classical SQL Injection login-attack as an example and applied "hard type" value objects in the same way as we have discussed before.

Secondly, I addressed a Cross-Site Scripting (XSS) scenario. Here I noted that even if XSS is often perceived as a problem with "bad indata", there are cases when those data are perfectly valid. So, instead I choose to look at it from an output-encoding perspective - not "bad indata validation", but "bad outdata encoding".

To be able to do something about the outdata encoding, you can think about the client side browser as a subsystem of your system. Seen that way, the presentation tier plays the role of the API to that subsystem. Then it becomes obvious that we should enforce proper encoding at the border of that subsystem - i e in the API we use for calling it.

Modifying the presentation tier API from "soft type" strings to "hard type" value objects made it obvious where proper encoding should go - thus solving the XSS problem.

The full video coverage from the conference has been released, and my presentation can be found at the OWASP website.

Yours

Dan

Demeter Saves Mocking Fairies

Dear Junior

In a Twitter post Damian Guy recently expressed his frustration with some hard-to-unit-test code as “”Every time a mock returns a mock, a fairy dies”. (Twitter @damianguy 2009 Oct 19)

I immediately fell in love with the quote and how it “backwards” refers to a guideline on how to design (or not design) your code for testability. What has struck me recently is that this design guideline is actually nothing but the battle-proven Law of Demeter (“Don’t Talk to Strangers”).

Let me give a rather sketchy example of how you could get a report of the shipping weight of an order composed of order items.

OrderService.reportTotalWeight(order)

   orderitemlist= order.items()

   weight = 0

   for(item in orderitemlist)

      weight.add(item.weight);

   return weight;

So, what is the problem? Well, imagine the unit test for the report functionality in OrderService. To properly unit test that functionality we need to rig a “test bench” with mocked versions of the “surrounding” objects. If we do not use mocks, we will actually also test the functionality in the classes of the surrounding objects as well, and then we are not really “unit-testing” any longer – more like “micro-integration-testing”.

So a setup would need to pass in a mocked order. But then comes the call “order.items()” upon which the mock need to pass back a list of objects. Further on, these objects will have their “weight” method called, so they need to be mocks as well …

OrderServiceTest.shouldSumOrderItemWeightsForOrderWeightReport()

   // Given

   order = new Mock()

   item1 = new Mock().expect(weight).return(4711)

   item2 = new Mock().expect(weight).return(42)

   order.expect(items).return([item1,item2])

   // When

   orderservice.reportTotalWeight(order)

   // Then

   …

Messy, messy, messy - and we have a mock that returns mocks. Not good for the fairy population.

The underlying design problem is that the OrderService “reaches beyond” the order object to work on the order item objects. By doing this, it "extends its contact surface" to the rest of the application – and makes it hard to write a simple and complete unit test.

Well, the Law of Demeter is a 20 years old design rule in object orientation that says that an object should only talk to its immediate neighbours. The definition found in e g Wikipedia is that the allowed objects are:

• The object itself
• Method parameters
• Any objects created/instantiated within the method
• The object’s direct component objects

In many OO-supporting languages build upon C syntax, this can be viewed as “just one dot” – this is not precise, but gives the right intuition about the rule.

And here we see that the order items do not fit into that list from the point of view of the order service.

Redesigning to follow Law of Demeter would limit OrderService to reach to order, but not to order items. This requires a new method in Order: “weight of all items”.

OrderService.reportTotalWeight(order)

   weight = order.weighOfAllItems()

   return weight

This piece of functionality is obviously trivial to unit test (in case you insist), and will only require you to mock one object, the order.

What has happened is of course that the complexity have moved elsewhere – into the order and its “weight of all items” method.

Order.weightOfAllItems

   orderitemlist= this.items()

   weight = 0

   for(item in orderitemlist)

      weight.add(item.weight);

   return weight

It is the same code as earlier – with “order” changed to “this” (and some obvious refactoring simplifications). Now unit testing of this only requires one level of mocks, as the code only talks to “its immediate neighbours”.

OrderTest.shouldSumOrderItemWeights ()

   // Given

   item1 = new Mock().expect(weight).return(4711)

   item2 = new Mock().expect(weight).return(42)

   // When

   order.add(item1).add(item2)

   // Then

And no fairies where killed in the testing of this code.

What I find beautiful is that these two ways lead towards the same goal. Law of Demeter was an object orientation theory invention to address coupling and coherence of designs. The save-the-fairies quote came from a pragmatic culture of wanting to produce high-quality code. I just love that they actually mean the same.

Well, when the Law of Demeter was coined, TDD was not around, “mock” meant something else, and fairies was not to be hip for a decade or two. So, finally I must praise Damian for formulating such a modern phrasing of a piece of ancient wisdom.

Yours

Dan

ps To be able to easily write unit tests without neither killing fairies nor "mocking yourself to death" is of course crucial for TDD to be fun.

pps Realising that redesigning your code for testability is of cause an effect of wanting to test the behaviour of the code, not the code per se.

Domain Models, Data Models, and How They Resemble Each Other

Dear Junior

In what ways domain models and data models should resemble each other is a really interesting topic. It is not uncommon for me to ask for what a “Foo” and a “Bar” really are and what their relationships are and upon that question being answered by the architect who with a smiling face show me the database schema, complete with join tables and everything. That was not what I had in mind.

So what is the difference between a database data model and a DDD domain model? Why do I care?

For me, it basically boils down to the purpose of the respective model. The purpose of the database data model is to provide the system with an efficient storage for persisting data, and for searching same data. The purpose of the domain model (a la Domain Driven Design) is to work as the focal point and a vehicle for communication about the system. In the database model you might have redundancy and pre-aggregated data for efficiency. In the domain model you try to make each concept and each relation as clear and unambiguous as possible.

True, in some places the database model is used as the domain model, and even the business side domain experts have learned to read the schemas. My experience is that even though this might work, for example by using ER-diagrams instead of the actual tables, it has its limitations. First, people tend to forget the distinction between the concepts per se, and their realisation in the database. So, if you want to re-conceptualise (which is good to try from time to time) you might be struck with counter arguments such as “but that will violate the constraints in the triggers”. Secondly, a database data model must by necessity cover every single piece of persistent data – however marginal its importance. To do strict modelling of a concept is a quite brain-expensive operation, thus it is simple waste of precious resources to do so on every minute detail. To get leverage out of your modelling efforts it usually suffices to concentrate on the most central parts. Thirdly, a database model will leave out data that is not persistent, or is extracted or imported from other sources - and those might well be very central to the conceptual understanding of the problem at hand.

For these reasons, I simply advice to do a separate domain model, even if you have a complete database model in place. They simply facilitate different kinds of discussions.

So, how much should these models resemble each other? Well, let us say that we have established that the domain model is the focal point of discussion. Further on, the domain model has grown out of a discussion where several types of input and considerations have been taken into account: how the customers of the product (e g end users) think, standards within the domain, GUI experts, and database experts. All those inputs are needed to give a model that fulfils its purpose – to enable us to build a good system. In that way the domain model is the result of a negotiation between several parties with the goal of meeting each party’s need as well as possible. So, as the database expert has his or her say in that discussion, there should be some level of resemblance.

But, how much should they resemble each other? Well, if the domain model is the common and central model, then each party will need to shrew it slightly to realise it into its context. The programmers might need to translate concepts to a different language, to make working code. The interface designers might need to lay components in the design in a slightly different way. And, the database designer might need to add an aggregation of data for efficiency, or add a join table to make ends meet. However, any unmotivated difference should be considered a technical debt and a target for a refactoring effort.

For the motivated adaptations, the burden of maintaining the respective mapping falls back on each party. If we need to reconceptualise and change the domain model, then each party will need to do both that change and the changes in their adaptations. So, in order to survive each party should keep the adaptations to the minimal motivated level.

Domain models a la Domain Driven Design and data models for database persistence should in a sound system resemble each other. But, they are not the same as they fulfil different purpose, and should not be confused with each other.

Yours

   Dan

PS When trying to keep down the semantic difference between the domain model and the data model, you probably want to refactor the database continuously - something that gets a lot easier if you version your database.
PPS A related question is to what extent technical details are allowed to be part of the model.
PPPS Domain Driven Design is in general a large and interesting field in and of itself.