Writing a book - Secure by Design

Dear Junior

I'm sorry for not writing to you for quite some time. But, you see, I have been working on a book together with my two colleagues and friends Daniel Deogun and Daniel Sawano. Late this summer we decided to try to write down our collective experiences and insights in design and security as a book. We are far from finished, but by now the first few chapters are released from the publisher Manning as part of their "Manning Early Access Program" (MEAP).

The book is named Secure by Design and our main message is how good design and good code can help developers avoid security problems. We have found that lots of security vulnerabilities are due to bugs that could have been avoided if the design of the code had been different. 

You as well as I know that it is a lot easier to think about "good design" when developing features, that it is to think "security" at the same time.

Of course not all good design lead to avoiding security vulnerabilities. So we have gathered our experience on what design-tricks that are most effective in giving security as a side-effect. It's probably no big surprise to you that Domain-Driven Security is an important part of this, but there are many other parts as well.

We still have a lot of material to cover. But, what we have started with things that are close to the code, such as building Domain Primitives, how to structure validation, immutability etc. I guess you will find several of these ideas familiar, as we have discussed them earlier - but there are also some new insights I have not yet had the time to write to you about. Well, not to mention all the ideas from Daniel and Daniel which you and I haven't had possibility to reflect on earlier.

In the later parts there will be more material on integration, security benefits from cloud thinking. We also want to share our ideas around architecture such as security concerns for microservice architecture or how to handle legacy codebases.

We will continue to write during the spring and early summer. Hopefully we will see the book in print sometime during the fall this year.

Yours

   Dan

PS The early-access was released last night; you can check the book at https://www.manning.com/books/secure-by-design

DDS Value Object Presentation Video

Dear Junior

At the OWASP AppSec conference I had a presentation on value objects and Domain Driven Security. It turned out into a 35 minute code kata with refactorings, where I used value objects "DDD style" in two ways.

Firstly, I used them to create a design that makes indata validation come natural in the context of Injection Flaws. I used the classical SQL Injection login-attack as an example and applied "hard type" value objects in the same way as we have discussed before.

Secondly, I addressed a Cross-Site Scripting (XSS) scenario. Here I noted that even if XSS is often perceived as a problem with "bad indata", there are cases when those data are perfectly valid. So, instead I choose to look at it from an output-encoding perspective - not "bad indata validation", but "bad outdata encoding".

To be able to do something about the outdata encoding, you can think about the client side browser as a subsystem of your system. Seen that way, the presentation tier plays the role of the API to that subsystem. Then it becomes obvious that we should enforce proper encoding at the border of that subsystem - i e in the API we use for calling it.

Modifying the presentation tier API from "soft type" strings to "hard type" value objects made it obvious where proper encoding should go - thus solving the XSS problem.

The full video coverage from the conference has been released, and my presentation can be found at the OWASP website.

Yours

Dan

The Problem with XSS

Dear Junior

I guess that one of the first things you have to do to address security risks like Cross-Site Scripting (XSS) is to learn what they are about. Here I must admin that even though I have been interested in security, I have lived in ignorance about several of the most common attack techniques. OK, I had some vague idea what a XSS attack was like, but would not be able to explain it. Really I have to thank my colleagues Michael Boman and John Wilander for finally setting the details for me.

Deep down XSS is really about making someone run a program that they did not intended to run. That program might be a harmless pop-up “You have been XSS”, it might be a local port-scan, or it might be installing a keylogger.

The road to make this happen is to mix up the distinction between code and data. This is typically in a web browser which is fooled to accept something that looks like content (data), but actually is JavaScript (code) that gets executed in the browser.

There are several ways to make a XSS happen but the one I have found most easy to understand is what is known as the “stored” attack.

Imagine that you have a community site with some discussion forum on – say – how to raise children. Member parents can enter posts and comments to a discussion. When you go to the discussion page, all posts and comments are sent to you as an HTML page and rendered onto your screen by your browser.

Let us imagine what happens if some evil pretend-to-be-parent enters a comment saying “I think you should <script>alert(‘Spank them!’)</script>be nice to kids”; what would happen? When you, the unsuspecting victim, browse the discussion you get back an HTML page. To start with, the page you get back is valid HTML – it is allowed to drop in a javascript almost wherever you want to. Your browser will render the text “I think you should be nice to kids”, and the script will be executed popping up an alert saying “Spank them!”. Of course, a yet more evil pretend-to-be-parent would substitute ‘alert’ for less pleasant, and now the computers of all unsuspecting parents are infected with keyloggers stealing their credit card numbers and uploading them to some obscure server in unknown jurisdiction.

This variant is called “stored” because the attacking script is stored on the forum server. There are other variants for getting the javascript into the poor browser, but I have found this the easiest to understand.

So, what is the problem? A simple way out is to blame it on bad indata validation. The forum comment “I think you should <script>alert(‘Spank them!’)</script>be nice to kids” should simply not have been accepted – and in this case that might be true. If it had been a discussion on JavaScripts, it might have been a little trickier.

Another way to view the problem is that the browser has mixed up data and code. What was data (the forum comment) was interpreted as code and executed. No good. But that would be a case of blame-shifting, because we (the programmers developing the forum site) are perfectly aware about how browsers work.

So really, the problem is in the forum application that mixes up domains. It takes what is a comment in the in-data domain (text field and http request), and happily let it go over to the presentation domain in which it is interpreted not as content, but as code. How come? Well – if we want the string “<script>alert(Spank them!)</script>” to be interpreted as content in a browser we must encode it properly. So, the long-and-winding answer is that we have missed to apply the proper encoding when transforming from the comment domain to the html-presentation domain.

Now, here we have formulated XSS as a problem in Domain Driven Design terminology – then we can see what Domain Driven Design would do to address the problem.

But that must be the subject of another letter

Yours

Dan

SQL Injection is not an Indata Validation Problem

Dear Junior

If DDS style use of value objects solves the indata validation problem , and if DDS style indata validation does not solve SQL Injection, then there is only one logical conclusion to draw.

SQL Injection is not an indata validation problem. Yes, that might be contrary to popular belief, but it is obviously so.

If not an indata validation problem, what kind of beast is it? And what can we do about it?

Yours

Dan

Indata Validation is Not Enough for SQL Injection

Dear Junior

When we do indata validation through value objects we get an application tier that is water-proof. The model describes exactly those data that we think are meaningful and can handle (“username, identifier with which the user present herself to the system; regexp [a-z]+“. Each piece of indata is validated as part of the value object constructor (public Username(String s)). For indata to pass through to application, it cannot avoid validation, as the application methods require the value object type (void authenticate(Username uname…)). What more can you ask for?

- Well, you see, we do not want usernames like “danbj”, we would prefer our real names, like “Dan Bergh Johnnsson”.

No problem, we expand the regexp with uppercase letters and space, getting a regexp like [a-zA-Z\ ]+.

- Nice, but our respected colleague Fredrik Jägare-Lilja needs a username as well.

Fair enough – we stuff Scandinavian letters and hyphen into the regexp as well, giving [a-zåäöA-ZÅÄÖ\ \-]+.

- Now there is only one person left: our highly respected Irish colleague Oliver O’Hehir.

Well, well, we are almost finished then, we only need to put the apostrophe into the regexp ending up with something along [a-zåäöA-ZÅÄÖ\ \-\’]+.

Wait, wait, wait!!! Who the h*** just logged in with username “’ OR ‘a’ is not null --“?

Sure, we might have tightened up the regexp to block out that specific attack string and any other malicious use of the format we can think of. But, it is always those we did not think of that causes the trouble.

Well, as always we can think that “SQL Injection is solved by prepared statements”, but remember that Injection Flaw is much larger than SQL Injection. The same vulnerabilities might be there when doing LDAP access, using parameters to construct file names (e g Directory Traversal), or if you have some Domain Specific Language (DSL) which you interpret. In any of these cases there might be a string that might well be fully legally formatted, but attacks the structure of how the underlying resources are used.

Over to a completely different domain: FM broadcast and music radio. In the FM radio broadcast system you must be able to shut down the transmitters from a remote site. Unfortunately, there is only one way to communicate with the transmitters – via radio. The problem was solved by defining a specific sequence of audio blips (very precise on frequencies, duration, and interval) and denoting that sequence the meaning “shut down the transmitter”.

The pioneering Swedish rap group JustD put that exact sequence as the final beat on one of their songs, without telling anyone. They must have laughed all the way home from the studio. That song has been played on Swedish radio exactly once.

The JustD track hack is a wonderful example of exploiting an Injection Flaw, there is no way to escape it “in band”. I have the same gut feeling about indata validation and SQL Injection.

No matter how we structure the indata model, there might always be some data that actually is valid indata, but causes the system to crash.

So, indata modelling and validation in all its glory: However necessary it is for upholding security, it is not sufficient.

Yours

Dan

Ensuring Indata Validation

Dear Junior

Creating a username class and a validation method has taken us a fair amount towards solving SQL Injection by focusing on a domain model API that is both easy to use correctly and hard to use incorrectly. I would say that we have this far achieved to make the API easy to use.

  Integer authenticate(Username username, String passwordMD5)  
 public class Username {
  // final making it immutable
  public final String username; 
  public Username(String username) { this.username = username; }
  public boolean isValid() { return username.matches("[a-z]+"); }
}
What remains is to ensure that indata validation actually is done. I can see two choices: either putting validation inside the authentication service, or to enforce validation before the call to the authentication service.

Let us first look at putting validation inside authentication.

  /** Authenticates a user with a given password.
   * @throws IllegalArgumentException if username invalid
   */
  Integer authenticateWithUsernameValidation(Username username, String passwordMD5)
          throws IllegalArgumentException, SQLException {
    if(!username.isValid())
      throw new IllegalArgumentException(
              "Cannot authenticate with invalid username: " + username);
    ...
  }

This definitely hardens the interface – now there is no possibility to not validate upon authentication. However, the same trick has to be used in every service method around, including the “create new account”, the “change account username” and all those that are to come in the future. Risks are high that the small isValid-call will be missed somewhere – and one hole is all an attacker needs.

Another drawback is the rather awkward “throws IllegalArgumentException” which feels like a very late validation – should not such validation be made much earlier, preferably up in the presentation and client tiers?

An alternative is to not allow invalid usernames to be constructed at all:

  @Test(expected = IllegalArgumentException.class)
  public void shouldNotCreateUsernameFromInjectionAttackString() {
    new Username("' OR 1=1 --");
  }

This request the constructor to do the validation on the inside, responding with an exception if given an invalid username candidate.

  public Username(String username)
    throws IllegalArgumentException {
    this.username = username;
    if(!isValid())
      throw new IllegalArgumentException();
  }

Now we also need some way to validate from the outside without taking the pain of provoking and handling an exception, so finally there will be a static method after all:

  public static boolean isValid(String username) {
    return username.matches("[a-z]+");
  }

Of course the old methods and constructor should be refactored to uphold the don’t-repeat-yourself (DRY) principle. Interesting enough this will lead the isValid() method to consistently return true – so I guess we can delete it from the class and inline it wherever it was used. That is, unless we for some bizarre reason want to have a method that explicitly tells the rest of the world that “this object is always valid”.

I definitely prefer this latter “strictly-validated-value-object” style before the "validating-service-methods" style. It creates an API that besides being easy to use correctly, also is hard to use incorrectly. It “guides” the client side programmer without being intrusive or obstructive about it.

In some sense, it "enforce" a behaviour upon the client side programmer. However, that does not trouble me. If someone just is nice to me, and don’t cause me trouble, I see no obstacle in letting her have her way.

Yours

Dan

Validating away SQL Injection

Dear Junior

So, if we want to ban ’OR 1=1 from being used as a username, we have to put that restriction into the model.

We have noticed that viewing the username as just a string does not help us much.

Integer authenticate(String username, String passwordMD5)

So, to be able to talk about usernames in a meaningful way , we introduced the class Username. This actually makes a huge difference as the authentication method is now explicitly distinguishing the username from other kind of indata.

Integer authenticate(Username username, String passwordMD5)

With username as an explicit part of the model, we now can say something like “‘ OR 1=1 is not a valid username”. The beautiful part is that this sentence both speaks immediately about a property in the conceptual mode, and about a property of the code.

The sentence is even well-formed to the degree that it can be turned into a unit test case.

  @Test
  public void shouldNotRegardInjectionAttackStringAsValid() {
    assertFalse(new Username("' OR 1=1 --").isValid());
  }

Not bad, if you can write a requirement as a unit test, then you are pretty well off. And, if the test is failing: the better, we have a licence to code, and we know when we are done. Not bad at all.

Implementing, adding a few related tests and going back to green will probably take us to a Username class looking roughly like this.

public class Username {
  // final making it immutable
  public final String username;
  public Username(String username) {
    this.username = username;
  }
  public boolean isValid() {
    return username.matches("[a-z]+");
  }
}

Hold the horses! We have now modified a class that encodes a central part of our domain at hand. So, we have actually changed the model. That is nothing to take lightly. Now we have to walk the round checking with the other stakeholders in the domain that this change is something we all can agree upon. Is it really ok that usernames should be only lowercase letters and that there must be at least one? In other words, should they look like “danbj”, “danberghjohnsson” and “ilovemylittlepony”?

In the case of usernames people are used to strange formats and restrictions, so my guess is that no one will object, but we have to check – otherwise the model turns from being a shared and agreed model to being “the developers stuff”.

Upholding the model as the ubiquitous language for talking about the system is essential for the quality of the system – it was that language that enabled us to turn a one-phrase requirement and turn it immediately into code. Further on, the quality of the system is essential for its security.

So – for the sake of security, if nothing else, work hard with that model.

Going back to the API we are building, it turns out pretty well. We have achieved that it is easy to use correctly – i e, when doing the right thing there is no much doubt. At the moment, you have to create a Username object (constructor), check that it is valid (isValid method) and then pass it to the authentication.

However, we have not yet achieved that the API is hard to use incorrectly, it is not “stable to erroneous use”. One really weak spot is that it is easy to miss calling the isValid method, and there is nothing in the rest of the API that catches that mistake.

So, we are not finished yet.

Yours

Dan

ps Of course, what we want to do is to ensure validation, not hoping that our fellow programmers will magically do the right thing.