Thursday 27 March 2014

Deep Learning as a Strategy for Information Age

Dear Junior

Last friday we had the pleasure of having CS students Sofie Lindblom and Anton Arbring as guests visiting the monthly competence day at Omegapoint. 

After this visit, Sofie have done me the honour of musing on the theme of our letters by writing an open letter "Dear Senior, Letter to a Senior Programmer". 

That post is so full with interesting topics it would take a day just to briefly discuss them. But those topics are also way too important to leave uncommented. So, to do something, let us pick one important thing and discuss it. I pick the topic about "what to learn".

Too much information out there - as always have been

Sofie writes:
But there is too much information out there to know where to start. I am not stupid, I did very well in all programming courses and is a fast learner. But I feel exhausted by the amount of information available.
To start somewhere, let us start with the vast amount of information, technology, frameworks, etc that are out there. Obviously there is no way to take in all of that. If we want to use metaphors, it does not suffice saying "drinking from the fire-hose", it is rather to try to gulp the Nile. 

Good part is that the situation is not new. Of course there are more information out there now compared to 15 years ago when I left university. But even then the amount of information available was too much for any individual to comprehend. And the situation is even older. The proverb "so many books, so little time" is not fresh-out-of-the-presses. 

For those leaving university today, there will be truckloads of technology you will be using at work that you did not learn in class. But that situation is not new either. When I left university, I had not used a relational database in any single class or lab. Still, most systems (not all), I have worked with professionally have included SQL databases of some sort. Actually, one of my first jobs was to teach a class on the Java database API JDBC. How did I manage?

The obvious solution is to replace "know everything" with "able to comprehend". We cannot know everything beforehand, but we need to be able to understand any technology we come across with just spending a reasonable amount of work.

Killing a meme

There is a meme around in this information age that basically goes "you do not need to know, you need to be able to find information". I want to kill that meme in the context of system development.

The meme might very well be true, with Wikipedia and the rest of the web at our fingertips we will be able to find data like "first historically recorded solar eclipse" (5th of May 1375 BC in Ugarit). Nevertheless true, it is worthless to us as software professionals. Because what we need is not data or information, but understanding.

Deep knowledge feed deep knowledge

Now, this is only my own meandering experience, but I have found it invaluable to know a few things really well. Deep knowledge has interesting side effects. Suddenly you see some pattern apply to a new domain. It seems like no domain of knowledge is an island. Even if facts do not carry across boarders, some structures of thinking and reasoning actually apply.

This is really vague, so let me throw some examples to clarify. When studying law I suddenly found that my studies of formal logic really helped me. I studied negotiation theory and found how it applies to finding a good architecture for a software system. I studied compiler technology and found it helpful when studying linguistics. Through my lifelong studies of math, I see wonderful aspects of beauty in the world every day. (OK, the last is a little bit off topic - but it makes my life richer, and that is worth something)

The strategy I try to apply myself is to study subjects in depth, to the level when I have to think hard about them. The specific knowledge might not be immediately applicable - I will probably not have any specific use of knowing e g how to count the number of ways to paint a cube using several colours. However, thinking hard has probably etched new traces in my brain - and those ways of thinking will probably pop up applicable in a new domain.

To fall back on metaphors again. As software developers we need to dig deep to understand a new technology. To get down to depth we are not very helped by having dug a meter deep over a large area. But if we have dug a few 20 m deep holes in other places, there is a good chance that we can dig a short tunnel at the 20 m level from the bottom of some other hole. 

How did I survive that first job-gig teaching an API that I had myself never used before? Well, having studied functional programming in depth (using e g ML) had made me comfortable with the ideas of abstract datatypes. So the idea of an API was not unfamiliar. Having studied linguistics I was very familiar with formal grammars of languages so SQL syntax was not strange. Having studied compiler technology I could understand the semantics of SQL. Having studied algebra and set theory I could easily pick up how SELECT and JOIN worked.

It took me a few days to read the JDBC API and specification combined with some small hacks to validate that I had got it right. And after those few days I did not only knew about JDBC, I actually understood it well enough to be able to teach it in a reasonable way. Not being an expert, but reasonably competent.

Without the deep knowledge in some very obscure subjects (linguistics, set theory, compiler technology etc) I would have been utterly lost. No skill in "searching information on the web" would have helped me the least.

Sofie writes

The more I learn, the more I realize how little I know. It creates contradictory feelings towards the field I love. To twist it even further the part I love the most is that you can never be fully learned and that there is never a ”right” answer.
I understand the frustration. But I am not sure I would like to have a field where there was a right answer, a proven best practice - many in our field dream of such. 

However, to me a large portion of the beauty of the field is that we are constantly pacing unchartered terrain. The challenge is to constantly search your tool-box for something that seems applicable, to adapt, to improvise, to search, to try, to fail, to back up, to learn, to grow, to try again, to discuss, to exchange ideas, to finally nail it.

This is nothing but my own personal experience, but if I should offer an advice to handle the world of information we have around us it would be the following:

Find things to learn that you find interesting and that challenge your intellect. Take the time, pain, and pleasure to learn a few of those things to depth. The deep thinking will etch your brain in ways that will help you enormously whenever you approach a new field. And enjoy the pleasure of deep understanding when it dawns on you.

Yours

  Dan

PS Should you come across Sofie and Anton, take the time to have a discussion with them. And do not stop at a chat about everyday things - they have really interesting ideas to delve into.