In this video, Glenn Vanderburg explains how software engineering differs from other engineering fields. I liked it so much that I decided to blog about it with a particular focus on waterfall and civil engineering.
As problems we could solve with software grew in complexity , it was becoming harder and harder to write quality software and deliver projects on time.
At the time, people in the industry concluded that it was due to the methods used for software development. They thought that their methods were no longer fit to manage this increase in complexity.
To solve this, they came up with the waterfall approach that is highly inspired by other engineering disciplines.
The waterfall model takes successful principles from the construction and manufacturing industries and applies them to software development.
As a result, for years, software development was treated like a construction process where engineers first make a design and hand it to programmers so they can assemble the code required.
Just like in construction, where engineers would design a bridge and the workers would follow that design to build a bridge for example.
Except it never worked for software... Why?
To better understand why the waterfall approach is not playing well with software development, let's look at some fundamental differences.
The code is not the product
Waterfall implicitly considers code as the product like the building blocks of a bridge in construction. We learned that it's not really comparable because it's a lot cheaper to move code around than physical materials. It's therefore cheaper to experiment with code than it is with constructions.
Also, your clients pay for working software, they don't actually care about code. Only your team cares about code.
So what is the code if it's not the product? It is a description of what your working software does and it encapsulates your understanding of the requirements.
The code should instead be compared to the design of a bridge, and the working bridge should be compared to the deployed solution and that is something waterfall never acknowledged.
The level of abstraction
For large scale concrete projects like civil and aerospacial engineering, the costs are dominated by physical stuff like materials and labor.
In small scale abstract projects like industrial, electrical and software engineering, it's the opposite. The cost of material and labor is negligible compared to the costs of design R&D and experimentations.
Abstract fields are about solving small problems with experiments and incrementing on them until something actually useful comes out.
In other words, abstract fields require (and can afford) prototyping while concrete fields cannot afford prototyping because it's too expensive.
Software engineering is often used to solve new problems
When we use software to solve a new problems like sharing your home with travellers (Airbnb) or helping everyone find stuff on the web as quickly as possible for example (Google), it's very different than allowing cars to transit between 2 islands (a bridge).
We know how to solve the latter, we have done it several times in the past. However, we have never solved the first two and history taught us that trying to plan it all in advance rarely works out. Why? This is mainly because there are too many unknown parts.
When starting a project with so much risk, it's clearly not a good time to predict the budget and deadlines. It's wiser, to reduce the risk by prototyping on small increments. At one point the risk will be reduced enough to commit on a small increment of working software.
Waterfall might be ok for building well known and predictable software projects like company websites or simple apps. When it comes to software engineering though, it fails to acknowledge some key aspects that distinguish this field from other engineering fields.