At some point in software engineering, we get across the term premature optimization. Premature optimization is the practice of improving your software without an actual need, hence prematurely. In many cases, this happens when clever developers want to optimize their code for performance or create abstractions before they're needed. Over the years, I observed this clever behavior not only when writing code, but also when designing systems and team structures. I think of this phenomenon as a premature abstraction and I think it might be one if not the most costly mistake when building software.
Premature abstraction can happen on different layers of the software development process. On the lowest level, it occurs when developers introduce abstractions such as interfaces or abstract classes without a clear, immediate need. Such an abstraction can lead to overly complex and rigid code structures, hindering maintenance and scalability. In practice, you will find it difficult to modify this code, because you don’t just need to change the implementation but as well the interface. Oftentimes, such a premature optimization is justified with common best practices. e.g. adding an unnecessary interface could be argued with the open-closed principle (SOLID).
One abstraction layer above, premature abstraction can happen when creating a system design. An example is the microservice hype of the 2010s that led to many small companies prematurely applying an architectural pattern that evolved at Big Tech companies like Amazon and Netflix. Microservices are designed for flexibility in environments, where components need to be scaled or maintained by individual teams. Most small to mid-sized companies neither have the scale nor the team size to justify the overhead of this. Yet, when I started building software in 2017, I was convinced this is what everyone does, so I also did it. After all, we wanted to be ready for Netflix-like scale. Fast forward a couple of months, my team had made every possible mistake and we learned the hard way what consistency, availability and partition tolerance (CAP) look like in the presence of networks, NoSQL databases and prematurely defined boundaries. It turns out that what could have been an SQL join, can be made exponentially harder when every microservice has its own database.
Even before we design a system, we define how the teams that should implement our future design are structured. Many times, this happens before having a clear picture of the architecture of the system to build. Socio-technical systems theory describes how people and technology interact. Within this theory, there is the famous Conway’s law which states that
Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.
— Melvin E. Conway, How Do Committees Invent?
So, whatever team design we choose to start out with, will eventually result in an architecture that resembles that team design. In practice, companies tend to split their teams by product domain, function (frontend, backend, platform engineering, etc.), or project. Many times, they once again look at what successful companies have been doing before them. Whether that is the famous Two Pizza Team approach by Amazon, Squads from Spotify or Microsoft’s F-Crew. No matter the approach, whoever decides how to split teams makes assumptions about who needs to work together and which problems/domains they will own. The further those assumptions are from reality / the actual needed solution, the more overhead this design will introduce in terms of communication, planning and alignment.
In short, premature abstractions happen when assumptions are made. The more assumptions we make, the more likely we are to be wrong. Anticipating future requirements is easy and exciting and it gives us an excuse to use the latest cool framework, methodology or team setup. However, it will almost always fail and introduce an overhead in code, system design, or team collaboration. So, if we want to avoid premature abstraction, we want to make as little assumptions as possible and keep things simple for as long as possible. Start with
a single concrete implementation
a single monolithic architecture
a single team
and iterate. Once you actually need to abstract, the abstraction will emerge naturally and you can refactor. Whether that is the interface in your code, a component of your architecture that can be split out of our monolith, or a new team. Wait for it to evolve instead of trying to fix a problem that does not exist (yet). After all, we do need to abstract to manage complexity, but abstracting prematurely swings the pendulum to the other side.