Developing software is hard and frequently there are many different pitfalls for the unprepared. As a result, there are many different ways in which we try to make the practice of software development more predictable. There are now lots of disciplines that exist to support the practice of software development (I mean outside of actually writing code). I won’t enumerate all of those disciplines here, but some common disciplines are: business analysis, project management, testing, and last but not least, solution architecture.
Today we’ll take a look at the solution architecture.
The first thing that you might ask: “Dany, why do I care about solution architecture?”
The reason is actually pretty simple. It turns out that much like architecture, in the traditional sense, understanding what you want to build, what it should look like, and how it all fits together are important properties. The practice of solution architecture helps us with that. Unfortunately, a lot of material related to solution architecture is protected by pay-walls, or its protected by large corporations that have profited from the practice and have tried to retain the knowledge within the boundaries of their own organizations. If you look up the WikiPedia page for solution architecture, yyou will see that it is rather sparse.
I want to change that; I’ve spent the last decade building software, from small projects to large enterprise systems. I’ve spent much of that time working on design and solution architecture. I’m going to try and capture some of my experiences and put them down on “paper” as it were. This is the first article in a series that will discuss the practices and principles related to solution architecture and how those principles apply to various environments. Initially, I will primarily focus on how solution architecture applies within the context agile software development practices.
Like most topics, there is a benefit to understanding the history of how and why something came to be. Before we dive in to the topic itself, we’ll spend a little bit of time talking about the origins of solution architecture.
The origins of solution architecture are shrouded, like all things, in mystery. I kid, they really aren’t. A few consulting firms, such as Gartner, Forrester, and EY figured out that in order to deliver complex projects within large enterprises they needed to understand what they were actually delivering. Looking back, this seems rather obvious. So taking a page out of the practice of building complex physical projects (like buildings, bridges, etc) they figured out that an architect is required. Projects became known as solutions, as in “a solution to a problem” and the people who would design and oversee them became known as solution architects. The corporate world is really creative like that.
Before there were solution architects, large projects were primarily driven by project managers and business analysts. This was useful to understand what bits needed to be built and the order in which those bits needed to be built. However, there were concrete problems with taking complex technical projects and appointing people that dealt with logistics, execution, and translation of business language into formal specifications. I will highlight one major drawback: these people are not concerned with the design and foundation of the overall solution. As a result, there was no single person that was responsible for the overall technical structure of such a project. So the corporations that had to develop these projects started to understand that there is a gap in the way these projects ran. The gap, in their eyes could be summed up in the gap between gathering requirements and execution.
As an aside, software development is a discipline that hasn’t been around for that long. As such we’re still figuring things out. Initially it was between a bunch of mathematicians and engineers. So the practices around software development originate there. However, it turns out that engineers and mathematicians are not really interested in solving problems that are related to humans, they are more interested in technology, puzzles, and mathematics. This becomes an issue when the problems that are being solved are primarily human ones; like communication, management, and cooperation.
To be able to describe a framework around solution architecture we need to talk about the software development lifecycle. Which is really just a series of steps that are used to plan and develop software. A little WikiPedia action about the SDLC if you’re interested. We’re going to talk about waterfall and agile and how solution architecture applies to those development models.
The waterfall development model indicates that each phase of a project is discrete and needs to be completed before the next phase can begin. For large projects each of these steps might take several months to complete. The image below show an example of what a waterfall model might look like.
This image may be familiar to some of you, but for the uninitiated, it’s easy to understand. There is a flow, everything is linear, and the hand-offs are well defined. However, there is one problem with the depiction above, life is not quite that neat. It turns out that if you follow this approach, the failure rate for IT projects was around 50%. Might as well go to Vegas with those odds. So, some very smart folks added the rational unified process on top of the waterfall model, which dealt with addressing the gaps between each step and adding visibility for other stakeholders. Without getting further bogged down by a history lesson, solution architecture evolved from that process as a separate discipline. So the image turned into something that looked more like the image below.
So the world of building large applications improved. The problem with the approach above was the amount of overhead required to run a project in this way. With the rate of change on the applications being built on the web, this approach didn’t scale well. So again, the world of software development reverted back to chaos.
In the late 80s and early 90s, a gentleman by the name of Kent Beck was busy applying practices that will later be termed XP, or eXtreme Programming; what is now considered the source of the Agile movement. This is a radically different approach to developing software. An approach that focuses on building smaller chunks of functionality that touch the entire end-to-end process. So instead of each step being completed for the entire project in the diagrams above, each step was completed for a single component. In the mid 90s, another wise human that goes by the name of Martin Fowler started learning and adopting the notions of extreme programming and was seminal in the birth of the agile software development movement. If you are curious, read the agile manifesto, and the principles behind the agile manifesto for software development. However, that is a story for another time; back to the task at hand: Solution Architecture within the agile world.
The agile software development process challenged a lot of the common concepts and roles. The roles specific to business analysis, systems analysis, and project management got turned upside down. Seemingly overnight, the norms shifted and people struggled to redefine themselves within the context of the agile software movement. However, the corporate world is slow to react and slow to embrace agility. As the agile software development process evolved; solution architecture stayed firmly planted within RUP and the waterfall worlds.
This entire post, is a long winded way of saying that we’re going to talk about solution architecture and the agile development process. We’ll talk about how we can create a solution architecture framework that will take a practice that was built around the waterfall process into something that resembling structured chaos, I mean agile software development.
I have no certification, and no specific credentials to wave around, except for experience earned in the field. However, the world is a pragmatic place and I will endeavor to make this a pragmatic set of articles. Hopefully this brief history of the discipline has given you a peak behind the curtain. In the next article we’ll talk a little bit about how we can iterate over the solution architecture of a project and some tools that we can use to make that process manageable. Note, that when I say tools, I mean concepts, not actual pieces of technology that will make this process automatic.