Softare Engineering is not Software Architecture
Until we all understand that we will be confusing ourselves and our clients.
The industry is in crisis. That’s the dialectic. There’s a need for critical qualitative information to accumulate before everybody gets quality results. The well-known transition from quantity to quality. Blaming software failure on particular people or difficulty of "changing requirements" is merely symptomatic of the lack of true architecture. When the owners and builders see what is being build, they realize that it is wrong and start to make changes in a post hoc attempt to bring the reality of the structure in line with their expectations. It is equally erroneous to blame software failure on poor management. Without a profession of true architecture (not just on words), it is the… architecture that is missing. Even the best managers cannot produce a satisfying result from a bad architecture or the lack of architecture. And management cannot compensate for the lack of a standardized construction process, vague roles, and confusing lexical. Only true, skilled architects, in partnership with their clients, can design "the thing actually to be done".
The MBAs call it "business process reengineering". We architects call it "architecture" and we mean it. We don’t mean "technical design". You don’t build a plane by designing its engines and wings separately, only considering the point in which they would be attached together. The power of the engines has to be in a precise accord with the wings’ ability to make the plane become air-borne.
Today at OpenFrame Technologies we are more convinced than ever: conceptual integrity is central to product quality. Having a system architect is the most important single step toward conceptual integrity. These principles are by no means limited to software systems, but to the design of any complex construct, whether a computer, an airplane, a Strategic Defense Initiative, a Global Positioning System.
A builder or an engineer don’t see the concepts of the architecture. Because they are not architects. And therefore they are not architects. You can blame an architect for being incapable to purge the plumbing. But an architect is able clearly to see a sloppy work of dilettantes.
Great design results from the architect’s understanding of the client’s world, as well as architectural vision and skill; it does not emerge from committees or the collective effort of builders. It does not grow from casual regard or from the hands of those who see design as a piecemeal development phase.
Architecture is based on patterns. Good architecture, good design, must be congruent with our individual and collective patterns of behavior and perception. But what software engineers and builders don’t realize is that architecture patterns are not the same as technical implementation design patterns, for instance, GoF (Gang-of-Four) patterns, such as Abstract Factory, Observer, Façade, Decorator et cetera. These are software engineering, implementation design patterns. We are pretty sure software engineers have never heard of such software architecture patterns as Application Controller, Coarse-Grained Lock, Domain Model, Front Controller, Metadata Mapping, Plugin, Gateway, Service Stub, Unit of Work, Separated Interface, Standardized Request Triplet, Special Case – to name a few.
The role of the architect is to design structures to meet clients’ needs. The architect is, and must be, a client advocate. Clients can take different forms, such as a committee, especially when public building and software projects are involved. Whatever form they take, the architect’s accountability is to the client, and the architect makes all decisions in that light.
For the preceding sentence to be more that just an ideal or a meaningless mission statement, the architect typically needs to be a professional with a status of independent of the builders, especially in complex, expensive projects where conflicts of interest are large in scale. There are exceptional individuals who have been very successful in the dual role of architect/builder, but often, a person in the dual role becomes influenced by profit and cost in design considerations. Corners are cut, and the purity of the architectural design suffers. The architectural quality is unmistakable, but it is fragile and dies when the design is not the architect’s paramount concern. What would you say if you knew an architect was not let to the client?
Also, when software engineers refer to themselves as architects… hmm... "solution architects", they frequently are alluding to design done at the technical, "in the walls" level, not the true architecture level. Would you say it is not so? The engineering worldview is from the perspective of the technical aspects of a system, and engineers use the term "architecture" to describe the design of what would be analogous to engineering elements and beams, bearing walls, and other structural systems.
Engineers typically do not see from the classical architectural vantage point of the client and the entire enterprise or domain. So, when they use words "architecture" and "architect" to describe their work, they bring the mass of the terms but fail to bring the meaning, leaving the real architectural work undone and the classical role unfulfilled. They are looking at the software as engineers and functioning largely as engineers, but persisting in believing that what they do is architecture.
As a result of this Herculean mission of trying to be architects-engineers-builders, engineers are biting off more than they can reasonably chew. They believe that simple answers would not do justice to the complexities of technology. They prefer to create even more exhaustive reams of end-user requirements and even more reliable techniques in their tireless, methodological search for project success. All their efforts rest on the premise that flawless engineering will solve the problem.
The high-tech industry has inadvertently put programmers and engineers in charge, so their hard-to-use engineering culture dominates. Despite appearances, business executives are simply not the ones in control of the high-tech industry. It is the engineers who are running the show. In our rush to accept the many benefits of silicon chip, we have abdicated our responsibilities. We have let the inmates run the asylum.
When the inmates run the asylum, it is hard for them to see clearly the nature of the problems that bedevil them. When the creators of software-based products examine their handiwork, they overlook how bad it is. Instead they see its awesome power and flexibility. They see how rich the product is in features and functions. They ignore how excruciatingly difficult it is to use, how many mind-numbing hours it takes to learn, or how it diminishes and degrades the people who must use it in their everyday lives.
There’s a fundamental problem of engineers not being able to see beyond their own ethos and worldview. Their assumption is persistent: "software construction only needs perfect engineering to stand". Despite the increasing emphasis on an architectural approach and design, software engineers have failed to see that the classical role of the architect and the guiding principles of the analogy are missing and that omission is the cause of the failures – not bad engineering. We have been trying to build elaborate, complex software structures without the organizing power of the Vitruvian Triad: Utilitas - Venustas - Firmitas (from Latin: Function – Beauty – Structure). Software engineers are absolutely correct in their assessment that more reliable software engineering practices are important. With that mission alone they will have their hands full. And with their profession and role defined and organized around that mission – not on architecture and building – they can achieve exactly that. Software engineers may enable software skyscrapers to stand solidly, but it will be the architects who determine that a structure will look like the new World Trade Center. And it will be in the hands of the "developers" to build. I could (and would be willing) also to elaborate on the "developers" role to build software but not at this time.
The confusion found in the developer/programmer nomenclature, though, is added and abetted by human resources departments who use titles as an odd form of compensation. If you deserve more money, they believe, because of merit and/or seniority, you also deserve an overloaded title, even if it bears no relation to your role. There are even movements afoot to ascribe numerical grades to the programming ranks, with title changes as you climb the rungs. And the title that sits at the top is, yes, architect. Excuse me, gentlemen, but this is akin to a talented plumber receiving promotions for his or her fine plumbing work until, finally, being promoted to architect.
A design process begins with the client and an architect. The client in building construction is not a mere stakeholder. The client plays a key, active role. Imagine Michelangelo referring to the Pope as a stakeholder or an end-user. The architect is always a client advocate, first and foremost, and intervenes on the client's behalf throughout the project. The architect has a foot in both worlds - the client's and the technical builder's. The architect understands both worlds and, in this way, is able to bring the design intact through the difficult construction phase.
The client can be wholly ignorant of the whole idea of design patterns, bearing loads, footings, and building codes but still can validate that what is being built is true to the architecture. The tradesman involved have their own lingo, methods, and tools, but the client does not need to learn them to remain in control of the project.
Without the architect/advocate, a carpenter could tell the client, for example, that a desired curved staircase is impossible, given the space and angles at hand. The client would have to entertain several possibilities:
- maybe this is the truth
- maybe the carpenter realized he lacks the skill or tools needed to build a curved staircase and does not want to admit that;
- maybe the carpenter believes the staircase can't be built, whereas it really can;
- maybe the carpenter underbid the job and decided the curved staircase is too expensive for the agreed-upon price;
- maybe a more lucrative job is waiting, and the curves would be too time consuming;
- maybe the carpenter is merely lazy or in a bad mood.
The carpenter could use many technical terms in support of the argument - even geometric theorems - but the client, unless a skilled carpenter as well, would be unable to validate the truth. The owner's only way to settle all those "maybes" would be to hunt for a valid second opinion from another carpenter or construction consultant. Both those options are expensive in terms of fees, project delays, and the aggravation factor.
Now imagine this scenario with a trained, professional architect involved in the project. The carpenter would be unlikely to ever present an argument for a simpler staircase unless it truly was unfeasible or beyond his ken. And it is unlikely that the architect would have designed an unbuildable staircase in the first place. The architect could tell if the carpenter simply lacked the skill, or if there truly was a problem. In either case, the architect would determine the truth without the client's being put into such an untenable situation. In the software industry we use to go the other way: we let carpenters and plumbers "assess" the architectural design...
In the software field builders rule the roost, and clients can seldom validate the design, process, cost, or outcome. Vision? Strategic use of technology? Design unity and elegance? Clients only hope the software structure will stand and not break the bank.
But once the analogy is accepted and the classical role of the architect understood and installed in the minds of people in the software industry, clarity of role and accountability will follow, cascading over to builders who will make the vision a reality. Then we will be able to move toward a meaningful lexicon that mirrors the standardized terms: engineer will mean engineer, not architect. Programmers will write code., the building blocks of software structures, not create architectures. We only mystify what we say when we use commonly understood words to mean wholly different things.
The architect's plan, or blueprint, is the high-level design, but during construction, the builders design as well. For example, the architectural plan could indicate the placement of electrical outlets and light fixtures, the number of telephone circuits, and even the requirement that the circuits be "home runs" without splices. The plan could also detail the brand, style, and even part number of light textures, outlets and switches. The master electrician, in turn, would design the circuit boxes and the general layout of the wiring. Indeed, it would be up to the electrician to design anything not specifically detailed by the architect, such as the type of wire, circuit breakers, and so forth. This is "low-level design - but it doesn't stop there. The electrician's helper, in turn, might design the way the wires run through the rafters and down the inside of the walls. Just as the master electrician design anything not included into the architectural plan, the helper designs anything not specifically addressed by the master electrician. And each level of design is supervised by the person at the next higher level.
Contrast this organized approach with that of the software industry. Have you seen software projects where design was done ad hoc, concurrently with construction? Where a representation of a high-level design outlining the true scope, organization, and behaviors of an information system did not exist, was incomplete, or was not followed? Or where the client did not understand or was surprised by the final result? Or where future needs for flexibility or expansion were not integrated into the initial construction and a part of the overall plan?
In the software industry, words have become the subject of great debate and controversy. These arguments are made all the more heated because the words in question are frequently divorced from clear, predictable processes and outcomes.