On Full Stack Development

Q: How does a full stack developer sleep?
 A: They don't sleep, they REST.

Almost every single company delivering some kind of product on the web since the early 2010s seems to be looking to hire full-stack developers.  Full stack development is all the rage.  If you are an employer, you can't possibly be doing it right unless you are hiring full-stack developers to do the job.  If you are a programmer, you are not particularly employable unless you can work with the full stack.

It is almost as if software architectures requiring each individual programmer to work on every single layer of the entire stack are a good thing. It is almost as if the the job market is full of programmers who are actually capable of working on every layer of the stack.  And it is almost as if they could possibly be good at that. Or at any part of it.

How did it all go so wrong?

The technological difficulties we are facing on the internet nowadays have to do with how to deliver a decent user experience on lame technologies like web browsers running javascript, how to write our server software in ways that allow us to massively scale the delivery of our services to huge numbers of concurrent users, and how to keep our systems running securely and reliably. 

The unfortunate combination of market forces and technological limitations results in the invention of unorthodox techniques out of sheer necessity, in order to dodge the roadblocks and keep delivering at least as much market value as the shop next door. They are not necessarily the best techniques, they do not even have to be particularly good, they just happen to have achieved some results somewhere, somehow, (high profile, cash-rich cases in the USA usually,) and then everyone over the entire world rushes to copy them without much thought, without even first examining whether they are necessary, or even applicable to their case. These techniques often involve some patently absurd choices, with immediately obvious drawbacks, so those who evangelize them resort to giving them nice and positive-sounding names in an attempt to downplay their deficiencies.

As Tertullian wrote, "prorsus credibile est, quia ineptum est." ("It is certainly believable, because it is absurd.")

The term "Full-Stack Developer" brings to mind programmers with a can-do attitude on their faces, who know how to do everything, shy away from nothing, are on top of their game, right? It is a very positive picture. The only flaw with the picture is that it flies in the face of Specialization and Division of Labor, which are some fairly old and tried inventions that have been the driving forces behind the Scientific Revolution and the Industrial Revolution, and pretty much the shape of the world as we know it today is owed to them. But modern techno-hipsters think they know better.

Full stack development requires work to be divided vertically rather than horizontally: each development team works on a separate feature of the product, which spans multiple software layers, instead of having different teams working on different layers. That's a very unfortunate direct consequence of the choice of the technology stack, where the modern trend is to have to put application code on the client, application code on the interface between client and server, and application code on (multiple layers of) the server. This usually means application programmers having to deal even with the transport protocol that the client and the server use to communicate with each other.  Adding insult to injury, the protocol of choice is REST, which is a particularly capricious protocol, requiring client and server code to be built with specific knowledge of the fact that they are communicating using REST, they cannot just use some abstraction where messages go into one end of a pipe, and come out of the other end of the pipe, or even better, an object is changed here, and the changes are automatically reflected there.

Due to these reasons, full stack development means that in order to add the tiniest new feature in the product, or to modify the slightest existing behavior of the product, you have to touch everything, from the abyssal depths of the persistence layer on the server to the wuthering heights of the cascading style sheets on the client, as well as every. single. layer. in-between. The job cannot be localized on a single layer, because the chosen technology stack employs no abstractions: it is application software all the way down.  And the job cannot be handed out to different developers with different specializations, because everything is so inextricably complexed together, that only someone with the entire picture in their mind can do the job.

Essentially, full stack development means "we have no frameworks and no abstractions; we are all goalkeepers, and every single one of us is playing all over the entire soccer field."

One might argue that this is not a problem, all you need to do is to hire lots and lots of bright people who will manage all this complexity. Unfortunately, this comes at a huge cost. That which used to require two teams of developers to build, now requires five or ten teams. With an increase in development effort you have a manifold increase in management and bureaucracy, and this is how you get overwhelmed development departments trying to manage huge code bases that despite their efforts are turning into trash due to accumulated technical debt. From the user's perspective, this is how you get standing annoyances and bugs that remain on the product for months under the excuse that they are not blockers. When each team works on the same layers as every other team, this also means that each team is re-inventing wheels on its own. Surely any healthy company encourages communication between development teams, but communication is also tantamount to overhead, and there is still a lot that is not communicated, because nobody can communicate the entirety of their work experience every day.

And then there is the much more severe problem of security. Have you noticed how many high profile hacks are surfacing in the news every few days lately? Ever wondered why this is happening? It is happening because it is so easy!  When the technology stack requires that every single feature must have a client footprint, this means that each application developer gets to work on their part on the client side, which in turn means that each developer has to be an expert in security. How familiar are your developers with the dozens upon dozens of well known attacks that are available to the bad guys on the client side? (https://www.owasp.org/index.php/Category:Attack) How familiar are they with just the top 10 of those attacks? (https://www.owasp.org/index.php/Top_10_2017-Top_10) Once a vulnerability is found, how easy is it to fix it?  When every single product feature has a footprint on the client, how many separate pieces of code need to be audited and fixed each time a new attack vector is discovered?

The number one difficulty in our discipline is the management of complexity. Asking developers to do full stack development is underestimating the complexity of working on a single layer. Full stack development extends the scope of the complexity that we deal with to include the complexity of multiple layers of the technology stack, and multiplies it by the extra dimension of managing how these layers interact with one another. That's what happens when you throw away all of the accumulated wisdom of the discipline and think you are re-inventing everything from scratch.

No comments:

Post a Comment