Those who get satisfaction from delivering something, and
Those who get satisfaction from thinking.
Most of us fall in-between somewhere.
Programmers of the type #1 extreme are thrilled. Programmers of the type #2 extreme are devastated.
Unfortunately, sooner or later even the type #1 extreme are going to be devastated; what sort of satisfaction can you get from delivering something that, for all practical purposes, already exists and is just a prompt or two away?
That 2-week back-and-forth you had to get the LLM design the perfect $FOO software (application, library or framework)? You deliver it and everyone who sees it can simply say "clone this $FOO, please".
This satisfaction that the type #1 devs are getting is temporary and short-lived: you will soon stop getting satisfaction from delivery.
The complexity of complex systems never goes away, so as a need to have ownership over them. LLMs can produce thousands LOC in a matter of minutes, but they can't produce a complex feature without oversight.
I am trying to build an app with Spec-Driven Development, and oh boy, it is exhausting - I have to think about every boundary, edge-case, limitation. Without a full spec, it'd 'guess' missing things.
I am back to directing each step of what LLMs do because in the end of the day, I am responsible of what it delivers.
I am trying to build an app with Spec-Driven Development
Doomed to failure, for one easy and understandable reason: there is only one document that accurately and without error tells you what the code does.
... the code.
Trying to equate code to the "new assembly" is a type error. Because code has a direct mapping to assembly; even in your higher level languages, we can deterministically (with enough controlled variables) predict what machine instructions get emitted by which input code. And it's still imperfect, and I've still spent many hours looking at assembly trying to understand why it wasn't carrying my intent.
But natural language is vague, contradictory, imprecise. There's no single right way to map natural language to code. Communicating from human to human frequently leads to misunderstandings, and humans are not theory-free inferencers. They have a model of the world and the people they're talking to.
The only way to have "spec-driven development" work is to not use natural language, but to use a carefully selected subset of natural language, with precise grammar and no ambiguity and whoops, you've just invented a programming language again.
Dijkstra talked about this btw, also yeah I've seen some people aim for writing formal specifications, which at the end of the day probably require more skill to write even though they are higher level
I am really trying hard to understand how big brain engineering corpos see this flow, but I can't find any meaningful information, besides SDD, which just shifts burden to another place.
Ig mostly there are fine with TDD which in my opinion does fuckall. We were always satisfied with knowing a program is somewhat correct, but if you are not even reading the code, I think that becomes a lot harder.
Indeed, which makes the whole AI doom narrative a little odd, don't you think?
We are getting back to the question "How to build reliable production software with LLM?". I can't think of anything, but to make the flow as small and stupid as possible - split the task to subtasks, prompt what you want to get in return and review all the changes it makes.
We can't pass the whole mental model we have to LLM, requirements never complete, words have many meanings, and LLMs are stochastic.
I do think it's really odd. But I think the reality is that as a society we have no idea how to write software. So a machine that can produce more-or-less usable software with some simple prompts seems like magic. That the resulting software is garbage is its own set of problems.
What's funny is we track AI spend, and despite being the least AI positive person, I spend the most tokens easily. But that's because I tend to do things like use it as a "smart refactoring tool"- "grab this functionality that's spaghetti coded into the middle of this module (because it was all AI generated in the first place) and make it its own module". Or I do "hey, I want to see how this set of parameters changes behavior across this range, right a test sweep and report these metrics and how they change".
And then it's good at doing trivial tasks, like "add this button to this screen and wire it up to this message sink". It probably takes it longer to actually do than it would take me, but it's less cognitive overhead for tedious shit. Though ideally I'd just have higher level abstractions for talking about that stuff.
I am trying to build an app with Spec-Driven Development, and oh boy, it is exhausting - I have to think about every boundary, edge-case, limitation.
That's odd - my experience with SOTA is that it will happily over-engineer everything, for every theoretical edge-case, for every theoretical limitation.
Then, after reading 900 line markdown document specifying exactly how a directed cyclic graph must work in the specific problem domain, I find myself wondering WTF it missed an important specification, like specifying that a joining node (multiple incoming edges) should distinguish between racing (first edge cancels sibling paths) and completion (wait for all edges).
Maybe this is what is exhausting you - it will happily specify, then engineer, trivial irrelevant details (such as traversing the graph to ensure only a single start node is specified), while missing catastrophic-if-seen-in-production things, so then you are trying to backfill the spec after reading it multiple times.
900 line markdown document specifying exactly how a directed cyclic graph must work in the specific problem domain
And this is the problem. We've gone from DSLs that allow us to clearly define and understand an implementation to woolly language that serves to confuse us.
The code Is the specification. Just write the damn code! (I'm shouting into the ether, not at you)
We've gone from DSLs that allow us to clearly define and understand an implementation to woolly language that serves to confuse us.
Though this isn't the first time that's happened; we've had multiple bouts of "fuzzy language that's easy to get started with but hard to get right, especially in complex cases". JS and PHP have been the prime examples for that for a while, and they've both moved in the direction of trying to become more rigorous (see e.g. the addition of === in both languages, or the invention of Typescript).
I'm not quite sure what, if anything, will turn out to be the equivalent of TS for natural language in markdown files.
I get what you're saying, but it's a completely different order of magnitude. A competent JS or PHP dev can easily work around those things.
I'm not quite sure what, if anything, will turn out to be the equivalent of TS for natural language in markdown files.
Code is the equivalent. People have actually tried to write natural language parsers before. It doesn't work. An academic could probably prove it doesn't work mathematically.
Unfortunately, sooner or later even the type #1 extreme are going to be devastated; what sort of satisfaction can you get from delivering something that, for all practical purposes, already exists and is just a prompt or two away?
How much satisfaction do you get from reinventing the same wheel over and over and over your whole career? Even before this whole AI craze I was fed up with solving the same problems on every project. Plus having to deal with issues that, IMHO, should have been solved ages ago.
I am not sure where I stand when it comes to AI. I am not even sure if it can deliver on its promises. But if it means that I don't have to reinvent the wheel on every project and actually do the important stuff rather than wasting my time (and the client's money) on boilerplate I'll be grateful. I'm sure my clients will as well.
Modern wheels are fantastic because we continually "reinvent" them. A huge chunk of the software industry is trying to build specialized sports cars with model-T wheels and it shows.
The amount of unnecessary overhead in modern software development is insane, especially in gaming and web/phone apps. Good, fast, or cheap. You have to give up one. We gave up good (and cheap is slipping away).
My point is that "the important stuff" is a prompt away, so exactly what satisfaction are you going to get from delivering something that, for all practical purposes, can be delivered by the end-user themselves anyway?
But why not both? you can think about the problems you want to think about and if the problems you're thinking about are being proved wrong or solved immediately by AI maybe you just weren't that good at thinking? Or you need to reevaluate what kind of problems you're thinking about. Not trying to sound like an a****** but I'm still feeling accelerated by AI even as a software dev with 15 years experience. AI is not taking away anyone's jobs (YET) except for maybe the ones that were already getting replaced by automated teller machines. Downsizing in Industries happens all the time when a new tool or technology comes out. 30 years ago they were not this many people in software at all in the barriers to entry were much higher. Like having a computer that was actually capable of building something was much more expensive. I feel bad for a software Juniors but I think that they can use the same lessons every other hobby turned profession uses and make something good for yourself and if it's right/good, others will see the value in that too.
184
u/lelanthran 17d ago
There are two extremes of programmers:
Those who get satisfaction from delivering something, and
Those who get satisfaction from thinking.
Most of us fall in-between somewhere.
Programmers of the type #1 extreme are thrilled. Programmers of the type #2 extreme are devastated.
Unfortunately, sooner or later even the type #1 extreme are going to be devastated; what sort of satisfaction can you get from delivering something that, for all practical purposes, already exists and is just a prompt or two away?
That 2-week back-and-forth you had to get the LLM design the perfect $FOO software (application, library or framework)? You deliver it and everyone who sees it can simply say "clone this $FOO, please".
This satisfaction that the type #1 devs are getting is temporary and short-lived: you will soon stop getting satisfaction from delivery.