Amazing analysis, thank you for sharing it.
Unfortunately I don’t think they are wrong about it changing everything, I just don’t think it will be a positive change for most people.
In my academic field of sustainability, AI still fails to explain what economic sustainability means. It’s very simple.
I lecture and explain to my students that economic sustainability is about how we assess and manage our knowledge, innovations, wealth, and other man-made capital. I stress that it is ABSOLUTELY NOT profit and revenue.
Sure enough, my students submit assignments clearly written by AI that state economic sustainability is profit and revenue.
AI is regurgitating decades of green-washing and even bad academic articles that state economic sustainability is about profits. I use that AI god mode site to test several AI’s at once and they’ve consistently gotten this wrong for 3 years.
On the contrary - the people who tell you AI doesn’t add any value and will disappear in a few years are either lying, dumb, or completely ignorant of what AI can already do.
Read the article
I’m a software developer since the 90s, basically before the internet, we had some C books for reference and that’s it. I can tell you that I started last year to use copilot in vscode and some chatgpt on a web page, and it basically changed my world, and all my 50+ years old coworkers are amazed by what it can do really.
You are right it will not fade at all in software development.
Yep I’ve got nearly 30 years of professional dev experience and Claude code/copilot/etc are absolutely groundbreaking.
Tech in general. I’m a sysadmin in research computing. You know how many clients at prestigious universities are using AI for mathematics, biology, etc? All of them
ive heard alot of posts subs about it, they are basically just graduating learning to use AI to bs through the whole degree and wonder why they arnt getting jobs, or they arnt learning anything, plus writing peer-reviewed papers?
Completely disagree. The difficult part of software development was never writing code OR speed of delivery. It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
An example of how harmful LLMs actually are to development can succinctly be described with an issue I had a few weeks ago. I found an issue in an open source project, code was fine if a bit hard to understand. I came up with a PR to fix the problem.
In the time from me checking out the code to submitting the PR, a little less than 24 hours, the maintainer had completely rewritten the entire project with Claude. It was complete nonsense. Incredibly difficult to understand. Abstracting things that didn’t need abstracting. My PR was useless, because the entire project was new. The maintainer definitely didn’t understand the changes either. If a bug came up there’s no way AI would be able to solve it (the bug was still there even though the code was entirely new).
LLMs don’t understand the code. They just make things that look like they will work. And then a human has to maintain it (or keep paying billions of dollars for Claude to try to fix it).
I’m sorry but to me your comment is a bit misguided. You raise extremely valid point and are completely right in what you say, and yet all your argument fails to prove that software development hasn’t changed.
The difficult part was understanding requirements and problem solving: absolutely true. Yet most of the time of a developer was spent in writing code. Now it’s spent refining the analysis so that the LLM stops producing slop. And many programmers are doing it, even with all its downsides, because for them the fun part is understanding the requirements and problem solving, not writing code nor delivering fast. They are delegating those tasks to a machine, even with all the risks and issues.
Your second point (and the anecdote) further proves how programming changed. Before it was unthinkable that some random person, likely with no clue about what they are doing, would refactor an entire codebase in a night.
Both are massive changes. For the best? Arguably not, but I seriously doubt there will be any going back now.
Yet most of the time of a developer was spent in writing code.
That’s what developers told the world. Now they’re exposed, it never really took that long to write the code. (Only partly joking.)
Actually, a whole lot of time went into reading other developers’ code, getting documentation in sync with the actual implementation. And if you didn’t do all that, you tended to have a lot more bugs / vulnerabilities, etc. The LLMs are wicked fast at reviewing code, they don’t find ALL the problems, a lot of problems they do find aren’t worth fixing, but they do find more actual actionable problems per minute than most developers can find per hour in a big code base.
The LLMs are wicked fast at reviewing code
That’s because it’s not actually reviewing the code, rather just producing text that statistically looks like review comments.
So it does look vaguely useful, e.g. if code has null checks, review comments usually won’t ask for them to be added, but it’s never actually reviewing, or understanding, or reasoning, or anything else people claim.
It’s like an actor playing a doctor in a TV show, they’re not actually diagnosing patients, they’re just following a script that looks like it.
Doctors “humanity” kills a lot of people, every day, that wouldn’t have died if they did a statistical matching of their symptoms to the available research literature.
Now it’s spent refining the analysis so that the LLM stops producing slop
The thing is that actually doing this isn’t faster than writing the code, robs the practitioner of learning, and more often than not doesn’t actually happen, so you have a harder to maintain codebase with more bugs and less knowledgeable developers to maintain it.
Edit: and as a fun bonus accelerates glacier melting!
Fully agree.
Unfortunately many people would rather spin the wheel for a chance to win magically produced functioning code, rather than doing the work themselves with sure results - even if it takes the same or more time. And the kind of current politicians there are around the world proves that most people don’t give a fuck about the ice-caps (though I would also argue that it’s not so much the random person calling an LLM that is poisoning the waters - even though it does have a non-negligible effect -, rather it’s massive sociopaths in charge of the companies creating LLMs that are perfectly fine with destroying the environment and other people’s money in a vain dream of being the owner of some kind of “new order”)
This is a point that really sticks with me. Using it for the sometimes spot on cakewalk segments is a fairly productive win. By the time you stubbornly insist on driving it entirely blackbox with chat and trying to get the right results without actually touching code… Well, even when it works, it’s often more work than just doing it yourself.
Someone rebased a UI I worked on in a new version of the UI framework. As a result, there was this one odd gap in the UI in one specific place. A vibe coder spent 3 hours back and forth with the AI trying to get it to correct the gap and finally submitted their merge request. Hundreds and hundreds of lines of CSS. So I declined the merge request, open the gui, looked at the gap, hit f12, adjusted a single padding statement, and it was all good. People are struggling with defining all sorts of criteria and rigging it to let it try and try and try again and hopefully laid out every contingency, every corner case, and spent hours laying the ground work and could have done similar in a more straightforward way.
Using it for the sometimes spot on cakewalk segments is a fairly productive win.
One thing that absolutely blows my mind is how many people will say how much time it saves then with repetitive or boilerplate code. It is obvious these people have never actually tried to optimize their workflow even a little bit before. Regex replace, snippets, and keyboard macros have existed in text editors forever and are actually deterministic.
I think you’re right in general. I think juniors and those who havent yet had experience are not going to understand a goddamn thing and produce broken, insecure, unmaintainable slop.
I’ve written my share of garbage code – completely by hand! And i’m much better for it.
Once you have that experience, once you’ve written a few backends and frontends, there’s not much left to understand. Move the data from here to there. Display it, transform it, slice it up. For webdev AI is a huge force multiplier. I can make a dozen features or apps in the time it used to take me to learn one framework I was curious about. It even helps me learn faster because of how quickly I can test new patterns and ideas.
There’s certainly a right way to use it if you want to continue being edified, burning the planet down aside.
burning the planet down aside
If we only used AI for codegen, this probably wouldn’t be much of an issue. Those cat videos take more energy than building a complete app. Also, it’s all pretty new and the newest tech 40 years ago would have filled a warehouse and had the computational power of a potato, but here we are now. I expect we’ll get more efficient at it and in the ways we use it. And there’s already a huge worldwide shift in energy capture (America aside…)
I’ve written my share of garbage code – completely by hand!
Whenever I look back at old code, mine or others, the first words that usually come to mind are “what you have to understand about this is… we were on a tight schedule, we never thought this was going to be used in production, we weren’t allowed to execute the planned and contracted refactor… etc. etc. etc.”
robs the practitioner of learning
Not at all. It gives the practitioner the option of skipping the learning.
Starting in the 1990s I started skipping the learning of assembly language, compilers got good enough that I just don’t need to know how the latest SIMD/MIMD/ whatever instructions work, I just express what I want in C and gcc or whatever handles the optimization for me.
Comparing it to an (almost) entirely complete abstraction where you (almost) never have any benefit to looking under the covers like c over assembler is completely dishonest.
entirely complete abstraction where you (almost) never have any benefit to looking under the covers like c over assembler is completely dishonest.
Is it, though? In the early 1990s I could still optimize compiler output by hand, here and there. In the 1980s it was common practice and necessary in many circumstances to make complex things happen on the constrained hardware. In the 1970s there were a lot of programmers who never touched Fortran, just practiced assembly all the time because Fortran was too inefficient for their needs.
I’ll say that LLMs, this year, are something like compilers were in the 1960s - a revolutionary improvement in accessibility of coding, being able to express what you want in “natural language” - like COBOL did starting in 1959.
LLMs have plenty of pitfalls that COBOL doesn’t today, but I’ll note that Borland Turbo C++ compiler in 1991 was too damn buggy to do anything much more complex than “Hello, World.” with.
the person in your scenario is just fkin stupid. When people say that llms are helpful for coding, they aren’t talking about telling an llm to rewrite an entire codebase.
You are right, but right now we are living in a very messy reality where it’s hard to know who’s being stupid and who is using it well. One person I know that, well, I was never a huge fan of his work but at least it was somewhat serviceable is now all-in on AI and his code has been rewritten in a similar manner as the parent poster comments. He’s got no idea how it works or how it should worked, the AI decided to rewrite it in an entirely different language, and it’s a buggy mess and it never fixes the bugs without making new bugs. Then he hit his token quota 3 weeks early and basically said he was going to stop working on it because he no longer could manually work the codebase. He didn’t ask for a rewrite, but AI advised him that his language choice was a poor fit and reworked things in another language, one that none of us use to that level of seriousness. It also made it largely based on super convoluted regular expressions.
The problem is that the leadership is singing the praises of these people, they were on their ‘leaderboards’ of AI adoption and much like the craze of praising “lines of codes”, we are neck deep in the most stupid non-technical evaluation of technical work you could imagine.
Dumb people use their tools incorrectly. That’s all you’re describing.
I gave one example. In fact that’s one of the least bad examples. If that was the only problem with LLMs it wouldn’t really be that bad. But the actual reality is so much worse. But I was arguing to the point that the person I replied to made, which was about development. The problems I have with AI are not fixable without literally every government on earth taking a stand, which just will not happen.
In other words, you’re arguing that their usage without discernment is detrimental.
Cory agrees on this very same article, and it also includes the same nuance this chain is trying to give voice to:
The other question Suresh implicitly raises is: “How can you reconcile the failure of AI in the enterprise with the individual claims of skilled technologists who insist that AI is helping them do great work?” The answer is that these AI users are “centaurs” – experienced workers who are assisted by automation on terms that they set for themselves.
Thanks to their skill and experience, these workers possess discernment, the ability to tell good code from bad, and (more importantly) good uses of code-generation tools from bad. They demonstrate the adage that worker-driven automation improves quality, while capital-driven automation improves throughput.
We wouldn’t be having this conversation if LLMs had been given the chance to grow into being the same way the web did. Whereas we would be having this same conversation with the letters swapped if corporations were the ones to spawn the WWW instead of the way it came about.
The reason is the same. There is only one war.
It’s not just “usage without discernment”, it’s that AI flattens some costs which are very obvious and very measurable (coding) in such a way that it introduces or amplifies other costs which are much more diffused and hard to measure (code and design reviewing, bug fixing, maintenance, adding new requirements), plus AI is totally incapable of doing the higher level tasks that shape what code needs to be done (technical analysis, requirements analysis and in bigger companies technical architecture).
People who are non-experts, aren’t really senior domain experts or have some kinds of unbalanced expertise (they’ve never really progressed beyond being a coder, or they don’t have full life-cycle experience with big projects or they’re in an industry or position where they just make the code, shove it out the door and it’s not their problem anymore) just look at the one thing they in their ignorance think is THE cost in programming - coding - and go “hey, this AI thing is amazing” even while AI is creating all sorts of much more time consuming problems which they don’t really understand formally (they think those things are just “bad luck”, “there’s nothing we can do about avoiding this” and “it’s just the way things are in programming”) that require the time of people with higher expertise levels (i.e. who are more costly) to solve and AI doesn’t even help with the kind of stuff which if done wrongly or not at all can condemn a software project to fail before it even starts like just half-way decent Technical and Requirements Analysis.
That’s why you get some programmers going “this AI shit is amazing” whilst the really senior software development types are just nodding their heads and thinking “these people are ignorant as fuck juniors”.
It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
I don’t know… I just made a scheduling / timesheet creation app. Multi-user, overlapping clients and providers, multiple funding sources. Took 10 calendar days to make the initial app working part time, maybe 2-3 hours a day. Initially written in Python, decided at that point I’d rather have it in Go. Because the initial app had robust requirements and design docs, the translation to Go happened in less than 5 calendar days, with almost zero human involvement beyond telling the agent “continue” at each stopping point. After the Go translation was done (and debugged by the LLM to a flawless translation - only difference is that it runs faster), I was given a new timesheet to use for some of the workers, weekly instead of bi-weekly. Pay weeks start on Monday instead of Thursday. Various wrinkles about how the employees and clients and services are identified, weird sub-totals by service. All I told the LLM was: “Here’s a new timesheet that we’ll be using for some workers, design the necessary modifications and extensions to accomodate it.” It did, independently. It highlighted three shortcuts it took and I told it not to take those shortcuts, it adjusted.
That’s not quite rocket science, but it’s still impressive: to dissect the given .pdf, determine what data goes in what fields, in what formats, with what calculations, based on just reading the page, then adapt the existing app to fill it out automatically.
I truly don’t want to try your app… I don’t want all my data leaked or my PC, laptop or phone bricked just because a dumb AI-slop code error.
It’s not for you, anyway.
In my professional work, I have watched AI code reviews catch 10x more dumb slop human errors than human reviews used to the same time a year earlier, consistently for about 8 months now.
I’m not a kid doing some vibe coding, of course I know what I want and basically know how to implement it, the AI is just helping me writing it, auto-complete and all.
The overwhelming majority of software development occurs on a team. You understanding what you’ve asked an AI to do doesn’t mean jack when you need to make sure an entire team can understand it five years from now. And LLMs tend to write code that is incomprehensible to another person the next day.
? developer can write 50 lines per day, LLM help us for some auto-complete or some easy 5 lines routine/functions, but we are not blind and do not accept them blindly without understanding them. And we have pull request and reviews etc and prior to this we have architecture and class diagram and SDD and acceptance criteria and all kinds of things.
You do think that in big companies a developer will use LLM to generate thousands of line per day by himself?
I’m my experience people give tech demos, everyone is impressed, my coworkers say it makes them multiples faster, then I have more work making sure the wheels don’t fall off.
I’ve spent the last 4 months using LLMs to review / ensure that junior coders’ wheels don’t fall off.
The echo chamber on this site about AI is really quite something. Someone confidently told me Mythos probably didn’t exist and they were upvoted for it and I was downvoted for pointing out the existence of Project Glasswing.
Like, Mythos isn’t real? Fable has been out for like two months lmao
This is me right nowI was talking about roof sheathing with the latest chat gpt.
It was confident the joists should go above the sheathing.
You might want a 2nd opinion, did you ask Grok?
Vibe construction
LLMs seem to be pretty bad at home improvement sort of questions in my experience. I suspect they don’t have a team of carpenters they use to RL the models, like they do for software engineers, etc
Software is easier because you can basically ‘inspect the house’ and the ‘home inspector’ via software.
Harder to do that with a house. Also a lot of different rules based on locality.
Reminder it doesn’t know anything.
It is just mimicking text it has been programmed for.
For it to be “good” at mimicking home improvement, it’s source needed to be full of home improvement talk. The best carpenters, plumbers, whatever aren’t on reddit, or wherever talking shop. They’re working, or making OSHA jokes at the bar.
How do people work? Are there ones born knowing how to use a hacksaw?
Mostly tradies are learning by hands-on apprenticeship with a master. All that discussion is not on the internet.
How did anyone ever learn anything before chatgpt?
The better question is how will anyone ever learn after ChatGPT
I can’t be the only one who will have a hard time for me to trust any professional who became qualified after 2023.
Personally I would ask my aunt and she would be wrong.
That is because carpenters and contractors talk shit at the job site, the bar, and the gas station instead of on reddit
I work in construction and so far I must admit that I have no idea what the hype is about.
But I have decades of experience, meaning I don’t know shit about anything BUT I know how to quickly find usefull information in the codes or in pertinents books.
If I were younger I might have fallen into the trap. If so I would probably be in jail by now, because in my tests I’ve seen LLM be dangerously wrong.
I’m gonna go ahead and assume that Doctorow’s actual stance isn’t as idiotic as the title suggests.
Well, he hasn’t been describing our future for at least two decades. /sarcasm
I typed into Google “potatoes pressure cooker” to get a reminder of how long to cook them. The AI told me, and correctly said that I should use the steamer sieve, but also told me to salt the water. The water that would be under the potatoes and that wouldn’t touch them.
I asked why, and the AI said that little splashes of salt water from the boiling would land on the surface of the potatoes and salt them gently, that the salt would increase the boiling temperature, and that the vapour would be aromatic.
I pointed out that the amount of salt landing on the surface of the potatoes would be negligible, that the vapour is distilled water in gas form, and that the increase in boiling temperature from salting the water is less than 1°C. The AI said, oh yeah, you’re right on all counts. Don’t salt the water.
What it has been doing was exactly what an LLM does. It gave me the received wisdom from the internet.
That aligns with what Cory Doctorow was saying here, namely that all the AI successes were from people who already knew what they were doing and could use the AI as a tool. Not from people who had no clue and just let AI do the job.
That “AI” is a completely dumbed down one with reasoning disabled. Of course it’ll make stuff up.
I use google “ai” in when thrifting for audio gear. Minimum 80% of the time it gets something GLARINGLY wrong. “Oh ho ho! You correctly pointed out that I was completely wrong on all counts! An astute observation! Let me try again but this time give you the CORRECT information.”
I mean, to be fair, that’s a dirt cheap model you’re talking to, and Google is not exactly at the forefront of any tier of model, cheap or otherwise.
Is that you being fair or is that you rushing to defend something?
As if you don’t know these “tools” don’t work.
Exactly. I use AI primarily as a natural-language search engine, when searching for things that can quickly be independently verified. One of the best uses for it is when you know a thing probably exists, but you don’t know the name for it. You can describe to the LLM the object or problem in detail, and it will give you a name. You can then take that word and do a non-AI search to instantly confirm if it got it right.
I’ll never use an AI for something I can’t at least verify without an AI.
Salting the water would increase the boiling temperature I guess, but it seems like a waste of salt because that info isn’t meaningfully changing the cooking time
Salting the water is relevant when not using a pressure cooker. It makes no sense when you’re steaming them in a pressure cooker. AI is a good tool, sometimes. But for everyday tasks, it is irrelevant and often wrong.
I really like a song that has intro and outro samples and I wanted to know where they were from, so I googled. Google AI promptly informed me that the song has no samples. Guess that settles that.
I use AI as a writing tool, not as a substitute for authorship.
The ideas, intent, perspective, and final judgment are mine. I decide what I want to say, what belongs in the piece, what does not, and whether the finished version accurately represents me. AI may help me organize my thoughts, clarify a sentence, improve the flow, or find wording that better expresses what I already mean. That is not fundamentally different from working with an editor, dictating to a transcriber, or revising a draft after receiving feedback.
What matters is that I review the final work, approve it, and put my name on it. By doing that, I take responsibility for every sentence. If the writing is thoughtful, accurate, and effective, I am responsible for those choices. If it is careless, misleading, generic, or full of slop, that is also my responsibility. Blaming the tool would be an attempt to avoid accountability.
AI does not decide what I believe. It does not decide what I am willing to defend. It does not decide what I publish under my name. I do.
The tool may assist with the writing process, but the authorship comes from intention, judgment, selection, revision, and responsibility. The final work is mine because I chose it, shaped it, approved it, and signed my name to it.
“I dont write anything but i take credit for everything”
You’re lost in the sauce. Do you know any other writers in real life? How do they feel about this?
It gave me the received wisdom from the internet.
I mean… it gave you the amalgamated wisdom of the internet after a very complex game of telephone.
I do potatoes in my instant pot but leave the water in, it just gets absorbed. So the salt is effective in that case. I salt the potatoes themselves and toss them before adding the water, too, so they should get reasonably seasoned.
Also i use soup stock.
Mashed, though.
The story itself is pretty ridiculous. He starts with a conclusion, and then looks for stories to support that narrative. What should happen, is first establish criteria to measure if an AI project has been successful or not and then measure projects against that criteria. Instead the author uses anecdotal stories of projects he has heard failed, so it must all wrong.
Not one mention of the advances in medical research using AI, because it goes against his narrative.
It’s changed the price of RAM, that much is assured.
I mean, I’m glad he said it but anyone who’s been paying attention already knows this. The entire country is collectively waiting for the bubble to burst as it is. You also have to separate things like Machine Learning from LLMs. Two totally different use cases. They’ve not even clearly articulated any end goals. Even NASA had ‘put a man on the moon’, not just ‘more space’. From what I can tell their goals boil down to either ‘make money’ or ‘make God’. The former works great for individuals but not a whole class of technology and the latter would surely not garner trillions in investment dollars. LLMs are certainly great productivity tools but what is their goal really for a CEO to be able to fire their entire staff so they can sit down at a computer and type in ‘make my business better’ and they think that’ll work?
The whole world is lying because that’s how we get fucking money. What a stupid hole we’ve dug ourselves.
I’m so happy everyone in the comments is engaging in a thoughtful and informed manner and not just spouts uninformed nonsensical talking points from both sides to see who can shout the loudest.
I mean, the article actually says that they are not lying.
But the truth they are telling is that this kind of tools is like Yuuzhan-Vong assistants in Star Wars NJO era. It’s just further in the direction of auto-complete and recommendations.
That’s good. It also means that a real job can’t be “done by AI”.
It’s another kind of data compression, in a very fundamental and general way.
It accelerates the most things that shouldn’t be done at all. Boilerplate code that’s written again and again because of wrong processes, which are taught as standard way of thinking and doing development. Other things that could be a higher-level programming/markup language or a library. Just a frontal way of solving things that were done by hand.
That’s useful, one can’t do everything by hand.
Just - things that require real human decisions, understanding, responsibility, choice, or semantic and semiotic context that has formed in the course of one’s own life for like 20 years - that they can’t do. And that’s what forms a professional. They can extrapolate stuff that doesn’t matter.
