> Reality Has A Surprising Amount Of Detail

Every time we make progress complexity increases and it becomes more difficult to make progress. I'm not sure why this is surprising to many. We always do things to "good enough", not to perfection. Not that perfection even exists... "Good enough" means we tabled some things and triaged, addressing the most important things. But now to improve those little things now need to be addressed.

This repeats over and over. There are no big problems, there are only a bunch of little problems that accumulate. As engineers, scientists, researchers, etc our literal job is to break down problems into many smaller problems and then solve them one at a time. And again, we only solve them to the good enough level, as perfection doesn't exist. The problems we solve never were a single problem, but many many smaller ones.

I think the problem is we want to avoid depth. It's difficult! It's frustrating. It would be great if depth were never needed. But everything is simple until you actually have to deal with it.

> As engineers, scientists, researchers, etc our literal job is to break down problems into many smaller problems and then solve them one at a time.

Our literal job is also to look for and find patterns in these problems, so we can solve them as a more common problem, if possible, instead of solving them one at a time all the time.

Very true. But I didn't want to discuss elegance and abstraction as people seem to misunderstand abstraction in programming. I mean all programming is abstraction... abstraction isn't to be avoided, but things can become too abstract

Alternate take: what agents can spit out becomes table stakes for all software. Making it cohesive, focused on business needs, and stemming complexity are now requirements for all devs.

By the same token (couldn’t resist), I also would argue we should be seeing the quality of average software products notch up by now with how long LLMs have been available. I’m not seeing it. I’m not sure it’s a function of model quality, either. I suspect devs that didn’t care as much about quality hadn’t really changed their tune.

how much new software do we really use? and how much can old software become qualitatively better without just becoming new software in different times with a much bigger and younger customer base?

I misunderstood two things for a very long time:

a) standards are not lower or higher, people are happy that they can do stuff at all or a little to a lot faster using software. standards then grow with the people, as does the software.

b) of course software is always opinionated and there are always constraints and devs can't get stuck in a recursive loop of optimization but what's way more important: they don't have to because of a).

Quality is, often enough, a matter of how much time you spent on nitpicking even though you absolutely could get the job done. Software is part of a pipeline, a supply chain, and someone is somehow aware why it should be "this" and not better or that other version the devs have prepared knowing well enough it won't see the light of day.

Honestly, in many ways it feels like quality is decreasing.

I'm also not convinced it's a function of model quality. The model isn't going to do something if the prompter doesn't even know. It does what the programmer asked.

I'll give a basic example. Most people suck at writing bash scripts. It's also a common claim as to LLMs utility. Yet they never write functions unless I explicitly ask. Here try this command

  curl -fsSL https://claude.ai/install.sh | less

(You don't need to pipe into less but it helps for reading) Can you spot a fatal error in the code where when running curl-pipe-bash the program might cause major issues? Funny enough I asked Claude and it asked me this

  Is this script currently in production? If so, I’d strongly recommend adding the function wrapper before anyone uses it via curl-pipe-bash.                

The errors made here are quite common in curl-pipe-bash scripts. I'm pretty certain Claude would write a program with the same mistakes despite being able to tell you about the problems and their trivial corrections.

The problem with vibe coding is you get code that is close. But close only matters in horseshoes and hand grenades. You get a bunch of unknown unknowns. The classic problem of programming still exists: the computer does what you tell it to do, not what you want it to do. LLMs just might also do things you don't tell it to...

You sound bored. If we triple head count overnight, we'd only slow our backlog, temporarily. Every problem we solve only opens up a larger group of harder problems to solve.

Why wouldn't we find new things to do with all that new productivity?

Anecdotally, this is what I see happening in the small in my own work - we say yes to more ideas, more projects, because we know we can unblock things more quickly now - and I don't see why that wouldn't extend.

I do expect to see smaller teams - maybe a lot more one-person "teams" - and perhaps smaller companies. But I expect to see more work being done, not less, or the same.

What new things would we do? I do contracting so maybe I'm lowest-bidder-pilled but I feel like drops in price in lean organizations sre going to eat the lunch of shops trying to make more quality software in most software disciplines.

How much software is really required to be extensible?

There is tons of stuff to do. Lots of technologies out there that need to be invented and commercialized. Tons of inefficient processes in business, government, and academia to improve.

None of this means that it will be the kinds of professional specialized software development teams that we're used to doing any of this work, but I have some amount of optimism that this is actually going to be a golden age for "doing useful things with computers" work.

I still think it's more likely to be more of the same thing but with less people.

One man shops being the ideal, and I don't think there will be proportionately more of them

This doesn't mesh with anything that has happened in the development of computing, or technology in general.

I dispute technology in general. There are plenty of examples where industrialisation led to a drop in quality, a massive drop in price, and a displacement of workers.

It hasn't happened in software yet. I suppose this has to do with where software sits on the demand curve currently.

I'm imagining a few more shifts in productivity will make the demand vs price derivative shift in a meaningfully different way, but we can only speculate.

I think you are misunderstanding the point I'm making. I agree that "writing code" is likely to be commoditized by AI tools, much like past industrialization disruptions. But I think there is going to be more things to do in the space of "doing useful things with computers", analogous to how industrialization creates new work further up the value chain.

Of course it often isn't the same people whose jobs are disrupted who end up doing that new work.

If LLMs are good at writing software, then there's lots of good software around written by LLMs. Where is that software? I don't see it. Logical conclusion: LLMs aren't good at writing software.

Are you trying to make a distinction between writing software vs writing code? LLMs are pretty great at writing good code (a relative term of course) if you lay things out for them. I use Claude Code on both greenfield new projects and a giant corporate mono repo and it works pretty well in both. In the giant mono repo, I have the benefit of many of my coworkers developing really nice Claude.md files and skills, so that helps a lot.

It’s very similar to working with a college hire SWE: you need to break things down to give them a manageable chunk and do a bit of babysitting, but I’m much more productive than I was before. Particularly in the broad range of things where I know enough to know what needs to be done but I’m not super familiar with the framework to do it.

Presumably they are writing the same quality software faster, the market having decided what quality it will accept.

Once that trend maxes out it’s entirely plausible that the level of quality demanded will rise quickly. That’s basically what happened in the first dot com era.

I'm not convinced. Honestly it seems like we're in a market of lemons and I don't know how we escape the kind of environment that is ripe for lemons. To get out requires customers to be well informed at the time of purchase. This is always difficult with software as we usually need to try it first and frankly, the average person is woefully tech illiterate.

But these days? We are selling products based on promises, not actual capabilities. I can't think of a more fertile environment for a lemon market than that. No one can be informed and bigger and bigger promises need to be made every year.

"(...) maybe growing vegetables or using a Haskell package for the first time, and being frustrated by how many annoying snags there were." Haha this is funny. Interesting reading.

While this is absolutely true and I've read this before, I don't think you can make this an open and shut case. Here's my perspective as an old guy.

The first thing that comes to mind when I see this as a counterargument is that I've quite successfully built enormous amounts of completely functional digital products without ever mastering any of the details that I figured I would have to master when I started creating my first programs in the late 80s or early 90s.

When I first started, it was a lot about procedural thinking, like BASIC goto X, looping, if-then statements, and that kind of thing. That seemed like an abstraction compared to just assembly code, which, if you were into video games, was what real video game people were doing. At the time, we weren't that many layers away from the ones and zeros.

It's been a long march since then. What I do now is still sort of shockingly "easy" to me sometimes when I think about that context. I remember being in a band and spending a few weeks trying to build a website that sold CDs via credit card, and trying to unravel how cgi-bin worked using a 300 page book I had bought and all that. Today a problem like that is so trivial as to be a joke.

Reality hasn't gotten any less detailed. I just don't have to deal with it any more.

Of course, the standards have gone up. And that's likely what's gonna happen here. The standards are going to go way up. You used to be able to make a living just launching a website to sell something on the internet that people weren't selling on the internet yet. Around 1999 or so I remember friend of mine built a website to sell stereo stuff. He would just go down to the store in New York, buy it, and mail it to whoever bought it. Made a killing for a while. It was ridiculously easy if you knew how to do it. But most people didn't know how to do it.

Now you can make a living pretty "easily" selling a SaaS service that connects one business process to another, or integrates some workflow. What's going to happen to those companies now is left as an exercise for the reader.

I don't think there's any question that there will still be people building software, making judgment calls, and grappling with all the complexity and detail. But the standards are going to be unrecognizable.

Is the surprising amount of detail an indicator that we do not live in a simulation, or is it instead that we have to be living inside a simulation because it doesn't need all this detail for Reality, indicating an algorithmic function run amuck?

Reality is infintely analog and therefore digital will only ever be an approximation.

I once wrote software that had to manage the traffic coming into a major shipping terminal- OCR, gate arms, signage, cameras for inspecting chassis and containers, SIP audio comms, RFID readers, all of which needed to be reasoned about in a state machine, none of which were reliable. It required a lot of on the ground testing and observation and tweaking along with human interventions when things went wrong. I’d guess LLMs would have been good at subsets of that project, but the entire thing would still require a team of humans to build again today.

Sir your experience is unique and thanks for answering this.

That being said, someone took the idea of you saying LLM's might be good at subsets of projects to consider we should use LLMs for that subset as well

But I digress because (I provided more in depth reasoning in other comment as well) because if there is an even minute bug which might slip up past LLM and code review for subset of that and for millions of cars travelling through points, we assume that one single bug in it somewhere might increase the traffic/fatality traffic rate by 1 person per year. Firstly it shouldn't be used because of the inherent value of human life itself but even from monetary sense as well so there's really not much reason I can see in using it

That alone over a span of 10 years would cost 75 million-130Million$ (the value of life in US for a normal perosn ranges from 7.5 million - 13 million$)

Sir I just feel like if the point of LLM is to have less humans or less giving them income, this feels so short sighted because I (if I were the state and I think everyone will agree after the cost analysis) would much rather pay a few hundred thousand dollars to even a few million$ right now to save 75-130 Million$ (on the smallest scale mind you, it can get exponentially more expensive)

I am not exactly sure how we can detect the rate of deaths due to LLM use itself (the 1 number) but I took the most conservative number.

And that is also the fact that we won't know if LLM's might save a life but I am 99.9% sure that might not be the case and once again it wouldn't be verifiable itself so we are shooting things in the dark

And we can have a much more sensitive job with better context (you know what you are working at and you know how valuable it is/can save lives and everything) whereas no amount of words can convey that danger to LLM's

To put it simply, the LLM might not know the difference between this life or death situation machine's code at times or a sloppy website created by it.

I just don't think its worth it especially in this context at all even a single % of LLM code might not be worth it here.

> we won't know if LLM's might save a life

I had friend who was in crisis while the rest of us were asleep. Talking with ChatGPT kept her alive. So we know the number is at least one. If you go to the Dr ChatGPT thread, you'll find multiple reports of people who figured out debilitating medical conditions via ChatGPT in conjunction with a licensed human doctor, so we can be sure the numbers greater than zero. It doesn't make headlines the same way Adam's suicide does, and not just because OpenAI can't be the ones to say it.

Great for her, I hope she's doing okay now. (I do think we humans can take each other for granted)

If talking to chatgpt helps anyone mentally, then sure great. I can see as to why but I am a bit concerned that if we remove a human from the loop then we can probably get way too easily disillusioned as well which is what is happening.

These are still black boxes but in the context of traffic lights code (even partially) feels to me something that the probability of it might not saving a life significantly overwhelms the opposite.

ChatGPT psychosis also exists so it goes both ways, I just don't want the negative voices to drown out the positive ones (or vice versa).

As far as traffic lights go, this predates ChatGPT, but IBM's Watson, which is also rather much a black box where you stuff data in, and instructions come out; they've been doing traffic light optimization for years. IBM's got some patents on it, even. Of course that's machine learning, but as they say, ML is just AI that works.

I've had good luck when giving the AI its own feedback loop. On software projects, it's letting the AI take screenshots and read log files, so it can iterate on errors without human input. On hardware projects, it's a combination of solenoids, relays, a pi and pizerow, and a webcam. I'm not claiming that an AI could do the above mentioned project, just that (some) hardware projects can also get humans out of the loop.

Don’t you understand? That’s why all these AI companies are praying for humanoid robots to /just work/ - so we can replace humans mentally and physically ASAP!

I'm sure those will help. But that doesn't solve the problem the parent stated. Those robots can't solve those real world problems until they can reason, till they can hypothesize, till they can experiment, till they can abstract all on their own. The problem is you can't replace the humans (unilaterally) until you can create AGI. But that has problem of its own, as you now have to contend with previously creating a slave class of artificial life forms.

  > until they can reason, till they can hypothesize, till they can experiment, till they can abstract all on their own

at that point we will have to let them vote...

I completely agree - my comment was sarcastic and in jest.

My bad. Getting hard to tell these days lol

No worries - you’ve added useful context for those who may be misguided by these greedy corporations looking to replace us all. Maybe it helps them reconsider their point of view!

If that is the only thing that you took out of that conversation, then I don't really believe that that job might've been suitable for you in the first place.

Now I don't know which language they used for the project (could be python or could be C/C++ or could be rust) but its like "python would have been good at subsets of that project", so some impact already and these python tools only get better

Did python remove the jobs? No. Each project has their own use case and in some LLM's might be useful, in others not.

In their project, LLM's might be useful for some parts but their majority of the work was doing completely new things with a human in feedback.

You are also forgetting trust factor, yes lets have your traffic lights system be written by a LLM, surely. Oops, the traffic lights glitched and all waymos (another AI) went beserk and oops accidents/crash happened which might cost millions.

Personally I wouldn't trust even a subset of LLM code and much rather have my country/state/city to pay to real developers that can be accountable & good quality control checks for such critical points to the point that no LLM in this context should be a must

For context, if LLM use can even impact 1 life every year. The value of 1 person is 7.5-13 million$

Over a period of 10 years in this really really small glitch of LLM, you end up in 10 years losing 75 million$

Yup go ahead save a few thousand dollars right now by not paying people enough in the first case to use LLM to then lose 75 million $ (on the least case scenario)