Quotes

Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.
…
When you take that behavior and you put it behind loops, you tend to amplify it. If each iteration adds another small defense, the system slowly becomes less understandable while appearing more robust. The more hands-off you are, the more that happens. It also teaches really bad practices when tools like this are given to juniors without clear guidance.
…
If a codebase is produced by loops, reviewed by loops, patched by loops, and kept alive by loops, what happens when you no longer have access to the same class of systems? What happens when some trade restrictions take away access to the most powerful models? What if just the cost becomes unbearable? What if you and your team just lose the last remaining ability to understand the code without using the machine?

More generally, liability concerns could mean that many current use cases for agents won’t be commercially viable. Companies may not be able to profitably operate AI lawyers, doctors and media influencers if they are held responsible for what they say and do.

We’re OK with this outcome. There’s nothing in the law that requires us to accommodate AI systems if they are fundamentally untrustworthy, just as we don’t need to accommodate untrustworthy human systems.

putting your art, writing, expression out to be judged by others is an act of bravery as much as talent, and a lot of people lack bravery.

The tooling as it exists today promotes absolutely nothing like the focus required to apply it judiciously.

moral status does not appear just because the machine can emit text in the first person.

Overall I think we’re going to see a much higher quality of software, ironically around the same level than before 2000 when the net became usable by everyone to download fixes. When the software had to be pressed to CDs or written to millions of floppies, it had to survive an amazing quantity of tests that are mostly neglected nowadays since updates are easy to distribute. But before this happens, we have to experience a huge mess that might last for a few years to come! Interesting times…

The real danger is that people stop thinking. The actual trap is engineers letting the tool carry the cognitive load they were meant to build – The abdication of reason from within.

I don’t see any of this as a tragedy. Because the capacity for good craftsmaship remains. The same tools that let someone drift into shallow work are the tools that let someone else build at a level that was previously impossible.

Your code is a strategy memo to strangers who will maintain it at 2am during an outage.

[Sandboxing ]isn’t just about LLMs, but it becomes even more important now there are so many more people writing code often without knowing what they’re doing.
…
Being able to read a detailed specification and transform it into lines of code is the thing that’s being automated away. What’s left is everything else, and the more time I spend working with coding agents the larger that “everything else” becomes.

I very often use AI for inspiration. In this case, I ended up keeping a lot (not all) of the UI code it made, but I will very often prompt an agent, throw away everything it did, and redo it myself (manually!). I find the “zero to one” stage of creation very difficult and time consuming and AI is excellent at being my muse.
[…]
If the agent figures it out and I don’t understand it, I back it out. I’m not shipping code I don’t understand. While it’s failing, I’m also tabbed out searching the issue and trying to figure it out myself.
[…]
AI is very good at fill-in-the-blank or draw-the-rest-of-the-owl. My pattern here of creating scaffolding with descriptive function names, parameters, todo comments, etc. is a really common one for me and it works very well.
[…]
My last prompt to an agent is always to ask what else I might be missing. I do this regardless of if I manually wrote the code myself or not.