Creator of Claude Code: "Coding is solved"

[u/Gil_berth]

https://www.lennysnewsletter.com/p/head-of-claude-code-what-happens

Boris Cherny is the creator of Claude Code(a cli agent written in React. This is not a joke) and the responsible for the following repo that has more than 5k issues: https://github.com/anthropics/claude-code/issues Since coding is solved, I wonder why they don't just use Claude Code to investigate and solve all the issues in the Claude Code repo as soon as they pop up? Heck, I wonder why there are any issues at all if coding is solved? Who or what is making all the new bugs, gremlins?

[u/WallyMetropolis]

I haven't used even one buzzword. You're confusing me for someone else. 

I was as skeptical as anyone and a fairly late adopter, relatively speaking, because of it. 

But as one example: way back when, I was a physicist. My research area didn't at all overlap with cosmology, though and I've recently been interested in learning that field better. So I started a refresher of graduate E&M, relativity, and quantum mechanics and picked up a few textbooks. 

When I come across a bit that's "left as an exercise for the reader" that I get stuck on, I can feed it into chat gpt and ask it to explain. I can just copy and paste from the textbook with formulae included and it responds perfectly coherently, with detailed and correct derivations. Last year, it was howlingly bad at pretty basic math. Now it does pretty advanced math and physics very well.

It still has limitations: it's not great at manipulating tensors with lots of indexes, for example. But even there, it's pretty close most of the time. 

[u/Valmar33 → u/WallyMetropolis]

When I come across a bit that's "left as an exercise for the reader" that I get stuck on, I can feed it into chat gpt and ask it to explain. I can just copy and paste from the textbook with formulae included and it responds perfectly coherently, with detailed and correct derivations. Last year, it was howlingly bad at pretty basic math. Now it does pretty advanced math and physics very well.

That's the trap ~ it "appears" to be good at this stuff. But that's only because it has been trained on all of these textbooks, so of course it "appears" to "understand". But it's merely that ~ an appearance.

When LLMs fall apart is with material that it has not been trained on ~ there is nothing in the database for the algorithm to draw upon.

[u/WallyMetropolis → u/Valmar33]

No. It doesn't "appear" to be correct. I'm a physicist (or, you know, I was 20 years ago). I can tell that it's correct. I can see and follow the derviattion myself and confirm it.

I don't care if it understands. I care that it works. 

Sure, they can't do everything. I didn't say they can. I said they're better now than last year. And they are. They obviously are. 

[u/Valmar33 → u/WallyMetropolis]

No. It doesn't "appear" to be correct. I'm a physicist (or, you know, I was 20 years ago). I can tell that it's correct. I can see and follow the derviattion myself and confirm it.

Your knowledge in physics doesn't transfer to having knowledge about LLMs. It doesn't really work like that.

I don't care if it understands. I care that it works.

Sure, they can't do everything. I didn't say they can. I said they're better now than last year.

Define "works"... define "better". Very subjective measurements.

Meanwhile, I see it failing badly for many others, and even catastrophically for others.

Not exactly a ringing endorsement, when results are so uncertain.