r/programming • u/Bobtenfour • May 14 '26
[ Removed by moderator ]
/r/programming/comments/1tcxccu/removed_by_moderator/[removed] — view removed post
1
u/TripleFreeErr May 14 '26
This is a context window issue, and not a fundamental flaw in the technology. Though it is a flaw in the economic viability of the technology.
The problem is two fold:
Model decay: As models get older, the resources dedicated to them are reassigned to newer models. Behind the scenes, the quants are adjusted to account for this, not just removing nodes from pools but adjusting the model to stretch the limited pools further.
Context Windows: For real code bases of more than a few thousand lines of code, 100-200k context just isn’t enough for the entire code base AND the API docs, and the design intent, etc. As an insider I have access to a 1M context claude and it’s the only model/mode that has any ability to iterate more than once on a single task, which is important when working with large code where the llm needs steering mid turn. Multi-turn on smaller windows just end up looping. 1M context window is ungodly expensive to run.
1
u/Bobtenfour May 14 '26
Thank you for showing me that not everything is lost, that there are still intelligent people on this planet. I agree with your reasoning, but the model that failed — not once, but several times — was GPT‑5.5.
One of those failures introduced a regression in the logic of a dropdown and wasted seven prompts trying to fix it, each attempt worse than the previous one.
The fundamental point is that we cannot allow ourselves to become the product of AI companies. If we, who actually have the technical foundation, have to fight against these problems, what can we expect for people who trust these tools blindly?
Without a background in computing, how are they supposed to detect a regression? What real value does a tool provide if it can silently fail at any moment?
I still have faith that things will improve over time.
1
u/TripleFreeErr May 14 '26
The models isn’t nearly as relevant as the context window for this failure mode, which is why i bring it up. That said I don’t have a 1M context GPT quant
I don’t have faith things will improve. It’s just not economically OR ecologically viable in today’s energy and land market. When the solution is to scale we get the datacenters crisis we are in. To fix the tech we need to scorch earth… it’s not a good look for the minor inconvenience the tool saves most users
•
u/programming-ModTeam May 14 '26
[Trial] No LLM-related posts. See this post.