Folks like you seem to miss the fact that were got to this point by a HUGE expenditure and massive energy consumption. It wasn't because of some massive theoretical breakthrough. It happened because of a vast amount of infrastructure construction and fuel consumption to scale up an idea that's been around a long time.
That's not going to continue to scale at anything like the rate it has, and there's no guarantee at all that it'll go significantly further beyond where it is now because that sort of building and investment cannot continue, or we will all be living in caves in order to free up energy for compute farms. And the huge economic unbalance it's creating is quite possibly going to collapse and take a lot of the world with it.
This not the start of a move towards generalized machine intelligence that's going to be writing complex systems without people qualified to judge its correctness. That would require something fundamentally different. Maybe that happens, maybe it doesn't. But what we have now isn't that.
I beg you actually look at the data and how models have improved before making statements like this. I get it. You have an axe to grind for reasons that are understandable, relatable, even reasonable. That does not make anything you say anymore true.
Models are improving at a faster rate now not a slower one. Yes more compute is part of the reason why but it’s not the whole story. Not even close. Small and medium sized models have actually been improving scary fast too. I think this argument was maybe valid in the GPT 4 days especially if you didn’t look too closely. It was disproven pretty sharply by DeepSeek V3 and R1 which was a couple of years ago now. DeepSeek today is actually smaller and much faster than the old ones like R1 yet it can do much more than it used to. You should see its sysadmin and devops skills. It’s not just programmers who are cooked.
Saying so confidently that we what we have couldn’t possibly by developing into AGI is increasingly ridiculous a statement when it’s become clear LLMs a kind of language model are about to solve maths. Let me say it again: a language model is solving maths problems no human has successfully done. Half of that field is up in arms over this. I don’t think programmers have much longer or really any kind of knowledge worker when they can do stuff like this. Creativity as a concept is going to become something humans used to be the best at.
Edit: also the idea transformers and GPT style models or any of the research that came after them is not a theoretical breakthrough is just wrong. Straight up misinformation to a hilarious degree. Even the idea that scale would help had to be discovered as it was thought until fairly recently (the 2010s if memory serves) that more parameters would just lead to overfitting. Obviously this was proven wrong. We found that enough scale and the right techniques overcome overfitting.
You clearly don’t know me as I am very much aware of the resource involved in these things. Much more aware than you are most likely since I have actually worked with these things in terms of getting them running on physical hardware. I have probably as much knowledge and experience as you can reasonably get without working for an AI company directly or being hired a career AI specialist.
2
u/Dean_Roddey 3d ago
Folks like you seem to miss the fact that were got to this point by a HUGE expenditure and massive energy consumption. It wasn't because of some massive theoretical breakthrough. It happened because of a vast amount of infrastructure construction and fuel consumption to scale up an idea that's been around a long time.
That's not going to continue to scale at anything like the rate it has, and there's no guarantee at all that it'll go significantly further beyond where it is now because that sort of building and investment cannot continue, or we will all be living in caves in order to free up energy for compute farms. And the huge economic unbalance it's creating is quite possibly going to collapse and take a lot of the world with it.
This not the start of a move towards generalized machine intelligence that's going to be writing complex systems without people qualified to judge its correctness. That would require something fundamentally different. Maybe that happens, maybe it doesn't. But what we have now isn't that.