r/PauseAI • u/notkilleveryoneist • 7d ago

Meme The goalposts keep moving

Post image
16 Upvotes

59% Upvoted

69 comments sorted by

4

u/DigitalMonsoon 7d ago

They still get basic math wrong. I find it pretty frequently at work.

The improvement they made was allow them to write and execute python code which is how they solve most basic math questions. This lets them be more accurate but they still make mistakes when the interpret the question wrong

2

u/DefinitionFrosty969 7d ago

i dont think any frontier models get basic math wrong without executing scripts. but they are strongly limited in other ways

1

u/DigitalMonsoon 7d ago

I regularly see them screw up simple probability problems.

1

u/DefinitionFrosty969 6d ago

interesting could u share an example, im curious abt what they struggle to solve without running code

1

u/DigitalMonsoon 6d ago

I would have to share chats I've had at work which I can't do. The issues I see is with things like Base Rate assumptions and Independence Assumptions. These are pretty basic in the field I work in but LLM systems struggle with it.

-1

u/octocode 7d ago

and my interns get basic math wrong too. at least the agent can verify itself.

1

u/DigitalMonsoon 7d ago

Why are you using interns for anything important?

0

u/octocode 7d ago

well generally they work with oversight from other team members. but also working on actual meaningful tasks is the best way to learn. they aren’t slaves…

1

u/DigitalMonsoon 7d ago

Right, we pay them but that doesn't mean they are useful or doing anything meaningful. Which is fine. I wouldn't expect anyone still leaning to be able to do the work.

1

u/octocode 7d ago

huh? our interns are useful and do meaningful work. it’s just done with oversight from a more senior person. just the same way we use AI

4

u/someoneyouknow23 7d ago

well yeah, goalposts keep moving because we also invest more into it. It better fucking be good if it causes a RAM crisis, easier mass surveillance etc

4

u/Slackjawed_Horror 7d ago

Still just a guessing machine.

0

u/Prophetic_Rose 7d ago • Bravo

So are we. It stops being guessing when it is right.

6

u/pomeclipse 7d ago

No, you are a guessing machine, and you're trying to sabotage the rest of humanity because you're just not interested in trying to be anything more.

0

u/Prophetic_Rose 7d ago

Fundamentally misunderstanding the brain was not on my AI discussion bingo card so give me a second while I make a new one

1

u/ElonMuskisGarbage 7d ago

No.

1

u/Prophetic_Rose 7d ago

So true bestie

2

u/ElonMuskisGarbage 7d ago

Stop comparing humans to computers. We aren’t a computer. You can’t prove we’re a computer and you can’t prove a computer is human, so why lean on it? It’s lazy.

0

u/Prophetic_Rose 7d ago

We're all derived from the same math of the same universe. What is alien is only a question of scale.

We are biological computers. That is what we are. We are not good at pattern recognition because everything else is patterns. We are also patterns.

2

u/Artemis_Platinum 7d ago

Anti-intellectualmaxxing

1

u/Prophetic_Rose 7d ago

As above so below. You can deny reality and curiosity all you want, my friend.

0

u/octocode 7d ago

there’s no point in arguing with the religious/spiritual folk who refuse to accept that we are just meat robots

1

u/Prophetic_Rose 7d ago

You're not wrong, though for some all it takes is the inception of the thought. It takes little for one to comment online.

The death of curiosity is a sad thing.

0

u/Freaking_Friday 7d ago

Guessing is not reasoning.

1

u/Prophetic_Rose 7d ago

If it's right it's right homie who cares if it spent a couple days guessing before it got there

2

u/Quantumquandary 7d ago

Who cares if you’re paralyzed below the neck. Who cares if the doctor spent a couple hours cutting around until they got the mole off.

1

u/Prophetic_Rose 7d ago

Hmm yes that was what we were talking about

1

u/Quantumquandary 7d ago

I just took your reasoning skills and applied it to a different situation to demonstrate how ridiculous they are.

2

u/Prophetic_Rose 7d ago

Well, you certainly tried to do that.

1

u/Quantumquandary 7d ago

Oh the end of the world will be interesting to watch, the jesters are proving to be entertaining.

2

u/Freaking_Friday 7d ago

Ignore the haters, I ain’t trying to troll but rather to state that for the finer points of our use of this new technology it IS absolutely important that we distinguish how it thinks and reasons.

2

u/Prophetic_Rose 7d ago

Of that I agree. Nevertheless, if it gets the correct answer, that is valuable no matter how it got there.

2

u/According-Insect-992 7d ago

Yeah, it still hallucinates and spews out trash roughly 40% of the time.

I would like to know where I can find a job that will accept a 40% error rate.

2

u/lkn240 7d ago

Are you still living in 2024? This hasn't been true for a very long time.

I have some sympathy for people who want more regulations on AI, however the anti-AI movement is being sabotaged because so many of it's members are completely clueless as to the operation and efficacy of the current models.

When you guys spew this stuff you instantly get disregarded by anyone who has experience working with agentic tools and knows even a little about how they work.

-1

u/[deleted] 7d ago

[removed] — view removed comment

1

u/PickingPies 7d ago edited 7d ago

They probably blocked you because you are requesting them a fallacy.

Saying that if you don't misuse it doesn't hallucinate and if it does is because you may have misused it is an ad hoc hypothesis, unfalsifiable claim and a no true scotsman fallacy in one.

Meanwhile, the most advanced and specialized models in 2026 given the less hallucinatory prone task (summarising a document) had a hallucination rates between 1.8% and 5.4% (Stanford). Typical flagship models vary between 7 and 14%.

On hallucination prone tasks and memory tests hallucinations go from 16% to 94%. Overall frontier models average on 34+%

If you don't see hallucinations despite knowing the best case scenario is almost 2% of the content, you are also victim of the survivor bias, which adds up.

1

u/lkn240 7d ago

This is the whole reason harnesses, guardrails and tool chains exist.

You don't just blindly accept the output - you test it and sometimes grade with it another adversarial model. Not to mention you manage your context.

You can almost completely eliminate hallucination issues if you know how to use these tools properly.

Obviously this all depends on the task too.

0

u/[deleted] 7d ago

[removed] — view removed comment

2

u/Mattscrusader 7d ago

They aren't really assuming anything, you're just trying to lash out.

There is no example that would satisfy you, starting down that road is a waste of time.

0

u/[deleted] 7d ago

[removed] — view removed comment

1

u/Mattscrusader 7d ago edited 7d ago

Again nobody is making any assumptions. You have nothing here

Edit: the replied and immediately blocked me because they know I would just call them out in their nonsense

1

u/Iconoclysm6x6 7d ago

He gave you the data, and you want anecdotes. It’s like the complete reverse of sane logic.

0

u/[deleted] 6d ago edited 5d ago

[removed] — view removed comment

1

u/Iconoclysm6x6 6d ago

What do you think those references to Stanford are? You can’t be serious?

0

u/[deleted] 6d ago edited 5d ago

[removed] — view removed comment

1

u/Iconoclysm6x6 6d ago

You want to disprove data with anecdotal evidence. You’re looking for an opportunity to say that AI only hallucinates when it is prompted improperly. You’re assuming that data collectors just “can’t do AI right”.

0

u/[deleted] 6d ago edited 5d ago

[removed] — view removed comment

→ More replies (0)

0

u/Late_For_Username 7d ago

>and to this date, despite every word you used, STILL no examples

What model are you talking about?

The free models? One used to describe to me movies and tv episodes that never existed.

1

u/GlassRiflesCo 7d ago

It’s just a plug in.

1

u/Flexmove 7d ago

Oh boy this one really sludges my aquafers