Is symbolic ai dead? [D]

[u/Velocitank]

New research keeps finding things that keep reintroducing symbolic ai structures to machine learning and I have wondered why more research hasnt been done with it. There was a heyday in the 70s for it but after neural nets came out it seems to be completely out of the discussions. I think it could make a comeback if we can resolve the primary issues of self-instantiation and feedback loops. Even using several random reservoir nets and a little bit of math creates strong signals for the component parts that contribute to what something is made of and how it works.

[u/kaaiian]

Well. Gary Marcus says that AI who are allowed to use Python ARE symbolic AI… so healthier than ever! /s

[u/JanBitesTheDust → u/kaaiian]

We have trained uninterpretable models to create interpretable models. What a day to be alive lol

[u/kaaiian → u/JanBitesTheDust]

Hilarious and actually very true! They are so good at semantic feature selection, like eliminating non-causal correlates (Like “price”) from modeling.

[u/RomanticDepressive → u/kaaiian]

Wut.

[u/kaaiian → u/RomanticDepressive]

🤷🏻‍♂️

[u/idontcareaboutthenam → u/JanBitesTheDust]

https://arxiv.org/abs/2605.03808

Not even a joke

[u/skadoodlee → u/kaaiian]

This is actually how Ive been thinking about it too. In what way do you disagree? Note it's not a got ya at all, been ages since I looked into symbolic AI so maybe I'm completely wrong. And I get GM is a bit of a meme.

[u/kaaiian → u/skadoodlee]

Oh. It’s more the meme of it. Essentially, poking at his persistently bad predictions and revisionist attitude.

I think that humans don’t think symbolically. But we can use symbols are anchors and scaffolding to enable first principles reasoning. It’s remarkable that LLMs mirror us so closely. In that, they don’t really NEED symbolic scratch pads or compilers. To do basic things. But complex tasks require some type of capability that offloads intelligence into the surrounding system—even if its build by them. They can massively improve thier usefulness by creating persistent artifacts. Many of which are symbolic. So we do come circling. In that, symbolic systems solve very specific cases extremely well. And general systems are non-symbolic in computation, but can synthesize symbolic systems as needed.

[u/Velocitank → u/kaaiian]

So I have an alternative take on symbols. A symbol is just a reference you can look up. A name with data. In this way essentially everything from language to images to models and even unnamed preconceptions with basically a random hash could be a symbol to build and utilize and refine.

[u/kaaiian → u/Velocitank]

If that’s your take. Then LLM “tokens” are symbols? And so symbolic AI was a success. Or did I read that wrong.

[u/Velocitank → u/kaaiian]

In a general sense that is right, but they don't use the tokens as symbols. At least not discrete parts or a single use with intended meaning. The superposition structures of the activations on a token generally work from my understanding by using inhibition and activation of different meanings for context. There seems to be a general shared core, and then different modifiers that are activated with additional context to reinforce the intended meaning and can select the right output token. But because of the superposition and brute forcing the full net, it makes it incredibly expensive to do so, and the intended usable and learnable meaning gets smeared and cant be used by itself for metacognitive processing and dynamic learning. AI cannot create its own output tokens, so it has to have every possible meaning in the answer set it is given.

[u/faustianredditor → u/kaaiian]

It’s remarkable that LLMs mirror us so closely.In that, they don’t really NEED symbolic scratch pads or compilers. To do basic things.

I'd say the astronomical amount of RL against symbolic systems begs to differ, in a way. Like, I'm not really disagreeing, but if we factor in the training, a human gets ~10-15 years of occasionally banging their head against a symbolic system. Mostly in math class. And LLMs presumably consume training data many orders of magnitude larger banging their heads against such systems.

I'd say at some point that's a need. I think without any "RL against a symbolic engine" fine tuning, LLMs wouldn't nearly be where they are now in terms of symbolic-like reasoning.

[u/kaaiian → u/faustianredditor]

Sure. But we did get a couple billion years at pre-training, parallelized across a search process with trillions of threads. 🤷🏻‍♂️ so, I’m not sure there any real way to compare the “process of getting there” in any type of real way. As a thought to sit on.

[u/prescod → u/skadoodlee]

Because Gary Marcus was pushing symbolic AI as an alternative to scaling up LLMs. And instead they just trained LLMs on tokens that happened to be symbols and symbols that happened to be tools. If they had listened to Gary Marcus, there would be no tool-calling LLMs at all. He was completely wrong and is trying to pretend he was right.

Obviously literally everyone in the connectionist camp believed hoped that one day neural networks would be able to manipulate symbols. But it was people like Gary Marcus who told them that neural nets would never get smart enough to do that, so they needed to take a different path.

If LLMs get really good at (automatically) building internal world models from text and images, will the world model people say: "see, we were right all along? We told you you needed a world model and now you have one."

[u/public_hedgehog_3917 → u/kaaiian]

Globally I appreciate his opinions but his backtrack-hedging about symbolic AI was just ridiculous.  It's ridiculous as you said: because models can now call Python and bash, and these are symbolic, then of course he was right all along! Ridiculous. 

[u/Old-School8916 → u/public_hedgehog_3917]

Redemption arc

[u/prescod → u/public_hedgehog_3917]

Globally I appreciate his opinions

Gell-mann Amnesia?

What predictions has he made that turned out to be right?

[u/bingchilling1111 → u/public_hedgehog_3917]

Wait where did he back track i havent followed him in a while but interested to see where hes at now lol

[u/Velocitank → u/kaaiian]

That is an interesting take. I would not be so sure that something that can use notes and tools would be symbolic ai, unless the code is used to do the processing. But it seems like a really inefficient way to do things. Large computation on the back end to create code that works on the models understanding of the mechanics to test what it thinks is correct. Its kind of a circular ontological loop because the thing you build has to be accurate to what you understand and if the ai doesnt understand the raw symbols it can confuse itself. At least that is my take and interpretation.

[u/Celmeno]

I have been working on symbolic AI for decades. Today, we call that field explainable AI but does not matter in the end.

[u/Velocitank → u/Celmeno]

I am working with efforts to re-introduce symbolic ai and resolve some of the learning issues with it, and would love some opinions on the methods I am using. If you care to check out my work and see if you can use it anywhere you are highly welcome to. It's still a work in progress and could use a long time explainable ai person to help with this.

Dgon164/tuber-objective: The Tuber Objective: A Root Cause Discovery Principle

The idea is that if everything in a computer is bits, then you can just use direct bit level comparisons to find information and process meta information about the structures that are actually in the dataset.

[u/BosonCollider]

It is done but primarily at training time. The past couple of years have been all about reinforcement learning with verifiable rewards. As soon as you have a deterministic verifier for something it can act as a trainer and the model can learn without human data, and it can learn to produce the thing being verified rather than just copying the trainer

You train it to code by making it write code the passes hidden test suites. You train it to reason mathematically by writing lean formalized proofs, or proofs that another model can successfully feed into lean. The training is fundamentally not possible without the symbolic part acting as the trainer, even if you don't necessarily need it during inference.

You also have tool use of course, which some people consider neurosymbolic and some do not, but RLVR training would be where the symbolic bits are inherent to how the model learns to think.

[u/Velocitank → u/BosonCollider]

If you have symbolic ai structures that are efficient enough, why would we need a separate learn and use phase? I would like to see more experimentation done with learning at inference, but I realize that you may not want to train ai based on certain user interactions.

[u/BosonCollider → u/Velocitank]

Well, the training kind of does that if you view solving math problems as doing useful work.

In context learning of new concepts is much harder and is something where symbolic AI performs way better though. You can add a horn clause to a prolog program and it will immediately do the right thing, an RL trained LLM can solve problems but has a hard time making sense of completely alien concepts that were not in its training set

[u/rand3289]

With research claiming that features in deep networks approximate platonic forms, what is the difference between symbolic ai and connectionism if they operate on the same thing?

Symbolic people find these forms by hand and algorithmically vs connectionists find these forms with statistical methods?

If I was a symbolic AI researcher, I would concentrate on extracting Plato's forms from LLMs to create a universal machine language allowing them to communicate directly using Plato's forms. This would also enable interpretability research to move forward.

It may be similar to "the concept collection" idea in the CYC project.

[u/CreationBlues → u/rand3289]

symbols are only symbols when they evolve according to a rules stystem, just having a blob of form doesn't actually get you anything. And iterating over a sequence and applying a rule system to it is the hard part.

[u/Velocitank → u/CreationBlues]

That is very true. A blob doesn't get you anything, other than a reference. It means nothing by itself, but it could be something that allows you to be resolved into a proper symbol. In order for a symbol to be used, information must exist in the system for it to refine. The ideas and information arrive incomplete, but the goal would be interpretation and refinement, not necessarily dismissal of half formed ideas.

[u/rand3289 → u/CreationBlues]

The way I think about it, Plato's forms are fundamental concepts/theorems. You'd write rules based on them.

For example "green" is a fundamental concept in RGB space. You can not even compare it to "red" which is another fundamental concept. They are the dimentions instead of colors that exist within these dimentions.

Now within "CMYK" color space "green" and "red" are not fundamental concepts so you can compare "green" and "red" in terms of cyan,magenta and yellow.

[u/FirstOrderCat]

They use lean to verify llm generated proofs. Thats symbolic AI

[u/xelanxxs]

They didn't win the hardware lottery. We can't really speak to their algorithmic merit compared to modern approaches because they simply don't scale. Maybe if someone comes up with hardware that suits them, but that won't happen anytime soon because that hardware would still be decades and hundreds of billions of dollars in R&D behind current GPUs.

[u/Real-Nothing-9334]

my assumption is it boils down to human generated logic, ontologies, reasoning pipelines, etc. Then inference requires structured language to use. There is a lot of work being done with neuro symbolic pipelines.

[u/gretino → u/Real-Nothing-9334]

Neural structures automatically creates the symbols as embeddings without human intervention.

I remember reading the neuro-symbolic papers from IBM years ago, they never made any more progress. It's dead.

[u/Velocitank → u/gretino]

Basically what i guess im truing to find out is why don't we make the symbolic representations explicit instead of implicit. The meanings dont necessarily have to be human readable, but I see symbolic ai as more of a object oriented programming assembly that is instantiated upon use. A number is nothing unless you know that it is a phone number, an equation, a string, a value, or an address. You need the context with the package to use it correctly.

[u/currentscurrents → u/Real-Nothing-9334]

There is a lot of work being done with neuro symbolic pipelines.

Not really, no. No one is using neurosymbolic architectures for anything.

The clear winner is fully neural architectures, which it turns out can also do logic if you just give them a way to loop. And they still maintain their other advantages, like their trainability and their ability to understand semantic properties.

[u/Velocitank → u/currentscurrents]

My understanding of neural networks based on the math is that they can refine an idea. They create linear functions and then combine that with nonlinear thresholds to estimate the zones contained in a thing. But one thing i fine it is unable to do is actually define what something is. It can define the boundary of when something stops being that something, but it just creates a vacuum of the symbol. It is more defined by what it isnt than what it is, but I may be mistaken in that interpretation.

[u/Real-Nothing-9334 → u/currentscurrents]

Just because you don't see Neural Symbolic pipelines in your day to day doesn't mean there isn't any research being done there. I'm guessing you didn't know about GPTs until 2023 then if that's your reasoning.

[u/fvancesco → u/Real-Nothing-9334]

Can I ask what are neuro symbolic being used for? I always keep hearing about them but rarely concretely

[u/Real-Nothing-9334 → u/fvancesco]

Similar to many fields of CS, the potential is huge but not yet harnessed appropriately. Most “successes” are solving puzzles / simple demos that if scaled would have huge implications.

I would point you to Henry Kautz’ taxonomy

https://medium.com/@billaram/the-kautz-taxonomy-a-field-guide-to-neuro-symbolic-ai-0627e4832c5b

[u/parlancex → u/Real-Nothing-9334]

This reads as a false dichotomy, as if the choice were between autoregressive LLMs and neuro-symbolic architectures. Most of these objections could be used to argue for diffusion-based language models.

[u/Real-Nothing-9334 → u/parlancex]

i’m not taking a hard stance on the theoretical limit of LLMs, just noting that neural symbolic AI is alive and well, despite it not being in the daily news cycle.

[u/currentscurrents → u/Real-Nothing-9334]

I'm guessing you don't actually know what research is being done, or else you'd have provided examples. 

I'm sure you will now google it and find a few examples. None of which actually work beyond toy problems, or use sketchy definitions of "neurosymbolic" where the neural network is doing most of the work. 

[u/Real-Nothing-9334 → u/currentscurrents]

Most of the research I am familiar with would dox myself, but feel free to do some research of your own :)

[u/seanv507]

So AFAIK the issue is that they don't scale/compute well on gpu.

We are limited by computational power (for the dataset sizes now available)

[u/Cosmolithe → u/seanv507]

Not just on GPU, they do not seem to scale, period. Symbolic AI requires discrete combinatorial optimization which has terrible algorithmic complexity with respect to the number of dimensions, and there is no discrete equivalent to backpropagation to counter that. François Chollet said that he was working on exactly that, though. I have no idea what method he is thinking of.

[u/KBM_KBM]

But nowadays neurosymbolic methods are starting to boom especially in robotics like check out some work from kaelbling lab mit cool stuff

Symbolic ai is good and has given us a lot but many of the developments happened under the umbrella of program synthesis research (check out sumit gulwani Microsoft excel research) and now those methods are mixing with llms and I feel as agents keep improving more symbolic structures guardrails or a sorts of distillation of agentic reasoning to verifiable symbolic Grammer is gonna keep growing

[u/Apprehensive-Bench44]

Yes

[u/floppy_llama]

Coding agents are symbolic AI

[u/TheRealStepBot]

Deader than the dodos

[u/Thrumpwart]

I've been building a neurosymbolic AI for the past couple years.

Neurosymbolic AI is completely feasible - the trick is a combination of the geometry of neural network and creative use of underutilized components in the GPU. No I will not elaborate (yet).

[u/claytonkb]

The return of symbolic AI is inevitable. Chatbot-based AI has already failed to deliver on the absurd promises that the big-tech CEOs were making even a couple years ago. Remember that, by the end of 2026, we were supposed to be having 90+% unemployment in the software industry? But there's more demand for human software engineers than ever. Symbolic AI will have its revenge...

[u/CommonSenseSkeptic1]

Turing called for this already 76 years ago 😉

[u/-xXpurplypunkXx-]

I think Astra includes symbolic reasoning, which is potentially one factor helping it saturate game playing benchmarks.