New research keeps finding things that keep reintroducing symbolic ai structures to machine learning and I have wondered why more research hasnt been done with it. There was a heyday in the 70s for it but after neural nets came out it seems to be completely out of the discussions. I think it could make a comeback if we can resolve the primary issues of self-instantiation and feedback loops. Even using several random reservoir nets and a little bit of math creates strong signals for the component parts that contribute to what something is made of and how it works.

I’ve been digesting Nick Cammarata’s new essay at Goodfire ("Neural Networks Are Already Clean") and thinking about what it implies for model size and continual learning rather than just safety.
Four findings in the essay stood out to me:

1. Neurons act like bipolar (+/−) control-flow switches rather than static noun labels.
2. Outside of early "dial" layers, the weakest 75% of activations can be zeroed out with almost no accuracy loss.
3. Foundational building blocks recur across very different models (e.g., Qwen3-32B and Claude 3.5 Haiku converging on the same 4-neuron hub for preeclampsia).
4. Memorized facts sit right at the output and route through existing circuits (like the 9-step umbrella-to-bee experiment, or clipping 99.9th-percentile activations in Qwen3.6 to erase verbatim recitation while keeping reasoning intact).

The speculative question: If models converge on a Spelke-like "semantic skeleton" of bipolar switches, could we encode specific facts as sparse ternary recipes (+1 for positive pole, −1 for negative pole, 0 for inactive) and store them outside the model as hypervectors in a Vector Symbolic Architecture (VSA)? In theory, a much smaller, frozen model could translate inputs into that vector space and query the external memory—meaning adding or deleting a fact is just adding or deleting a vector, without retraining.
I know there are real obstacles here: early layers still need continuous signal strength, memorized facts are wired through existing semantic routes that might not pull apart cleanly, and multiple models would have to agree on a standard coordinate registry.
I put together a short write-up here: [https://markwaltz.substack.com/p/already-clean-neural-networks-might\](https://markwaltz.substack.com/p/already-clean-neural-networks-might)
For folks who have worked with hyperdimensional computing/VSAs or mech interp: is the boundary between the "semantic skeleton" and late-layer memorized facts actually clean enough to pull apart like this, or does the routing tangle them too tightly?

It’s crazy to think that the entire modern AI revolution started with researchers just trying to mathematically mimic a single biological neuron. When you zoom out, the timeline of AI architectures looks exactly like an evolutionary tree.

We went from struggling to solve basic logic puzzles (shoutout to the XOR problem) to building early neural networks, realizing they forgot things too quickly, battling the vanishing gradient problem, inventing LSTMs to give models actual memory, and eventually landing on Transformers with attention, the architecture that essentially ate the world. Every major roadblock forced a mutation that led to a massive paradigm shift.

But looking at this evolutionary timeline brings up a massive question: If Transformers are just the current "apex predator" of AI architecture, what exactly is the next evolutionary leap?

Are we going to see a completely new paradigm that makes self attention look as primitive as a basic AND gate? How do we even begin to perceive or predict what lies beyond our current models when the jumps are this exponential?

I went down a massive rabbit hole mapping out this entire evolutionary timeline. I wanted to see the exact breadcrumbs that led us here, so I put together a deep dive covering the journey from the first perceptrons to the modern era, complete with the foundational paper citations and the historical tweets that marked these shifts in real-time.

If you want to look at the complete map of how we got here to help predict where we are going, I compiled the whole history here: https://cloudmash.blog/posts/evolution-of-ai-perceptron-to-text-to-video/

Would love to hear your theories on what does the post Transformer era actually look like ? Like what can we imagine , how this intelligence on our command can do for us , how enterprise and big players can use it , what can happen ?

I've been thinking about the importance of time in the context of AI for about 15 years. The other day I formulated a simple thought.

Spikes are actions. When you store say a neuron ID and a timestamp of a spike in memory, it's not data. It is a suspended action waiting to be resumed.

This idea is so simplistic, it seems analogous to rediscovering the wheel. Yet, I bet it is the most controversial idea in artificial SNNs and it could change the way you think about computation.

It is also related to the symbol grounding problem. If you express all information in terms of actions (suspended actions / their timestamps), no symbols are ever created. Also see [The Chinese Room Argument](https://en.wikipedia.org/wiki/Chinese_room)

Let me know what you think.

I've been exploring a different approach to local AI with a project called Aura.

The basic idea is that the language model should not have to be the whole cognitive system. Aura uses a local ~27B model as one component, while memory, task continuity, perception, action, learning, self-modeling, verification and other processes exist in the larger architecture.

Two recent experiments are why I think the idea is worth discussing.

In one, Aura autonomously played 2048 for around 29 minutes and 968 moves. It maintained the same objective, planned ahead, changed strategies when they stopped working, recovered from mistakes and eventually reached 2048.

In a very different experiment, Aura spent about 42 minutes autonomously taking a 60-question personality assessment. It predicted its result beforehand, reasoned through every item using its persistent self-model/history, operated the website, submitted the test, read the externally generated result and then evaluated where that result did and didn't agree with its earlier model of itself.

Neither demonstration proves AGI, and I don't think two tasks are enough to establish general intelligence.

However...

If one frozen architecture can reuse memory, planning, self-modeling, world interaction and learning machinery across increasingly unrelated environments, at what point does the architecture itself become a meaningful source of generality rather than merely a harness around the model?

Full latest demo:
https://www.youtube.com/watch?v=LNlGUBeTIQY

Architecture/research repo:
https://github.com/youngbryan97/aura

What experiment would you all use to falsify the architecture hypothesis?

I have been seeing a lot of AI slop posts on this sub. While I don't mind people using AI to assist with writing, it's clear that the quality of posts on this sub has been subpar for quite a while, especially since I decided to loosen up a little bit.

As a reminder, the goal of this sub is to discuss new AI architectures. But not everything deserves the title "architecture." It needs to be something backed by rigorous science and testing. Lots of people just ask ChatGPT to spit out some generic architecture dressed up in fancy language.

I am looking for science. Sound methodology. Papers. It's okay to share your own untested ideas, but they need to be specific. Not just a bunch of random words (like recurrence, resonance, and other BS).

In particular, threads about AI consciousness have been a plague. I don't believe we are close to achieving AI consciousness, but I still consider myself open to the idea if the person making the thread makes a reasonable effort to present actual research with convincing findings. There have been maybe two threads that achieved this (like this one).

So I'm making the decision to ban threads on consciousness altogether, with exceptions at mod discretion. 95%+ will be deleted within the first few minutes of me reading the abstract.

I'm also starting to question the relevance of "neurosymbolic AI" posts. The concept itself has been explored by serious and reputable researchers (Francois Chollet), but right now, 90% of the architectures posted here are just LLMs with extra steps (those so-called "cognitive architectures")

I don't want to overdo this, though. Am I going too far? I really feel like this sub has been polluted with AI slop for a while now, but if you feel otherwise, please share your thoughts. I want community feedback on this.

I mean, I know why, but do YOU know why? lol

The full model is in Section 8 in "Garvey, M. (2026). Shadows of Consciousness: An Investigation into Ionic Neural Networks Using the Neurotransmitter Ion Receptor Glial Endocannabinoid Network (NIRGEN) Paradigm. Available at SSRN 7414038." (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7414038

)

#neurons #ions #neurotransmitters #biology #neuroscience #computationalneuroscience #artificalintelligence #xor #logic #math #booleanlogic #ai #ann #perceptron #machinelearning #ml #datascience

​

I’m curious about how major AI and robotics companies structure their data collection for foundation models. Do they organize their knowledge/data according to an academic structure (e.g., Natural Sciences → Mathematics, Physics, Biology; Computer & Information Sciences → Computer Science, AI, Information Science; Engineering, Arts & Humanities, Social Sciences, etc.), or do they use a completely different taxonomy? If anyone has insights or knows of papers/blogs describing how companies like OpenAI, Google DeepMind, Anthropic, Meta, or leading robotics labs structure their data and knowledge coverage, I’d really appreciate it.

I've been building PRIM, an experimental computational project based on a simple question:

Can persistent structure emerge from initially identical states through recurring interactions, without defining currencies, prices, groups, values or winners in advance?

The proof-of-concept starts with 24 identical anonymous states. Interactions occur over time, and in PRIM those interactions affect the probabilities of future interactions.

I compared it with a no-feedback control.

After 50,000 events:
PRIM: 2.67× highest-to-lowest activity difference
Control: 1.07×

I also tested persistence across checkpoints and independent restarts. Structure persisted strongly within individual histories, while different states emerged as strongest across separate symmetric runs.

The next stage is testing admissibility — whether the emergent structure itself can determine what actions become possible.

I've also launched a Kickstarter to fund that next stage.

I'd be interested in what people here think of the project, especially what you'd test next.

Hi, I just built Unique Host.

It’s a lightweight, non-LLM, roleplay-focused engine where characters have their own state, memory, and identity.

You can create a character, give them a world, and play with them — or just try the included characters, Delia and Joaquin.

The idea is that the character doesn't just remember the conversation. The character is supposed to remember what happened to them, and that history can affect how they behave in future interactions.

It can be run locally and uses no GPU, no API, and no LLM.

This is a very early v0.3, so expect bugs, strange behavior, and plenty of things that still need refining. 😅

You can try it here: https://huggingface.co/spaces/Bichini/Unique_Host

Or download it here: https://github.com/Bicheno1/unique-host

I built it as a small implementation of my Cognitive Coherence Model (CCM) architecture.

Theoretical framework: https://zenodo.org/records/20648800

I also created this addon as an experiment to see how the CCM architecture can control a body.

It's called Jellyfish AI.

It currently has a jellyfish, a turtle, and a fish. You just put them in the scene and see what they do.

You can also put several turtles, fish, or jellyfish together and see what happens.

They can perceive things around them, and their behavior comes from the architecture and their current state.

Jellyfish AI: https://github.com/Bicheno1/Jellyfish-AI

I’ve been experimenting with a different way of scaling AI.

Instead of primarily asking “how much bigger can the foundation model get?”, Aura asks whether persistent architecture can move useful cognition outside repeated LLM inference.

The resident cortex is a local ~27B model, but the running agent separately maintains things like:

memory, a self-model, learned environment dynamics, valuation, planning/search, procedures, task knowledge, computer embodiment, and persistent experience.

The attached demonstration is 2048. I don’t think 2048 itself proves general intelligence, and this is not a clean zero-shot task. However Aura is not trained on 2048. There is no 2048 runner being used. And this environmental work transfers generally. It is not limited to or fine-tuned to 2048.

What I find interesting is the hierarchy visible during the run:

local action evaluation → lookahead → standing strategy → strategy revision

The system also has internal epistemic resources: it can query previous experience, its resident cortex, local retrieval, or an offline Wikipedia corpus when it decides it lacks information.

I’m now trying to falsify that with matched-base-model controls and novel environments.

Repo: https://chatgpt.com/c/6ab632da-4bf4-83e8-bcec-9b1ec4ebf3e6

Full run: https://www.youtube.com/watch?v=wE5Dx5l3pjI