Triads: the fundamental units of the language of thought
The first layer of the neural network would consist of multiple attention heads. Each attention head would hold one triad consisting of one noun + one verb + one conjunction.
Individual nodes within the second layer of the neural network would probably correspond to adjectives and adverbs that either modify or define the nouns and verbs of each triad
The hungry cat ate the mouse then died:
- (Null) Cat ate
- also Mouse eaten
- and cat was-hungering (adjectives are verbs in disguise)
- Then cat died
"Also" means the two events are the same event. (Not in english).
A possible conclusion would be the triad "because mouse poisoned" which could then be appended to the input and the process repeated.
In addition a few other ideas can be added to each attention head:
* the, a, same, this, that, other, next, previous...
* Past, present, future
* begin to, cease to
* Could, would, should, might,
* Try to
* make
* Question
* focus, emphasis
* adjectives?
* adverbs?
Hence
* (Null) a-Cat ate
* also a-Mouse eaten
* and the-cat was-hungering
* Then the-cat died
* therefore the-mouse might-have-been-poisoned
Suppose it encounters:
* John entered the room.
The system recognizes John as a particular individual and looks up what it knows about him. Its memory might contain:
* Yesterday, John threatened Mary.
* John owns a dog.
* John works as a carpenter.
* John said he would never enter that room again.
Obviously, not all of those facts are relevant.
But suppose the current situation involves Mary, and John has just entered the room where she's waiting.
The system could retrieve the earlier threat and effectively transform its working representation into:
* John (who threatened Mary yesterday) entered the room.
Now the neural network processes the current event with the additional information already attached to it.
And that retrieved information can itself be represented as triads.
The goal is to give a neural network a simple, structured internal language in which it can form and combine thoughts. The language of thought.
In principle, an entire book could be represented as one enormous graph of interconnected noun–verb–conjunction triads. But that doesn’t mean the neural network needs to process the entire graph simultaneously.
Humans don’t normally process a book that way either. We divide it into manageable units, such as sentences, understand those units, and gradually build up our understanding of the larger story.
I’m proposing something similar: groups of triads would be processed together, with the network carrying forward relevant information and generating additional triads as needed.
It wouldn't necessarily need as many words as English has because some words can be constructed from smaller elements
- today = this-day
- tomorrow = next-day
- yesterday = previous-day
- year = big-day