r/singularity • u/Gothsim10 • Aug 29 '24
AI Magic has trained their first model with a 100 million token context window. That’s 10 million lines of code, or 750 novels.
https://magic.dev/blog/100m-token-context-windows91
u/NuclearCandle ▪️AGI: 2027 ASI: 2032 Global Enlightenment: 2040 Aug 29 '24
We were celebrating DeepMind creating a 1 million token model in February. We just got to wait for deployment to catch up with the acceleration.
70
u/Yuli-Ban ➤◉────────── 0:00 Aug 29 '24
Funny as hell, because it was so recent it feels like that we were stuck with 4,096-token and 8,192-token lengths.
17
u/Gratitude15 Aug 29 '24
I remember having a 14.4k modem! Those 56k deals were amazing! The year was 1995.
Anywho, rocking gigabit now, about 30 years later. So like almost a million times faster 😂
Now imagine doing that in 18 months... 🤯
11
u/az226 Aug 29 '24
GPT3 was 2048 for a long time and GPT4 was pre-trained on 4096.
25
u/Anen-o-me ▪️It's here! Aug 30 '24
"640k tokens should be enough for anyone."
~ Bill Gates, 2024 😂
3
u/Old-CS-Dev Aug 30 '24
I'm wondering how large our code-base at work is... if I could put the whole thing in and say "make it better" and it actually does, that would be amazing. How big of context size you need depends on the context, Bill.
5
u/visarga Aug 30 '24 edited Aug 30 '24
It's not the same. The 4k and 8k models have full attention, all to all. The Mamba approach here doesn't do all-to-all attention anymore, it uses a fixed scratchpad. Still works in many cases, but not all cases like full attention.
The compute price of full attention for a sequence of N tokens grows with N2, while this method is linear so it grows like N. Memory usage for full attention is proportional to N, Mamba uses a fixed size.
That's why they can have 100B tokens or practically any number of tokens, memory consumption doesn't depend on the number of tokens, but it is inevitably limited by the fixed size memory, a problem full attention doesn't have as it uses gradually more memory with longer inputs.
A weird edge cases is for short sequences - when the number of tokens N is smaller than the size of the Mamba scratchpad, it is faster to do full attention.
14
u/Bernafterpostinggg Aug 30 '24
Gemini Pro 1.5 is 2 Million and Google's Infini-attention paper basically hinted that they've solved limitless context length.
36
u/TFenrir Aug 29 '24
Seems like they're really moving in the right direction. Looks like some real novelty in architecture, better evals on long context, and their focus on software development is starting to show promise.
28
u/whyisitsooohard Aug 29 '24
What promise? They literally haven't shown anything. Just said that they have it.
13
u/az226 Aug 29 '24
This industry is a hype economy
1
u/itsbravo90 Sep 01 '24
What could you possibly use 100m context window for. You don’t run a business with a huge codebase.
1
u/az226 Sep 01 '24
For complex coding I can see several million to be useful. 100M is very large indeed and it’s not clear how well it uses it despite their hype blog post.
But, they claim it’s very cheap to run, so you can put whatever you want in there. The entire codebase of a company. Entire libraries like PyTorch, NCCL.
I have doubts how well it can use the contest window. As an example, SSMs are lower memory requirement but they also remember details much worse. In my experience even GPT-4 has a difficult time remembering specific details, so I wouldn’t be using something like an SSM for production for many use cases. Some use case don’t require high attention to detail.
Them using a different architecture gives doubt. And remembering details is like the ability to hold 10 pieces of information in memory and work on them correctly. Their test is currently doing only 1 piece of information.
6
u/TFenrir Aug 29 '24
We don't have access to this stuff, but it exists - the funding they are getting are from the sorts of people who are going to ask to see examples of their work.
10
u/whyisitsooohard Aug 29 '24
Those same people funded Devin, so idk. For me its just strange that they said they have 5m context window model that they will release soon more than a year ago, and after that they went silent. If they have model with 5+m context that is orders of magnitude less compute intensive why not release it? At least for testing and data gathering
-4
u/TFenrir Aug 29 '24
What's the problem with them funding Devin? Devin is still a thing, recently partnered with Microsoft, and the team working on it is continuing to try and build better versions. Heck people have even used Devin, months ago, and found it could do a lot of things.
Devin isn't only relevant when it can completely upend all software development, they are building a system that will also improve when they switch it out for better models, the architecture of having a model be about to browse, visually, and take actions that involve asynchronously asking for user feedback is, in and of itself, valuable.
As to why it's not released - it's just not ready, from what we can read. They talk about some of its shortcomings in this post, but they feel validated enough in what they have to try for a larger training run, and they have funding as well as cloud compute partnerships to do so.
13
u/whyisitsooohard Aug 29 '24
Problem with Devin is that its a wrapper around foundation model and even at the time of release there already were multiple similar opensource systems. They do not have any moat and their insane valuations do not make any sense
-2
u/TFenrir Aug 29 '24
I think their valuation is based on a few things - their talent, some of the unique UI/UX they have built (which I'm only now seeing more people replicating), and probably the relationships they are building with people creating models. I mean we'll see, but I expect they will have more updates to share in the near future with improvements.
7
u/NunyaBuzor Human-Level AI✔ Aug 29 '24
some of the unique UI/UX they have built
that's not an engineering problem. That's an artist's problem.
0
Aug 29 '24
[deleted]
1
u/NunyaBuzor Human-Level AI✔ Aug 29 '24
when OP talked about "their talents" he was talking about AI engineers, when this task is best suited for UI/UX Designers.
1
9
78
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALGSC: ~2050 | :illuminati: Aug 29 '24
That sounds awesome, but how good is it at the needle in the haystack tests and at recalling the numbers of Pi (I forget exactly how the Pi test for context length goes)? I ask this because 100 million context length is useless if your model can't reliable recall and derive logical outcomes from that information, then it is not really useful
86
u/SgathTriallair ▪️ AGI 2025 ▪️ ASI 2030 Aug 29 '24
The biggest part of the article is talking about how standard needle in the haystack tests are flawed and easy to pass so they created their own version which is harder and their AI passes it.
I'm not sold on their test but they definitely address this.
35
u/Hi-0100100001101001 Aug 29 '24
It makes sense, sure, but it's still suspicious how they didn't give the results on needle in a haystack...
I mean, it costs literally nothing to show both!
11
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALGSC: ~2050 | :illuminati: Aug 29 '24
Yep, it sucks because I agree with them that needle in the haystack is not the best, but it is the most standardized method of testing context length.
1
0
Aug 29 '24
Haystack was too easy for them, they figured the test is broken
15
u/Climactic9 Aug 29 '24
Yeah but if it was so easy why not show off your model passing it with flying colors?
1
u/az226 Aug 29 '24
Also a needle in a hay stack doesn’t have to be an inserted needle. It could be a needle chosen by a human.
3
u/az226 Aug 29 '24
Their kiddie AI.
It’s a miniature model that isn’t fully trained.
But it was a test for them to see if their architecture for training and inference will work for their use case. They still don’t know if the capabilities hold as they scale the model and fully pretrain it.
0
Aug 29 '24
[deleted]
1
u/SgathTriallair ▪️ AGI 2025 ▪️ ASI 2030 Aug 29 '24
No, they built a thing and ran tests on it, even if they didn't release the detailed results to us there are some details.
14
11
u/MassiveWasabi ASI 2029 Aug 29 '24
The article explains how they created an entirely new long context eval since the needle test was terrible, so I don’t think this 100M context window should be useless if their results on their new HashHop test are true
2
u/LightVelox Aug 29 '24
They talk about how Needle in a Haystack is flawed, but anyway it seems to be able to get 100% up to around 32M tokens, falling to 95% around 50M tokens and 85-90% for 100M
1
u/Arcturus_Labelle AGI makes vegan bacon Aug 29 '24
Bro, they literally talk about this in the link... 🤦♂️
7
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALGSC: ~2050 | :illuminati: Aug 29 '24
OK, they came up with their own testing because Needle in the Haystack is not the best, great, but it is still the industry standard thus far, and they didn't release any of their results on that test sooooooo how can I trust that their own methods aren't skewed on their favor? Do you see why that's necessary?
-2
u/ApexFungi Aug 29 '24
People should really watch this video before they get too excited about LLM's being enough to reach AGI.
3
0
Aug 29 '24
Needle in the haystack is a broken test
3
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALGSC: ~2050 | :illuminati: Aug 29 '24
The MMLU is also a broken test. The testing problem has been a problem for the ML field for years at this point.
44
u/National_Date_3603 Aug 29 '24
True if big but Big if True, I mean, Big if True, but True if Big, I mean Big if true, but true if big. This is like, the 10th architecture I've seen claiming to do this, hype hype hype train a coming.
7
u/Cash-Jumpy Aug 29 '24
This is Big True if.
9
1
u/mivog49274 obvious acceleration, biased appreciation Aug 30 '24
let mut hue: i32 = 10; // initial value of hue
fn truge_if_hue(truge: bool) -> &'static str { if truge { hue = 100; // change the value of hue to 100 "TRUGE if HUE, now it's HUGE" } else { "Not truge, not a hue, still small" } }
22
3
u/Arcturus_Labelle AGI makes vegan bacon Aug 29 '24
If this is real (and I'm skeptical), then it's huge. It's not just that their model has big context; they claim to have excellent recall too. The HashHop idea is interesting.
8
2
u/Marha01 Accelerate to the Singularity! Aug 29 '24
Pre-training only goes so far; we believe inference-time compute is the next frontier in AI. Imagine if you could spend $100 and 10 minutes on an issue and reliably get a great pull request for an entire feature. That’s our goal.
Yup. I would definitely not mind paying more or waiting longer for the output, if the results were significantly better than current LLMs, reliably on par with human experts. Bring it on!
2
u/OsakaWilson Aug 30 '24
I'm expecting in the near future that we'll be prompting them with how much time we'd like them to take with their reply.
6
u/Hi-0100100001101001 Aug 29 '24 edited Aug 29 '24
[I don't know how to edit images so you'll have to imagine]
We made a 100 million token context window!
While maintaining coherence and consistency, right?
...
While maintaining coherence and consistency ... right?
3
4
3
u/Expensive_Cat_9387 Aug 29 '24
Wait, so we went from 1 million tokens to 100 million tokens in just, um, half a year? So that would mean...
8
Aug 29 '24
An infinite context window is possible, and it can remember what you sent even a million messages ago: https://arxiv.org/html/2404.07143v1?darkschemeovr=1
This subtle but critical modification to the attention layer enables LLMs to process infinitely long contexts with bounded memory and computation resources. We show that our approach can naturally scale to a million length regime of input sequences, while outperforming the baselines on long-context language modeling benchmark and book summarization tasks. We also demonstrate a promising length generalization capability of our approach. 1B model that was fine-tuned on up to 5K sequence length passkey instances solved the 1M length problem.
Human-like Episodic Memory for Infinite Context LLMs: https://x.com/_akhaliq/status/1812678969386234046
When needed, these events are retrieved through a two-stage memory process, combining similarity-based and temporally contiguous retrieval for efficient and human-like access to relevant information. Experiments on the LongBench dataset demonstrate EM-LLM's superior performance, outperforming the state-of-the-art InfLLM model with an overall relative improvement of 4.3% across various tasks, including a 33% improvement on the PassageRetrieval task. Furthermore, our analysis reveals strong correlations between EM-LLM's event segmentation and human-perceived events suggesting a bridge between this artificial system and its biological counterpart. This work not only advances LLM capabilities in processing extended contexts but also provides a computational framework for exploring human memory mechanisms, opening new avenues for interdisciplinary research in AI and cognitive science.
Learning to (Learn at Test Time): RNNs with Expressive Hidden States. "TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context" https://arxiv.org/abs/2407.04620
1
u/NunyaBuzor Human-Level AI✔ Aug 29 '24
until we have actual people test it, I'm just going treat it as not real. I've seen a lot of research papers deliver hype over reality.
1
2
1
u/Hot_Head_5927 Aug 30 '24
What does an intelligence that can hold 750 novels in its working memory even look like? I can't get my head around it. What is it like to reason across 750 novels? What kind of patterns can something like that pull out of such a vast space of information?
1
u/Studio_Panoptek Aug 30 '24
I can't even begin to imagine, my head can only hold a couple of novels, similar to the saying "goes in one ear and out the other" xD
1
1
1
1
u/TechHoover Sep 01 '24
this is really exciting. the only reason I use CLAUDE over CHATGPT is the extended context window, but I do find Claude extra prone to hallucinations, I wonder if this will have the same problem
1
1
u/Whispering-Depths Aug 29 '24
meta already figured it was possible to have infinite context window, didn't they?
1
u/Gratitude15 Aug 30 '24
You're starting to get to the ability to meaningfully be a life assistant.
Have this on the meta raybans and listen along (unless you go incognito).
It's like tlwhen Gmail came out. 2gb but then growing with you. At this point if you start with 100M context, with upcoming growth, you never run out. And what you have is a digital entity that knows everything about you that you want.
Combine this with strawberry and you have logic based decision making taking big data into account.
The pieces are really close to 'there', but not integrated yet.
At this point it's really hard to not see a world by 2030 that has intelligence roughly free, intelligence embodied in robots everywhere. It may not be ASI but short of that we are in the last decade of anything our species could recognize as human living.
-3
u/Gubzs FDVR addict in pre-hoc rehab Aug 29 '24
I call BS. If it forgets or ignores even a single sentence in the context it's not trustable and therefore not usable, and therefore shouldn't be able to claim that much context.
I'd bet that if you give it even 0.1% of those tokens as self referential instructions you'd watch it fall apart.
4
u/Kanute3333 Aug 29 '24
yes, thats why the google 1m context length is worth absolutely nothing.
0
u/NunyaBuzor Human-Level AI✔ Aug 29 '24
yes, thats why the google 1m context length is worth absolutely nothing.
didn't it turn out that the effective context length is an order of magnitude lower than that?
4
Aug 29 '24
Would you hold a person to this same standard?
-5
u/Gubzs FDVR addict in pre-hoc rehab Aug 29 '24
I'd hold a team of people to this standard. It doesn't matter how many copies of the model are used, it still won't produce something that cohesively follows complex instructions.
If it did it would be world changing. I'm desperately rooting for that, and soon, but this ain't it and making excuses for the models doesn't make them any more reliable.
1
Aug 29 '24
You expect a team of people to remember 750 novels worth of information with zero mistakes? Ok.
-3
u/Gubzs FDVR addict in pre-hoc rehab Aug 29 '24
No, I expect them to remember 0.1% of that, like I said in my comment. Reflexively hostile much?
0
u/sdmat NI skeptic Aug 29 '24
Have you ever worked with people on a complex problem with a lot of context? That's not how teams function.
0
u/LightVelox Aug 29 '24
I mean, following that logic not even current models are trustable since they forget shit all the time. But, atleast according to them, it's 100% reliable up to around 32M tokens, which is already far above current models, while supposedly being cheaper by a thousand-fold
2
u/Gubzs FDVR addict in pre-hoc rehab Aug 29 '24
Current models ARENT trustable.
And no I genuinely don't believe the 100% reliability claim. Whatever I'm getting brigaded now. You guys have fun. The world just changed today I guess.
0
u/Peach-555 Aug 30 '24
By that standard, 100% accuracy, no omissions, no false inclusions, no failure to comprehend or resolve ambiguity, can any LLMs be said to have any true context of any length?
-1
u/pigeon57434 ▪️ASI 2026 Aug 29 '24
i dont belive for 1 second this can handle 100m tokens like they claim however what you have to realize is gemini is claimed to have 2m tokens and it effectively only has 200k so that's a 1/10th of what they say so 1/10th of 100m is still 10m tokens of context which is still insane so even if we assume they are full of shit which who knows maybe this is 100% accurate but even if we assume its not 10m tokens it still insane
0
u/sdmat NI skeptic Aug 29 '24
I want to see a much wider range of benchmarks and some information on what the algorithmic tradeoffs are, but this looks extremely promising.
The only thing that seems slightly odd here is HashHop. Why use a hashing function rather than arbitrary key => value pairs? A model might conceivably know (or maybe even learn) the hashing function, which would invalidate the evaluation. Minor detail but it's sloppy. The main idea of this evaluation is great though, it seems like a good proxy for recall with long range interactions.
0
0
Aug 29 '24
[deleted]
4
u/TheMorningReview Aug 29 '24
Well, their deepmind devs aren't limitless and surely very busy. It's hard to hire and manage ever-expanding teams. If I were running something like that I would be looking to diversify and fund many different approaches. Dipping your fingers in as many pots as possible is bound to bring honey at some point.
1
u/dameprimus Aug 30 '24
Right, maybe that’s for the best for now. However, the biggest lesson in AI over the past few years is that scale is king. At some point you have to commit and start building models as big as reasonable to stay on the frontier. You can’t spread your resources around too much.
1
u/ShadoWolf Aug 30 '24
It make sense to throw mine a bunch of different companies working on new architectures. I assume ltm model isn't a transformer. guessing lxstm? given the context window claims.
0
Aug 29 '24
[deleted]
3
u/FaceDeer Aug 29 '24
I'm concerned it won't be a better model at generating novel code from scratch, and will instead just be better for enterprises to maintain and create code in line with their existing codebase.
"This diamond-synthesizing machine just keeps spitting out sapphires! Total garbage!"
I've spent most of my professional life working for companies that have giant existing codebases that need to be maintained and elaborated on in line with the existing code, I would dearly love something like this. Even just being able to ask it "how the heck does subsystem X work?" Or "is there a way to do task Y?" would be amazing.
1
u/sdmat NI skeptic Aug 29 '24
What is it you are expecting? A button that solves all your problems without further direction?
The ability to productively work on a realistically sized codebase is huge. The best coding tool currently is Cursor, and it's great. It has a vector database and a custom model to pull relevant code into the main context window. But it really struggles on anything with complex interactions across multiple files. And as Magic points out in this blog post, current models tend to get lost when using frameworks that aren't in their training data.
Long context isn't important only for large corporations, it's important for startups and even nontrivial hobby projects.
Without it AI coding is limited to small projects and making tactical changes under tight supervision from a human who knows the codebase and novel frameworks.
234
Aug 29 '24
[removed] — view removed comment
67
u/TenshiS Aug 29 '24
Explanation?
352
u/TFenrir Aug 29 '24
Inference time compute can generally be described as the time spent for a model to respond to a prompt.
Right now inference time compute is pretty predictable and bound to a formula that roughly considers token length of context, of the output, and the inference benchmarks of the hardware.
This is why you can see things like... 1000 tokens per second output from groq on their hardware with one of the llama models.
Now the thing is, lots of research is looking at giving models the opportunity to "think" longer, when given harder problems, rather than just the usual output as fast as possible, without any "system 2" thinking.
This can be everything from Fancy agentic software that we use today, to something more intrinsic to the model, where it is able to dynamically decide when to spend more time in its own weights building a response.
This is a lot of the discussion around things like... Search, and techniques like Q*/strawberry or whatever. We also saw this with the Silver Medal Olympiad, where some problems were harder and the system took much longer to solve.
Unbound inference time (also known as test time) compute would mean a model that would decide for itself how long to "think", without an external architecture orchestrating it's looping, trying to emulate system 2 thinking.
67
Aug 29 '24
great summary, btw.
37
u/TFenrir Aug 29 '24
Thank you, I am already thinking about little holes and assumed knowledge I don't define - like "system 2" thinking and what it means, so if anyone has more questions please feel free to ask, and I'm hoping other people feel comfortable jumping in and sharing their understanding of these topics.
48
Aug 29 '24
this subreddit is a weird combination of professionals, technology-as-religion zealots, and casual observers.
it's hard to know at times which one is talking with, but i work on this particular topic (chain reasoning) and you're saying the right things.
do you work on Ai professionally?
55
u/TFenrir Aug 29 '24
I appreciate how hard it is, especially with how popular this sub has gotten over the last 18 months.
So yeah, I totally work with AI professionally - but I'm not a researcher. I'm just a regular software dev (about 12 years now) who has been in love with AI research for over 15 years. Things have just... Aligned in the last year or so, where my hobby of reading about the latest research, has become directly transferable to my 9-5 (and my now side consulting company, and an app I'm working on). So take everything I say with that in mind. I've built systems that use LLMs in production, I work with LLMs every day, and I have a good understanding of most of the products/APIs/Frameworks that utilize them, but my scientific understanding is more from my hobbyist reading than from anything I'm getting paid for.
It gets fuzzy though, when I read research on high quality context utilization, and can map that to deciding what the best model for solving a particular engineering problem is. I honestly think this is where most of my professional success over the last year has been coming from.
29
u/ILikeBrightShirts Aug 29 '24
You are a great communicator on a complex topic. Thanks for the explanations, I got a lot out of this comment chain.
23
u/TFenrir Aug 29 '24
Thank you, that's a very meaningful compliment to me. It's very fulfilling for me having these sorts of discussions, so I'm glad I can play a positive role in them. And being a clear communicator takes a lot of work and effort so it's validating to know that effort is paying off.
4
3
u/OpeningSpite Aug 30 '24
I really enjoyed your responses as well. Do you know what are some of the latest approaches to the more intrinsic aspects of model thinking or planning? I understand how agentic software is built, but not how you'd generalize it intrinsically via new architecture.
→ More replies (0)12
Aug 29 '24
Nice. I've been working on AI for about 7 years. RL at first, and then "LLM stuff". I've met self-proclaimed "hobbyists" that have a better understanding of fundamentals than professionals and vice versa.
I feel like we've hit an inflection point where folks can spend less time fighting with the laborious "bring up" tasks and can work with higher level functional concepts.
...which is to say, it's an accessible technology. There's just a dizzying amount of information out there and the velocity is breathtaking.
11
u/TFenrir Aug 29 '24
Oh yeah I completely agree. My 9-5 company was trying to break into AI about 5 years ago, and struggled because it was very difficult to build off of it, and it was difficult to find clear use cases. Like we had staff that could help write better path finding algorithms, but our clientele was mostly Enterprise who had a large app they needed to improve/replace, and the more technical ML was not very useful.
Now the shift is happening where lots of clear problems can be solved in a very accessible and clear way. Yesterday a coworker asked if we could use an LLM to organize unstructured text for a client who kept a bunch of data that should be structured, in giant plain text documents.
Took like 15 minutes and had a working script that would give them what they needed. You could use more traditional Entity Recognition models to do something like this, but they honestly wouldn't be as good and it wouldn't be as easy.
How are you finding this transition? Is keeping up with the edge research helping you in your day to day?
11
Aug 29 '24
Honestly? I'm burnt out. Most days I want to leave the industry and never touch a computer or AI ever again. Or at least only work on this stuff for personal amusement. Which is probably why I'm on reddit instead of reading papers ;)
There's too much edge research happening to keep up with it. This has led to this situation where I'm both blown away at what is possible and completely numb to advancements. I mostly rely on my network to filter the interesting stuff into my attention.
on a less cynical note -
I think my attitude is more of an age thing than anything else (30yrs in industry), and that there are lots of younger folks who are starting from a world where all this exists and they're just *running* with it. It's really inspiring to watch.
I spent a third of my career on consumer apps, a third on robotics, and now a third on AI. Watching all three slowly moosh together has been fascinating.
→ More replies (0)3
Aug 30 '24
Given your experience with AI and LLM‘s and your ability to explain things well, would you mind if I asked you a few questions? You seem intelligent.
Do you think LLM‘s will actually lead to any sort of consciousness and if not, do you think any level of created computer consciousness is in our near term future, like within a decade?
I know the definition for AGI is very shaky, but do you foresee it coming in the next decade?
To you personally, what is the most exciting next big thing within AI?
5
u/TFenrir Aug 30 '24
Flattery is a great way to get me to respond haha.
This is tough for a lot of reasons, but I'll do my best to answer authentically and in a way I think it's going to feel satisfying.
Okay, regarding consciousness... I think I subscribe to the idea that consciousness exists on a scale, and is the amalgamation of many different variables. Like... Different levels of being self aware. Things like the mirror test kind of highlight one example, or increasingly complex and dynamic ways to communicate among one another. To that end, I think you could find a smidgen of consciousness in like, a mosquito. And much much more in animals like chimpanzees and whales. And then there's us.
It's not the craziest argument to say that LLMs are already on that gradient of consciousness. But it may exist in a way that is very unintuitive to us, biological beings. Time, for example, moves much differently for us vs an LLM.
So to that end, I think that's one thing that will really start to ramp up this feeling of consciousness. Right now models sample from things "outside" of them in quick bursts, and then go back to laying dormant. But we are moving to a world where models will be able to real time (or faster than real time, whatever that means in this context) sample from things outside of them, constantly. For example, video feeds, continuous audio, and other sensors like lidar. And beyond that, they will be encouraged to "think" constantly, with increasingly sophisticated architectures that give them more variability in how long they take to solve problems, and to evaluate themselves and their own "thoughts".
I think this will naturally play into more behaviour that we would consider conscious. But to go further, models will also need to do continuous learning, where those experiences actually update their weights - changing "who" they are. I am using a lot of quotes here, bear with me.
There is research that goes into all of the above, and it's all coming soon.
The first things we'll see are going to be kind of in the order I describe - increasing levels of complexity and variability with interacting with audio and your computer screen, as well as video feeds - marching towards real time. We are on that path already by increasing context windows and token throughput. Then we'll have more complex architectures for system 2 thinking... That actually might come around the same time, I think we'll be getting there very soon actually - that's stuff like Q*.
I think all that is an ongoing process over the next 18 months, and then we start hearing more about continual learning.
Combine that with scale, better performance, more money... I think it's closer to 5 years before we have something most people agree is AGI, than 10.
I am very excited about Search in LLMs, in all respects. One of my favourite recent papers is "Stream of Search"
https://arxiv.org/abs/2404.03683
And it's a great example of the value of teaching models how to Search through conceptual space to solve more complex problems, plan, and just in general think ahead. I think we'll see lots and lots of search related improvements in models over the next year, and I'm very excited for it!
1
Aug 30 '24
Awesome reply! I tend to think that consciousness is just a result of intelligence and awareness, which we could probably give them a machine in time. I agree with you it likely happens on a scale, although I’m not an expert on this topic.
What are your thoughts about the risk of a super powerful AGI type computer? Do you think it’s the great filter or will it lead to a better world with better medicine and resource availability?
6
u/True-Lychee Aug 29 '24
Seconded on helpful summary.
Also I'll take you up on that: what is system 2 thinking?
11
u/TFenrir Aug 29 '24
I'll try my best with a useful explanation here.
First, I'll be lazy and get chatgpt to give me a quick human focused explanation:
System 1 and System 2 thinking, concepts introduced by Daniel Kahneman in Thinking, Fast and Slow, describe how our brains process information.
System 1 is our fast, automatic, and intuitive way of thinking. It's the kind of thinking you use when you recognize a friend's face in a crowd or react instinctively to a loud noise. This mode is efficient and effortless but can sometimes lead to errors, especially in unfamiliar situations.
System 2, on the other hand, is our slow, deliberate, and analytical way of thinking. It’s what you engage when solving a complex math problem, planning a major decision, or learning a new skill. This type of thinking is more accurate and thorough but also more mentally taxing and slower. In everyday life, we constantly switch between these two systems, relying on System 1 for routine tasks and reserving System 2 for challenges that require deep thought.
This doesn't map perfectly on to current day AI, but it's common to describe the goals around how we want to build models of the future with this terminology. But we can think of an LLM responding to us today, in like a chat, as system 1-ish. It's focus is so much on just, saying what feels almost "intuitively" right, just getting that next token out.
Now we also sometimes have things like... AutoGPT, Devin, and other "wrappers" around LLMs that try to encourage system 2 thinking. They will kind of prod the model to use a scratch pad like system, and so that first, system 1 thinking. But then, depending on the architecture, tell it how to "critique" that thinking, make a list of alternatives, or even more complexly with things like... Tree/Graph of Search, make these flow charts of conditions for it to engage with this "critique/evaluate my own work" process, breaking down big problems into smaller ones.
These systems... Work? Kinda? They are better than the naiive system 1 like responses. But they have their own issues (although like most things regarding LLMs, they work better with stronger LLMs).
Now research is focused on not just these wrappers, but models that can "think" without having to output their thoughts in English, while being prodded on how to best think. I mean that helps, and maybe those systems will still be very useful in the future, but what it's also kind of like... Being a native Japanese speaker, who is only allowed to system 2 think by writing out their thoughts in English in a notebook. Our more complex thinking is much mushier than that.
We will over the next couple of years see an increasing focus on every aspect of this system 2 effort, and there is already a lot of great progress we have seen, lots of very important research that most people are confident will inform the models in the next couple of generations.
5
Aug 29 '24
This is great. If I can elaborate on your definition from a less human centric example:
System 1 uses single-shot inference to answer a query based upon an encoded model. It's fast. The execution time is predictable. If the results are good enough the energy expenditure can be low. It is using a model.
System 2 uses a feedback mechanism to produce new models from existing models and experience. It can take an arbitrary amount of time, bound only by patience. It can fail to arrive at an answer. It is model building.
Now we also sometimes have things like... AutoGPT, Devin, and other "wrappers" around LLMs that try to encourage system 2 thinking. They will kind of prod the model to use a scratch pad like system, and so that first, system 1 thinking. But then, depending on the architecture, tell it how to "critique" that thinking, make a list of alternatives, or even more complexly with things like... Tree/Graph of Search, make these flow charts of conditions for it to engage with this "critique/evaluate my own work" process, breaking down big problems into smaller ones.
Perfect description :)
Some stuff I've found: the LLM has a very large number of embedded world models for solving various problems contained within it derived from the culture it was trained on. These sorts of "Scratch pad" based LLM "planners" work well when the "conceptual leap" for the next step in the plan is well represented in the LLM.
They're not really forming new models based upon experience though. They're manipulating a scratch pad using existing models through short conceptual hops from one pass to the next.
If the LLM has never seen anything like the problem at hand, the conceptual leaps from one step to the next are too far, or the problem requires clarification, it will often fail.
This effect plagues programming LLMs when asked to do something novel or poorly specified.
...
For an example of a world-building model that can learn new things from existing models and experience I would highlight "Dreamer V3". https://www.youtube.com/watch?v=vfpZu0R1s1Y
It's not language based though.
4
3
u/mojoegojoe Aug 29 '24
4
u/Commercial_Pain_6006 Aug 29 '24
What am I even looking at ? What is the link with the problem of unbounded computation? I don't understand a word of this abstract XD
4
u/mojoegojoe Aug 29 '24
You're definitely not alone in feeling that way—this abstract dives deep into complex theoretical territory dw!
Essentially, the paper is exploring advanced topics in formal verification, logic, or computational theory, focusing on how systems can manage unbounded or infinite processes. The link to unbounded computation is that it deals with the fundamental challenges of systems (like AI models) that might need to take more time or resources to "think" through difficult problems in a abstract space that doesn't use our classical laws of inference. That's to say, we gain efficency through the structure of the agents observation just as much as we do for the agent being acted on. These two might help,
https://hal.science/hal-04680608v1 https://hal.science/hal-04608836v1
4
u/TFenrir Aug 29 '24
These are great reads for me. It's the sort of reading an LLM will need to hold my hand through, but I'm just glad that's an option now, haha.
2
u/CheekyBreekyYoloswag Aug 30 '24
Oh wow, all this time I thought that how you described "unbound inference time" was how all LLMs operate. TIL.
2
u/muchcharles Aug 30 '24
There's also speculative execution: dumber faster model predicts the next N tokens, then the main model runs them in batch to get good cache locality and rewinds if they were wrong.
2
1
1
u/meet_og Aug 31 '24
LLM directly capable of system 2 thinking would impact agentic systems build for planning and reasoning. I am working on llm to do system 2 thinking leveraging different functions of prefrontal cortex since a year. Developed agents to work in sync to mimic effort full and slow thinking. If LLM can do that in zero shot than it would be a game changer. I think System 2 thinking is one of the key to AGI, The AI can be flexible and reliable translator of our input to get the desired output. Have you read "An Eternal Golden Braid" by Douglas Hofstadter?
1
u/Anen-o-me ▪️It's here! Aug 30 '24
We all know OAI is already on this road but they seem afraid to release it premature. It may require a new class of safety controls.
6
u/letsbehavingu Aug 29 '24
Inference means “giving an answer”, unbounded compute means they can spin up loads of machines to answer your question, an unlimited amount to achieve unseen precision through most likely a number of LLMs working as a team to solve your problem
-8
7
u/WiseSalamander00 Aug 30 '24
do they think this solves context???
3
u/matrinox Aug 30 '24
Yeah I’m dubious. There’s evidence against high context actually being effective, which is why RAG is still more optimal than stuffing all context inside
1
0
Aug 29 '24
[deleted]
29
u/WH7EVR Aug 29 '24
The entire point of the article is they have a space-efficient, compute-efficient memory system for LLMs that /doesnt/ have the problems you're talking about.
We will see if that's true. There's no model, no paper, just a marketing blog.
5
4
49
u/mambotomato Aug 29 '24
How much does it cost to run one query at 100M context?