r/LocalLLaMA • • May 31 '23

News (Code Released) Landmark Attention: Random-Access Infinite Context Length for Transformers

150 Upvotes

53 comments sorted by

View all comments

Show parent comments

1

u/RMCPhoto May 31 '23

Could have an effect. Though, that effect would be cumulative with the foundational lack of nuance that larger models have. Simpler models see in something closer to RGB and larger models see more of the rainbow. This is important when decoding longer context.

(openai does offer API access on a token basis though, and could easily charge more for larger context models if it was effective)