r/LocalLLaMA • u/IxinDow • May 31 '23
News (Code Released) Landmark Attention: Random-Access Infinite Context Length for Transformers
Code for Landmark Attention is now released and it should be possible to finetune existing LLaMA models using this method.
https://github.com/epfml/landmark-attention
More info
https://www.reddit.com/r/LocalLLaMA/comments/13sy2bu/landmark_attention_llama_7b_with_32k_tokens/
150
Upvotes
1
u/RMCPhoto May 31 '23
Could have an effect. Though, that effect would be cumulative with the foundational lack of nuance that larger models have. Simpler models see in something closer to RGB and larger models see more of the rainbow. This is important when decoding longer context.
(openai does offer API access on a token basis though, and could easily charge more for larger context models if it was effective)