Extending the RoPE
How position interpolation, NTK-aware methods, and YaRN extend the context windows of language models using RoPE.
Meta
EleutherAI Second Retrospective: The long version
What we've been up to for the past year-and-a-half at EleutherAI.
The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective
(Some of) what we've been up to for the past year-and-a-half at EleutherAI.
What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective
A look back at the first year of EleutherAI.