Why Release a Large Language Model?
We believe the creation and open source release of a large language model is a net good to AI safety. We explain why.
EleutherAI's Research Blog
Previous Posts
On the Sizes of OpenAI API Models
Using eval harness, we can deduce the sizes of OpenAI API models from their performance.
Evaluating Different Fewshot Description Prompts on GPT-3
We evaluate different fewshot prompts on GPT-3 to see how it changes performance.
Finetuning Models on Downstream Tasks
We tuned GPT-Neo on eval harness tasks to see how it would change its performance.
Activation Function Ablation
An ablation of activation functions in GPT-like autoregressive language models.
Rotary Embeddings: A Relative Revolution
Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test.