2026
2025
2024
- SAEs trained on the same data don’t learn the same features
- Partially rewriting an LLM in natural language
- Third-party evaluation to identify risks in LLMs’ training data
- Mechanistic Anomaly Detection Research Update 2
- RLHF and RLAIF in GPT-NeoX
- The Practitioner's Guide to the Maximal Update Parameterization
- Mechanistic Anomaly Detection Research Update
- Open Source Automated Interpretability for Sparse Autoencoder Features
- Experiments in Weak-to-Strong Generalization
- Free Form Least-Squares Concept Erasure Without Oracle Concept Labels
- VINC-S: Closed-form Optionally-supervised Knowledge Elicitation with Paraphrase Invariance
- Pile-T5
- Yi-34B, Llama 2, and common practices in LLM training: a fact check of the New York Times
- The Foundation Model Development Cheatsheet
2023
- Least-Squares Concept Erasure with Oracle Concept Labels
- Diff-in-Means Concept Editing is Worst-Case Optimal
- The third New England RLHF Hackers Hackathon
- Extending the RoPE
- How the Foundation Model Transparency Index Distorts Transparency
- Llemma: An Open Language Model For Mathematics
- The second New England RLHF Hackers Hackathon
- Contributor Spotlight: Mohammad Aflah Khan
- The first New England RLHF Hackers Hackathon
- EleutherAI's Thoughts on the EU AI Act
- Minetester: A fully open RL environment built on Minetest
- 🐶Safetensors audited as really safe and becoming the default
- Alignment Research @ EleutherAI
- Transformer Math 101
- Exploratory Analysis of TRLX RLHF Transformers with TransformerLens
- EleutherAI Second Retrospective: The long version
- The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective
2022
2021
- A Preliminary Exploration into Factored Cognition with Language Models
- Multiple Choice Normalization in LM Evaluation
- Downstream Evaluations of Rotary Position Embeddings
- What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective
- Why Release a Large Language Model?
- On the Sizes of OpenAI API Models
- Evaluating Different Fewshot Description Prompts on GPT-3
- Finetuning Models on Downstream Tasks
- Activation Function Ablation
- Rotary Embeddings: A Relative Revolution
