EleutherAI Blog

#theme-toggle, .top-link { display: none; } @media (prefers-color-scheme: dark) { :root { --theme: #1d1e20; --entry: #2e2e33; --primary: rgba(255, 255, 255, 0.84); --secondary: rgba(255, 255, 255, 0.56); --tertiary: rgba(255, 255, 255, 0.16); --content: rgba(255, 255, 255, 0.74); --hljs-bg: #2e2e33; --code-bg: #37383e; --border: #333; } .list { background: var(--theme); } .list:not(.dark)::-webkit-scrollbar-track { background: 0 0; } .list:not(.dark)::-webkit-scrollbar-thumb { border-color: var(--theme); } }

2022

February

Announcing GPT-NeoX-20B

February 2, 2022 · Connor Leahy

2021

October

A Preliminary Exploration into Factored Cognition with Language Models

October 25, 2021 · Leo Gao, Kyle McDonell, Laria Reynolds, Stella Biderman

Multiple Choice Normalization in LM Evaluation

October 11, 2021 · Leo Gao

August

Downstream Evaluations of Rotary Position Embeddings

August 16, 2021 · Leo Gao

July

What A Long, Strange Trip It’s Been: EleutherAI One Year Retrospective

July 7, 2021 · Connor Leahy, Eric Hallahan, Leo Gao, Stella Biderman

June

Why Release a Large Language Model?

June 2, 2021 · Connor Leahy

May

On the Sizes of OpenAI API Models

May 24, 2021 · Leo Gao

Evaluating Different Fewshot Description Prompts on GPT-3

May 24, 2021 · Leo Gao

Finetuning Models on Downstream Tasks

May 24, 2021 · Leo Gao

Activation Function Ablation

May 24, 2021 · Leo Gao

April

Rotary Embeddings: A Relative Revolution

April 20, 2021 · Stella Biderman, Sid Black, Charles Foster, Leo Gao, Eric Hallahan, Horace He, Ben Wang, Phil Wang