Blog

Discussing and disseminating open-source AI research.

2023

November

The third New England RLHF Hackers Hackathon

November 26, 2023 · Will Beddow, Matthew Bernstein, Chase Blagden, Louis Castricato, Jacob Makar-Limanov, Arjun Prakash, Sid Verma

Extending the RoPE

November 13, 2023 · Honglu Fan, Bowen Peng, Jeffrey Quesnelle

October

How the Foundation Model Transparency Index Distorts Transparency

October 26, 2023 · Nathan Lambert, SE Gyges, Stella Biderman, Aviya Skowron

Llemma: An Open Language Model For Mathematics

October 16, 2023 · Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer', Albert Q. Jiang, Jia Deng, Stella Biderman, Sean Welleck

The second New England RLHF Hackers Hackathon

October 13, 2023 · Suraj Anand, Chase Blagden, Louis Castricato, Yeganeh Kordi, Francisco Piedrahíta-Vélez, Arjun Prakash, Logan Riggs, Benjamin Spiegel, Kevin A. Wang, Benjamin Wright, Yong Zheng-Xin, Max Zuo

September

Contributor Spotlight: Mohammad Aflah Khan

September 21, 2023 · Mohammad Aflah Khan

The first New England RLHF Hackers Hackathon

September 19, 2023 · Suraj Anand, Stephen Casper, Louis Castricato, Arjun Khurana, Alyssa Loo, Shukai Ni, Benjamin Spiegel, Jiayu Zheng, Yong Zheng-Xin

July

EleutherAI's Thoughts on the EU AI Act

July 26, 2023 · Aviya Skowron, Stella Biderman

Minetester: A fully open RL environment built on Minetest

July 8, 2023 · Curtis Huebner, Robert Klassert, Stepan Shabalin, Edwin Fennell, Delta Hessler

May

🐶Safetensors audited as really safe and becoming the default

May 23, 2023 · Nicolas Patry, Stella Biderman, Garry Jean-Baptiste

Alignment Research @ EleutherAI

May 3, 2023 · Curtis Huebner

April

Transformer Math 101

April 18, 2023 · Quentin Anthony, Stella Biderman, Hailey Schoelkopf

Exploratory Analysis of TRLX RLHF Transformers with TransformerLens

April 2, 2023 · Curt Tigges

March

EleutherAI Second Retrospective: The long version

March 26, 2023 · Stella Biderman, Shivanshu Purohit, Curtis Huebner, Leo Gao, Connor Leahy, Eric Hallahan

The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective

March 2, 2023 · Stella Biderman, Curtis Huebner, Connor Leahy, Eric Hallahan

2022

February

Announcing GPT-NeoX-20B

February 2, 2022 · Connor Leahy

2021

October

A Preliminary Exploration into Factored Cognition with Language Models

October 25, 2021 · Leo Gao, Kyle McDonell, Laria Reynolds, Stella Biderman

Multiple Choice Normalization in LM Evaluation

October 11, 2021 · Leo Gao

August

Downstream Evaluations of Rotary Position Embeddings

August 16, 2021 · Leo Gao

July

What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective

July 7, 2021 · Connor Leahy, Eric Hallahan, Leo Gao, Stella Biderman

June

Why Release a Large Language Model?

June 2, 2021 · Connor Leahy

May

On the Sizes of OpenAI API Models

May 24, 2021 · Leo Gao

Evaluating Different Fewshot Description Prompts on GPT-3

May 24, 2021 · Leo Gao

Finetuning Models on Downstream Tasks

May 24, 2021 · Leo Gao

Activation Function Ablation

May 24, 2021 · Leo Gao

April

Rotary Embeddings: A Relative Revolution

April 20, 2021 · Stella Biderman, Sid Black, Charles Foster, Leo Gao, Eric Hallahan, Horace He, Ben Wang, Phil Wang