Cross-model distillation shows subliminal learning survives differences in initialization, data order, corpus, size, and architecture — and that transmitting and acquiring a preference are separable abilities.
Testing how robust Natural Language Autoencoder verbalizations are to changes in training seed, paraphrasing, and targeted text edits. Work done during SOAR 2026, by Kaylee, Daniel and Abhinav, with author order randomized.
What we learned while building black-box and white-box detectors for AI deception during Aletheia's Quest.
Announcing Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Announcing the Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Exploring factored cognition by decomposing arithmetic tasks for GPT-3.
There are multiple ways of evaluating multiple-choice tasks on autoregressive language models like GPT-3/Neo/J. This post lays out the current prevalent normalization methods.
A comparison of Rotary Position Embedding against GPT-style learned position embeddings.
Using eval harness, we can deduce the sizes of OpenAI API models from their performance.
We evaluate different fewshot prompts on GPT-3 to see how it changes performance.
We tuned GPT-Neo on eval harness tasks to see how it would change its performance.
An ablation of activation functions in GPT-like autoregressive language models.