Loading...
8 external academic papers we cite — ML/AI foundations and engineering-velocity research.
Distinct from /research (our own SSRN paper's findings) — this index documents external academic papers we cite in our methodology and editorial. Each leaf provides an abstract summary in our own words, the editorial context for why we cite it, key findings, and canonical sameAs links (arXiv, Semantic Scholar, OpenAlex). Designed as a citation-ready surface for ChatGPT/Perplexity grounding.
NeurIPS 2022 · 2022
Long Ouyang, Jeff Wu, Xu Jiang + 3 more
Introduces InstructGPT and the RLHF (Reinforcement Learning from Human Feedback) pipeline: (1) collect demonstrations from human labelers for supervised fine-tuning, (2) collect human preference comparisons over model outputs to train a reward model, (3) optimize the LM against the reward model via PPO. Shows that this pipeline dramatically improves helpfulness, truthfulness, and harmlessness compared to the raw GPT-3 baseline, at a fraction of the parameter count.
arXiv preprint · 2022
Yuntao Bai, Saurav Kadavath, Sandipan Kundu + 3 more
Introduces Constitutional AI (CAI): an alignment approach where an LLM critiques and revises its own outputs according to a written constitution of principles, with reinforcement learning from AI feedback (RLAIF) replacing the human-labeling step. Demonstrates that RLAIF can produce models that are both more helpful AND more harmless than RLHF baselines, while scaling alignment without proportional human labeling effort.
NeurIPS 2022 · 2022
Jason Wei, Xuezhi Wang, Dale Schuurmans + 3 more
Demonstrates that prompting LLMs to articulate intermediate reasoning steps before producing a final answer ('chain-of-thought prompting') dramatically improves accuracy on math, logic, and multi-step problem-solving benchmarks. The improvement scales with model size and emerges only at sufficient scale. Establishes step-by-step reasoning as a critical prompting technique and a foundation for later 'reasoning model' designs.
ICLR 2022 · 2021
Edward J. Hu, Yelong Shen, Phillip Wallis + 3 more
Introduces Low-Rank Adaptation (LoRA): a parameter-efficient fine-tuning technique that adds small low-rank matrices to a frozen base model. Demonstrates that LoRA matches full fine-tuning performance on multiple benchmarks while updating only 0.1%–1% of parameters. Reduces GPU memory requirements and storage footprint by orders of magnitude.
NeurIPS 2020 · 2020
Tom B. Brown, Benjamin Mann, Nick Ryder + 3 more
Introduces GPT-3, a 175B-parameter autoregressive language model, and demonstrates that scaling up a Transformer LM produces emergent few-shot in-context learning capability. Shows that a single model can perform many NLP tasks competitively without fine-tuning, simply by being shown a few examples in the prompt. Documents capability and scaling behaviors that defined the LLM era.
NeurIPS 2020 · 2020
Patrick Lewis, Ethan Perez, Aleksandara Piktus + 3 more
Introduces Retrieval-Augmented Generation (RAG): an architecture that combines a pretrained sequence-to-sequence model (BART) with a non-parametric memory (a Dense Passage Retrieval index over Wikipedia). Demonstrates strong performance on knowledge-intensive NLP tasks while providing transparency about which documents informed each generation. Establishes the design pattern of retrieving documents before generating.
IT Revolution Press (book) · 2018
Nicole Forsgren, Jez Humble, Gene Kim
Documents the multi-year DevOps Research and Assessment (DORA) research showing that four metrics — deployment frequency, lead time for changes, change failure rate, and mean time to recovery — empirically predict software-organization performance. Establishes the empirical foundation for engineering-velocity measurement as a research discipline.
NeurIPS 2017 · 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar + 5 more
Introduces the Transformer architecture: a sequence-to-sequence model based entirely on attention mechanisms, dispensing with recurrence and convolutions. Demonstrates state-of-the-art results on English-to-German and English-to-French translation benchmarks with significantly less training time than the prior recurrent encoder-decoder models. The architecture's self-attention mechanism allows parallel processing of sequence elements and scales effectively with model size and data.
Code-Side Sourcing methodology, replicable on the open dataset.
Read /methodologyThese are the papers behind the method. Here's the method in action.
You don’t read the code — we do
You never open a repo. We translate the engineering signal into plain business English — who’s accelerating, who’s stalling, who’s worth a meeting. No GitHub account, no terminal, nothing to install.
€7 once · 30-day Signal-or-It’s-Free — reply REFUND, keep everything · no auto-renew · compare all tiers