Home

Paper Reviews

Paper Reviews

Research takeaways worth keeping

Close readings of research papers and technical reports, with the authors’ claims, their evidence, and my interpretation kept distinct.

Where to start with speech

New to speech models? This order builds each idea before the next one needs it. Blog posts explain the basics; reviews read the papers.

  1. What a computer hears: samples, frames and mel spectrograms
  2. Residual vector quantization: turning sound into tokens
  3. SoundStream: the neural codec most others descend from
  4. VALL-E: text to speech as next-token prediction
  5. Flow Matching: the training loss behind today's fast generators
  6. F5-TTS: flow matching applied to speech, with no aligner
  7. Whisper: speech recognition from 680,000 hours of the web
  8. Moshi: one model that listens while it talks
  9. Delayed Streams Modeling: streaming recognition and speech on one clock
  10. MoshiRAG: looking facts up without breaking the conversation
  11. Pocket TTS and its Khmer fine-tuning: putting it together

Browse the reviews

TypeAllPaper reviews 16Technical reports 1
TopicAllSpeech 7Audio codecs 2Generative models 4Architecture 2Attention 1Training 10RL 3Inference 2Kernels 1Evaluation 4

17 entries

PaperTraining

Chinchilla: how much data a model should see, and how firmly the paper shows it

Chinchilla argues that parameters and training tokens should grow together with compute, roughly 20 tokens per parameter at its optimum. Two of its three estimation me...

DeepMind · 11 Nov 202610 min min

PaperRL · Training

DeepSeekMath and GRPO: a cheaper RL algorithm, and what RL actually changed

DeepSeekMath introduced GRPO, which replaces PPO's value model with the average reward of a group of samples. The paper's lasting value is twice over: a simpler algori...

DeepSeek-AI, Tsinghua University and Peking University · 11 Nov 20269 min min

PaperTraining · RL

DPO: preference learning without a reward model, and how strong the evidence was

DPO shows that the optimal policy of the KL-regularized reward objective defines its own reward, so preferences can train the policy directly. The derivation is exact ...

Stanford University · 11 Nov 20269 min min

PaperKernels · Attention · Training

FlashAttention: exact attention made fast by counting memory traffic

FlashAttention computes ordinary softmax attention in blocks that stay in on-chip memory, never writing the N × N score matrix. The paper backs this with an IO-complex...

Stanford University and University at Buffalo · 11 Nov 20269 min min

PaperTraining

LoRA: low-rank adaptation, and where low rank stops being enough

LoRA freezes a pre-trained model and trains a rank-r update to a few weight matrices, cutting trainable parameters by up to 10,000 times with no inference cost after m...

Microsoft · 11 Nov 20268 min min

Tech reportTraining · RL · Evaluation

Tülu 3: an open post-training recipe and what its ablations show

Tülu 3 releases every piece of a modern post-training pipeline, from prompts and decontamination to SFT, preference tuning and RL with verifiable rewards, together wit...

Allen Institute for AI and University of Washington · 11 Nov 202611 min min

PaperTraining · Evaluation

FineWeb: what each stage of a web-data pipeline is worth, and how the paper measures it

FineWeb builds a 15-trillion-token English web corpus by ablating every stage: text extraction, language and heuristic filters, MinHash deduplication and an educationa...

Hugging Face · 10 Nov 202614 min min

PaperTraining

Megatron-LM: tensor, pipeline and data parallelism, and what the comparisons control

Megatron-LM combines tensor parallelism inside a server, pipeline parallelism across servers and data parallelism on top, reaching 52% of peak on 3,072 GPUs. Its layou...

NVIDIA, Stanford University, Microsoft Research · 09 Nov 20269 min min

PaperTraining

ZeRO: what state partitioning proves, and what the experiments test

ZeRO removes the redundant copies of model state that data parallelism keeps on every GPU. The memory arithmetic is exact; the speedups come from bundled configuration...

Microsoft · 09 Nov 20267 min min

PaperSpeech · Generative models · Inference

Delayed Streams Modeling: Streaming ASR and TTS from One Shifted Clock

Kyutai's DSM lines text and audio up on a shared 12.5 Hz clock using word timestamps, then delays whichever stream is the output. Delay the text and you get streaming ...

Kyutai · 14 Oct 202618 min

PaperSpeech · Generative models

F5-TTS: Speech From Text Padded With Blanks

F5-TTS writes speech by flow matching on a mel spectrogram, with the text simply padded with filler tokens to the audio's length. A small ConvNeXt text branch makes th...

Shanghai Jiao Tong University · 14 Oct 202628 min

PaperGenerative models · Training

Flow Matching: Teaching Noise Where to Go

Flow matching trains a network to output a velocity by plain regression, one random time and one data point at a time. With straight-line optimal transport paths it be...

Meta AI (FAIR) · Weizmann Institute of Science · 14 Oct 202630 min

PaperSpeech · Architecture · Audio codecs

Moshi: A Model That Listens While It Talks

Moshi reads your audio and writes its own every 80 ms, writes a silent transcript of itself one step ahead of its voice, and answers within a theoretical 160 ms. Inner...

Kyutai · 14 Oct 202629 min

PaperSpeech · Audio codecs

SoundStream: One Codec, Every Bitrate

SoundStream turns 24 kHz audio into 75 small vectors a second, squeezes each with a stack of residual codebooks, and rebuilds the sound with a GAN-trained decoder. Tra...

Google Research · 14 Oct 202627 min

PaperSpeech · Generative models

VALL-E: Text to Speech as Next-Token Prediction

VALL-E turns speech into a grid of EnCodec tokens, predicts the first row word-by-word like GPT and fills the other seven rows in seven parallel passes. Trained on 60K...

Microsoft · 14 Oct 202620 min

PaperSpeech · Evaluation

Whisper: Speech Recognition from 680,000 Hours of the Internet

Whisper trains a plain encoder-decoder transformer on 680,000 hours of audio paired with web transcripts, in 97 languages and with translation mixed in. One token form...

OpenAI · 14 Oct 202628 min

PaperSpeech · Architecture · Inference · Evaluation

MoshiRAG: Retrieval That Fits Inside a Sentence

MoshiRAG lets a 7B full-duplex model trigger a lookup, keep talking, and absorb the answer before it reaches the fact. On TriviaQA, accuracy goes from 22.8% to 69.6%, ...

Kyutai · TTIC · 12 Oct 202619 min