Finding · 2026-07-07

MNIST-1D: a minimal playground for the science of deep learning

A procedurally generated 1D dataset ~100× smaller than MNIST that still separates model classes the way real benchmarks do — so hypotheses about deep learning become testable in minutes on a CPU.

The finding

Most deep learning experiments are bottlenecked by compute before they are bottlenecked by ideas. MNIST-1D flips that: a dataset roughly 100× smaller than MNIST that still separates model classes the way real benchmarks do. Hypotheses about deep learning — not just about vision — become testable in minutes on a CPU.

What MNIST-1D is

Sam Greydanus built MNIST-1D by starting from ten fixed one-dimensional template patterns, one per digit class, and procedurally transforming them: padding, translation, shear-like warping, added noise. Each example is a 40-point 1D sequence. The generative process is fully controlled — you can dial the difficulty, regenerate the dataset from scratch, or ablate individual transformations.

The surprising part: despite being tiny, it discriminates between architectures better than MNIST does.

Model	MNIST	MNIST-1D
MLP	≈ peers	lags — no temporal bias
CNN	≈ peers	wins — exploits structure
GRU	≈ peers	wins — exploits structure

On MNIST the three land within a few points; on MNIST-1D the gaps are wide, stable, and match what theory predicts.

Why this matters

There is a growing "science of deep learning" — treating networks as objects of empirical study, running controlled experiments, isolating variables. The limiting factor is iteration speed. If a single run takes a GPU-day, you get a handful of data points per week and no error bars.

With MNIST-1D the loop becomes: hypothesis → experiment → answer, in minutes. That changes what kinds of questions are worth asking. Sweeps over dozens of seeds, full ablation grids, and second-order experiments (experiments about experiments, like metalearning) become routine instead of heroic.

What it enables

The original paper and repo demonstrate a range of classic phenomena, all reproducible on a laptop:

Lottery tickets — finding sparse subnetworks that train to full accuracy, and showing they can transfer across tasks.
Double descent — the test-error curve that dips, rises, then dips again as model capacity grows past the interpolation threshold.
Metalearning — because a full training run is so cheap, you can differentiate through it: learning learning rates, or even learning activation functions.
Inductive-bias probes — clean comparisons of MLP vs CNN vs GRU on identical data, isolating what each architecture actually buys you.

Reproduction notes

The dataset is a single pip install mnist1d (or one file from the repo). Generation is deterministic given a seed; the default configuration produces 4000 train and 1000 test examples. A useful control experiment: regenerate with the shuffle flag, which permutes each sequence identically — this destroys local structure while keeping pointwise statistics, and the CNN advantage disappears on cue.

References

Repo: github.com/greydanus/mnist1d
Paper: Greydanus & Kobak, Scaling down deep learning with MNIST-1D, ICML 2024.