Emergent abilities

Scaling laws promised smooth loss curves, and delivered. Then researchers looked past the aggregate at individual skills, and the smoothness shattered. Three-digit arithmetic: near-zero accuracy across model after model, then a jump to competence somewhere past 10B parameters. Same pattern for translation, instruction following, multi-step reasoning.

These jumps got a loaded name: emergent abilities. Skills absent at one scale, present at a larger one, with nothing in the smaller models' loss curves warning they were coming.

Remember: every one of these models trained on the same next-token exam. Nobody added an arithmetic module at 13B parameters. The capability condensed out of scale, the way the BPE algorithm condensed suffixes out of raw counting: unprogrammed structure, appearing when the pressure is high enough.

In 2023, a Stanford paper (Schaeffer and colleagues) put a needle in the balloon: part of the "emergence" is an artifact of HOW we score. Grade three-digit addition as all-or-nothing and a model going from "2 digits right" to "3 digits right" looks like a leap from zero to hero. Grade partial credit and the underlying skill often climbs smoothly. Some celebrated jumps are the metric's cliff, not the model's.

Real, and also not the whole verdict. Two things stay true at once:

• Under the hood, competence mostly accumulates gradually, tracking the smooth loss curve.
• At the point of USE, thresholds exist anyway: an arithmetic that is 60% right is broken, a code generator that compiles half the time is a toy. Utility is all-or-nothing even when skill is not.

My position for you as a builder: skip the metaphysics, keep the operational rule. Capabilities you cannot see in today's models keep showing up in bigger ones, on no announced schedule. Test models empirically at each scale, and never assume the ceiling you observed is the ceiling.