An empirical relation between tokens N and observed vocabulary size V(N) commonly expressed as V(N) = K · N^β (with constants K>0 and 0<β<1 for many natural language corpora), meaning vocabulary grows sublinearly with text length.

Principle

As token count increases, new distinct types continue to appear but at a decelerating rate: incremental vocabulary growth per additional token declines approximately as a power law governed by exponent β, so doubling tokens typically increases vocabulary by a factor 2^β rather than twofold.

Demonstration

Illustrative scenario: A corpus with N=1,000,000 tokens fitted to V(N)=K·N^β with β=0.5 implies that doubling corpus size to 2,000,000 tokens increases V by factor ≈2^0.5≈1.414; recognition: observed vocabulary expands but with diminishing marginal novelty per token.

Misapplication

Treating K and β as universal constants for all languages, domains, preprocessing pipelines, or tokenisation choices is incorrect. Extrapolating the relation far beyond observed data without checking fit, or ignoring lemmatisation and orthographic normalisation, leads to misleading predictions.

Consequence

Heaps’ law provides a practical basis for estimating expected vocabulary sizes, planning lexicon storage, and setting vocabulary cutoffs for language models; causally, it implies that open‑vocabulary systems must manage a long, slowly accumulating tail of rare types as data increases.

Reversal

At very small N, vocabulary growth can appear approximately linear (β≈1) before the sublinear regime emerges. In restricted or closed vocabularies (specialised lexica, taxonomies) V(N) can saturate and deviate from a power law; preprocessing (lemmatisation, token normalisation) alters estimated K and β.

Boundary

Clearly within: large, tokenised natural language corpora where new token types continue to appear with increasing N. Boundary case: multilingual or heavily noisy corpora where fitted β varies by language/subdomain. Clearly outside: closed finite lexicons, toy languages with bounded vocabularies, or analyses that count lemmas after heavy normalisation and therefore reach saturation.

Semantic Tension

Heaps’ law is a compact empirical summary useful for engineering trade‑offs (storage, vocabulary truncation) but competes with the desire for theoretically causal explanations of lexical growth (e.g., generative vs sampling accounts); choosing pragmatic fit versus explanatory model creates tension.

Synthesis

Heaps’ law summarises the common empirical pattern that vocabulary growth slows with corpus size and provides actionable scaling expectations for NLP and corpus design, but its parameters are descriptive and domain‑sensitive rather than universal laws.