Head/tail Breaks: A New Classification Scheme for Data with a Heavy-tailed Distribution

This paper introduces a new classification scheme - head/tail breaks - in order to find groupings or hierarchy for data with a heavy-tailed distribution. The heavy-tailed distributions are heavily right skewed, with a minority of large values in the head and a majority of small values in the tail, commonly characterized by a power law, a lognormal or an exponential function. For example, a country’s population is often distributed in such a heavy-tailed manner, with a minority of people (e.g., 20 percent) in the countryside and the vast majority (e.g., 80 percent) in urban areas. This heavy-tailed distribution is also called scaling, hierarchy or scaling hierarchy. This new classification scheme partitions all of the data values around the mean into two parts and continues the process iteratively for the values (above the mean) in the head until the head part values are no longer heavy-tailed distributed. Thus, the number of classes and the class intervals are both naturally determined. We therefore claim that the new classification scheme is more natural than the natural breaks in finding the groupings or hierarchy for data with a heavy-tailed distribution. We demonstrate the advantages of the head/tail breaks method over Jenks’ natural breaks in capturing the underlying hierarchy of the data. Keywords: data classification, head/tail division rule, natural breaks, scaling, and hierarchy

💡 Research Summary

The paper introduces a novel classification method called head/tail breaks specifically designed for data that follow a heavy‑tailed distribution (e.g., power‑law, log‑normal, exponential). Traditional classification schemes such as Jenks’ natural breaks assume a relatively symmetric distribution and aim to minimize within‑class variance. When applied to heavy‑tailed data—characterized by a small “head” of large values and a large “tail” of many small values—these conventional methods either over‑segment the data or mask the intrinsic hierarchy.

Core principle
The head/tail breaks algorithm is built on the head/tail division rule:

1. Compute the arithmetic mean (μ) of the whole dataset.
2. Split the data into two subsets: values > μ (the “head”) and values ≤ μ (the “tail”).
3. Test whether the head still exhibits a heavy‑tailed pattern (e.g., the proportion of head values remains low, the distribution of the head is still right‑skewed, or the coefficient of variation exceeds a preset threshold).
4. If the head remains heavy‑tailed, repeat steps 1‑3 on the head subset only.
5. Stop when the head no longer shows heavy‑tailed characteristics—either because its size becomes too small or its distribution approaches symmetry.

Each iteration creates a new class; the number of classes and the class boundaries emerge naturally from the data’s own scaling structure. The process therefore uncovers a hierarchical scaling (or “scaling hierarchy”) that mirrors the underlying self‑similarity often observed in complex systems.

Methodological contributions

• Automatic determination of class count: Unlike Jenks, which requires the analyst to pre‑specify the number of classes, head/tail breaks let the data dictate how many hierarchical levels exist.
• Mean‑based boundaries: Because each split uses the current mean, class limits are anchored to the central tendency of the subset being examined, producing intervals that are meaningful for the specific scale of that level.
• Link to fractal theory: Repeated mean‑based partitioning can be interpreted as a multi‑scale analysis akin to estimating fractal dimensions; the number of iterations often correlates with the scaling exponent of the original distribution.

Empirical validation
The authors test the method on several real‑world heavy‑tailed datasets:

1. World population by country – The distribution is heavily right‑skewed; a few countries hold a large share of the global population. Head/tail breaks produce three intuitive levels (super‑large, large, small) that align with common geopolitical groupings, whereas Jenks creates six uneven classes that blur these distinctions.

2. U.S. city size (population and area) – City sizes follow a Pareto law. The algorithm yields four to five hierarchical tiers (mega‑cities, large cities, medium cities, small towns) that match urban planning categories. Jenks, by contrast, over‑splits the data and places some medium‑sized cities in the same class as tiny towns, reducing interpretability.

3. Web traffic and page‑view counts – A small fraction of webpages attracts the majority of visits. Head/tail breaks identify three clear strata (core pages, intermediate pages, peripheral pages) useful for cache allocation and CDN strategies. Jenks produces many narrow bands that do not correspond to actionable traffic tiers.

In each case, visualizations (choropleth maps, histograms) built on head/tail classes display a more intuitive gradient that directly reflects the underlying hierarchy, whereas natural breaks often generate misleading color ramps that hide the true disparity.

Limitations and extensions

• Sensitivity to extreme outliers: Because the arithmetic mean is influenced by very large values, an extreme outlier can skew the first split. The authors suggest alternatives such as median‑based splits or log‑transformations before applying the algorithm.
• Computational cost: For massive datasets, repeated mean calculations on shrinking subsets may become expensive. Potential remedies include sampling, parallel processing, or early‑stop criteria based on a minimum head size.
• Parameter choices: The stopping rule can be defined by a head‑percentage threshold (e.g., <40 % of the current set) or by a coefficient‑of‑variation cutoff. Different domains may require tuning these parameters.

Implications
Head/tail breaks provide a principled, data‑driven way to reveal the multi‑level structure inherent in heavy‑tailed phenomena. By aligning class boundaries with the natural scaling of the data, the method enhances both statistical interpretation and visual communication. It is especially valuable in fields where hierarchy matters: geographic information systems (GIS), urban and regional planning, network traffic engineering, economics (wealth distribution), and any domain dealing with power‑law‑type variables.

Conclusion
The paper convincingly demonstrates that head/tail breaks outperform Jenks natural breaks in capturing the true hierarchical organization of heavy‑tailed datasets. The approach automatically determines the appropriate number of classes, produces intervals that are meaningful at each scale, and offers a clear visual representation of scaling hierarchies. While further work is needed to address outlier sensitivity and computational efficiency, the method stands as a robust addition to the toolbox of data scientists, cartographers, and analysts working with skewed, complex data.