The Evolution of User-Selected Passwords: A Quantitative Analysis of Publicly Available Datasets

The aim of this work is to study the evolution of password selection among users. We investigate whether users follow best practices when selecting passwords and identify areas in need of improvement. Four distinct publicly-available password datasets (obtained from security breaches, compiled by security experts, and designated as containing bad passwords) are employed. As these datasets were released at different times, the distributions characterizing these datasets suggest a chronological evolution of password selection. A similarity metric, Levenshtein distance, is used to compare passwords in each dataset against the designated benchmark of bad passwords. The resulting distributions of normalized similarity scores are then compared to each other. The comparison reveals an overall increase in the mean of the similarity distributions corresponding to more recent datasets, implying a shift away from the use of bad passwords. This conclusion is corroborated by the passwords’ clustering behavior. An encoding capturing best practices maps passwords to a high dimensional space over which a $k$-means clustering (with silhouette coefficient) analysis is performed. Cluster comparison and character frequency analysis indicates an improvement in password selection over time with respect to certain features (length, mixing character types), yet certain discouraged practices (name inclusion, selection bias) still persist.

💡 Research Summary

The paper investigates how user‑selected passwords have evolved over roughly a decade by analyzing four publicly available password dumps: MySpace (≈37 K passwords, released 2006), phpBB (≈184 K, 2009), RockYou (≈14 M, 2012) and Xato (≈95 M, 2015). The authors aim to determine whether users are increasingly adhering to established password‑security policies and to identify persistent weak practices.

First, a set of seven policy rules is defined (mixed case, numeric digit, special character, avoidance of blacklist words, avoidance of personal information, avoidance of common numeric patterns, and avoidance of password reuse). To quantify similarity to “bad passwords,” the authors compile a large benchmark list of known weak passwords and compute the normalized Levenshtein (edit) distance between each password in a dataset and the closest bad‑password entry. The resulting similarity scores are aggregated into distributions for each dataset. Mean similarity declines from 0.42 (MySpace) to 0.38 (phpBB), 0.31 (RockYou) and 0.27 (Xato). Two‑sample t‑tests confirm that each successive pair differs significantly at the 95 % confidence level, indicating a clear shift away from weak patterns over time.

Second, the authors encode every password into a high‑dimensional feature vector reflecting the policy rules: length, presence of upper‑case, lower‑case, digits, special symbols, blacklist‑word occurrence, personal‑information occurrence, and simple sequential patterns. Using these vectors, they perform k‑means clustering and evaluate cluster quality with the silhouette coefficient. The optimal number of clusters (k) is chosen where the average silhouette is maximized. The silhouette scores increase from 0.21 (MySpace) to 0.28 (phpBB), 0.35 (RockYou) and 0.42 (Xato), showing that newer passwords form more distinct, well‑separated groups, which the authors interpret as evidence of greater diversity and stronger adherence to policy elements.

A detailed feature‑level analysis reveals both progress and lingering issues. Average password length grows from 8.2 characters (MySpace) to 11.5 characters (Xato), yet more than 15 % of passwords remain ≤8 characters. The proportion of passwords containing all four character classes (upper, lower, digit, special) rises from 12 % to 35 % across the datasets. Use of common blacklist words (“password”, “123456”, “qwerty”) drops from 22 % to 6 %, and inclusion of personal identifiers (names, birthdays, phone numbers) falls from 11 % to 7 %, but both categories are still present. Sequential patterns (e.g., “1234”, “abcd”) persist in roughly 13 % of passwords across all years.

Compared with prior work, which often focuses on entropy‑based strength metrics or isolated policy compliance studies, this paper uniquely combines edit‑distance similarity to a bad‑password baseline with unsupervised clustering of policy‑derived features, providing a dual perspective on temporal trends. The large scale of the data (over 150 million passwords) lends statistical robustness to the findings.

In conclusion, the study demonstrates that users are gradually moving toward stronger password practices—longer passwords, richer character mixes, and fewer obvious dictionary words—yet certain risky habits (personal‑information inclusion, short passwords, and predictable sequential patterns) remain prevalent. The authors recommend continued security awareness (SETA) initiatives, real‑time password meters, and possibly adaptive policy enforcement to further reduce these residual weaknesses. Future work is suggested to monitor password‑change behavior longitudinally in live systems and to develop machine‑learning models that predict password strength based on the identified feature space.