Evaluating Password Advice

Password advice is constantly circulated by standards agencies, companies, websites and specialists. But there appears to be great diversity in terms of the advice that is given. Users have noticed that different websites are enforcing different restrictions. For example, requiring different combinations of uppercase and lowercase letters, numbers and special characters. We collected password advice and found that the advice distributed by one organization can directly contradict advice given by another. Our paper aims to illuminate interesting characteristics for a sample of the password advice distributed. We also create a framework for identifying the costs associated with implementing password advice. In doing so we identify a reason for why password advice is often both derided and ignored.

💡 Research Summary

The paper “Evaluating Password Advice” investigates the bewildering variety of password recommendations that users encounter across the Internet, corporate policies, standards bodies, and security specialists. The authors begin by collecting a large corpus of advice: 269 distinct pieces drawn from 21 sources, including multinational companies, universities, security experts, and general‑interest articles. Academic papers were excluded because they are not readily accessible to end‑users. The corpus is split into 155 pieces aimed at users and 114 aimed at organizations.

The next step is systematic categorisation. The authors iteratively group the advice into 29 high‑level categories (e.g., “Password Reuse”, “Composition”, “Expiry”, “Personal Information”, “Storage”, etc.). Within each category they further distil the raw statements into 78 concise “statements” that capture a unified sentiment (e.g., “Never reuse a password”, “Include special characters”, “Do not use published phrases”). For each statement they count how many source excerpts agree and how many disagree, thereby exposing contradictions. For example, the category “Phrases” contains both “Do not use published phrases” and “Choose a line of a song that no one else would associate with you”, which are directly opposed.

Having identified the logical landscape of advice, the authors introduce a cost‑analysis framework. They argue that any password policy imposes concrete costs on users and on organisations, and that these costs are often ignored in the literature. Ten cost categories are defined:

1. Increased risk of forgetting (user memory burden).
2. Need to pick a new password (selection effort).
3. Possible multiple attempts to enter a valid password (time and frustration).
4. Inconveniences of using personal password‑generation systems.
5. User time spent complying.
6. Reduced “entropy” (i.e., smaller key‑space, easier guessing).
7. Organisation time required to implement or program the rule.
8. Difficulty or impossibility of enforcement.
9. Creation of new security holes (e.g., password‑reset mechanisms).
10. Increased computational power required (e.g., for verification).

The authors note that many of these costs are hierarchical; for instance, “increased risk of resets” is a sub‑cost of “increased forgetting risk”. They deliberately keep “user time” as a separate category because it can be the sole cost in some cases.

The paper then focuses on four representative categories—Phrases, Composition, Reuse, and Expiry—to illustrate how the cost framework interacts with real‑world advice.

Phrases: Advice about using or avoiding dictionary words, song lyrics, or other published text is highly contradictory. While some sources forbid any published phrase, others encourage selecting a line that is obscure to the user. The authors point out that the most common passwords in leaked datasets are still simple dictionary words, indicating that advice to avoid them is often ignored because the perceived cost (e.g., memorability) outweighs the security benefit. The “substitute symbols for letters” recommendation is also split: two sources endorse leet‑style substitutions, while a third warns they provide little extra security. The cost analysis shows that the user burden is low, which may explain why the advice persists despite limited efficacy.

Composition: Restrictions such as “must include special characters”, “no repeated characters”, and “enforce character class rules” are widespread but not uniform. Some sites ban spaces, others ban all special characters, and a few (e.g., NIST 2016 draft) argue against composition rules altogether. The authors calculate that mandatory special characters reduce the effective key‑space (e.g., from 96 to 62 possible characters for one position), thereby “reducing entropy”. However, the security gain is modest compared to the increased memorability cost and user frustration, especially when multiple composition rules are stacked.

Reuse: The classic recommendation “never reuse passwords” is contrasted with advice that allows reuse of certain passwords across low‑risk accounts. The cost framework shows that strict reuse bans dramatically increase the risk of forgetting and the time needed to generate and remember many unique passwords, while the security benefit (preventing credential stuffing) is substantial. The authors argue that the lack of a unified stance leads to user confusion and policy non‑compliance.

Expiry: Periodic password changes are intended to limit the window of exposure after a breach. Empirical studies, however, reveal that forced expiry often leads users to make minimal modifications (e.g., appending a digit), which reduces effective entropy and may even introduce predictable patterns. The cost analysis highlights a high user‑time burden and increased likelihood of password fatigue, which can drive users toward insecure coping strategies.

Overall, the authors conclude that many current password recommendations impose significant cognitive and operational costs while delivering modest security improvements. This aligns with Herley’s earlier economic argument that users rationally reject security advice when the perceived cost exceeds the benefit. By quantifying the costs, the paper provides a practical tool for policymakers: they can prioritize recommendations that offer a favorable cost‑benefit ratio and discard or redesign those that are too burdensome.

The discussion also points to future work: developing quantitative metrics for each cost category, conducting user studies to validate the perceived burdens, and extending the framework to newer authentication mechanisms such as passphrases, password managers, and multi‑factor authentication. The authors suggest that a cost‑aware approach could lead to more coherent, user‑friendly, and ultimately more secure password policies.