DuoLungo: Usability Study of Duo 2FA

Multi-Factor Authentication (MFA) enhances login security by requiring multiple authentication factors. Its adoption has increased in response to more frequent and sophisticated attacks. Duo is widely used by organizations including Fortune 500 companies and major educational institutions, yet its usability has not been examined thoroughly or recently. Earlier studies focused on technical challenges during initial deployment but did not measure core usability metrics such as task completion time or System Usability Scale (SUS) scores. These results are also outdated, originating from a time when MFA was less familiar to typical users. We conducted a long-term, large-scale Duo usability study at the University of California Irvine during the 2024-2025 academic year, involving 2559 participants. Our analysis uses authentication log data and a survey of 57 randomly selected users. The average overhead of a Duo Push task is nearly 8 seconds, which participants described as short to moderate. Overhead varies with time of day, field of study, and education level. The rate of authentication failures due to incomplete Duo tasks is 4.35 percent, and 43.86 percent of survey respondents reported at least one Duo login failure. The Duo SUS score is 70, indicating good usability. Participants generally find Duo easy to use but somewhat annoying, while also reporting an increased sense of account security. They also described common issues and offered suggestions for improvement.

💡 Research Summary

This paper presents a comprehensive, real‑world usability evaluation of Duo two‑factor authentication (2FA) conducted at the University of California, Irvine over the 2024‑2025 academic year. While Duo is widely deployed across Fortune‑500 companies and major universities, prior research has focused mainly on deployment challenges and user perceptions during initial rollout, leaving core usability metrics such as task completion time and System Usability Scale (SUS) scores largely unmeasured.

The study combines two data sources. First, authentication logs from the university’s IT department were collected for nine months (August 2024 – April 2025), yielding 96,048 entries. After cleaning (removing entries without user IDs, invalid IDs, and opt‑out participants), 45,541 Duo push events linked to 2,558 unique users remained for quantitative analysis. Each Duo push event records the timestamp when the push notification is sent and the timestamp when the user clicks “Approve” or “Deny” in the Duo app. The difference defines the “overhead” or task‑completion time.

Second, a post‑study survey was sent to 800 randomly selected participants; 57 completed it (≈7 % response rate). The questionnaire gathered demographic data, perceived difficulty, annoyance, security benefit, failure experiences, and included the standard ten‑item SUS. Respondents received a $5 Amazon gift card for participation.

Key quantitative findings:

• Average Duo push overhead = 7.82 seconds (≈8 s). Participants described this as “short‑to‑moderate.”
• Overhead varies significantly with time of day (≈9.3 s during late‑night hours vs. 6.9 s in the morning), academic discipline (engineering/science users are ~0.8 s faster than humanities/social‑science users), and education level (upper‑classmen are about 1 s faster than freshmen).
• Incomplete Duo tasks (push sent but no approval/denial recorded) occur in 4.35 % of push events.
• 43.86 % of survey respondents reported at least one Duo login failure, citing missed notifications, network latency, device changes, and accidental dismissal of pushes.

Usability assessment: The SUS average score is 70, which maps to the “Good” category on the standard adjective scale (51.7–73.5). Individual SUS items reveal a mixed picture: users strongly agree that the system is easy to learn and use, yet also perceive it as unnecessarily complex and cumbersome. This duality aligns with qualitative comments that Duo is “easy but annoying.”

Qualitative insights: Survey participants overwhelmingly believe Duo improves account security. Common pain points include push‑notification latency or loss, cumbersome re‑enrollment after phone number or device changes, and the extra step required for users without smartphones who must generate OTPs. Suggested improvements involve enhancing push reliability, streamlining device‑swap workflows, and refining the UI to reduce perceived annoyance.

Limitations acknowledged by the authors: (1) The analysis excludes nearly half of the log entries lacking user IDs, potentially under‑estimating overall failure rates. (2) The low survey response rate may introduce selection bias. (3) OTP‑based second‑factor interactions were not measured due to logging constraints, limiting the study’s coverage of all Duo usage modes. (4) The participant pool is dominated by university students and staff, which may not fully represent corporate environments.

Implications: The study demonstrates that Duo’s average 8‑second push delay does not materially disrupt typical academic workflows, and a SUS score of 70 suggests that most users find the system acceptable despite occasional annoyance. Organizations considering MFA deployment can cite these findings as evidence that security gains can be achieved with modest usability impact, provided that push reliability and device‑transition processes are continuously refined. Future work could compare Duo’s push‑based flow with alternative 2FA methods (e.g., hardware tokens, SMS codes), evaluate accessibility for users with disabilities, and test adaptive push‑timing algorithms to further reduce perceived latency.

In summary, this large‑scale, longitudinal field study fills a gap in the literature by delivering concrete, data‑driven metrics on Duo’s real‑world usability, highlighting both its strengths (short overhead, good overall SUS, perceived security benefit) and areas for improvement (failure handling, annoyance, device management). The findings are valuable for security practitioners, UX designers, and policy makers aiming to balance strong authentication with user‑friendly experiences.