Detecting Malicious Entra OAuth Apps with LLM-Based Permission Risk Scoring

This project presents a unified detection framework that constructs a complete corpus of Microsoft Graph permissions, generates consistent LLM-based risk scores, and integrates them into a real-time detection engine to identify malicious OAuth consent activity.

💡 Research Summary

The paper addresses the growing problem of malicious OAuth applications in Microsoft Entra ID by introducing a unified detection framework that leverages large language models (LLMs) to assign risk scores to Microsoft Graph permissions. The authors first construct a comprehensive corpus of 769 Graph permissions, enriching each entry with metadata such as read/write scope, global versus resource‑specific access, and functional category. Using eight open‑source LLMs—including GPT‑OSS‑120B, GPT‑OSS‑Safeguard‑120B, and Qwen‑3‑235B—they generate a consistent risk score (1 = low, 5 = high) and a natural‑language justification for each permission. The resulting dataset, which is publicly released, reveals that broad “.Read.All” and “.Write.All” scopes consistently receive the highest scores, while narrowly scoped permissions like “User.Read” are rated low.

The detection pipeline consists of five stages. Stage 1 collects application registration events and consent logs from Entra ID. Stage 2 aggregates the LLM‑derived scores for all permissions requested by an app, producing an “Aggregated Application Risk (Rapp)” metric. Stage 3 applies a stateful spike‑logic algorithm that monitors short‑term fluctuations in Rapp; a rapid increase triggers a spike state stored in an in‑memory buffer. Stage 4 sends real‑time alerts via Slack Webhooks and persists the event, risk scores, and metadata in a SQLite database for auditability. Stage 5 updates the system state and cleans up temporary data.

Empirical evaluation shows clear separation between high‑risk and low‑risk permission sets, with statistical analysis of mean, standard deviation, and distribution across models. The authors also perform N‑gram and tri‑gram analyses of LLM reasoning texts to assess consistency and identify model‑specific biases. Limitations include dependence on prompt design, the need for periodic corpus updates as new permissions are introduced, and the current focus on static permission risk without modeling inter‑permission correlations or user behavior.

Future work proposes multi‑model ensemble scoring, correlation‑aware risk adjustment, extension to other identity platforms (Azure AD, Google Workspace), and integration with dynamic threat intelligence feeds to enable adaptive risk scoring and automated response playbooks. In summary, the study demonstrates that LLM‑based permission risk scoring, when coupled with a real‑time detection engine, provides a practical and explainable method for identifying malicious Entra OAuth applications, filling a critical gap in cloud identity security.