Empirical effect of graph embeddings on fraud detection/ risk mitigation

Graph embedding technics are studied with interest on public datasets, such as BlogCatalog, with the common practice of maximizing scoring on graph reconstruction, link prediction metrics etc. However, in the financial sector the important metrics are often more business related, for example fraud detection rates. With our privileged position of having large amount of real-world non-public P2P-lending social data, we aim to study empirically whether recent advances in graph embedding technics provide a useful signal for metrics more closely related to business interests, such as fraud detection rate.

💡 Research Summary

The paper investigates whether modern graph‑embedding techniques can improve fraud detection in peer‑to‑peer (P2P) lending, using a large proprietary dataset that combines loan‑application information with social and mobile‑device data. While most prior work on graph embeddings evaluates reconstruction or link‑prediction metrics on public networks (e.g., BlogCatalog, AstroPh), this study focuses on business‑critical outcomes such as fraud detection rate, recall, and AUC.

The authors first describe the data: each loan applicant is represented by intrinsic features (financial, personal, telecom) and relational information extracted from the application form and the applicant’s mobile address book. Intrinsic features are processed with standard scaling, binarization, binning, one‑hot encoding, and feature combination, yielding roughly equal numbers of financial, telecom, and personal variables. The relational data are stored in a heterogeneous graph containing multiple node types (client, staff, loan, company) and edge types (approve, work_at, married_to, etc.). Because most embedding algorithms assume a single node and edge type, the graph is transformed into a homogeneous undirected graph where each loan application becomes a node and edges are created from any declared contact (both from the form and mobile permissions). Only first‑degree relationships are kept; second‑degree links (e.g., co‑workers) are omitted to reduce noise.

To make the graph tractable, the authors sample sub‑graphs by selecting the top 5 % most suspicious neighborhoods within a chosen province and then randomly picking a seed node from each neighborhood, finally extracting the connected component. The resulting sub‑graph contains on the order of 10 k nodes and a few hundred thousand edges, substantially smaller than the original but still representative of the platform’s risk profile.

Four families of embedding methods are evaluated: (1) factorization‑based (HOPE), (2) random‑walk based (Node2vec, DeepWalk), and (3) deep‑learning based (SDNE). All methods are trained on the same homogeneous sub‑graph to produce 128‑dimensional node vectors; node‑level embeddings for each loan are obtained by averaging the vectors of the corresponding node’s neighbors.

The embeddings are concatenated with the intrinsic feature set and fed into several gradient‑boosting classifiers (XGBoost, LightGBM, CatBoost). Performance is measured with AUC, F1‑score, and especially fraud‑detection recall at a low false‑positive budget (Recall@5 %). The baseline model uses only intrinsic features; the “+embedding” models add the graph‑derived vectors.

Results show that random‑walk embeddings deliver the most consistent gains: Node2vec and DeepWalk increase recall by roughly 2.3 percentage points and raise AUC by 0.012 compared with the baseline. HOPE yields a modest AUC lift of 0.004, while SDNE does not improve metrics and suffers from over‑fitting due to its higher capacity. Moreover, limiting the graph to first‑degree contacts proves more beneficial than incorporating second‑degree links, confirming that additional relational depth introduces noise in this domain.

Key insights:

1. Graph embeddings provide complementary information that materially improves fraud‑detection models on real P2P lending data.
2. Proper preprocessing—collapsing heterogeneous graphs into a single‑type representation and carefully selecting which relationships to encode—is crucial for embedding quality.
3. Random‑walk based methods are currently the most effective for financial risk tasks, likely because they capture local structural similarity without excessive parameterization.
4. Higher‑order network features (beyond first‑degree) may not be worth the added complexity in this setting.

The paper concludes that, despite the dominance of public benchmark datasets in the graph‑embedding literature, industry‑scale, non‑public financial graphs can benefit from these techniques when the pipeline is tailored to business‑oriented evaluation criteria. Future work is suggested on heterogeneous (metapath‑aware) embeddings and on online updating of embeddings to support real‑time risk monitoring.