LogSyn: A Few-Shot LLM Framework for Structured Insight Extraction from Unstructured General Aviation Maintenance Logs

Aircraft maintenance logs hold valuable safety data but remain underused due to their unstructured text format. This paper introduces LogSyn, a framework that uses Large Language Models (LLMs) to convert these logs into structured, machine-readable data. Using few-shot in-context learning on 6,169 records, LogSyn performs Controlled Abstraction Generation (CAG) to summarize problem-resolution narratives and classify events within a detailed hierarchical ontology. The framework identifies key failure patterns, offering a scalable method for semantic structuring and actionable insight extraction from maintenance logs. This work provides a practical path to improve maintenance workflows and predictive analytics in aviation and related industries.

💡 Research Summary

The paper introduces LogSyn, a novel framework that leverages off‑the‑shelf large language models (LLMs) such as GPT‑4 and Gemini to transform unstructured general‑aviation (GA) maintenance logs into structured, machine‑readable records. The authors start by highlighting the critical safety value embedded in maintenance logs, which traditionally consist of two free‑text fields—“Problem” (symptom description) and “Action Taken” (repair steps). Because these narratives are jargon‑heavy and lack a fixed schema, conventional keyword‑based NLP or rule‑based extraction methods fail to capture causal relationships and nuanced domain terminology.

LogSyn addresses this gap through a five‑stage pipeline: (a) preprocessing (whitespace cleaning, abbreviation normalization, concatenation of problem and action text); (b) few‑shot prompt construction that embeds task instructions and 2–3 representative examples; (c) deterministic LLM inference (temperature = 0.1) to generate a JSON object; (d) post‑processing that parses, validates, and flags structural anomalies; and (e) aggregation for macro‑level analysis. The core of the system is Controlled Abstraction Generation (CAG), which simultaneously produces a concise summary of the problem, a concise summary of the action, identifies the failed component, and assigns the record to a hierarchical ontology.

The dataset used is the publicly available Aircraft Historical Maintenance Dataset (2012‑2017) from Kaggle, comprising 6,169 GA entries (mostly Cessna 172). The authors first enriched the raw data with ontology labels generated by Gemini and subsequently corrected them manually for evaluation purposes.

A data‑driven ontology emerges from the LLM’s own clustering of recurring themes. Eight top‑level categories are derived, with “Powerplant – Sealing & Gaskets” accounting for 55 % of the logs (3,454 entries). Other categories include “Powerplant – Mechanical”, “Powerplant – Structural Components”, “Ignition System – Component Failure”, and several service‑oriented groups. This bottom‑up taxonomy reflects the actual maintenance workload of GA fleets more faithfully than generic taxonomies.

Performance is evaluated in two complementary ways. First, a qualitative manual review confirms that CAG outputs are accurate and that the JSON schema is consistently adhered to. Second, an “LLM‑as‑a‑Judge” approach uses a second LLM to rate each output on a 1‑5 Likert scale for summary accuracy (4.7), component accuracy (4.5), and category relevance (4.8). Quantitatively, LogSyn’s few‑shot approach is benchmarked against three baselines: (1) zero‑shot LLM classification, (2) rule‑based NER (regex + spaCy), and (3) a supervised BERT NER fine‑tuned with cosine similarity. LogSyn achieves an overall accuracy of 0.9021 versus 0.8899 for zero‑shot, and markedly higher macro‑precision (0.7455 vs. 0.6427) and macro‑F1 (0.7614 vs. 0.6891). The macro‑averaged metrics demonstrate LogSyn’s robustness on rare fault classes, a critical advantage for safety‑critical domains where infrequent failures can have outsized impact.

Visualization of the structured data using Sankey diagrams reveals clear problem‑to‑action pathways; for example, the “Powerplant – Sealing & Gaskets” category overwhelmingly leads to “Component Replacement” actions. Such visual analytics enable maintenance managers to prioritize training, inventory stocking, and predictive modeling based on empirically observed failure patterns.

The authors acknowledge limitations: performance varies by 2‑4 % with different prompt phrasings, indicating prompt sensitivity; and the few‑shot examples can bias the model, especially for low‑frequency classes. They suggest future work on automated prompt optimization, lightweight fine‑tuning (e.g., preference‑based methods), and extending the framework to other high‑safety sectors such as rail, maritime, and energy.

In conclusion, LogSyn provides a reproducible, cost‑effective pipeline that converts free‑text GA maintenance logs into structured JSON records via controlled abstraction generation and a data‑derived hierarchical ontology. The resulting structured dataset unlocks quantitative analysis, trend detection, and integration into real‑time predictive maintenance dashboards, thereby offering tangible improvements to maintenance workflows, safety oversight, and operational efficiency in aviation and beyond.