Continuous Patient Monitoring with AI: Real-Time Analysis of Video in Hospital Care Settings

This study introduces an AI-driven platform for continuous and passive patient monitoring in hospital settings, developed by LookDeep Health. Leveraging advanced computer vision, the platform provides real-time insights into patient behavior and interactions through video analysis, securely storing inference results in the cloud for retrospective evaluation. The dataset, compiled in collaboration with 11 hospital partners, encompasses over 300 high-risk fall patients and over 1,000 days of inference, enabling applications such as fall detection and safety monitoring for vulnerable patient populations. To foster innovation and reproducibility, an anonymized subset of this dataset is publicly available. The AI system detects key components in hospital rooms, including individual presence and role, furniture location, motion magnitude, and boundary crossings. Performance evaluation demonstrates strong accuracy in object detection (macro F1-score = 0.92) and patient-role classification (F1-score = 0.98), as well as reliable trend analysis for the “patient alone” metric (mean logistic regression accuracy = 0.82 \pm 0.15). These capabilities enable automated detection of patient isolation, wandering, or unsupervised movement-key indicators for fall risk and other adverse events. This work establishes benchmarks for validating AI-driven patient monitoring systems, highlighting the platform’s potential to enhance patient safety and care by providing continuous, data-driven insights into patient behavior and interactions.

💡 Research Summary

The paper presents an AI‑driven platform for continuous, passive patient monitoring in hospital rooms, developed by LookDeep Health. Using RGB or near‑infrared video captured at one frame per second by dedicated LookDeep Video Units (LVUs), the system processes each frame locally (compression to JPEG at 80 % quality, resizing to 1088 × 612) before securely uploading inference results to a Google Cloud database. Privacy is protected through a two‑step face anonymization (manual bounding‑box labeling followed by Gaussian blur) in compliance with HIPAA and institutional Business Associate Agreements.

The computer‑vision pipeline consists of four main components. First, a YOLOv4‑based object detector, fine‑tuned on more than 40 000 manually annotated frames, identifies “person”, “bed”, and “chair” objects with a macro F1‑score of 0.92. Second, each detected person is further classified into “patient”, “staff”, or “other” using role‑specific labels added during training; the role classifier achieves an F1‑score of 0.98, providing high‑confidence distinction between patients and caregivers. Third, motion estimation is performed with the Gunnar‑Farneback dense optical‑flow algorithm on down‑sampled grayscale frames (480 × 270). Average motion magnitude is computed for predefined regions of interest (full scene, bed area, safety zone), yielding a quantitative activity score without additional training. Fourth, logical predictions such as “person alone”, “patient alone”, and “supervised by staff” are derived from the combination of object detections, role classifications, and motion scores. A five‑second smoothing filter mitigates transient detection errors, and the resulting high‑level states are stored alongside raw detections for downstream analysis.

Data were collected from 11 hospitals across three states, focusing on high‑risk fall patients identified by standard mobility assessments. The dataset comprises three subsets: (1) a single‑frame set for model development (≈40 000 frames, with 10 000 held‑out for testing), (2) observation logs for ten patients who experienced falls (54 patient‑days) with manually timestamped “alone” periods, and (3) a publicly released anonymized dataset covering over 300 patients and more than 1 000 patient‑days of continuous monitoring. All video frames were anonymized, and metadata were stored in Google BigQuery under strict security controls.

Performance evaluation was conducted on two levels. Frame‑level metrics measured precision, recall, and F1 for object detection, role classification, and the derived logical states. Trend‑level evaluation compared AI‑derived hourly metrics with ground‑truth observation logs, aligning predictions at the per‑second level. Logistic regression accuracy was used to assess the system’s ability to predict “patient alone” behavior during daytime (6 am–9 pm), nighttime (9 pm–6 am), and the full 24‑hour period, yielding an average accuracy of 0.82 ± 0.15. In cases where only one class was present, a manual match‑rate was computed. An “assisted” analysis demonstrated that integrating AI predictions with manually logged “alone” periods further improved trend accuracy, highlighting the complementary value of AI and human observation.

The authors emphasize several contributions: (1) deployment of state‑of‑the‑art vision models on edge devices capable of real‑time inference at 1 fps, reducing bandwidth and latency; (2) rigorous real‑world validation across multiple hospitals, showing that continuous video analytics can reliably flag patient isolation, wandering, and unsupervised movement—key risk factors for falls; (3) creation and public release of a large, anonymized dataset that establishes a benchmark for future research in continuous patient monitoring. The study also discusses challenges such as varying lighting conditions, camera placement, and the need for transparent, privacy‑preserving pipelines to gain clinical acceptance.

In conclusion, the LookDeep Health platform demonstrates that AI‑enabled continuous video monitoring can provide actionable, data‑driven insights into patient behavior, supporting early detection of fall risk and other adverse events. By achieving high detection and classification performance, ensuring privacy compliance, and offering an openly available dataset, the work paves the way for broader adoption of vision‑based monitoring systems in healthcare settings and invites further research on multi‑camera integration, advanced activity recognition, and domain adaptation to diverse clinical environments.