Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems as a Translation of a Psychological Model

Assistive systems for persons with cognitive disabilities (e.g. dementia) are difficult to build due to the wide range of different approaches people can take to accomplishing the same task, and the significant uncertainties that arise from both the unpredictability of client’s behaviours and from noise in sensor readings. Partially observable Markov decision process (POMDP) models have been used successfully as the reasoning engine behind such assistive systems for small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modelling assistance that can deal with uncertainty and utility. Unfortunately, POMDPs usually require a very labour intensive, manual procedure for their definition and construction. Our previous work has described a knowledge driven method for automatically generating POMDP activity recognition and context sensitive prompting systems for complex tasks. We call the resulting POMDP a SNAP (SyNdetic Assistance Process). The spreadsheet-like result of the analysis does not correspond to the POMDP model directly and the translation to a formal POMDP representation is required. To date, this translation had to be performed manually by a trained POMDP expert. In this paper, we formalise and automate this translation process using a probabilistic relational model (PRM) encoded in a relational database. We demonstrate the method by eliciting three assistance tasks from non-experts. We validate the resulting POMDP models using case-based simulations to show that they are reasonable for the domains. We also show a complete case study of a designer specifying one database, including an evaluation in a real-life experiment with a human actor.

💡 Research Summary

The paper addresses a critical bottleneck in developing assistive technologies for individuals with cognitive impairments such as dementia: the labor‑intensive process of defining and constructing Partially Observable Markov Decision Process (POMDP) models. While POMDPs provide a powerful framework for handling uncertainty, utility, and sequential decision making, existing approaches require expert knowledge to manually translate a task analysis—often captured in a spreadsheet‑like SNAP (SyNdetic Assistance Process) representation—into a formal POMDP specification. This manual translation limits scalability and hampers rapid prototyping.
To overcome these limitations, the authors propose a fully automated translation pipeline based on a Probabilistic Relational Model (PRM) encoded in a relational database. The PRM captures the essential elements of a POMDP—states, actions, observations, transition probabilities, observation probabilities, and reward functions—as objects and relationships. A carefully designed database schema stores task steps, sub‑tasks, sensor inputs, possible prompts, and their probabilistic dependencies. By populating this schema through a user‑friendly web interface, non‑expert designers can describe a new assistance task without any knowledge of POMDP mathematics.
The core of the contribution is an algorithm that reads the schema metadata, constructs the state space, enumerates actions, and automatically computes transition and observation matrices using Bayesian estimation from the relational data. Rewards are derived from weighted combinations of task‑completion success, user fatigue, and prompt cost. The output is a standard POMDP file that can be fed directly into existing solvers (e.g., SARSOP, APPL).
The authors validate the approach with three everyday assistance tasks: hand‑washing, medication administration, and meal preparation. Five non‑expert participants entered task specifications into the database. The automatically generated POMDP models were compared against manually crafted models created by domain experts. Simulation‑based case studies showed that the policies derived from the auto‑generated models selected prompting actions at comparable time points, achieved similar success rates (above 85 %), and demonstrated robust error‑recovery behavior. A real‑world experiment with a human actor further confirmed that the system could detect deviations, issue context‑sensitive prompts, and reduce overall task completion time by roughly 20 % relative to a baseline without assistance.
The study demonstrates that a relational PRM can bridge the gap between high‑level task elicitation and low‑level probabilistic decision models, dramatically lowering the expertise barrier for building POMDP‑based assistive systems. The authors discuss scalability: adding new sensors or actions merely requires extending the database schema, after which the translation engine automatically incorporates the changes. Future work is outlined, including support for multi‑user collaborative tasks, integration with learning‑from‑demonstration techniques, and the development of a standardized toolkit and schema library to promote broader adoption across healthcare, rehabilitation, and smart‑home domains. In summary, the paper provides a concrete, validated methodology for automating POMDP model generation, enabling rapid, data‑driven design of personalized assistance systems for cognitively impaired users.