A systematic review of smartphone-based human activity recognition for health research

1

A systematic review of smartphone-based human activity recognition for health research

Marcin Straczkiewicz1*, Peter James2,3, Jukka-Pekka Onnela4

1 Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA; mstraczkiewicz@hsph.harvard.edu 2 Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA 02215, USA; pjames@hsph.harvard.edu 3 Department of Environmental Health, Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA; pjames@hsph.harvard.edu 4 Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA; onnela@hsph.harvard.edu

• Corresponding author Abstract Background: Smartphones are now nearly ubiquitous; their numerous built-in sensors enable continuous measurement of activities of daily living, making them especially well-suited for health research. Researchers have proposed various human activity recognition (HAR) systems aimed at translating measurements from smartphones into various types of physical activity. In this review, we summarize the existing approaches to smartphone-based HAR. Methods: We systematically searched Scopus, PubMed, and Web of Science for peer-reviewed articles published up to December 2020 on the use of smartphones for HAR. We extracted information on smartphone body location, sensors, and physical activity types studied and the data transformation techniques and classification schemes used for activity recognition. HUMAN ACTIVITY RECOGNITION FOR HEALTH RESEARCH

2

Results: We identified 108 articles and described the various approaches used for data acquisition, data preprocessing, feature extraction, and activity classification, identifying the most common practices and their alternatives.
Conclusions: Smartphones are well-suited for HAR research in the health sciences. For population-level impact, future studies should focus on improving quality of collected data, address missing data, incorporate more diverse participants and activities, relax requirements about phone placement, provide more complete documentation on study participants, and share the source code of the implemented methods and algorithms.

1. Introduction Progress in science has always been driven by data. More than 5 billion mobile devices were in use in 2020 1, with multiple sensors (e.g., accelerometer and GPS) that can capture detailed, continuous, and objective measurements on various aspects of our lives, including physical activity. Such proliferation in worldwide smartphone adoption presents unprecedented opportunities for the collection of data to study human behavior and health. Along with sufficient storage, powerful processors, and wireless transmission, smartphones can collect a tremendous amount of data on large cohorts of individuals over extended time periods without additional hardware or instrumentation.
   Smartphones are promising data collection instruments for objective and reproducible quantification of traditional and emerging risk factors for human populations. Behavioral risk factors, including but not limited to sedentary behavior, sleep, and physical activity, can all be monitored by smartphones in free- living environments, leveraging the personal or lived experiences of individuals. Importantly, unlike some wearable activity trackers 2, smartphones are not a niche product but instead have become globally available, increasingly adopted by users of all ages both in advanced and emerging economies 3,4. Their adoption in health research is further supported by encouraging findings made with other portable devices, primarily wearable accelerometers, which have demonstrated robust associations between physical activity and health outcomes, including obesity, diabetes, various cardiovascular diseases, mental health, HUMAN ACTIVITY RECOGNITION FOR HEALTH RESEARCH

3

and mortality 5–9. However, there are some important limitations to using wearables for studying population health: (1) their ownership is much lower than that of smartphones 10; (2) most people stop using their wearables after 6 months of use 11; and (3) raw data is usually not available from wearable devices. The last point often forces investigators to rely on proprietary device metrics, which lowers the already low rate of reproducibility of biomedical research in general 12, and makes uncertainty quantification in the measurements nearly impossible. Human activity recognition (HAR) is a process aimed at classification of human actions in a given period of time based on discrete measurements (acceleration, rotation speed, geographical coordinates, etc.) made by personal digital devices. In recent years, this topic has been proliferating within the machine learning research community; at the time of writing, over 400 articles had been published on HAR methods u

…(Full text truncated)…