SENDoc Project
(Ulster University & Tyndall National Institute)
Nominated Award:
Best Application of AI in an Academic Research Body
Website of Company (or Linkedin profile of Person):
https://sendoc.interreg-npa.eu/

The Smart sENsor Devices fOr rehabilitation and Connected health (SENDoc) project, funded by EU Interreg Northern Periphery and Artic (NPA) Programme, was developed based on common territorial challenges that exist across the different countries involved in the programme. The main aim of SENDoc was to identify technical solutions to problems in the healthcare sector in sparsely populated areas. In particular, the project aimed to apply connected health concepts through innovative wearable sensor technology to improve the quality-of-life of aging communities in northern rural remote areas. The project is comprised of four research institutions from UK, Finland, Sweden, and Ireland. The institutions from Sweden and Finland provided healthcare expertise and are outside of the scope of this award. The remaining two technological institutions, Ulster University and Tyndall National Institute, provided technical expertise and developed several AI-based solutions in the project and are the focus of this nomination.
The Ulster University team is comprised of researchers from the Intelligent Systems Research Centre (ISRC). Research within the ISRC is enabled by over 20 years of experience and focus on developing fundamentally new AI algorithms in areas such as self-organising fuzzy neural networks, type-2 fuzzy logic systems, spiking neural networks, predictive modelling and analytics, computer vision, human behaviour analytics, evolutionary algorithms, accelerated computing on FPGAs and bioinspired hardware self-repair. The research conducted in the ISRC focuses on creating models and technologies for complex issues that face people and society. To accomplish this, a variety of research strategies and applications are used including big data and machine learning (ML), brain imaging and neural interfacing, human-computer interaction and robotics. The overall SENDoc project was led by ISRC member Prof. Joan Condell while the AI and analytics stream of the project was led by ISRC member Dr. Daniel Kelly.
Tyndall National Institute is one of Europe’s leading research centres in integrated ICT hardware and systems, and the largest facility of its type in Ireland. Tyndall’s strategy is to focus on addressing the world’s major societal challenges including energy and climate change, clean water, healthcare, disease prevention, and gender equality using deep-tech R&D. The Human-Centric System Cluster in Tyndall, in particular, has the vision to enable sustainable well-being for healthy populations by developing next-generation human-centric wearable systems by leveraging the core technology platform capabilities available in body-centric communication, human-computer interaction, flexible electronics, and AI. The technological advances to build such wearables, the unique role of underlying AI, and how these can transform our everyday lives are the main research focus of Dr. Salvatore Tedesco’s Wearable-AI team.
The collaboration between the teams from Ulster University and Tyndall National Institute enabled critical elements from sensor hardware design, signal processing, ML and system usability to be considered when developing novel AI based remote healthcare solutions for problems faced by aging rural communities. The team collaborated and shared expertise closely. AI researchers from both institutions worked together to solve key problems which were identified by clinicians, physiotherapists and other healthcare professionals working within the overall SENDoc project over 4 years.
Reason for Nomination:
Advancements in healthcare have resulted in longer life expectancy. This ageing population, accompanied by an increasing number of the elderly becoming physically inactive, is placing an additional burden upon healthcare systems and directing research towards early detection or prevention of future medical issues. Remote digital health technology presents a potential solution to relieve strain on healthcare systems, reducing long-term care costs whilst improving patient outcomes and quality-of-life for older adults.
A critical factor that sets the SENDoc project apart from previous remote monitoring research is that it prioritized feasibility, usability, and acceptability of remote monitoring technology deployed in real-world conditions. The project developed several AI-based technologies to provide older adults and clinicians with novel insights/predictions about health conditions. These predictions can enable more timely and personalised interventions and ultimately prevent health conditions deteriorating. Three types of health predictions were developed: 1) Health Status Prediction, 2) Fall Risk Prediction, and 3) Mortality Prediction.
To increase the likelihood that any developed technology could be deployed in real-world conditions, solutions needed to be usable and feasible from a cost and technology deployment point-of-view as well as being acceptable by older adults. All solutions were therefore based on a single wearable sensor. This reduces the cost while also limiting invasiveness. As a consequence, the quantity and quality of data recorded about the user was also reduced. With the sensor being deployed in uncontrolled real-world conditions, sensor data were also likely to include significant noise levels. Thus, a critical technical challenge was developing AI techniques that could accurately extract insights from these limited and noisy sensor signals. Descriptions of the three individual elements of the project are described below:
Health Status Prediction: This work aims to predict the subjects’ health status as defined by the standardized SF-36 questionnaire. SF -36 is a commonly used tool in healthcare to measure health status. Self-reported questionnaires have a number of disadvantages due to their subjective nature and due to their inability to continuously measure health status over time. The developed technology continuously measures health status allowing healthcare providers to evaluate potential changes in health status over time and enabling more personalised and timely interventions.The SENDoc project harnessed the power of smartphones and AI to act as a remote health-status sensor. The developed technology uses GPS, accelerometer and microphone sensors embedded in a smartphone to measure human behaviour and generate behaviour profiles which can be used to make objective predictions related to health status. Motion sensors are utilised to measure physical activity, location sensors are utilised to measure travel behaviour, and sound sensors are used to measure voice activity related behaviour. Sensor fusion, through a genetic algorithm, was performed to find complementary and co-operative features. Using a behaviour measurement composed of motion, sound, and location data, results showed that a Support Vector Machine (SVM) can predict ten different health metrics with an error that did not exceed a clinical error benchmark. Evaluations of the technology, using 186 participants, showed that utilising all three sensors produced the overall best prediction performance with an average mean absolute error and correlation of 10.9% and 0.771, respectively. Taking the minimum clinically important difference into account, error rates were below the suggested benchmark of half a standard deviation and therefore deemed accurate enough to detect clinically significant changes in health status.
Fall Risk Prediction: Globally, falls are one of the costliest population health issues in terms of reduced quality-of-life and economic burden on healthcare service. Screening of older adults for fall risks can allow for earlier interventions and ultimately lead to better outcomes and reduced public health spending. The SENDoc project developed a novel solution to limitations in existing fall screening techniques by utilizing a hip-based accelerometer worn at-home. The technology used AI techniques to extract novel fall risk features from periods of free-living ambulatory activity. Analysis of the techniques were conducted and compared with existing screening methods using functional tests and lab- based gait analysis. The technology was evaluated using data obtained from 1705 older adults. Data consisted of 1 week of hip-worn accelerometer data, gait measurements, and performance metrics for three functional tests. Retrospective and prospective fall data were also recorded based on the incidence of falls occurring 12 months before and after the study commencing respectively. ML techniques, using novel accelerometer-based measures, performed best when predicting falls compared to more traditional lab-based measures. Prospective falls had a sensitivity and specificity of 0.61 and 0.66, respectively, while retrospective falls had 0.61 and 0.68, respectively.
Mortality Prediction: The application of proper prognostic indices in clinical decisions regarding mortality prediction has assumed a significant importance for personalized risk management. This requires identifying patients who are at high or low risk of death to help ensure effective healthcare services are given to at risk patients. The project developed ML models for all-cause mortality prediction in a cohort of healthy older adults. The models are based on features based on signals from a single wearable sensor device in free-living conditions as well as anthropometric variables, physical and lab examinations, questionnaires, and lifestyles collected from 2291 recruited participants.
A novel ensemble model for was developed. Results (AUC-ROC: 0.88) were in line with previous investigations. Accelerometer-related metrics, with demographics, may predict all-cause mortality in older adults with sufficient performance (AUC-ROC: 0.763) and cancer-related mortality with even higher performance (AUC-ROC: 0.857). Results showed that ML models could provide comparable results to standard epidemiological models while being completely data-driven and disease-agnostic, and prove that wearables may have a role in mortality prediction modelling in older adults.
Overall Impact: After testing the developed technologies in real-world uncontrolled conditions, the main conclusion was that wearable sensor systems and the developed AI, were able to make predictions about health risk with accuracy levels that either matched or exceeded traditional infrequent and costly measures. The implications of this could have major impacts in the way health is managed. Rather that fitness trackers just providing feedback on activity metrics, inexpensive and readily available wearable sensors could be deployed to give feedback on health risk.
Additional Information:
Two key data-sets were used during the development of AI solutions for the three aforementioned Health Status, Fall Risk, and Mortality Risk problems.
Health-U App Dataset:
A custom Android App was developed by Ulster University team and published on Google Play (https://play.google.com/store/apps/details?id=com.csri.ami.health _u). The App comprises four main components: 1) Sensor Recording, 2) Data Processing, 3) Data Communication, 4) User Activity Feedback User Interface.
The Sensor Recording component recorded signals from GPS, inertial and microphone sensors. The data processing component performed novel signal processing and feature extraction on the different signals. A novel location entropy feature , developed by Dr. Kelly, is used to describe user location behaviour and the randomness/predictability of their travel. Location entropy features are computed using probability distributions of several different geographical areas of interest for each user. Temporal transitions of the user between these geographical areas are modelled using a Markov process and travel entropy is calculated. Sound processing algorithms are used to perform voice activity detection. A Support Vector Machine is used to classify sound samples as voice, music or ambient. Profiles are then built up of voice activity to computer a proxy measure for user social activity. Finally, inertial data are processed to build patterns of physical activity . All features are then uploaded to an AWS cloud storage system where it was further analysed. Users were also asked to complete an in-App health status questionnaire and these measures are used as the dependent variables for ML prediction. The app also features a user-friendly UI to provide feedback to users on data collected and to increase the likelihood of continued app usage. Over 5000 users downloaded the app with a total of 620,000 hours of sensor data uploaded.
Health Aging Initiative Dataset:
The “Healthy Ageing Initiative” (HAI) dataset was collected by SENDoc partners in Umeå, Sweden. HAI is an ongoing primary prevention study conducted at a single clinic with the aim of identifying traditional and potentially new risk factors for cardiovascular disorders, falls, and mortality among 70-year-olds in Umeå. The data collection involved a 3-hour health examination for each participant, who was then asked to wear an ActiGraph GT3X+ accelerometer device on the hip for one week. Future status of all participants was monitored after the initial data collection period. For example, incidents of falls were monitored used 6 and 12 month follow up phone calls.
Mortality was tracked using population registers in order to know which patients passed away in the time between their data collection and the end of the study date. The overall dataset consisted of 2291 recruited participants with 156 parameters plus raw accelerometer data for each participant. Raw accelerometer data was on average 910mb per participant, totalling (2291 x 910mb) 2TB. This was a significant challenge when evaluating techniques to extract features from accelerometer signals. To overcome the big-data challenges, the project utilized the Tier 2 High Performance Computing resources provided by the Northern Ireland High Performance Computing (NI-HPC) facility.