About the project

Objective
The main aim of this project is to increase the diagnostic power of PET images for detecting lung cancer lesions at an early stage by overcoming the loss of contrast and spatial resolution caused by respiratory motion during data acquisition. Traditional ways of tackling this problem are computationally too demanding to be useful in clinical practice. For this reason, we will implement algorithms deploying a mixture of modelling of the data-acquisition set-up and machine-learning-based tools for image registration. The challenges of this project consist mainly in tackling the size of this four-dimensional image reconstruction problem and in being able to deliver images to radiologists in a time frame compatible with the hospital workflow.
In collaboration with our clinical partners at Karolinska Hospital in Huddinge, we will collect gated PET line-of-response activity data and corresponding 3D CT attenuation maps of the thorax region of 400 patients and test and optimise our 4D algorithms and gating strategies on this data. We will evaluate the resulting reconstructions with a particular focus on their diagnostic power for small lung lesions and make the reconstruction software package openly available.

Background
PET (Positron Emission Tomography) is a medical imaging modality that reconstructs the 3D distribution of metabolic activity by detecting photons emitted during the in vivo annihilation of free electrons with positrons from an injected radioactive tracer. In principle, cancer lesions will be visible in the reconstructed image with high contrast compared to the surrounding healthy tissue thanks to their peculiar metabolic fingerprints (e.g. higher sugar metabolism). PET is, in fact, one of the most powerful imaging modalities for cancer diagnosis and staging. However, the long acquisition time required to collect projection data with an acceptable noise level leads to motion artefacts that strongly affect the contrast-to-noise-ratio of lesions. This is of particular interest when trying to detect tumours that are of a size comparable to the system resolution (~5 mm) and that are continuously moving because of respiratory motion. Those lesions are the most important ones to detect for early-stage diagnosis, leading to a better prognosis for the patient.

Crossdisciplinary collaboration
This is a project in which state-of-the-art mathematical research for solving large inverse problems meets the clinical practice of medical imaging and brings together faculty from the Department of Mathematics at the SCI school at KTH with faculty from the Department of Biomedical Imaging of the CBH school.

Watch the recorded presentation at the Digitalize in Stockholm 2023 event:

About the project

Objective
The project objectives:

Background
Micromobility refers to lightweight, typically electric, transportation modes like bicycles, e-scooters, e-bikes, and similar small vehicles. To simulate riding experience in micromobility, researchers typically design and construct indoor simulators to advance research, development, and understanding of the behaviour of micromobility riders.

While many bicycle simulators have been previously developed and investigated in terms of safety, realism, and motion sickness, other micromobility simulators still need to be explored. Particularly, four new types of micromobility are slowly growing within urban spaces: e-scooters, segways, electric unicycles, and one-wheeled skateboards. This opens new research opportunities focused on improving the safety of riders and understanding their behaviour in urban environments. This demonstrator project is focused on addressing how a VR micromobilty simulator can be designed to accommodate the current and future micromobility vehicle innovations and what the implications are for research and innovation in the domain of VR simulators.

To investigate this research question and understand riders’ behaviour on four types of micromobility vehicles safely, one indoor stationary Virtual Reality Micromobility Simulator (MicroVRide) will be designed and built. Different riding modes concerning four types of vehicles will be accommodated under safe and controlled conditions by this simulator. The design and construction of this simulator will involve the utilization of two main elements: (1) a fixed hardware platform (with a dismountable handlebar) with tracking of body weight distribution and (2) Virtual Reality (VR) simulation (presented in a wireless VR headset) to experience a virtual world. Following the construction of MicroVRide, experiences with users of different levels of riding proficiency will be investigated to understand their riding behaviour, performance, and experience.

The development of MicroVRide is motivated by a commitment to advancing VR simulator technology, establishing a secure and realistic environment for safe vehicle practice.

Crossdisciplinary collaboration
The researchers in the team represent the KTH School of Electrical Engineering and Computer Science and RISE Research Institutes of Sweden, Digital Systems Division.

About the project

Objective
The project aims to model and predict the dynamic urban road traffic noise by integrating the data-driven microscopic traffic simulation and instantaneous noise emission and propagation models. It will use passive traffic and publicly available built environment data to demonstrate a high-fidelity dynamic traffic noise simulator with geographic information system (GIS) tools to predict and visualize noise levels at a time scale and geographic granularity previously unattained.

The DIRAC model and tool will pave the way for complex, dynamic urban road traffic noise modeling using passive traffic data, bridge the digital and physical urban world, and support ongoing efforts and collaborative decision-making of noise mitigation measures for a livable and healthy city, particularly for the growing demand of urban mobility for both people and goods in cities.

Background
Noise pollution is increasingly considered a major environmental issue in urban areas, with rapid urbanization (projected to reach 68% of the world’s population by 2050) in the context of the growing demand for mobility for people and goods in cities. It is recognized as a major cause of public health concerns, e.g., annoyance, sleep disturbance, other health effects (e.g., depression, anxiety, and mood swings), and decreased productivity. In particular, road traffic is deemed the major noise source in urban areas. Some 125 million people in the EU (32% of the total population) are estimated to be exposed to harmful traffic noise levels.

Several initiatives have, therefore, followed the European Noise Directive, mandating the development of urban noise maps. However, strategic noise maps exhibit strong limitations in view of noise exposure mitigation measures: long-time (yearly) averages based on traffic flow, static representations, source-rather than receiver-centric, and non-representative of transitioning vehicle fleet. They are limited in modeling and predicting fluctuations of noise levels over time under planning and management interventions, yet they are a fundamental tool to address specific dimensions of human health effects.

Built on the multidisciplinary team’s expertise and project in traffic simulation and traffic noise modeling, the DIRAC project aims to develop and demonstrate a high-fidelity road traffic noise simulation model in urban areas empowered by ubiquitous passive traffic and open-source data and Digital Twin models. For this, data-driven models work in parallel with real-life measurements to reproduce findings and predict the results of response actions. The models are agent-based (ABM) and open-source, enabling city stakeholders to recognize their roles and management models for informative decision-making of noise mitigation measures toward more livable and healthy cities.

Crossdisciplinary collaboration
The researchers in the team represent the KTH School of Architecture and Built Environment (ABE), Civil and Architectural Engineering Department, Transport Planning Division and KTH School of Engineering Science (SCI), Engineering Mechanics Department, The Marcus Wallenberg Laboratory for Sound and Vibration Research. The project is supported by strategic research partners at the KTH Center of Traffic Research (CTR), VTI (Swedish National Road and Transport Research Institute), Linköping University, and the University of Tartu.

Watch the recorded presentation at the Digitalize in Stockholm 2023 event:

About the project

“Advancing real-time exoskeleton control for human-in-the-loop optimization” is a continuation of the completed project Real-time exoskeleton control for human-in-the-loop optimization.

Objective
Our proposal is to build a physical prototype of a modular lower-limb exoskeleton system with a digital interface to a real-time variable controller. The prototype will be equipped with motors that provide variable control to different joints and joint ranges of motion while being capable of supporting real-time control of its kinetic properties. By varying the assistive strategies in the exoskeleton system via a digital interface, we will enable human-in-the-loop (HILO) optimization in order to find optimal control strategies for different users and different goals.

Background
Persons with physical disabilities are the largest minority group in the world, and global trends in ageing populations indicate an expected increase in the population affected by disability. In Sweden, musculoskeletal disorders are one of the most common causes of long-term disability.

Wearable robotic assistive exoskeletons have undergone rapid developments in the past decades, yet only a handful of products are used frequently, either within or outside of research environments. A major reason for this is that a device must be adjusted for user compliance and efficacy for optimal effect and comfort. Using methods for automatically discovering, customizing, and continuously adapting assistance could overcome these challenges, allowing exoskeletons and prostheses to achieve their potential.

Crossdisciplinary collaboration
The researchers in the team represent the KTH School of Engineering Science, Department of Engineering Mechanics and KTH School of Industrial engineering and management, department of Machine Design.

Watch the recorded presentation at Digitalize in Stockholm 2022 event:

About the project

Objective
The main goal of the dBrain collaborative project is to develop an interdisciplinary approach to improve early diagnosis and prognosis for brain diseases and to facilitate the development of therapies by combining computational modelling of the brain and advanced artificial intelligence (AI) based data analyses.
The computational approaches will initially focus on Parkinson’s (PD) and Alzheimer’s diseases (AD), the two most common neurodegenerative diseases. These disease cases are strategically selected in collaboration with the Karolinska Hospital (KS) and Karolinska Institute (KI).

This research is important because the incidence of brain diseases will only increase with increasing human longevity. It is estimated that by 2030, mental disorders will be the largest contributor to the total disease burden.

Background
With increased longevity, the number of patients with brain-related diseases will grow. In 2030, mental illness is estimated to be the main imposition on health care. New advances in data analysis and computational modelling of the brain can use the potential of patient data to prevent and treat brain diseases.

Cross-disciplinary collaboration
The research team represents the School of Electrical Engineering and Computer Science (EECS), the School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH) and the School of Engineering Sciences (SCI).

Watch the recorded presentation at the Digitalize in Stockholm 2023 event:

Activities & Results

Find out what’s going on!

Activities, awards, and other outputs

Results

dBrain addresses two major challenges in our understanding of the brain in health and disease: a) what is the relationship between the structural connectivity and functional interactions between different brain regions and within local neuronal networks?; and b) how do disease-related slow changes in the chemical balance, gene expression and neuronal degeneration shape the brain activity dynamics? We address these challenges via four work packages (WPs), using digital approaches comprising AI-based data analysis, math, computational modelling and simulations.

With the help of Digital Futures, we created a vibrant environment involving three KTH schools and collaborators at KS and KI. In addition to regular weekly meetings at the Digital Futures for the whole consortium (i.e. PIs, the recruited postdocs, and other students), several subgroups also meet more frequently in one of the labs or via zoom. Also, several Master’s Student projects have been completed in addition to the work of PhD students and postdocs. In the weekly meetings, postdocs have particular roles – regularly presenting updates on their work and journal articles selected for general interest. These activities provide the postdocs with training in presenting their work and leading discussions. The postdocs also have arranged team-building activities.

Publications

We like to inspire and share interesting knowledge!

Publications

  1. Sanna Persson, Rodrigo Moreno, Bounding tractogram redundancy, Frontiers in Neuroscience. 18:1403804, 2024, https://doi.org/10.3389/fnins.2024.1403804
  2. Malin Siegbahn, Daniel Jörgens, Filip Asp, Malou Hultcrantz, Rodrigo Moreno, Cecilia Engmér Berglin. Asymmetry in cortical thickness of the Heschl’s gyrus in unilateral ear canal atresia. Otology & Neurotology. 45(4),e342-e350, 2024. https://doi.org/10.1097/MAO.0000000000004137
  3. Caroline Dartora, Anna Marseglia, Gustav Mårtensson, Gull Rukh, Junhua Dang, J- Sebastian Muehlboeck, Lars-Olof Wahlund, Rodrigo Moreno, José Barroso, Daniel Ferreira, Helgi Schiöth, Eric Westman. A deep learning model for brain age prediction using minimally preprocessed T1w-images as input. Frontiers in Aging Neuroscience 15, 1303036, 2024. https://doi.org/10.3389/fnagi.2023.1303036
  4. Antonia Hain, Daniel Jörgens, Rodrigo Moreno. Randomized iterative spherical-deconvolution informed tractogram filtering. Neuroimage 278, 120248, 2023. https://doi.org/10.1016/j.neuroimage.2023.120248
  5. Jingru Fu, Antonios Tzortzakakis, José Barroso, Eric Westman, Daniel Ferreira, Rodrigo Moreno. Fast three-dimensional image generation for healthy brain aging using diffeomorphic registration. Human Brain Mapping 44 (4), 1289-1308, 2023. https://doi.org/10.1002/hbm.26165
  6. Kajsa Ahlgren, Christoffer Olsson, Inna Ermilova, Jan Swenson. New insights into the protein stabilizing effects of trehalose by comparing with sucrose. Physical Chemistry Chemical Physics 25 (32), 21215-21226, 2023. https://doi.org/10.1039/D3CP02639F
  7. Zhou Zhou, Chirstoffer Olsson, T Christian Gasser, Xiaogai Li, Svein Kleiven. The White Matter Fiber Tract Deforms Most in the Perpendicular Direction During In Vivo Volunteer Impacts. J Neurotrauma. 2024 41(23-24):2554-2570, 2024. https://doi.org/10.1089/neu.2024.0183
  8. Jingru Fu, Daniel Ferreira, Örjan Smedby, Rodrigo Moreno. A deformation-based morphometry framework for disentangling Alzheimer’s disease from normal aging using learned normal aging templates. arXiv.2311.08176, 2023. https://doi.org/10.48550/arXiv.2311.08176
  9. Daniel Jörgens, Pierre Marc Jodoin, Maxime Descoteaux, Rodrigo Moreno. Merging multiple input descriptors and supervisors in a deep neural network for tractogram filtering. arXiv:2307.05786, 2023. https://doi.org/10.48550/arXiv.2307.05786
  10. Jingru Fu, Simone Bendazzoli, Örjan Smedby, Rodrigo Moreno. Unsupervised Domain Adaptation for Pediatric Brain Tumor Segmentation. MICCAI 2024 Workshop on Advancing Data Solutions in Medical Imaging AI (ADSMI’24) (in press). https://doi.org/10.48550/arXiv.2406.16848
  11. Sanna Persson, Xinyi Wan, and Rodrigo Moreno. Randomly COMMITting: Iterative Convex Optimization for Microstructure-Informed Tractography. MICCAI 2024 Workshop on Computational Diffusion MRI (CDMRI’24) (in press)
  12. Yu Zhou, Jingru Fu, Örjan Smedby, Rodrigo Moreno. Synthesis of Pediatric Brain Tumor Images with Mass Effect. SPIE – Medical Imaging: Image Processing 12464, 699-707, 2023. San Diego, USA, February 2023. https://doi.org/10.1117/12.2654366
  13. I Carannante, Y Johansson, G Silberberg, J Hellgren Kotaleski. Data-Driven Model of Postsynaptic Currents Mediated by NMDA or AMPA Receptors in Striatal Neurons. Front Comput Neurosci. 2022; 16: 806086.
  14. J Fu, A Tzortzakakis, J Barroso, E Westman, D Ferreira, R Moreno. Generative Aging of Brain Images with Diffeomorphic Registration. arXiv preprint. 2022; arXiv:2205.15607. https://doi.org/10.48550/arXiv.2205.15607
  15. A Hain, D Jörgens, R Moreno. Assessing Streamline Plausibility Through Randomized Iterative Spherical-Deconvolution Informed Tractogram Filtering. arXiv preprint 2022; arXiv:2205.04843. https://doi.org/10.48550/arXiv.2205.04843
  16. M Siegbahn, C Engmér Berglin, R Moreno. Automatic segmentation of the core of the acoustic radiation in humans. Front Neurol. 2022 : 934650. https://doi.org/10.3389/fneur.2022.934650
  17. C Dartora, A Marseglia, G Mårtensson, G Rukh, J Dang, JS Muehlboeck, LO Wahlund, R Moreno, J Barroso, D Ferreira, HB Schiöth, E Westman. Predicting the Age of the Brain with Minimally Processed T1-weighted MRI Data. medRxiv preprint. 2022; medRxiv:2022.09.06.22279594. https://doi.org/10.1101/2022.09.06.22279594
  18. Chakravarty K, Roy S, Sinha A, Nambu A, Chiken S, Hellgren Kotaleski J, Kumar A. Transient Response of Basal Ganglia Network in Healthy and Low-Dopamine State. eNeuro. 2022 Mar 18;9(2):ENEURO.0376-21.2022. doi: 10.1523/ENEURO.0376-21.2022.
  19. Hjorth, J.J.J., Hellgren Kotaleski, J. & Kozlov, A. Predicting Synaptic Connectivity for Large-Scale Microcircuit Simulations Using Snudda. Neuroinform 19, 685–701 (2021). https://doi.org/10.1007/s12021-021-09531-w
  20. G Colombo, R Cuber, L Kanari, A Venturino, R Schulz, M Scolamiero, J Agerberg, H Mathys, L Tsai, W Chachólski,, K Hess, S Siegert. Microglial morphOMICs, a tool for mapping microglial morphology, reveals brain-region-and sex-dependent phenotypes. Nature neuroscience, 2022.
  21. W Chachólski, A Guidolin, I Ren, M Scolamiero, F Tombari. Effective computation of relative homological invariants for functors over posets. arXiv preprint 2022; arXiv:2209.05923
  22. W Chachólski, A Jin, F Tombari. Realisations of posets and tameness. arXiv preprint 2022; arXiv:2112.12209
  23. W Chachólski, René Corbet, Anna-Laura Sattelberger. The Shift-Dimension of Multipersistence Modules. arXiv preprint 2022; arXiv:2112.06509.
  24. W Chachólski, B Gunti, C Landi, Decomposing filtered chain complexes: Geometry behind barcoding algorithms. Computational Geometry, Volume 109, 2023.
  25. Georgios Moschovis. NeuralDynamicsLab at ImageCLEF Medical 2022. ImageCLEF, 2022.
  26. Helson Pascal, Lundqvist Daniel, Svenningsson Per, Vinding Mikkel, Kumar Arvind (2023). Cortex-wide topography of 1/f-exponent in Parkinson’s disease. Nature Parkinson’s Disease, 9, Article number: 109.
  27. Zang, Jie, Liu Shenquan, Helson Pascal, Kumar Arvind (2024). Structural constraints on the emergence of oscillations in multi-population neural networks. eLIFE, 12(Feb 5).
  28. Wärnberg Emil, Kumar Arvind (2023). Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia. PNAS 32(120):e2221994120

Master School Theses

  1. Yuqi Zheng. Predictive MR Image Generation for Alzheimer’s Disease and Normal Aging Using Diffeomorphic Registration. MS thesis KTH 2023. Supervisors Rodrigo Moreno, Jingru Fu. Diva diva2:1833589
  2. Xinyi Wan. Assessing the Streamline Plausibility Through Convex Optimization for Microstructure Informed Tractography(COMMIT) with Deep Learning. MS thesis KTH 2023. Supervisor Rodrigo Moreno. Diva diva2:1754277
  3. Teodor Pstrusiński. Impact of the autoencoder-based FINTA tractogram filtering method on brain networks in subjects with Mild Cognitive Impairment. MS thesis KTH 2023. Supervisors Rodrigo Moreno, Fabian Sinzinger. Diva diva2:1799683
  4. Erika Bengtsdotter. Towards Anatomically Plausible Streamline Tractography with Deep Reinforcement Learning. MS thesis KTH 2022. Supervisors Rodrigo Moreno, Fabian Sinzinger. Diva diva2:1695872
  5. Filippo Maschio. Influence of tractogram filtering in the analysis of tractography data in rat brains. MS thesis University of Padua, Italy 2022. Supervisor Rodrigo Moreno.
  6. Yu Zhou. Synthesis of Pediatric Brain Tumor Image With Mass Effect. MS thesis KTH 2022. Supervisors Rodrigo Moreno, Jingru Fu. Diva diva2:1686799
  7. Antonia Hain. Assessing individual streamline plausibility through randomized spherical deconvolution-informed tractogram filtering. MS thesis University of Saarland, Germany 2021. Supervisor Rodrigo Moreno.
  8. N Hulst, Exploring persistent homology as a method for capturing functional connectivity differences in Parkinson’s Disease. Master Thesis. DiVA, id: diva2:1687257.
  9. Marvin Köpff. Impact of tractogram filtering and graph creation for structural connectomics in subjects with mild cognitive impairment. MS thesis KTH 2020. Supervisors Rodrigo Moreno, Joana Pereira. Diva: diva2:1463915
  10. Johannes Wennberg. Longitudinal assessment of functional connectivity impairment in rat brains. MS thesis KTH 2019. Supervisor Rodrigo Moreno, diva2:1327919
  11. Cristina Zanin. Graph Convolutional Networks for static and dynamic functional connectivity: an application to the Autism Spectrum Disorder. MS thesis University of Padua, Italy 2020. Supervisor Rodrigo Moreno.
  12. Lucas Höglund. Analysis of Eye Tracking Data from Parkinson’s Patients using Machine Learning. MS thesis, KTH.
  13. Leo Bergman. Feature extraction with self-supervised learning on eye-tracking data from Parkinson’s patients and healthy individuals. MS thesis, KTH.
  14. Emma Lind. Analysis of Brain Signals from Patients with Parkinson’s Disease using Self-Supervised Learning. MS thesis, KTH.
  15. Wilhelm Ågren. Feature extraction from MEG data using self-supervised learning. MS thesis, KTH.
  16. Giulia Tuccio. Parameter estimation in a cardiovascular computational model using numerical optimization. MS thesis, KTH.
  17. Paolo Calderaro. Patient simulation. Generation of a machine learning “inverse” digital twin. MS thesis, KTH.

About the project

Objective
This project aims to develop adaptive social robots that can understand humans’ communicative behaviour and task-related physical actions and adapt their interaction to suit. We aim to investigate and demonstrate fluid and seamless adaptation of intelligent systems to users’ contexts, needs or preferences. To achieve fluidity, such adaptation needs to happen with minimal interruption to the users’ ongoing interaction with the system, without requiring user intervention, while providing accountability and control of the adaption in a task-appropriate, timely, and understandable manner. This will be explored in multiple embodiments: smart speakers, back-projected robotic heads, and dual-arm robots.

Our use case scenario is an adaptive intelligent kitchen assistant that helps humans prepare food and other kitchen-centric tasks, focusing on supporting ageing in place. Our systems will engage in face-to-face spoken and physical collaboration with humans, track the users’ affective states and task-related actions in real-time, adjust performance based on previous interactions, adapt to user preferences, and show intention using a self-regulation perception-production loop. The project will use the Intelligence Augmentation Lab that TMH and RPL plan to set up.

Background
Intelligent systems built around big datasets and machine learning techniques are becoming ubiquitous in people’s lives – smart appliances, wearables, and, increasingly, robots. As these systems are intended to assist an ever wider range of users in their homes, workplaces or public spaces, a typical one-fits-all approach becomes insufficient. Instead, these systems will need to take advantage of the machine learning techniques upon which they are built to adapt to the specific task, user constellation continually, and shared environment in which they are operating. In long-term deployments, the state of the environment, user preferences, skills, and abilities change and must be adapted. This is relevant for socially assistive robots in people’s homes, education or healthcare settings, and robots working alongside workers in small-scale manufacturing environments.

Crossdisciplinary collaboration
The research team represents the School of Electrical Engineering and Computer Science (EECS, KTH), the School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH, KTH) and the Department of Computer and System Science (DSV) at Stockholm University.

Watch the recorded presentation at the Digitalize in Stockholm 2023 event:

Articles

Avancerade adaptiva intelligenta system

Activities & Results

Find out what’s going on!

Results

The main results achieved in the first half of the project towards the 3 main objectives enumerated in the proposal:

  1. Understanding and Design for the Smart Kitchen

Deeper understanding of the cooking process (Kuoppamäki et al, 2021) led to a number of studies on interaction in and around cooking, including list advancement (Jaber & McMillan, 2022), content navigation (Zhao et al., 2022), designing conversational interaction (Kuoppamäki et al., 2023), the impact of contextual awareness on command construction and delivery (Jaber et al., in submission) and an ongoing study comparing proactive organisational support between young adults and those over 65 (Kuoppamäki et al.).

  1. Perception and Representation of Long Term Human Action

Research showing that thermal imaging is a modality relevant for detecting frustration in human-robot interaction (Mohamed et al., 2022). The models to predict frustration based on a dataset of 18 participants interacting with a Nao social robot in our lab were tested using features from several modalities: thermal, RGB, Electrodermal Activity (EDA), and all three combined. The models reached an accuracy of 89% with just RGB features, 87% using only thermal features, 84% using EDA, and 86% when using all modalities. We are also investigating the accuracy of frutration prediction models using data collected at the KTH Library where students ask directions to a Furhat robot.

  1. Input, Output, and Interaction for Smart Assistive Technologies

In order to fulfil the goal of adaptation of virtual agent and robot behaviour to different contexts, we have researched interlocutor-aware facial expressions in dyadic interaction (Jonell et al, 2020) as well as adaptive facial expressions in controllable generation of speech and gesture: we have developed techniques to generate conversational speech, with control over speaking style to signal e.g. certainty or uncertainty (Wang et al, 2022, Kirkland et al 2022) as well as models that are able to generate coherent speech and gesture from a common representation (Wang et al 2021). Another direction concerns adaptation to spatial contexts and environments, where we have used imitation learning and physical simulation to produce referential gestures (Deichler et al, 2022).

Publications

We like to inspire and share interesting knowledge!

  1. Parag Khanna, Mårten Björkman, Christian Smith. “Human Inspired Grip-Release Technique for Robot-Human Handovers”, 2022 IEEE-RAS International Conference on Humanoid Robots, Ginowan, Japan. (Accepted for publication)
  2. Sanna Kuoppamäki, Mikaela Hellstrand, and Donald McMillan (2023, forthcoming). Designing conversational scenarios with older adults digital repertoires: Graphic transcript as a design method. In: Hänninen, R., Taipale, S., Haapio-Kirk, L. (eds). Embedded and everyday technology: Digital repertoires in an ageing society. UCL Press.
  3. Youssef Mohamed, Giulia Ballardini, Maria Teresa Parreira, Séverin Lemaignan, and Iolanda Leite. 2022. Automatic Frustration Detection Using Thermal Imaging. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’22). IEEE Press, 451–460
  4. Razan Jaber and Donald McMillan. 2022. Cross-Modal Repair: Gaze and Speech Interaction for List Advancement. In Proceedings of the 4th Conference on Conversational User Interfaces (CUI ’22). Association for Computing Machinery, New York, NY, USA, Article 25, 1–11.
  5. Yaxi Zhao, Razan Jaber, Donald McMillan, and Cosmin Munteanu. 2022. “Rewind to the Jiggling Meat Part”: Understanding Voice Control of Instructional Videos in Everyday Tasks. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 58, 1–11.
  6. Kirkland, A., Lameris, H., Gustafson, J., Székely, É. (2022) “Where’s the uh, hesitation? The interplay between filled pause location, speech rate and fundamental frequency”, Interspeech 2022, Incheon, Korea.
  7. Wang, S., Gustafson, J., Székely, É. (2022) “Evaluating Sampling-based Filler Insertion with Spontaneous TTS”, 13th Edition of the Language Resources and Evaluation Conference (LREC 2022), Marseille.
  8. Deichler, A., Wang, S., Alexanderson, S., & Beskow, J. (2022). Towards Context-Aware Human-like Pointing Gestures with RL Motion Imitation. In Context-Awareness in Human-Robot Interaction: Approaches and Challenges, workshop at 2022 ACM/IEEE International Conference on Human-Robot Interaction.
  9. Donald McMillan and Razan Jaber. 2021. Leaving the Butler Behind: The Future of Role Reproduction in CUI. In Proceedings of the 3rd Conference on Conversational User Interfaces (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 11, 1–4.
  10. Sanna Kuoppamäki, Sylvaine Tuncer, Sara Eriksson, and Donald McMillan. 2021. Designing Kitchen Technologies for Ageing in Place: A Video Study of Older Adults’ Cooking at Home. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 2, Article 69 (June 2021), 19 pages.
  11. Katie Winkle, Gaspar Isaac Melsión, Donald McMillan, and Iolanda Leite. 2021. Boosting Robot Credibility and Challenging Gender Norms in Responding to Abusive Behaviour: A Case for Feminist Robots. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’21 Companion). Association for Computing Machinery, New York, NY, USA, 29–37.
  12. Wang, S., Alexanderson, A., Gustafson, J., Beskow, J., Henter, G. and Székely, É. (2021) “Integrated Speech and Gesture Synthesis”, 23rd ACM International Conference on Multimodal Interaction (ICMI 2021), Montreal
  13. Jonell, P., Kucherenko, T., Henter, G. E., & Beskow, J. (2020, October). Let’s Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents (pp. 1-8).

About the project

Objective
Adaptive Intelligent Homes (AIH) aims to develop a suite of demonstrators, combining the cutting-edge advances made within the parent collaborative research project (AAIS) on user state recognition, human-robot handovers, multimodal referring expression generation, controllable speech synthesis, and proactively supporting users in complex tasks. AIH will extend the context-awareness and adaptiveness of those conversational robots from a laboratory environment to a real-life environment and from individual to collaborative settings, where multiple users interact with the system simultaneously. Through engaging with societal and municipal actors, the project contributes to more inclusive and accessible robotic systems with potential applications to promote healthy lifestyles and provide home care services for older adults.

Background
AI-powered agents provide an opportunity to manage, coordinate and initiate activities within and across households, contributing to healthy lifestyles and improved quality of life for users of all ages. To achieve this goal, these robotic systems must be aligned with the user’s needs, preferences, and interests, and they must adapt themselves to complex environments involving multiple users performing overlapping and interwoven household tasks, such as cooking.

Cooking in the home is not simply instruction-giving; it is a multi-party, parallelized activity requiring timing, action relevance, and adaptability from both the human and any AI assistance. The project has envisioned an adaptive intelligent kitchen assistant that could help humans prepare food and other kitchen-centric tasks to enhance healthy ageing and improve quality of life.

Cross-disciplinary collaboration
The interdisciplinary approach will allow us to synthesize the human-centric understanding of the home and kitchen environment to the technical capabilities of dialogue systems and robotic agents. This could indicate, for instance, new modalities for personalising the intelligent kitchen assistant for different user groups or linguistic styles that could be adapted to different family members’ skill sets and preferences. An important societal aspect is to explore and design for improved inclusivity of dialogue systems by recognizing diverse ways of interacting with the system based on user’s age, gender, ethnicity, or disability and investigate the long-term impact of deployment of such systems in the home environment to the quality of life and self-efficacy among end-users. Designing for improved accessibility of dialogue systems has the potential to result in more equal access to intelligent and assistive home environments.