Through scientific research and in an international context of interdisciplinary collaboration, Casa Paganini develops new technology and interactive multimedia systems combined with Art to assist with therapies and rehabilitation.
Lugano, 15th February 2021
Casa Paganini – a research centre where Art meets Artificial Intelligence for Health
Casa Paganini, an interdisciplinary research centre, was set up in Genoa in 2005. It is the result of a framework agreement between the Liguria Region, the City and the University of Genoa, which is participating through DIBRIS (a Department of the Polytechnic School).
One of the main aims of the scientific research being conducted at Casa Paganini is the challenge of reading and interpreting the affective, emotional and social qualities of movement through the development of new technology. To be able to see the secret landscape of passions that gestures reveal, using technological tools, is an ambitious goal that opens the door to a wide range of application developments.
The centre designs interactive multimedia systems that analyse both individual movement and movement in groups. It supports the creation of experiences for therapy and rehabilitation (for the elderly or children). It also designs new interfaces and systems for the active enjoyment of cultural content in museums and science centres, as well as for experiments and the validation of research in artistic projects.
“In the past, for example, we worked with the composer Luciano Berio to create the interfaces and interactive systems for two of his works: Outis (Teatro alla Scala, Milan) and Cronaca del Luogo (Salzburg Festival). The opportunity to participate in artistic projects allows us to rethink and see the results of our scientific research through different eyes,” says Prof. Antonio Camurri, Director of the centre.
Trans-disciplinarity and cross-fertilisation
Despite being part of a polytechnic school, the centre has a workforce with a wide range of skills that are usually perceived as very distant.
“I am a software engineer, others come from cognitive science and experimental psychology backgrounds or from the world of performing arts (dance, music), sonic design, and other scientific disciplines, such as operations research and neural networks. Nicola Ferrari is a lecturer in comparative literature and has a degree in composition, like his colleague Corrado Canepa, a composer and expert in interactive systems. Paolo Coletta and Simone Ghisio are computer engineers and software architects”.
The approach is based on trans-disciplinarity, uniting scientific aspects that stem from Human Sciences, the Arts, Engineering and Cognitive Sciences, viewed from a rigorous scientific perspective with the aim to achieve concrete results.
Besides being verbal, more than 80% of communication between human beings is made up of non-verbal signals. When speaking, we use prosody, gestures, arm and body movements, postural attitudes and facial expressions.
“Our work focuses on the analysis and automatic prediction of clues related to non-verbal behaviour, expression and emotions, and to non-verbal social signals expressed through body movements: these are the keywords of our research. Our goal is to develop technology that investigates the nature of these complex phenomena”.
This research in the field of Health is aimed at detecting and analysing non-verbal signals in frail elderly people or in children with cognitive and motor problems. With the ultimate goal to support their residual abilities in forming relationships with other people, or to support rehabilitation activities.
Therapies and projects
The Centre is part of a joint laboratory with the Giannina Gaslini Children’s Hospital in Genoa, Ariel – Augmented Rehabilitation Interactive Environment/Ecological Lab (Unit 1 of the Gaslini Hospital), headed by Dr Paolo Moretti.
“In the field of physical and cognitive rehabilitation, we work together to transform the quality of gestures into sound (interactive sonification), sound into play and play into therapy. We work with physiotherapists and children, developing interactive applications. Using interaction design, gamification and interactive sonification techniques, we develop interactive systems used by physiotherapists in cognitive-motor rehabilitation, with children suffering from cancer or cerebral palsy or children who are bedridden for a long period of time, who, for example, need to exercise in order to regain muscle tone”.
Serious games support physiotherapists in their activities with children with cerebral palsy that are wheelchair-bound, helping with postural realignment and avoiding further damage to the spine, neck and skeletal structure. Through a “reward” given by movement-controlled sonification, the child’s favourite music is modified in real time to encourage them to hold their chest and head upright. The child learns to perceive the relationship between the processing of sound and their effort to stay upright. Some games, such as the one just described, HTCT – Head Trunk Control Trainer, have been approved by the ethics committee and used on the ward.
More recently, Casa Paganini developed several serious games within the ICT ASC-Inclusion European project to teach children with autism to recognise and express emotions through body language (video).
In the European Project Care Here, the centre also worked on the development of interactive multimedia technologies for autism, such as game-based interactive installations later used in two therapy centres in Barcelona and London.
Studies on cross-modality indicate the scientific grounds of the correspondence between different sensory channels, such as, for example, how a visual stimulus can be ‘translated’ into an equivalent
auditory stimulus (from a perceptual point of view). “On this front we recently coordinated a three-year European Horizon 2020 ICT project – Dance. One of the activities involved the Virgilio Sieni dance company, who we worked with to study technology on how to ‘make a blind person see dance’, and more generally on how to increase the awareness of one’s own movement to a person who is not an expert dancer”.
A pathway for the treatment of chronic pain
Together with Prof. Alberto Pilotto, senior consultant of the Geriatrics department at the Galliera Hospital in Genoa, Casa Paganini has started a programme to support the care of frail elderly people, using interactive technology and cultural content. The work is inspired by a previous project in 2016 with London’s UCL Hospital on the design of interactive systems to treat chronic pain, which often lingers after the cause has been eliminated.
“We created exercises to get people moving and distract them from their pain, through real-time sonification of movement quality. Imagine an exploration of the soundscape of a forest, where you can hear the leaves blowing in the wind and the birds, until you reach a final situation (corresponding to the correct execution of a certain movement and achieving a certain posture) that is rich and airy and stimulates curiosity, creativity and physical exercise.”
With Galliera, Casa Paganini is working on the design of artificial intelligence tools that integrate multimodal and interactive multi-sensory aspects, inspired by the visual arts and sound, aimed at elderly people at risk of loss of independence and dementia.
“Our idea is to work on the potential of art as a tool to address frailty – which is a growing problem in the elderly, from both a physical and a cognitive perspective – in both rehabilitation and prevention, in aesthetically-resonant interactive environments that support and encourage lifestyle changes, for example by stimulating motor activity.”
Art and artificial intelligence are confirmed as great allies of Health, in both prevention and the provision of care.
Written by:
-Antonio Camurri
Founder and Scientific Director of Casa Paganini and InfoMus Lab
-Catterina Seia
Member of the IBSA Foundation Advisory Board; President of the CCW Cultural Welfare Center; Co-Founder and Vice-President of the Fitzcarraldo Foundation and the “Medicina a Misura di Donna” Foundation
