CopyRobot: Interactive Mirroring Robotics Game for ASD Children

Laura Santos, Alice Geminiani, Ivana Olivieri, José Santos-Victor, Alessandra Pedrocchi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The family of disorders commonly known as autism is characterized by a deficit in social interaction and restricted repetitive and stereotyped patterns of behaviours, activities and interests. Motor disturbances are not part of the diagnosis of the children with autism but some studies have estimated that between 80 and 90% of children with Autism Spectrum Disorder (ASD) demonstrate some degree of motor impairments. Several therapies have been used for the improvement of motor skills, always leading to behavioural improvements as side-effects, demonstrating the importance of motor interaction and stimulation for the case of autism. Recent studies have shown that motor, imitation and social abilities are all related in people with autism. In this work, a humanoid robot is used to create a therapy that unites all these areas. The system involves a robot (NAO), a Kinect camera and Personal Computer, with the goal of facilitating the interaction between therapist and a child with ASD during a physical therapy session. To improve the imitation abilities of the child, the robot was programmed to mirror both the child and the therapist movements. After testing different tracking methodologies, the Kinect sensor was selected as the best compromise of quality and cost. Two protocols were developed, depending on who plays the role of the main actor. In the first protocol, the robot is the master and leads the interaction. It decides the exercise to execute and gives feedback to both the therapist and the child. In the second protocol, the choice of the exercise sequence is the therapist’s responsibility. To promote interaction further during clinical tests, the protocol was changed to include gesture imitation. For the robot master protocol, the space theme was chosen. For the therapist master protocol, the theme of sports, that was already performed by the children in the usual therapy, was adopted. The system was tested in realistic conditions with two different autistic children. The reaction was different in each case but it demonstrated the importance of these imitation games in the treatment of this disease.

Original languageEnglish
Title of host publication15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019
EditorsJorge Henriques, Paulo de Carvalho, Nuno Neves
PublisherSpringer
Pages2014-2027
Number of pages14
ISBN (Print)9783030316341
DOIs
Publication statusPublished - Jan 1 2020
Event15th Mediterranean Conference on Medical and Biological Engineering and Computing, MEDICON 2019 - Coimbra, Portugal
Duration: Sep 26 2019Sep 28 2019

Publication series

NameIFMBE Proceedings
Volume76
ISSN (Print)1680-0737
ISSN (Electronic)1433-9277

Conference

Conference15th Mediterranean Conference on Medical and Biological Engineering and Computing, MEDICON 2019
CountryPortugal
CityCoimbra
Period9/26/199/28/19

ASJC Scopus subject areas

  • Bioengineering
  • Biomedical Engineering

Cite this

Santos, L., Geminiani, A., Olivieri, I., Santos-Victor, J., & Pedrocchi, A. (2020). CopyRobot: Interactive Mirroring Robotics Game for ASD Children. In J. Henriques, P. de Carvalho, & N. Neves (Eds.), 15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019 (pp. 2014-2027). (IFMBE Proceedings; Vol. 76). Springer. https://doi.org/10.1007/978-3-030-31635-8_239

CopyRobot : Interactive Mirroring Robotics Game for ASD Children. / Santos, Laura; Geminiani, Alice; Olivieri, Ivana; Santos-Victor, José; Pedrocchi, Alessandra.

15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019. ed. / Jorge Henriques; Paulo de Carvalho; Nuno Neves. Springer, 2020. p. 2014-2027 (IFMBE Proceedings; Vol. 76).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Santos, L, Geminiani, A, Olivieri, I, Santos-Victor, J & Pedrocchi, A 2020, CopyRobot: Interactive Mirroring Robotics Game for ASD Children. in J Henriques, P de Carvalho & N Neves (eds), 15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019. IFMBE Proceedings, vol. 76, Springer, pp. 2014-2027, 15th Mediterranean Conference on Medical and Biological Engineering and Computing, MEDICON 2019, Coimbra, Portugal, 9/26/19. https://doi.org/10.1007/978-3-030-31635-8_239
Santos L, Geminiani A, Olivieri I, Santos-Victor J, Pedrocchi A. CopyRobot: Interactive Mirroring Robotics Game for ASD Children. In Henriques J, de Carvalho P, Neves N, editors, 15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019. Springer. 2020. p. 2014-2027. (IFMBE Proceedings). https://doi.org/10.1007/978-3-030-31635-8_239
Santos, Laura ; Geminiani, Alice ; Olivieri, Ivana ; Santos-Victor, José ; Pedrocchi, Alessandra. / CopyRobot : Interactive Mirroring Robotics Game for ASD Children. 15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019. editor / Jorge Henriques ; Paulo de Carvalho ; Nuno Neves. Springer, 2020. pp. 2014-2027 (IFMBE Proceedings).
@inproceedings{4aa2acc049f54840bd2ad1d04c242112,
title = "CopyRobot: Interactive Mirroring Robotics Game for ASD Children",
abstract = "The family of disorders commonly known as autism is characterized by a deficit in social interaction and restricted repetitive and stereotyped patterns of behaviours, activities and interests. Motor disturbances are not part of the diagnosis of the children with autism but some studies have estimated that between 80 and 90{\%} of children with Autism Spectrum Disorder (ASD) demonstrate some degree of motor impairments. Several therapies have been used for the improvement of motor skills, always leading to behavioural improvements as side-effects, demonstrating the importance of motor interaction and stimulation for the case of autism. Recent studies have shown that motor, imitation and social abilities are all related in people with autism. In this work, a humanoid robot is used to create a therapy that unites all these areas. The system involves a robot (NAO), a Kinect camera and Personal Computer, with the goal of facilitating the interaction between therapist and a child with ASD during a physical therapy session. To improve the imitation abilities of the child, the robot was programmed to mirror both the child and the therapist movements. After testing different tracking methodologies, the Kinect sensor was selected as the best compromise of quality and cost. Two protocols were developed, depending on who plays the role of the main actor. In the first protocol, the robot is the master and leads the interaction. It decides the exercise to execute and gives feedback to both the therapist and the child. In the second protocol, the choice of the exercise sequence is the therapist’s responsibility. To promote interaction further during clinical tests, the protocol was changed to include gesture imitation. For the robot master protocol, the space theme was chosen. For the therapist master protocol, the theme of sports, that was already performed by the children in the usual therapy, was adopted. The system was tested in realistic conditions with two different autistic children. The reaction was different in each case but it demonstrated the importance of these imitation games in the treatment of this disease.",
author = "Laura Santos and Alice Geminiani and Ivana Olivieri and Jos{\'e} Santos-Victor and Alessandra Pedrocchi",
year = "2020",
month = "1",
day = "1",
doi = "10.1007/978-3-030-31635-8_239",
language = "English",
isbn = "9783030316341",
series = "IFMBE Proceedings",
publisher = "Springer",
pages = "2014--2027",
editor = "Jorge Henriques and {de Carvalho}, Paulo and Nuno Neves",
booktitle = "15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019",

}

TY - GEN

T1 - CopyRobot

T2 - Interactive Mirroring Robotics Game for ASD Children

AU - Santos, Laura

AU - Geminiani, Alice

AU - Olivieri, Ivana

AU - Santos-Victor, José

AU - Pedrocchi, Alessandra

PY - 2020/1/1

Y1 - 2020/1/1

N2 - The family of disorders commonly known as autism is characterized by a deficit in social interaction and restricted repetitive and stereotyped patterns of behaviours, activities and interests. Motor disturbances are not part of the diagnosis of the children with autism but some studies have estimated that between 80 and 90% of children with Autism Spectrum Disorder (ASD) demonstrate some degree of motor impairments. Several therapies have been used for the improvement of motor skills, always leading to behavioural improvements as side-effects, demonstrating the importance of motor interaction and stimulation for the case of autism. Recent studies have shown that motor, imitation and social abilities are all related in people with autism. In this work, a humanoid robot is used to create a therapy that unites all these areas. The system involves a robot (NAO), a Kinect camera and Personal Computer, with the goal of facilitating the interaction between therapist and a child with ASD during a physical therapy session. To improve the imitation abilities of the child, the robot was programmed to mirror both the child and the therapist movements. After testing different tracking methodologies, the Kinect sensor was selected as the best compromise of quality and cost. Two protocols were developed, depending on who plays the role of the main actor. In the first protocol, the robot is the master and leads the interaction. It decides the exercise to execute and gives feedback to both the therapist and the child. In the second protocol, the choice of the exercise sequence is the therapist’s responsibility. To promote interaction further during clinical tests, the protocol was changed to include gesture imitation. For the robot master protocol, the space theme was chosen. For the therapist master protocol, the theme of sports, that was already performed by the children in the usual therapy, was adopted. The system was tested in realistic conditions with two different autistic children. The reaction was different in each case but it demonstrated the importance of these imitation games in the treatment of this disease.

AB - The family of disorders commonly known as autism is characterized by a deficit in social interaction and restricted repetitive and stereotyped patterns of behaviours, activities and interests. Motor disturbances are not part of the diagnosis of the children with autism but some studies have estimated that between 80 and 90% of children with Autism Spectrum Disorder (ASD) demonstrate some degree of motor impairments. Several therapies have been used for the improvement of motor skills, always leading to behavioural improvements as side-effects, demonstrating the importance of motor interaction and stimulation for the case of autism. Recent studies have shown that motor, imitation and social abilities are all related in people with autism. In this work, a humanoid robot is used to create a therapy that unites all these areas. The system involves a robot (NAO), a Kinect camera and Personal Computer, with the goal of facilitating the interaction between therapist and a child with ASD during a physical therapy session. To improve the imitation abilities of the child, the robot was programmed to mirror both the child and the therapist movements. After testing different tracking methodologies, the Kinect sensor was selected as the best compromise of quality and cost. Two protocols were developed, depending on who plays the role of the main actor. In the first protocol, the robot is the master and leads the interaction. It decides the exercise to execute and gives feedback to both the therapist and the child. In the second protocol, the choice of the exercise sequence is the therapist’s responsibility. To promote interaction further during clinical tests, the protocol was changed to include gesture imitation. For the robot master protocol, the space theme was chosen. For the therapist master protocol, the theme of sports, that was already performed by the children in the usual therapy, was adopted. The system was tested in realistic conditions with two different autistic children. The reaction was different in each case but it demonstrated the importance of these imitation games in the treatment of this disease.

UR - http://www.scopus.com/inward/record.url?scp=85075865820&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075865820&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-31635-8_239

DO - 10.1007/978-3-030-31635-8_239

M3 - Conference contribution

AN - SCOPUS:85075865820

SN - 9783030316341

T3 - IFMBE Proceedings

SP - 2014

EP - 2027

BT - 15th Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019 - Proceedings of MEDICON 2019

A2 - Henriques, Jorge

A2 - de Carvalho, Paulo

A2 - Neves, Nuno

PB - Springer

ER -