Notes on using Troy in therapies from Maggie and Aersta

Maggie and Aersta gained a lot of practical experience in conducting meaningful therapies using Troy as a tool. We asked them to share some lessons learned, with an eye toward helping the next therapists and improving the technology design. Clearly, I (Mike) don't have a lot of experience judging clinical talent, but from what I saw it appeared that both Maggie and Aersta were very talented clinicians so I think it's useful to record their thoughts. Here's what they said:

• Maggie
  • What is needed to make the robot more useful? Clinicians need to be able to program the robot and need to know Troy's limitations and capabilities.
  • What skills do clinicians need in therapy to benefit the child? Clinicians need he ability to sense when the robot is necessary to instigate something and to know when it's time to ignore the robot so that the child and therapist can practice an emerging social skill.
  • What other technologies would be helpful? If Troy could move its head to track the child then this might help. The ideal system would be for Troy to be able to visually track 1) the pertinent events in an interaction (watch Chris push the truck, watch the clinician push it back), and also to track when the child has left the interaction (paying attention to the activity until the child stops it by leaving the interaction).
• Aersta
  • What is needed to make the robot more useful? To be more useful in the real world, it needs to be cost effective.

In order for this technology to ever be used in a clinical or school setting, clinicians need access to a cost effective robot. To be more useful in the study, it would have been nice if he was more portable. It was a little limiting that he had to be connected to an outlet and a laptop computer. ** What skills do clinicians need in therapy to benefit the child? Clinicians need to be able to think on their feet. Children with language impairment, especially children with Autism, can be impulsive. It's important for a clinician to be able to shape a child's impulse and interests in therapy in order to accomplish the child's specific therapy goals. ** What other technologies would be helpful? (or, what features do you wish the robot or user interface had while you were using it?) A more user-friendly interface would have been nice. Something that was more intuitive with less bugs (I realize it was a new program and bugs were inevitable). It was especially hard to create complex routines for the robot in the interface. Once a routine was fairly complex, it was hard to fix a mistake or make a change because it was hard to see what boxes were connected to what. (I hope that makes sense. It's been a long time since I've used the interface and I can't remember what we called certain functions). We really liked using the Wii remote to control the robot, so that's definitely a keeper. ** Other Thoughts? I really enjoyed using Troy in therapy and it worked really well once Maggie and I figured out the system.

Lee's notes on AVATAR TEST 1 10/02/09

1 S101 (4 YRS TYPICAL) TEST1 AVATAR: S101 is a typically developing 4 year-old. First clip was our first attempt at presenting the avatar. The avatar wasn’t working so we had to start again. Bonnie “scolded” the avatar for not working and to engage S101 in “buying” into playing with the avatar.

2 S101 (4 YRS TYPICAL) TEST1 AVATAR: The second attempt was more successful. S101 thought it was silly and fun and said she would play with us again.

1 S102 (3YRS, POSSIBLE ASD) AVATAR TEST1: Our first attempt with S102. Lee works the avatar and Bonnie provides full prompt with S102. Mom and student clinician move to avatar to try to direct S102’s attention to it.

2 S102 (3YRS, POSSIBLE ASD) AVATAR TEST1: Our second attempt with S102. Mom and student clinician take him to the avatar while Lee works the avatar and Bonnie models language.

3 S102 (3YRS, POSSIBLE ASD) AVATAR TEST1: Our third attempt with S102, lights off and mom’s feedback.

Bonnie's notes on AVATAR TEST 1 10/02/09

We tried a basic imitation game (hands up—hands on hips) with a typically developing 4-year-old girl and with a 3-year-old boy with significant developmental disabilities, including behaviors associated with autistic spectrum disorder. Here are our impressions:

1. The Avatar: We were able to get the Avatar up and running without difficulty. It crashed from time to time, and we restarted it. We noticed the following

• a. The response time is quite slow. It takes the avatar several seconds to perform a movement.
• b. The avatar would not extend its arms upward more than about shoulder level. So, the arms would move to a straight out position and then down in response to our putting our arms above our heads and on our hips.
• c. The avatar rotates somewhat even if we stand still. This is disconcerting.
• d. Sometimes the arm movement is unreliable. (e.g. When we moved our arms up, it might move one had part way up and the crook the other in an odd position.)
• e. The positioning of the child and clinicians in the room is challenging. Ideally, we’d like to position ourselves (child, Avatar, and clinician) as an interactive triad. This is difficult to do with one person directly facing the camera.

2. The typically developing child (S101: age 4 years)

• a. Imitation game: Lee stood in front of the camera, S101 faced Lee and the screen, and Bonnie knelt behind S101. Bonnie used a hand-over-hand technique with S101 moving her hands up and then down to her hips. Lee imitated S101.
• b. S101 focused easily on the avatar and seemed to enjoy watching it “imitate” her. In the few minutes we worked, Bonnie used the hand-over-hand, but S101 did not perform the motions on her own.
• c. In response to questions after the activity, S101 said that the Avatar was “silly” and “fun.” She said she’d like to play again.

3. The child with the disability (S102: age 3 years)

• a. We tried the same positioning and hand-over-hand technique with S102. He resisted the hand-over-hand direction.
• b. We tried various techniques to help S102 focus on the Avatar including pointing, verbally directing his attention, having his clinician and his mother sit next to the screen, etc.
• c. S102 rarely focused on the Avatar.
• d. He approached the screen a couple of times and touched the image, but we could not engage him in the imitation game. (Context: we were able to engage him in the imitation game without the Avatar.)
• e. We asked S102’s mother what she would suggest to make the Avatar more compelling. She said, increased speed of movement and sound.
• f. It was impossible to evaluate how S102 perceived the Avatar. Could he resolve the image? Did he view it as a responsive entity?