'Baldi' Grows on Kids With Autism
Virtual Talking Head Teaches Autistic Children Language Skills
Feb. 15, 2008 (Boston) -- Children generally love to play with computers, but many children with autism prefer interacting with a computer to conversing with a person. Many scientists have been working on ways to take advantage of that to help autistic kids learn.
A virtual talking head, or "embodied conversational agent," nicknamed Baldi by its creators, is one of the latest technologies being used to teach language skills to autistic kids. Dominic Massaro, PhD, a language researcher at the University of California, Santa Cruz, demonstrated Baldi at this week's annual meeting of the American Association for the Advancement of Science.
If you've seen virtual characters in video games, you may notice that as realistically rendered as they may be, their mouths typically don't move in a natural way when speaking. The difference with Baldi is the extent to which Massaro has realistically modeled the virtual mouth movements. Baldi's whole face -- including the tongue, jaw, and lips -- moves the way a real person's does when making different spoken sounds. This is important for learning, Massaro says, because language is learned not only by ear but also by sight. "People use cues from both the face and the voice," he says.
In experiments with Baldi, Massaro showed that children recognized many more words in sentences when watching the talking head than by listening only.
To accurately model mouth movements for the computer graphics, Massaro used ultrasound images of people's tongues during speaking. Measurements are taken with sensors placed all over the face and a special sensing plate placed inside the mouth. The inside of Baldi's mouth can be revealed to see how he's making a certain sound, like an "r" sound or "th" sound.
Learning with Baldi occurs in the context of a software program with features that are friendly to autistic kids.
The program uses a style called "errorless" learning. "The child is not told he's wrong if he makes a mistake," Massaro says. Instead, the program will suggest how to get to the right answer in positive terms. For example, the program prompts, "Click on elephant." If the child clicks on a picture of a giraffe instead, it will say, "You clicked on giraffe. A giraffe is an animal with a long neck. An elephant is an animal with a long trunk. Can you click on elephant?"
'Baldi' Helps Kids With Autism
Virtual Talking Head Teaches Autistic Children Language Skills
Feb. 15, 2008 (Boston) -- Children generally love to play with computers, but many children with autism prefer interacting with a computer to conversing with a person. Many scientists have been working on ways to take advantage of that to help autistic kids learn.
A virtual talking head, or "embodied conversational agent," nicknamed Baldi by its creators, is one of the latest technologies being used to teach language skills to autistic kids. Dominic Massaro, PhD, a language researcher at the University of California, Santa Cruz, demonstrated Baldi at this week's annual meeting of the American Association for the Advancement of Science.
If you've seen virtual characters in video games, you may notice that as realistically rendered as they may be, their mouths typically don't move in a natural way when speaking. The difference with Baldi is the extent to which Massaro has realistically modeled the virtual mouth movements. Baldi's whole face -- including the tongue, jaw, and lips -- moves the way a real person's does when making different spoken sounds. This is important for learning, Massaro says, because language is learned not only by ear but also by sight. "People use cues from both the face and the voice," he says.
In experiments with Baldi, Massaro showed that children recognized many more words in sentences when watching the talking head than by listening only.
To accurately model mouth movements for the computer graphics, Massaro used ultrasound images of people's tongues during speaking. Measurements are taken with sensors placed all over the face and a special sensing plate placed inside the mouth. The inside of Baldi's mouth can be revealed to see how he's making a certain sound, like an "r" sound or "th" sound.
Learning with Baldi occurs in the context of a software program with features that are friendly to autistic kids.
The program uses a style called "errorless" learning. "The child is not told he's wrong if he makes a mistake," Massaro says. Instead, the program will suggest how to get to the right answer in positive terms. For example, the program prompts, "Click on elephant." If the child clicks on a picture of a giraffe instead, it will say, "You clicked on giraffe. A giraffe is an animal with a long neck. An elephant is an animal with a long trunk. Can you click on elephant?"