Social robot in training to express human feelings through art

Usually Baxter lives at the School of Information Technology at Halmstad University, where he is in training to understand and interpret people’s needs and emotions in order to help them feel better. But one day a few weeks ago, Baxter took part in a public event well suited for a social robot. People had gathered on campus to see Baxter try to read, interpret and draw the feelings of well known local artist Peter Wahlbeck, also assisted by artist and author Dan Koon. They have both been part of Baxter’s training by coaching the robot research team in an ongoing master thesis project where the robot is trained to pick up on feelings and express them through painting.

Baxter, who had taken the name ”Rob Boss” fort the art competition, met his audience blindfolded.

“We did it to show that this is not about the robot reading facial expressions or body language. A person can smile without being happy. So we want to take it one step further by getting the robot to sense emotions through brain waves, to understand how a person truly feels,” says Martin Cooney, researcher in social robotics at Halmstad University.

Transferring emotions through brain waves

With a sensor measuring brain waves attached to his head, Peter Wahlbeck proclaimed to be the guinea pig for the event. The emotions he was trying to convey to the robot were decided by chance. And as it turned out, they were to be two extremes – happiness and misery. Peter Wahlbeck produced his happy thoughts by thinking of a hot sunny summer day on the beach. Misery and irritation was evoked by remembering a recent event:

“I thought about something I had ordered online that turned out to not be what I expected, which was very irritating at the time!”

The robot reading Peter Wahlbeck’s emotions painted his impressions with a bright happy yellow and a dark blue and black. Even though the robot chose colours to represent the ”right” emotions, Dan Koon was disappointed after the public event, seeing the robot perform at a more advanced level before.

“Baxter had stage fright today, the robot has done better on other occasions. But we still think that the event has proved what a large potential this research has,” says Dan Koon.

“It is positive that the robot chose the colours matching the emotions, according to the colour scheme we have been using. It makes for a promising future,” says Martin Cooney.

The public event was not the first time Dan Koon and Peter Wahlbeck painted with Baxter – and the images entered in the Robot Art Competition are from those other occasions.

Learning to help humans takes time

Maria Luiza Recena Menezes is a postgraduate student in affective computing, a subject exploring the emotional dimension in the interaction between human and computer. She is the co-supervisor for the master thesis – which she describes as unique.

“Robot systems usually copy. But here we want the robot to learn how to interpret signals into images,” she says, explaining the reason why the robot has been programmed only with colours and emotions, no images, “So the robot can’t copy something it has seen before.”

But the learning process must be given time.

“You have to remember that the robot is like a child learning. It must be given the time to pick up on and understand a person’s feelings. Every time we do these attempts the robot enhances its abilities.”

Helen Fuchs, lecturer in Art science in the School of Education, Humanities and Social Sciences was especially invited by the research team to give her views on the project. She feels that the endeavour is interesting and that robots learning to sense and express human emotions could lead to a new way of communication for people who suffer from various disabilities.

“The cooperation between different disciplines is interesting. The researchers take help from artists in exploring the robot’s possibilities,” says Helen Fuchs.

Peter Wahlbeck thinks that the project and the collaborations with the research team have been very interesting.

“This is an elaborate idea, that a robot can sense an emotion and express it through art. In the future this research can be of value to the health care sector in giving help to persons whom for different reasons can’t express themselves.”

Text: Lotta Andersson and Louise Wandel
Photo: Anders Andersson