They’re robots, and they’re here to help: Computer scientist improves robot interactions with human beings

Friendly robots, the ones people love to love, are quirky: R2-D2, C-3PO, WALL-E, BB-8, Marvin, Roz and Baymax. They’re emotional, prone to panic or bossy, empathetic and able to communicate like humans do — even when they communicate in only beeps and bloops.  

At Purdue University, Sooyeon Jeong, robot communication and behavior expert, is working to make robots as friendly and helpful as possible in nonfictional settings.

An assistant professor of computer science, Jeong leads a lab of researchers who work at the intersection of human behavior and artificial intelligence. Her goal is to create robots that make the world a better place for humans. Her research has helped cancer patients, people with aphasia, hospitalized children and geriatric adults. Now, new research projects, including a robot study buddy and a robot concierge, are improving the ways humans and robots communicate and collaborate.

Jeong’s research is on software, but she runs it on a variety of hardware robots — small, fubsy humanoid robots; robots that are a humanoid head with a face projected upon it; robots that don’t resemble a living creature at all, like a Roomba; or AI agents that don’t have a physical body and exist only as a software program that people communicate with inside a computer.

“My goal, and the goal of my research group, is to design robots and AI that can have socially and emotionally natural interactions with people,” Jeong said. “I want anything I make or design to have a measurable positive impact on people’s lives.”

The ideal study buddy

One way Jeong is working to help humans is by creating robot software that will help students learn  — or help anyone complete a task.

“For a lot of people, working in a study group helps provide peer accountability,” Jeong said. “But for some people, that can be difficult in terms of logistics or social anxiety. There are existing technologies like productivity apps that do things like block YouTube, but they’re mostly prohibitory. They don’t encourage or support you the way peers would. So we got interested in seeing if we could build a robot that would provide both companionship and support to help someone study and be productive.”

Jeong and her team tested several different iterations of the robot. In one, the robot was merely present, pretending to work at its own task in parallel with a student studying or working on homework. In a second, the robot would periodically remind the student of the student’s stated goals: “Remember! You said you wanted to complete the exam study guide by 2 p.m.” And in the third situation, the robot offered more emotional support, along the lines of an exercise instructor in a difficult class.

“In that third one, the robot would offer emotional support, saying things like, ‘You’re doing a great job! We can do this together!’ ” Jeong said. “It would encourage the student to take short breaks, stretch, move around and then refocus.”

What the study found was that the successful strategy depended not just on the student’s personality, but also on the student’s mood that day, and even what subject they were studying and what they were trying to accomplish.

“It was interesting; we had mixed and nuanced results from the study,” Jeong said. “Some of the students who participated said the robot should be a little sterner, should scold them if they got off task. But this also was a one-time experiment: That might change if they had repeated sessions with the robot. This has some interesting design implications for future work because the robot would need to be able to read the student’s body language and decide what is going to work best in each situation.”

Friends, robots, countrymen, lend me your ears

Putting people at ease is a huge hurdle for robots. A tired person wouldn’t necessarily confide about a hard day to the toaster, after all. But that person might very well talk to the cat. Or even a fish. Jeong is trying to create a robot that’s more like the cat and less like the toaster. More like another person would be even better.

Jeong explains that robots don’t just need to listen to what a person is saying; they need to engage in active listening. They need to mirror behavior and tone and understand how to react and respond.

One project Jeong’s lab is tackling is understanding the best way for robots to respond to conversation from a human. Active listening does not just mean sitting in silence.

“Many of the robot assistants people are used to — Siri, Alexa, Google — are discrete in their interactions with humans,” Jeong said. “You ask them a question, and they give you the answer. But the way people interact with one another is very different. As I’m talking, you’re nodding, and saying ‘right’ and responding. These things are called backchannels, and they’re part of how we as humans keep a conversation going, how I know you’re really listening. We are working on how to generate appropriate backchannels for robots to use in conversations.”

Not only is nonverbal communication like a backchannel signaling an advanced social cue, it is also a staggeringly difficult programming task. Jeong and her team are using large language models and recordings of human voices and communications to identify changes in rhythm, pitch and wording in conversations to teach computers what they call empathetic listening behaviors.

“These robots and AI agents really need social and emotional intelligence to help people,” she said. “And we’ve found that the more empathetic listening behavior we can implement, the more personalized support we’re able to provide to people, whether it’s in a health care, therapeutic or an educational setting. When people are more comfortable, the robot is more able to help. As robots become more available in people’s daily lives, we need to make sure they’re helping in real ways, long term, in the real world.”

Jeong’s work is part of Purdue Computes, a comprehensive initiative that spans computing departments, physical AI, quantum science and semiconductor innovation. Leveraging Purdue’s signature strengths in materials science, engineering, microelectronics, computer science, agriculture and life sciences, the Institute for Physical Artificial Intelligence (IPAI) is committed to solving the world’s toughest challenges.

About Purdue University

Purdue University is a public research university leading with excellence at scale. Ranked among top 10 public universities in the United States, Purdue discovers, disseminates and deploys knowledge with a quality and at a scale second to none. More than 106,000 students study at Purdue across multiple campuses, locations and modalities, including more than 57,000 at our main campus locations in West Lafayette and Indianapolis. Committed to affordability and accessibility, Purdue’s main campus has frozen tuition 14 years in a row. See how Purdue never stops in the persistent pursuit of the next giant leap — including its integrated, comprehensive Indianapolis urban expansion; the Mitch Daniels School of Business; Purdue Computes; and the One Health initiative — at https://www.purdue.edu/president/strategic-initiatives.