conversational ui in driverless cars | Spring 2017

Daya Lee, Sydney Choi, Jarret Lin



REMi is an intelligent conversational user interface in a driverless car. In conventional cars, it is assumed that users are awake. Additionally, typically business people schedule calls or meetings during long car rides and short naps during this time are not very practical.

REMi challenges these ideas. It introduces a way to optimize long car rides for business men and women that need a quick nap by waking a user based on the proximity to the final destination. Also REMi's humor and unpredictability makes it more efficient at waking up its users and more delightful to interact with.


Domain Selection

We were given four choices of service-contexts to prototype a novel CUI: In-car interaction with driverless car service, hotel agent, retail store dressing room, and meal ordering. We chose in-car interaction with driverless car service due to the fact that the service has not become common yet, which would allow us to be more creative and future-oriented.

After selecting our domain, we conducted a competitive analysis of current CUIs and in-person interviews to drivers and non-drivers. Also, we performed literature reviews on conversational models and agents. Additional research was done in various areas in order to stretch out ideas further to create more than just a conventional system.


Questions to ask:

Why would we need driverless cars and conversational UI?
Who would be our target user?

Background Research

Drowsy Driving.jpg


- Hinders drivers from paying attention to the road.

- Slows reaction time if they have to brake or steer suddenly.

- Affects drivers’ abilities to make good decisions.

Drowsy driving

An estimated 1 in 25 adult drivers (aged 18 years or older) report having fallen asleep while driving in the previous 30 days. The National Highway Traffic Safety Administration estimates that drowsy driving was responsible for 72,000 crashes, 44,000 injuries, and 800 deaths in 2014.

Business travel

According to GBTA, $302.7 billion was spent on business travel in 2015.Their research with Visa predicts an even greater increase in the business travel industry in the following years.



We thought that a CUI that has the capability to optimize time by allowing business men and women to take naps while traveling would be a viable product. Users would then be able to leave the car with excellent condition, which will improve work efficiency.


Interviews with drivers and non-drivers

Q. What features are important to you?

CUI Design.png

Competitive Analysis

스크린샷 2018-02-23 오전 2.04.40.png

I. A natural language processing system that can fulfill simple commands such as dimming lights, turning on music, etc,

II. Although the system is always listening, it only processes the trigger words like “Alexa”, “Echo”, “Computer”. Everything else is not considered.

III. Alexa can only take one command at a time. You can’t combine two actions into one sentence and expect Alexa to do both.

스크린샷 2018-02-23 오전 2.00.35.png

I. Uses Google Assistant, the same one on Google’s Pixel phone, only 2 microphones to capture sound.

II. Contains a better search system because of its contextual design.

III. Can only access one google account; process one command at a time.

스크린샷 2018-02-23 오전 1.47.43.png

I. Animated persona (somewhat like domestic partner) that responds to voice interactions

II. Can send/receive text messages when its “master” is not at home and also control smart-home appliances, such as lights and robotics vacuums

III. .Vinclu is planning multiple possible personalities


Visual Feedback is necessary (ex: Alexa's orange ring)
Must react to Trigger words only
Having personality can create more fun and pleasant interaction

Experience Prototyping


At first, we focused on the conversation aspect because we thought CUI should be able to produce quality conversations with the user.

However, we received critiques about how the conversations are keep going back and forth without producing any real value that only our CUI can make.

During user testing, the user fell asleep and the existing script for the CUI failed to wake him up.

Other feedback we received was that specific accents or attempting to replicate the user’s mother waking him up was awkward and unnecessary.



The Element of Surprise and Humor in Interactions

The incongruity theory explains the benefit of adding elements of surprise or humor into user interactions. This theory states that the moment in which a joke is told, it marks an unexpected, sudden shift in perspective that is pleasant to experience. As it turns out, jokes lend a pleasant way that helps users to understand what is occurring while making it funny by bringing in the emotional factor of people.

Based on this theory, we added elements of surprise and humor to engage users and conversation models to improve REMi’s performance. Instead of letting users to choose REMI's character (ex: mom), we left it unpredictable. By threading unexpected humor into the CUI we believed it will be more successful at keeping the user engaged and help the user to wake up when groggy. Also, we redesigned REMi to ask more provocative questions the harder it is to wake up a user.




Jessica is a businesswoman who is on her way to the airport. 

She has been traveling non-stop for the past 2 weeks and is very jetlagged. 

Jessica wants to be woken up 20 minutes before she arrives at the airport so she has some time to go over her notes

Light sleeper


Tom is a businessman who is on his way to the airport. 

He has been traveling non-stop for the past 2 weeks and is very jetlagged. 

Tom wants to be woken up 20 minutes before she arrives at the airport

Heavy sleeper


Error Handling and Recovery

For REMi’s error handling, we focus on the most common scenario in which the user’s words are recognized, but incorrectly. Our solution is to recognize the mistake, evaluate if there is an opportunity to enhance the CUIs AI, repeat the correct version of the user’s words, wait for confirmation, and provide appropriate feedback.

Excerpt from the Final Script:

Remi: Based on the app, your airline is Korean Air Airline. Is that correct?

Jessica: Yea it’s KAL.

Remi: I’m sorry, Let me reroute to the China Airlines terminal.

Jessica: NO! I meant Korea Air Airlines, KAL is just a short nickname.

Remi: Okay I will KAL to my airline database. We will be arriving in about 10 minutes



Scenario 1 deals with the situation where the CUI misunderstands the user and how it recovers.

Scenario 2 depicts a situation where the CUI’s humor and unpredictability wakes up a user who is a deep-sleeper.



스크린샷 2018-02-23 오전 2.20.16.png