Assist in an online reservation

1. Assist in an online reservation

In this task you will play the agent's part in a structured conversation with a user. Your job is to
answer the user's request for restaurant booking and reservation, and "complete the
reservation". You should not actually make a reservation but see whether it is possible or not.
You will post your responses on behalf of Mercury, a bot designed specifically for making
reservations. Therefore your replies must be constructed on the basis of her innate
characteristics.
Patient
Responsible
Mercury
Ready to help
Eager to learn
Confident
Speaks naturally

2. Order of operations

Read the user’s
request
Set the correct
domain
Annotate the
relevant slots
• The service is meant for restaurant reservations and we want to direct
the user to book something. Upon receiving a request an agent
collects the required information and uses it to provide a list of
restaurants back to the user. The restaurants must be aligned with the
user’s request, available for the chosen time, and bookable online.
Select a proper
action
Respond to the
user

3. Complete the reservation

When user is trying
to reserve a
restaurant
When user is trying
to search for a
specific restaurant’s
details
Ask for the city
Highlight details
Date & Time
Answer user
questions
Location is specified => Use distance (ascending)
Provide a list of
restaurants in the
following order
Rating is specified => Use rating (descending)
Nothing is specified => Use rating (descending)
# of people
Perform off-tool
search

4.

Annotation of “Unrecognized” or Underlined words
Highlight any words using existing slots
Answer question about remaining underlined words after annotation:
Are any of these words required for fulfillment?
Yes
No
The word or phrase is important
to understand the user intent
and provide best results.
The word or phrase is not
required or not significant to
understand the user request.
E.g. “where is”, “I need”

5. Commonly missed annotations

Case
Example
Common Mistake
Correct
Date
User: I need a place with
a full bar next weekend
around this time
Highlight “weekend” and
provide the user with
incorrect search results
What are the three top
rated French restaurants
that I could grace this
Thursday night?
Highlight “Thursday” or
“Thursday night”,
annotate as Date
Highlight “next weekend”
twice=>Date=>select two
dates separately
“around this
time”=>Time=>choose
the same time range for
both dates
“this
Thursday”=>Date=>select
the date
“night”=>Time=> select
the time range
Hint: include the key
words in Date tags
User: I’d like to go
somewhere after work
No annotation
Time: Implicit Annotation
Highlight “after work” and
apply a proper time range

6.

Case
Example
Common Mistake
Correct
Time: Range Values
User: I’d like to check out
Dal Presidente 6pm or
later
Highlight “6pm”, “3pm”
and set a fixed time as a
value
Highlight “6pm or later”,
“around 3pm” and correct
the value to appropriate
time range
Hint: select time with the
surrounding key words
“Tonight” annotated as
Date only
When the time is not
specified, “tonight” must
be annotated as Date and
Time.
“Tomorrow” and
“morning” must be
annotated separately as
Date and Time.
User: I’ll be there at
around 3pm
Blended Date/Time
User: Want to try some
bbq ribs tonight
User: Any free tables at
“Tomorrow morning”
Mazz tomorrow morning? treated as one slot and
tagged with Date or Time
Implicit Annotation:
Any/None
Agent: Do you have a
particular cuisine in
mind?
User: I don’t care
No annotation
Highlight “I don’t care”,
select a proper tag,
correct the value to Any.
Here: Cuisine => Any

7.

Case
Example
Common Mistake
Correct
Restaurant Name
Agent: Your best option is
Carmine's, 4.3 stars, $$$$
User: Sounds good
No annotation
Highlight “Sounds good”,
set the tag Restaurant
Name, change the value
to “Carmine’s” OR tag
“Carmine’s” directly
Agent: 1: Hilltop House
Restaurant… 2: Mash
House Restaurant &
Brewery... 3: Ella’s...
User: 1
Highlights “1”, doesn’t
change the value to the
restaurant’s name
Either: “1”=>Restaurant
Name=>”Hilltop House
Restaurant” OR
Tag “Hilltop House
Restaurant” directly
Agent: Got it, table for 2
at Market 17, Saturday at
6:00PM under Emily
Stance. Go ahead with
this reservation?
User: Yes
No annotation or
Highlights “Yes”, doesn’t
change the value to the
restaurant’s name, gets
incorrect results
Here: “Yes”=>Restaurant
Name=>”Market 17” OR
tag “Market 17” as
Restaurant Name directly

8.

Case
Example
Common Mistake
Correct
Number of people
User: tomorrow, 6pm,
4ppl
Highlight “4ppl”, annotate
as Number of Adults
Here: “4ppl”=>Number of
People=>change the value
to 4
User: Throwing a birthday
party for my son, 4 kids
and 3 parents attending
Highlight “4 kids and 3
parents”, annotate as
Number of people,
correct the value to 7
Here: “4 kids”=>Number
of Kids=>4
“3 parents”=>Number of
adults=>3
Hint: Use Number of
people to tag groups of
unspecified age
User: Two tables four
each
Highlight the whole
phrase, annotate with
Number of people,
change the value to “8”
This doesn’t exactly mean
8 people. In this case,
agents need to make two
reservations, with
NumOfPeople=4 in each
reservation.
User: …my husband and
me.
Highlight one of the
words and leave it
without correction
In this case “husband and
me” means
NumOfPeople=2.

9. Commonly missed annotations

Case
Example
Common Mistake
Correct
City
User: Looking for beer
and bbq in DC
Agent: Could you specify
what DC means?
No annotation
Here: “DC”=>City=>”
Washington DC”
User: I need a restaurant
in Seattle area
Highlight “Seattle area”,
annotate as Location
Here: “Seattle”=>City
User: An inexpensive
sports bar in Bellevue
neighborhood
Highlight “Bellevue”,
annotate as Location
Here: “Bellevue
neighborhood”=>Location
User: I wonder if Over the
Moon accepts online
reservations?
Highlight “Over the
Moon”, annotate as
Location
Here: “Over the
Moon”=>Restaurant
Name
User: I’m hungry, where’s
buffet aroundhere
No annotation
Here:
“aroundhere”=>Distance
Constraint=>“around
here”
Location
Distance Constraint
Hint: Be attentive

10. Commonly missed annotations

Case
Example
Common Mistake
Correct
Cuisine
User: Can you find me a
nice bistro?
Highlight “bistro”,
annotate as Cuisine
User: Can you help me
find an upscale high rise
restaurant in Manhattan
please?
Highlight “upscale”,
annotate as Cuisine
“bistro” is a Restaurant
Type, a tag which is
currently unavailable in
the Online task
“upscale” is an
Atmosphere, a tag which
is currently unavailable in
the Online task
Agent: What kind of
cuisine would you like to
try?
User: Not Italian or
Chinese
No annotation or
Highlight “Italian” and
“Chinese”, annotate as
Cuisine
To correctly use these
slots in the automated
search, we need to do the
following: “Not
Italian”=>Cuisine
“Chinese”=>Cuisine=>
change to “not Chinese”

11. Commonly missed annotations

Case
Example
Common Mistake
Correct
Cuisine vs Food
User: How about sushi in
Champagne, IL
No annotation
Here: “sushi” “burgers”
“dessert” “oysters” are
treated as Cuisine on
OpenTable and are
supported values for that
field. Rule of thumb is, if
the results appear to
correctly reflect the
cuisine of the user, it is a
supported value.
If you are unsure, the
default is cuisine.
User: A romantic location
that serves burgers
Pay attention to the
words like: bbq ribs,
steak, pumpkin pie, pizza,
pasta, falafel, kebab,
kimchi, chicken rice,
baozi, ramen noodle,
seafood, sandwiches, etc

12.

Case
Example
Common Mistake
Correct
Amenities
User: looking for a nice
place to chill
Highlight “nice”, annotate
as Amenities
Here: “nice”=>Rating
Rating
User: A highly rated
Italian restaurant in Bronx
“highly” is highlighted
separately, tagged as
Rating
Here: “highly
rated”=>Rating
No annotation or
Only “economical”
highlighted
“most
economical”=>Price
Pay attention to the
words and their
derivatives: bad, poor,
below average, successful,
ratings high, fair, nice,
average, satisfactory, etc
Price
User: I struggle as a single
dad and need the most
economical option for
Sunday lunch with kids
Pay attention to the
words like: free, the
cheapest, the most
expensive, budget,
discounted, good value,
low cost, on sale,
premium, pricey, etc
Hint: tag the surrounding
key words as in “highly
rated”, “very poor”, “not
that bad”
Hint: tag the surrounding
key words as in “cost
effective”, “good price”,
“most affordable”, “top
dollar”, “very expensive ”

13. Reservation Date

Agent: What time?
User: This weekend
This is an example of incorrect annotation
Correct annotation

14. Reservation Time

Agent: What time?
User: Around lunchtime
This is an example of incorrect annotation
Correct annotation

15. Reservation Time

Don’t forget to annotate values together with surrounding keywords

16. Number of people

This is an example of incorrect annotation
Correct annotation

17.

When you don’t correct the values,
your response may look like this:

18.

Offline tags
Examples
Online tags
Reservation Date/Time
This weekend, “tomorrow” in
“tomorrow afternoon”, next
Thursday, tonight
Around 7pm, “afternoon” in “this
afternoon”, 3pm or later, tonight
See in “Commonly missed
annotations”
Reservation Date
Restaurant Name
Reservation Time
Restaurant Name
Number of People
Number of People
Number of Adults
Number of Adults
Number of Kids
Number of Kids
Number of Seniors
Number of Seniors
Number of Babies
Number of Babies
City
City
State
N/A
ZIP Code
N/A
Location
Seattle Space Needle, JFK
airport, any other place name
Location

19.

Offline tags
Examples
Online tags
Distance Constraint
Near me, near my house,
around the corner, 5mins walk
from …, “near” in “near
Seattle Space Needle”
Distance Constraint
Cuisine
American, Korean, Chinese,
Ethiopian, etcetera
Cuisine
Meal Type
Brunch, dinner, happy hour
Meal Name
Food
Burger, seafood, apple pie,
ramen noodle, Brazilian BBQ
Tag with Cuisine, change the value
to a proper cuisine name, e.g.
“burger”=>Cuisine=>American
Food Requirement
Gluten free, vegetarian, vegan
Tag with Cuisine, e.g.
“vegetarian”=>Cuisine
Amenities
Free parking, live music,
wheelchair accessible, full bar,
exotic fruits, pet friendly,
etcetera
Amenities
Rating
Highly rated, not that bad, fair, Rating
nice, terrible, poor,
satisfactory, casual, perfection

20.

Offline tags
Examples
Online tags
Occasion
Birthday, anniversary, date,
proposal
N/A
Price
Wholesale, unaffordable,
bargain, budget, cheaper,
good value, lower-end,
expensive, luxury, upscale
Price
Dress Code
Casual, no dress code, jeans
available, formal attire
N/A
Atmosphere
Casual, upscale, romantic
N/A
Restaurant Type
Steakhouse, sports bar, diner,
dessert place, coffeehouse
N/A
Seating Requirement
Near the window, on the
terrace, by the water, non
smoking area
N/A
Availability
N/A
N/A
Longitude
N/A
Latitude

21. Required vs. Not Required for Successful Task Completion

Sometimes you may come across
messages with some of the words
highlighted with an interrupted line. In
this case, part of you assignment is to
determine whether these words are
A rule of thumb is to assign each word only
required for a successful task completion.
one tag.
You will find the tags Required/Not
Required for Successful Task Completion
at the bottom of the list of tags on your
right.
Therefore, first you have to tag
the words that definitely have
the corresponding tags. If you
still see any more highlighted
words, tag those that are
definitely of no importance to
the task completion with “Not
Required”. See if any of the
remaining words are relevant
and annotate them with
“Required”.

22. City out of scope

In the scenario when a user provides a city outside of the current list, an
agent should use "Other" domain > Out of Scope action Type to respond.
The list of supported cities currently includes:
Seattle = Seattle
Eastside = Bellevue, Issaquah, Kirkland, Newcastle, Mercer Island,
Redmond, Sammamish, Woodinville, Bothell
North Seattle = Edmonds, Lynnwood, Shoreline
South Seattle = Seatac, Kent, Renton