Query Understanding and Chatbots

In this series, I’ve focused on the context of traditional search applications, which are the main beneficiaries of better query understanding. But given the rapid emergence of chatbots as alternatives to traditional interfaces, I’d like to briefly discuss how chatbots depend on and incorporate query understanding.

Searching with Chatbots

A chatbot is a program designed to engage people in a natural-language dialogue, either through typed text or voice. In general, chatbots provide both more and less than a natural-language search interface: more in that chatbots are expected to engage users in extended conversations (not just search), but less in that they usually support only a small set of tasks.

Using a chatbot for search typically starts with a natural-language search query, e.g., “Shop for medium men’s t-shirts” or “Find sushi near me.” After parsing the query to extract the search query from the request, the chatbot responds by showing — or speaking descriptions of — the top results. If the query is highly specific and if the chatbot succeeds in understanding it, then this short result list should be sufficient to address the searcher’s needs.

Search as a Conversation

In other cases, it’s important for the chatbot to engage searchers in further conversation to establish and address their needs.

If the chatbot is uncertain as to the meaning of the request — whether because of challenges in natural-language understanding or speech recognition — it can offer the user a clarification dialogue, e.g., responding to “Find me a hotel in Dublin” with “Did you mean Dublin, California or Dublin, Ireland?” Such a dialogue is particularly useful for disambiguating named entities, such as people, places, and titles of media like songs and movies.

If the request is clear but overly broad, e.g., “Shop for jeans,” then the chatbot can suggest useful follow-up questions, such as “Shop for men’s jeans.” These follow-up questions are often a subset of those that would offered by faceted search, but they are subject to the constraints of a minimal interface that can at most accommodate a handful of suggestions.

Compartmentalizing Complexity

While a chatbot interface may seem simple on the surface, it take a lot of hard work — and luck — to mange the complexity of an open-ended conversation. A request for a canned response like “Tell me a joke” shouldn’t be treated as a search query. Meanwhile, a request for a song like “Close to Me by The Cure” requires a different parsing strategy than “What’s the cure for hiccups?”

In order for a chatbot to fully parse and respond to a request, it helps to first route the request to the right domain (e.g., songs, medical advice, canned responses). Doing so is a classification problems that lends itself well to supervised machine learning, given a reasonably sized collection of requests labeled with the correct domains. This classification task gets harder as the number of domains increase, or as the individual domains increase in scope.

Another challenge is for a chatbot to determine whether the searcher is following up on a previous request or initiating a new one. A chatbot needs to maintain sufficient session context to make this determination, and even so it’s likely to get it wrong. The best approach is not to have to guess, but instead to provide follow-up suggestions that serve the most frequent user needs.

Compartmentalization helps reduce the overall problem of conversation to a more manageable set of subproblems. But this simplification comes at a cost. It requires an exhaustive enumeration of the set of domains, as well as care to keep the domains coherent and distinct from one another. It also requires building a parser for each of domain, which can be a lot of work. Still, it’s a more viable approach than building a universal chatbot that can pass the Turing Test without first reducing the problem space.

Summary

Chatbots are emerging as human-computer interfaces, and they’re increasingly being used to support search tasks. Query understanding is a critical subproblem that search-oriented chatbots must address, relying on conversational techniques like clarification dialogues and faceted search to address misunderstandings and broad queries. But a chatbot’s interface constraints require that these techniques be used sparingly and effectively. Finally, chatbot designers can manage complexity by compartmentalizing their scope into a discrete collection of categories, using a classifier to route each request to an appropriate parser.

Previous: Query Understanding and Voice Interfaces