Designing chatbots to fight information overload

A Google News Lab fellow‘s investigation into categorised news for Facebook Messenger.

We live in a world of transactions. Money, goods, and importantly, information. Our social media feeds, news websites and internet-driven messaging apps churn out so much content that finding exactly what you want is now akin to the proverbial needle in the information haystack.

This is where a transaction like the above would make sense. Karen, I’m sure you were intelligent enough to figure out, is a chatbot — an artificially programmed entity that (in most cases) trawls the web or a repository and fishes out exactly the information you need. While a Google search is programmed to give you thousands of results, bots are programmed to give you a handful.

The BBC Mundo Messenger bot, which was built by News Labs in 2016, is programmed to send headlines and links to the top stories of the day to tens of thousands of subscribers. My task was to build on this and add a feature where users could search for stories by category (for instance, ‘sports’) and get links to the day’s top stories in that category.

The project came with a few challenges — not all of them technical. The Mundo bot is programmed to speak Spanish, and the closest I got to Spanish was ‘Hasta la vista, baby’ from Terminator 2 (a bot reference for another time), one of my favourite movies growing up.

Because the feature would take free text input from users, the bot also needed to intuitively understand and account for typos and misspelled or missing words. Most of us at some point type something like ‘cst videos’ into the search bar and the search engine has a gotcha moment, ‘Did you mean, cat videos?’

So what would happen behind the scenes if a user typed ‘categories’ (or ‘cstogaried’) into a Facebook Messenger conversation with our Mundo bot? Ideally, the bot would give you a list of categories to select from, corresponding to the nine major topic tabs on the BBC Mundo website: News, Latin America, International, Business, Technology, Science, Health, Culture and Sports.

To get you your stories, the bot needs to talk to the BBC News feed through an Application Programming Interface, or API. The API is like a waiter who fetches your order.

In this case, the bot fetches all the stories in a particular category from the API and does two things:

The immediate utility of the first step is clear — the bot just gives you the stories that you asked for. But what does the second step do? Why do we even need a database?

Well, the API is the busy waiter at a popular restaurant during the holiday season. He/she has a lot of requests from a lot of other people, so we don’t want to bother the waiter every time we want another helping — we can serve ourselves with our nifty bot once the food is on our table (database).

The first time a user asks for stories, we would want to retrieve a set of links from the API and store them in the database for future use. This means we wouldn’t need to bother the API for the rest of the day.

If a user wants to read more stories, they can just type ‘more’ and the bot goes back to the database and fetches four more stories. This continues until all the links provided by the API are delivered to the user. The number of stories served by the API to the bot depends on the number of stories in that category present in the news feed, at the time the request is made.

If it’s possible to deliver categorised news to users, it would also be possible to implement a dashboard for administrators to track which categories users requested the most. Eight weeks being eight weeks, I didn’t have enough time to get started on this feature of the bot publishing system.

One of the challenges of working on an innovation team is that sometimes your ideas don’t make it in front of audiences, for reasons outside your control. I produced a working prototype while completing my fellowship, but the systems underlying the BBC’s chatbots changed before my code was able to be implemented. But like all good experiments, I still learned a lot from it.

The project gave me the opportunity to interact with three different types of people: software developers, who helped me develop and optimising code; user interface specialists, who discussed what the interface should look and feel like; and social media managers from BBC Mundo, who helped me understand how their audience perceived and interacted with chatbots.

The project also helped me add some technical skills to my arsenal: the Node.js framework, the Facebook Messenger Platform and MongoDB.

By corollary, news organisations also use Facebook and Instagram in this manner, and hence the flood of information our audiences experience is continuous, relentless even.

In contrast, chatbots help channel this information and give you only what you ask for. If companies selling products to consumers can leverage chatbots to answer queries, why can’t news organisations provide news to their consumers the same way?

Related posts:

Trecento Blockchain Capital is convinced that Blockchain technology is a disruptive innovation which will have an impact across all industries. Before becoming mainstream, Blockchain technologies…

Psychological Safety has been identified by Google via a two-year study Code-named Project Aristotle, as one of the most important elements of building a successful team. The ability to express…

Leasing aims to emerge because the market leader in generating short-term property income , where profits are paid to token holders within the sort of fair deflation for all. . this may allow anyone…