What's cool vs what's useful

What's really cool is an AI you can have a seamless conversation with. What's useful is a service that can answer your questions.

A long time ago, before everyone was working on a chatbot, a colleague asked me for help on his personal project. He had built a website where he aggregated some data he scrapped from a public server. The data was raw and unfiltered. He painstakingly went through hundreds of CSV files, cleaned up the data, then uploaded it in a database.

Now he wanted to present it as a website where people could query the data. The only issue was he wasn't a programmer. He worked in marketing. He built the website by himself by following tutorials from all over the web. For someone who was not technical, he was pretty good. But how was he supposed to let people search through the database? He built a prototype then sent it to me to help fix it.

What I saw was a text box where the customer could write in plain English whatever data they wanted to get back. For example they could write: “Give me the products that were added between January and March.” I wrote that exact query and to my surprise it worked. I instantly became jealous and insecure about my own abilities. “How did he do it?” He worked in marketing, he had little programming insight. How did he write a sophisticated natural language processor? My many years of experience were no match to him. But then I looked at the code that powered it and my ego was appeased.

In the code he had some specifically hard coded conditions that allowed it to work. For example, the string had to start with “Give me,” or “Show me.”

if (strpos($text, "Give me") !== 0 || strpos($text, "Show me") !== 0) {
    return false;
}

Then he looked for the word “product”, followed by “between”, then an array of months. The year was always set to the current. In fact, the tool was not sophisticated at all. Anything out of the ordinary would break it. For example, if “Give me” was not capitalized, the script would fail. If you added an extra space, it would fail. If you reworded the sentence, it would fail. It wasn't an NLP; it was a bunch of nested conditions. There was no way I could help him debug his script. Instead, I proposed something less cool.

“But I want the user to feel free to type whatever they want. They don't have to be experts in order to query the data.” He told me.

Sure, NLP is great. But it is an order of magnitude more difficult to build. Note that spaCy was a few years away from being released. Even so, NLP would have been overkill for what he was trying to do. My solution was to use boring old filter drop downs.

When the user visits your website, they usually have a pretty good idea what they are looking for. Especially when it's a niche website. It may seem like they want to have a conversation with an AI driven chatbot, but the reality is that they have a problem and you have promised to solve it. Whether you use a checklist, an AI or a mind reading device, the result should be the same.

So instead of a free form text box, we ended up with 3 or 4 dynamic drop downs that allowed users to select the product type and the date range. It wasn't sexy, It wasn't complex, and it did not help him get VC funding. But it was simple and practical. That's all that mattered.

It's always fun to get lost in technology and build things in the most complex way possible. But if you actually want to build a tool that is useful from the very beginning, it's best to avoid being fancy.

---