Machine Amplified Human Intelligence

When we first started work on MAHI, we wanted to build a system that listened to conversations and was able to pull up relevant data. We focused our efforts on making the MAHI engine robust to understand text from speech and remove the dependence on punctuation. The MAHI engine was able to extract knowledge from speech text, examine the topic webs, and then integrate with other API's to give you more info on what you were talking about.

One of the cool things about MAHI was that it picked up on the subtle topics and topic shifts in a piece of text, and it could come up with related topics for it.

Text to Knowledge

MAHI was designed to be able to work even if it only understood a user with 70% accuracy (we used Google speech recognition). MAHI could extract topics from any text, such as a news articles and Tweets, not just speech.

Listening

API's and Data Blocks

After MAHI deciphered topics, it figured out what kind of object the topic was (e.g. restaurant, person, car, etc.) and then had to gather more info to augment the topic for the user.

In order to do this, we integrated MAHI with 13 different API's to let it access a wide array of info about a topic. I created the data blocks on the front end to display info from each API.

The Information Hierarchy

In the web app, I wanted to bring background information about the main topics in your conversation and be able to give the user a feel of what it's like to interact with this web of knowledge. With the MAHI IA, I was trying to create a subtle tree of how the presented info related to a conversation.

The left column constantly updates as you talk with topics that are directly mentioned in your conversation. You can click on one of them to explore it further. This becomes a timeline that you can scroll to see what has been said so far.

In the middle column, you are presented with the data blocks that were gathered for a selected topic. By putting it all in a column, it made it easier to scroll through and find information quickly.

In the right column, you see topics related to the selected primary topic, as well as outward links to the news sites covering it.

Primary Info

Secondary Info

Tertiary Info