Thursday, November 19, 2015

IBM WATSON: Part 1

Hello there and welcome to my first blog post, in a new series, highlighting the amazing power of the Supercomputer: Watson!

First of all, what is Watson?

Watson, simply put, is an intelligent, NLQA (Natural Language Question Answering) system developed by IBM. It has the capability to answer questions posed to it in natural language. Watson has enough cognitive ability to learn about any topic, with a "corpus" of knowledge fed into Watson by humans. Such corpora include Healthcare, Travel and Cooking - its current 3 main uses. Watson is also continuously trained by professionals in the area that he is trying to learn.

Examples of an NLQA system could be Siri, Google Now, Sirius, or Cortana to name of a few.

However,  Google, Bing, Yahoo, DuckDuckGo don't fall into this category.

The difference is that NLQA systems actually extract the answers and give them to the user. However, search engines just give back results that may hold the answers, and humans have to dig into the results and find the answers themselves. Also, if you, in general, don't know what "Natural Language" is, it's what we're using right now, to talk and to communicate.

Also, in the past, Watson had an extremely difficult test, to see if it actually has the power to be the most advanced NLQA system out there.

Watson played a game of "Jeopardy!", a game show where contestants are shown clues, and they have to come up with questions as answers. Watson played against the game show's two best contestants: "Ken Jennings" and "Brad Rutter" (imagine ANY other NLQA system even trying to do that).
To everyones surprise, Watson won that game with $1,000,000.

Watson's big win actually meant and indicated that it should be put to an appropriate use elsewhere and therefore IBM decided to put it to use in Healthcare, Travel and Cooking at the moment.

You might want to know: The project manager for Watson is "David Ferrucci"
You might also want to know: Watson was built on IBM's "POWER7" Supercomputer.

Watson also has the capability to "learn", using algorithms classified under "Machine Learning". Now, what this allows Watson to do, is to make sure that it never repeats a mistake. I will take an example here. Young kids have an urge to draw on the wall. When they do it, parents usually scold them. Over time the kids link drawing on the wall with a negative outcome of their actions. Their brains don’t want the negative outcome, so they learn not to draw on the wall . Similarly Watson links mistakes with a negative outcome so, it'll learn not to make mistakes. 

Furthermore, Watson works by using these four steps, to determine the answer to a question:

1. Question Analysis - In this step, Watson will try to figure out what the question is asking for, and what the answer type (should) be.

2. Hypothesis Generation - Here, Watson creates hundreds of different possible candidate answers. These are mostly answers that are wrong by a LOT, but again, it NEEDS a lot of answers to choose from. Because later, Watson will have an extreme variety of answers to choose from, and will probably score the correct answer as correct.

3. Hypothesis and Evidence Scoring - Now, Watson will try to weigh each answer. It will downgrade wrong answers, by looking at the evidence that does NOT support the hypothesis, and upgrade correct answers, with evidence that DOES support the hypothesis.

4. Final Merging and Ranking - Finally, Watson ranks all the candidate answers, and displays the top 3 answers on the answer panel. It gives confidence scores for each candidate answer and says out the final, first ranked answer, in hopes that it is right.

This blog post was an introduction to the IBM Watson NLQA system.

I will continue to write and take you into depth on how to use Watson, and also release a few articles on how to use Watson APIs as well.

My due credits and thanks to IBMWatson.com

10 comments:

  1. I really enjoyed reading your blog, Tanmay! I look forward to reading more of your insights on Watson, and how to use the APIs.

    ReplyDelete
  2. Good intro..neatly written.helps som done to understand quickly what Watson is about

    ReplyDelete
  3. Really great ...a layman can understand as well

    ReplyDelete
  4. Awesome!! Super excited to see your demo tomorrow at the IBM Dev Connect event in Bangalore tomorrow. Keep Coding!! Also, how do I subscribe to your blog so I get automatic feeds in my email whenever you publish a new post? I would love to follow your cognitive journey.

    ReplyDelete
  5. Nice compilation of information on IBM watson. I will share this with my students.

    ReplyDelete
  6. Hi Tanmay,
    Your blogs and videos are amazing! I'd like to know why did you choose Watson over Api.ai/wit.ai as these platforms also help you build conversational bots.

    ReplyDelete