Jeopardy!?? Elementary my dear Watson.

[sclitheroe]

Originally Posted by Bring_Back_Shantz

Where we've always had a huge advantage is being able to hear someone say something like "Hey what's the name of that Beatles song where they sing about all the stuff that guy should or souldn't do" and coming up with "Hey Jude". Those are the types of things it would seem we're on the brink of being able to ask a computer.

You can already do this. Plug "Hey what's the name of that Beatles song where they sing about all the stuff that guy should or shouldn't do" into Google, and its the second link down. Google found it in 0.46 seconds, roughly the reaction time of a human Jeopardy contestant.

[MarchHare]

Sorry. Watson is all about great internal architecture (plumbing) that lets it work in speeds approaching real-time on truly staggering data sets. The algorithms its running to compete at Jeopardy are so specialized and tweaked for the game it plays that Watson is useless outside that arena.

Otherwise, we could just keep feeding it data and asking questions like you proposed. Not gonna happen.

IBM spent millions of dollars over four years and dedicated some of their top computer scientists to this project. They didn't make that kind of investment just for the PR of appearing on a prime time game show. The project lead specifically cited the computer from Star Trek as his inspiration; he wants to build a computer to which a human can pose a question in natural language, have the computer quickly search its massive databases, and then return the answer in natural language. Helping doctors diagnose conditions by parsing through all the available literature was an example he used as a practical application of this technology.

[Bring_Back_Shantz]

Originally Posted by sclitheroe

Hahaha... As if they'd waste a breakthrough of that magnitude on a game show and not finding Osama.


Sorry. Watson is all about great internal architecture (plumbing) that lets it work in speeds approaching real-time on truly staggering data sets. The algorithms its running to compete at Jeopardy are so specialized and tweaked for the game it plays that Watson is useless outside that arena.

Otherwise, we could just keep feeding it data and asking questions like you proposed. Not gonna happen.

Just because something is specialized doesn't mean it isn't a huge stepping stone to greater achievements.

The first space capsules were pretty specalized, but they were a massive leap towards putting a man on the moon.

Watson was built to play Jeopardy, but what the people who built him learned about how to build a computer to interact and work in that way is the first step to computers that can do much more.

[Bring_Back_Shantz]

Originally Posted by sclitheroe

You can already do this. Plug "Hey what's the name of that Beatles song where they sing about all the stuff that guy should or shouldn't do" into Google, and its the second link down. Google found it in 0.46 seconds, roughly the reaction time of a human Jeopardy contestant.

Fair enough, but that's now how the question was actually asked on Jeopardy.
Try getting google to get he right answers using puns, or clues instead of search parameters and it won't fare so well.

[Cecil Terwilliger]

Originally Posted by Burninator

You are giving far too little credit to the computer figuring out the actual answers to the questions. Buzzer speed is nothing if it doesn't know the answer. That's the hard part, not the buzzer.

No I'm not.

The amazingness of the computer figuring out Jeopardy answers has absolutely nothing to do with the ridiculous buzzer advantage.

These aren't mutually exclusive. Just because I don't think the buzzer thing is fair, doesn't mean I'm somehow unimpressed with the computer's ability to answer questions.

Frankly it boggles my mind how you even came to that conclusion. They have nothing to do with each other.

Originally Posted by MarchHare

Saw this on another forum and thought it was apt:

Complaining that Watson has a buzzer advantage in Jeopardy is like complaining that a giraffe playing in an NBA game has a height advantage. You should be amazed that a giraffe is able to play basketball at a competitive level against top human players at all, not that it has a physical advantage!

Same thing. Who said I wasn't impressed?

It doesn't make Watson's buzzer advantage any less true or any less prohobative to a fair game.

[sclitheroe]

Originally Posted by MarchHare

IBM spent millions of dollars over four years and dedicated some of their top computer scientists to this project. They didn't make that kind of investment just for the PR of appearing on a prime time game show.

IBM's R&D budget is in the 6 billion dollar area. You are right - they didn't spend millions of dollars on this just for PR, they also did it because its as cost effective as any other national ad campaign.

What Watson does is nothing new - question answering (QA) systems are out there and in use now. Take a look at Wolfram Alpha as an example.

I stand by what I said - all IBM demonstrated was they could build a really big, really high performing one using commodity hardware. It's the plumbing that is the news here - that systems this large and parallel are feasible using current tech. That's the sales pitch for these other systems you are talking about, like medical.

Edit: I'm no disparaging what the Watson team accomplished. Just saying its not as revolutionary as some here are claiming. It's just the biggest public example of a QA system most people have seen.

[Dan02]

Originally Posted by Bring_Back_Shantz

That's a huge benefit. For example, now instead of referencing a limited number of resources about a few symptions, a doctor can ask a computer "Hey, what kind of disease would cause someone to XXXX with spots on their XXXX" and the computer can reference ALL the literature in the world and come to a conclusion that the doctor can then use as a starting point.

What stopping a computer now adays from doing that? the breakthrough here is Watsons language abilities not it's search abilities. Im no medical professional but i would hope there is already a system in place where you can input symptoms and it would return a list of most probable matches.

[Dan02]

Originally Posted by J pold

But I agree that arguing about Watson's buzz speed is a moot point, what it can do is amazing.

I think the arguing about buzz speed has more to do with the integrity of the "Jeopardy" game then anything to do with Watson.

They said it was taking Watson 2.6 seconds to come up with the answer on average after it finished reading the question. If it took Ken or Brad that long they wouldn't have won a single game yet alone be the winningest players ever. Watsons got the advantage of being able to process the entire question before Ken or Brad could even recognize the first letter of the first word. Furthermore it then got the advantage of having nearly perfect timing with the buzzer.

remove those 2 advantages and although Watson would have eventually reached the right answers, it would have taken him long enough that he would have gotten white washed.

[Mazrim]

Originally Posted by sclitheroe

You can already do this. Plug "Hey what's the name of that Beatles song where they sing about all the stuff that guy should or shouldn't do" into Google, and its the second link down. Google found it in 0.46 seconds, roughly the reaction time of a human Jeopardy contestant.

It wasn't the second link down for me. It wasn't even on the results page. The first result is this thread!

[SebC]

Originally Posted by sclitheroe

Watson is good at parsing natural language and looking up facts contextually, but it doesn't infer, deduce, or synthesize information it doesn't have. Have you ever correctly answered a question on a multiple choice test by looking at your answer to previous questions and deducing the correct answer? Watson can't do that.

If you programmed all the questions into a computer, it would solve the self-referential aptitude test in no time. That's easy.

[Kerplunk]

While not new, I haven't seen a natural language parser and search engine that grabs relevant data to the degree of accuracy that Watson did. Natural language processing is a bitch, and hopefully this will push the field to develop even better parsers for everyday use.

[sclitheroe]

Originally Posted by Bring_Back_Shantz

Fair enough, but that's now how the question was actually asked on Jeopardy.
Try getting google to get he right answers using puns, or clues instead of search parameters and it won't fare so well.

It was a self-referential joke - the answer to your hypothetical question is right on page one of google because it points to this thread

[Cecil Terwilliger]

So I had a chance to watch all 3 on my PVR. I had seen most of the 2nd episode when it aired on Tuesday.

First, because people seem to be pretty defensive about this, I want to make it very, very clear that I am as amazed and impressed as anyone else about what IBM has been able to do here.

Watson is not connected to the internet and the fact that he can distinguish all the oddities of the english language and find the answers is nothing short of amazing. It was cool to see the behinds the scenes stuff from the first episode because it showed that this was not an easy process and took a long time.

All that being said...

This is nothing short of a, not sure what the right word is here, farce? set-up? hustle?

I'm a massive jeopardy fan and these episodes were almost unwatchable. I like jeopardy because I can play along. With Watson on the show it was almost unwatchable. He's just way, way, way too fast at buzzing in.

What I find a little silly about this and the reason I say farce or set-up is because we all know that a computer has a crazy advantage when it comes to ringing in. Computers can transmit way faster than a human. While it takes x number of seconds to ring in for a human, a computer can do it in milliseconds.

And I get that that isn't the impressive part but that is totally moot.

IBM knew they had this massive advantage of buzzer speed. The huge disadvantage was getting a computer to be able to answer Jeopardy questions.

What I don't like and what made the show totally pointless IMO is that they obviously did enough testing before the show to know that Watson was going to absolutely destroy all the competition.

And I totally get Jeopardy had it in their best interests to let Watson mop the floor with Ken and Brad but it took all the fun out of it. Jeopardy got record ratings for their recent shows which is fine and probably good for the show.

However from a Jeopardy lover the initial "wow" factor wore off after about 10 minutes. After it became obvious that the computer could answer Jeopardy questions and that it's buzzer speed was unmatchable, what's the point? I started to skip all the play and just watch the behinds the scenes stuff. Why watch if there is absolutely no doubt who's going to win and especially if everyone already knew the outcome?

It isn't like this was a surprise that Watson would win, at least for everyone involved. The behind the scenes stuff made it obvious that IBM, Jeopardy and Ken/Brad knew full well that Watson was unbeatable.

Personally I can't wait until they go back to regular shows.

[Sidney Crosby's Hat]

Originally Posted by Cecil Terwilliger

It isn't like this was a surprise that Watson would win, at least for everyone involved. The behind the scenes stuff made it obvious that IBM, Jeopardy and Ken/Brad knew full well that Watson was unbeatable.

Personally I can't wait until they go back to regular shows.

If you swap the Final Jeopardy question from the first match with the question from the second match, Ken wins the second match so I wouldn't call Watson unbeatable.

[Cecil Terwilliger]

Originally Posted by Sidney Crosby's Hat

If you swap the Final Jeopardy question from the first match with the question from the second match, Ken wins the second match so I wouldn't call Watson unbeatable.

Yeah ok, I definitely would have been much more interested if it had been closer, I hardly watched the 3rd day. He did beat them like $36k to 10k and 8k or something in the first game though. That was kind of a letdown.

I want to be very clear that I found the concept fascinating. It just didn't hold my interest long because a)it wasn't fun to play along with, half the fun is seeing if I can beet the contestants or whoever I'm watching it with and b)it just didn't seem like much of a contest.

[pylon]

Until it is an android standing up there, reading the questions through camera eyes, and holding a buzzer in a rubber skinned mechanical hand, playing the game like the other 2 contestants.... it's not a fair contest.

To me this was really just two players playing against what will probably be google in the next couple years. It was impressive to an extent. But I guarantee all of you inside of 5 years you will look at this, like we look at bagphones and Tandy 286 computers now.

[corporatejay]

I will be worried when Watson can build another Watson...then we're in trouble.

[worth]

Originally Posted by corporatejay

I will be worried when Watson can build another Watson...then we're in trouble.

This is how you colonize a galaxy!

http://www.daviddarling.info/encyclo...mannprobe.html

[jar_e]

Originally Posted by Dan02

What stopping a computer now adays from doing that? the breakthrough here is Watsons language abilities not it's search abilities. Im no medical professional but i would hope there is already a system in place where you can input symptoms and it would return a list of most probable matches.

Google?

[Bobblehead]

It is the natural language interpretation that is astonishing. Not the fact recall.

The sprinter rand down the track.
The pen ran out of ink.
The stream ran to the river.
The nose ran all day.

English speaking humans easily understand all those. But to program a machine to understand this sort of thing with minimal context is radically difficult. It is why the Turing Test is so often referenced - to get interactions that make sense and appear natural is tough. Watson is another stepping stone on this path.

And this type of research has been going on for years and years and IBM are not alone. Jeopardy is simply a target to allow the team to focus their research and a way to measure their progress.