When I stepped up to lead the team at I.B.M. that would create this computer, called Watson, I knew the task would be formidable. The computer would have to answer an unpredictable variety of complex questions with confidence, precision and speed. And we would put it to the test in a publicly televised “human versus machine” competition against the best players of all time.
It was not easy finding people to join the Watson team in the mid-1990s. Most scientists I approached favored their own individual projects and career tracks. And who could blame them? This was an effort that, at best, would mingle the contributions of many. At its worst it would fail miserably, undermining the credibility of all involved.
Scientists, by their nature, can be solitary creatures conditioned to work and publish independently to build their reputations. While collaboration drives just about all scientific research, the idea of “publishing or perishing” under one’s own name is alive and well.
I remember asking some researchers how long they had been working in natural language processing — the field of computer science focused on getting computers to interact in ordinary human language. For many, it had been well over a decade.
I asked them if they preferred spending the next 10 years as they had the first 10, publishing isolated research results and earning modest acclaim within a niche community. Or would they like to see whether the technology that had been their life’s work could accomplish something monumental?
For the scientist in me, it was an irresistible challenge. I believed it was a rare opportunity to counter conventional wisdom and advance technology. I was willing to live with possible failure as a downside, but was the team?
A few people were extremely hesitant to join the project and later left, thinking that the whole enterprise was insane. But a majority bought in. We eventually pulled together a core group of 12 talented scientists, which over time grew to 25 members. It was a proud moment, frankly, just to have the courage as a team to move forward.
From the first, it was clear that we would have to change the culture of how scientists work. Watson was destined to be a hybrid system. It required experts in diverse disciplines: computational linguistics, natural language processing, machine learning, information retrieval and game theory, to name a few.
Likewise, the scientists would have to reject an ego-driven perspective and embrace the distributed intelligence that the project demanded. Some were still looking for that silver bullet that they might find all by themselves. But that represented the antithesis of how we would ultimately succeed. We learned to depend on a philosophy that embraced multiple tracks, each contributing relatively small increments to the success of the project.
Technical philosophy was important, but so were personal dynamics. Early on, I made the unpopular decision to bring the entire team together in a war room, to maximize communication. The shared space encouraged people with wildly different skills and opinions to exchange ideas.
The early practice rounds for “Jeopardy” were downright disappointing. Many of Watson’s answers were stupid and irrelevant, some laughably so. Each wrong answer demonstrated the profound failings of simple search-based technologies and showed how sophisticated Watson needed to become.
We had to keep the team’s collective intelligence from being overcome by egos, or dragged down by desperation. Leadership had to be steadfast and persistent but grounded in optimism. Through it all, the team developed a culture of trust that let creativity flourish.
IN the end, the hero was the team, not any individual member or algorithm. Eventually, everyone came to appreciate that. Well into the throes of the project, one researcher commented, “Compared to the way we work now, it’s like we were standing still before.”
Watson went on to win “Jeopardy” a year ago, but its work is far from over. Now we and other research and development teams at I.B.M. are busy developing ways to put Watson to work in several different areas, most notably health care.
As for the members of the original Watson team, they’d tell you that never in a million years could they have imagined what we accomplished. Just like Watson itself, we all learned that the sum is much greater than the parts.
E-mail: preoccupations@nytimes.com.
Article source: http://feeds.nytimes.com/click.phdo?i=59572eaa64d081ea9390e633739aa22a
Speak Your Mind
You must be logged in to post a comment.