It hasn't worked out that way. And the reason may be not just a paucity of the data AI requires, but skepticism about its efficacy from the humans attempting to use it.
The challenge may be as much psychological and organizational as technological. JL
Daniela Hernandez and Ted Greenwald report in the Wall Street Journal:
Artificial intelligence has the potential to reinvent the world, from how businesses operate to the types of jobs people hold to the way wars are fought. In health care, AI promises to help doctors diagnose and treat diseases. Watson’s struggles suggest that revolution remains some way off. In many cases, the tools didn’t add much value. In some cases, Watson wasn’t accurate. Watson can be tripped up by a lack of data. Six years and billions of dollars later more than a dozen IBM partners and clients have halted or shrunk Watson projects.
Can Watson cure cancer?
That’s what International Business Machines Corp. IBM -1.23% asked soon after its artificial-intelligence system beat humans at the quiz show “Jeopardy!” in 2011. Watson could read documents quickly and find patterns in data. Could it match patient information with the latest in medical studies to deliver personalized treatment recommendations?
“Watson represents a technology breakthrough that can help physicians improve patient outcomes,” said Herbert Chase, a professor of biomedical informatics at Columbia University, in a 2012 IBM press release.
Six years and billions of dollars later, the diagnosis for Watson is gloomy.
More than a dozen IBM partners and clients have halted or shrunk Watson’s oncology-related projects. Watson cancer applications have had limited impact on patients, according to dozens of interviews with medical centers, companies and doctors who have used it, as well as documents reviewed by The Wall Street Journal.
In many cases, the tools didn’t add much value. In some cases, Watson wasn’t accurate. Watson can be tripped up by a lack of data in rare or recurring cancers, and treatments are evolving faster than Watson’s human trainers can update the system. Dr. Chase of Columbia said he withdrew as an adviser after he grew disappointed in IBM’s direction for marketing the technology.
No published research shows Watson improving patient outcomes.
Artificial intelligence has the potential to reinvent the world, from how businesses operate to the types of jobs people hold to the way wars are fought. In health care, AI promises to help doctors diagnose and treat diseases as well as help people track their own wellness and monitor chronic conditions. Watson’s struggles suggest that revolution remains some way off.
IBM said Watson has important cancer-care benefits, like helping doctors keep up with medical knowledge. “This is making a difference,” said John Kelly, IBM senior vice president. “The data says and is validating that we’re on the right track.”
In health care, AI software is starting to help radiologists and pathologists analyze X-rays and digital images of biopsies. Companies and clinicians are developing simple text-based chatbots that aim to help people manage mental-health issues like anxiety and depression in therapy-like conversations.
Recommending personal medical treatment is a taller order. The software needs to be trained with data on what has worked in the past, including details on patients’ medical histories and treatment outcomes. That information is often recorded in different formats and owned by different companies, and isn’t always complete or consistent.
Moreover, human doctors still have a lot to learn about the science of disease, including cancer.
Oncology won’t be “a great space for making [AI] products” until there’s better data about patients, spanning genetic, environmental, lifestyle and health information, said Bob Kocher, a medical doctor and partner at venture-capital firm Venrock in Palo Alto, Calif. In the near term, most of the benefits from AI in the health-care field will come in administrative tasks such as billing, he added.
IBM promotes Watson—which powers AI applications such as online customer-service assistants to a range of industries—as a symbol of its technological prowess and the engine of the company’s growth. IBM spent $15 billion on Watson and related efforts as of 2015, according to that year’s annual report.
Big Blue has been banking on artificial intelligence to help drive its transformation from building and maintaining information technology systems on customers’ premises to providing newer services based on cloud computing. IBM’s total revenue has fallen every year since 2012, though it has ticked up in the company’s three most recent fiscal quarters compared with a year earlier, thanks largely to sales of mainframe computers.
Sales in the cognitive solutions division, which includes Watson, grew just 1% in 2017 after adjusting for currency changes. In its latest quarterly earnings report, the division’s sales fell 1.3% to $4.58 billion after adjusting for currency changes.
Health care is IBM’s most important target for AI, said Mr. Kelly. In 2017, he told the Journal the company had “bet the ranch” on it. The company has spent almost $5 billion on acquisitions since 2015 to feed health-related expertise into Watson, including Explorys Inc. for electronic health records and Truven Health Analytics Inc. for insurance claims.
Giant rivals including Alphabet Inc.’s Google, Amazon.com Inc., Microsoft Corp. and Philips NV have also entered the market, along with health-care-focused startups, medical-record companies and insurers.
Watson’s health products include non-cancer offerings such as workflow-management systems for health-care providers and wellness apps. Cancer-related products have “touched” roughly 84,000 patients and have been used by 230 institutions, Mr. Kelly said in a recent interview. He wouldn’t disclose revenue from oncology-related offerings but said it has been doubling annually in recent years.
In June 2017, Ginni Rometty, IBM’s chief executive, president and chairwoman, told CNBC that Watson will be able to diagnose and treat most of “what causes 80% of the cancer in the world.”
The largest AI product in the Watson Health portfolio is Watson for Oncology, for which IBM typically charges $200 to $1,000 per patient, plus consulting fees in some cases, according to Mr. Kelly.
After a doctor enters information about a patient’s medical status, the application recommends treatments by analyzing published research that might be relevant.
New York’s Memorial Sloan Kettering Cancer Center has been helping IBM train the software since 2012. (It doesn’t use the software for patient care.) The hospital’s specialists work with IBM engineers to rank relevant features of medical histories like tumor location and coexisting conditions. They also rank medical studies of a given therapy. Then they evaluate Watson’s ability to match test cases with treatments and help the engineers tune the output until it agrees with the doctors’ judgment.
“It’s still a work in progress,” said oncologist Mark Kris, who coordinates Watson’s cancer training. He said keeping pace with rapidly evolving cancer treatments has proved to be tougher than he imagined.
IBM’s Mr. Kelly agreed it was difficult “for anything to keep up” with the pace of discovery but said that machines will be able to learn faster than humans.
Dr. Kris still believes in the technology. “Is it as nuanced as we’d like? Is it 100% up-to-date? No,” he said. “But for what it is, it’s pretty darned good.”
IBM initially planned to meld medical-claims databases and electronic health records, among other data troves, from companies it bought into one cohesive system that would make Watson’s health-care products smarter. It never did, in part because it underestimated the complexity, according to a former employee and a hospital executive who was a former client.
Mr. Kelly said IBM was actively working on the integration. Among patients who went through the system, he said, Watson’s treatment recommendations changed the course of care in 2% to 10% of cases—or roughly 1,680 to 8,400 patients so far.
Jupiter Medical Center in Jupiter, Fla., began using Watson for Oncology in clinical care for lung, breast and gynecological cancer patients in March 2017. Today, Watson provides an additional voice on a cancer-care committee that evaluates 10 to 15 cases a week, said K. Adam Lee, head of the thoracic-surgery unit.
Humans generally agree with Watson’s recommendations, so it rarely affects treatment, he said, but it has helped to attract patients to the center. He declined to disclose what Jupiter pays IBM.
Watson for Oncology has made the most headway in Asia—a less lucrative health-care market than the U.S. In India, Manipal Hospitals, a network of 15 facilities, adopted the product in 2016 for a multimillion-dollar fee over five years, according to Ajay Bakshi, Manipal’s chief executive from 2014 to 2017, who now heads a Singapore-based hospital network.
At first, Manipal used Watson to recommend treatment options for all cancer patients, said oncologist S.P. Somashekhar. It found the software agreed with doctors most of the time, so Manipal stopped using Watson on every patient, he said. It now uses Watson only in difficult cases, or roughly 30% of patients. Watson’s recommendations influence care in 9% of those cases, he said.
IBM said the number of Manipal patients using Watson has remained steady since January 2017.
Other key cancer applications—Watson for Genomics and Watson Clinical Trial Matching—aim to pair the genomic data of patients’ tumors to cancer drugs or drug trials, among other functions.
Watson for Genomics has been piloted at multiple cancer centers in the U.S. Doctors at several of these centers said results weren’t always accurate, and when they were, they often provided information oncologists already knew.
“The discomfort that I have—and that others have had with using it—has been the sense that you never know what you’re really going to get…and how much faith you can put in those results,” said Lukas Wartman of the McDonnell Genome Institute at the Washington University School of Medicine in St. Louis. Dr. Wartman said he rarely uses the system, despite having complimentary access.
IBM said it has gotten good feedback about the product from clients and research partners, and that its product provides supporting evidence for the recommendations it makes. Mr. Kelly said he was glad Watson agreed with physicians most of the time.
Dr. Wartman’s organization was one of 14 IBM partners announced in May 2015, with a goal of enabling clinicians to use Watson for Genomics on a broad range of patients by the end of that year. The system was not in clinical use at the 11 centers the Journal reached for comment, including Cleveland Clinic, Yale Cancer Center, the University of Southern California’s Center for Applied Molecular Medicine and the University of North Carolina Lineberger Comprehensive Cancer Center.
Perhaps the largest clinical user of Watson for Genomics has been the U.S. Department of Veterans Affairs.
In June 2016, IBM formed a partnership with the VA with the goal of using the product on 10,000 veterans within two years. So far, doctors there have used it on more than 3,000 patients, according to Michael Kelley, a VA oncologist.
To use Watson for Genomics, the VA sends tumor samples to a third-party vendor that does a genetic analysis. Dr. Kelley uploads the results, plus the cancer type, to Watson, which returns a ranked list of possible treatments based on studies describing what has worked on cases with similar genetics.
Dr. Kelley said Watson’s recommendations can be wrong, even for tried-and-true treatments. On the other hand, he said, it is fast and useful at finding relevant medical articles, saving time and sometimes surfacing information doctors aren’t aware of.
VA doctors don’t discuss Watson’s suggestions with patients. Patrick McGuire, a patient of Dr. Kelley’s, was unaware it recommended an immunotherapy for his cancer type until recently. The 45-year-old retired Navy veteran, who lives in Kenly, N.C., said he was happy AI played a role in his current treatment, which seems to be working although it hasn’t completely shrunk his tumor. He said his doctors will re-evaluate its effectiveness in a couple of months.
In another instance, Watson identified a therapy for a type of gastric cancer associated with a certain genetic alteration and cited a paper Dr. Kelley said he didn’t think he would have found on his own. He sent that information to the patient’s physician and the patient was treated accordingly. The patient didn’t respond to the medication, he said.
The unpaid project, originally slated to conclude in July, will continue for another year. The VA is also testing the reliability of other similar services.
“There’s a lot of promise for AI,” Dr. Kelley said, but for now, “that promise is not realized. "
0 comments:
Post a Comment