Report: IBM Watson AI Recommended Unsafe Cancer Treatments
napocska/Shutterstock.com
The system was trained using hypothetical scenarios.
IBM has put its Watson artificial intelligence system to work on a variety of problems, including caring for cancer patients.
Watson's oncology system has helped care for more 84,000 patients around the world, and while the tech company says the program is a success, a Stat News report said internal documents tell a different story.
The publication reviewed the documents from 2017 in which medical experts discuss "multiple examples of unsafe and incorrect treatment recommendations."
In one example, a patient was recommended a drug that could lead to severe or fatal hemorrhage while he was already dealing with severe bleeding due to his condition. In another example, a Florida doctor who reviewed the system told the company that the technology is "a piece of shit."
IBM engineers and doctors who helped train the AI provided Watson with data about hypothetical patients and treatments instead of data from real, living patients, the report says.
The company plans to take client feedback into consideration and plans to update the system.
"We have learned and improved Watson Health based on continuous feedback from clients, new scientific evidence, and new cancers and treatment alternatives,” an IBM spokesperson told Gizmodo.
Training AI is a major concern for other organizations as well. In the intelligence community, the Intelligence Advanced Research Project Activity has examined ways to improve the data sets, virtual environments and other training materials to help evolve AI algorithms.
"Many additional artificial intelligence problems may be solvable in the near-term, without significant innovations in the underlying algorithms, if the right training resources become widely available,” IARPA said in 2016.
If problems persist, organizations are sometimes forced to retrain AI with new data sets.
IBM Watson for Genomics Technology has recently extended its partnership with the Veteran Affairs Department. The partnership has run since 2016, with Watson analyzing the DNA of veterans with Stage 4 cancer. The partnership will continue until 2019.