MIT Disavows Ph.D Research Claiming AI Productivity Benefits

This does not necessarily mean that AI does not now - or will not in the future - deliver productivity benefits, but it does suggest that, in this instance, they are not yet provable. 

The larger implication is that anxious attempts to prove AI benefits at this time may be premature so that doing so raises more concerns, not less, and might ultimately set back adoption of the technology seen as too aggressively trying to market itself in order to increase corporate customer uptake, which has lagged expectations due to costs, reliability and impact. JL

Anthony Ha reports in Tech Crunch:

The paper in question was written by a doctoral student in the university’s economics program. It claimed to show that introduction of an AI tool into a large materials science lab led to the discovery of more materials and more patent filings, but at the cost of reducing researchers’ satisfaction with their work. Economists say they now have “no confidence in the provenance, reliability or validity of the data and in the veracity of the research.” MIT says that due to concerns about the “integrity” of the paper about the effects of artificial intelligence on research and innovation, the paper should be “withdrawn from public discourse.”

MIT says that due to concerns about the “integrity” of a high-profile paper about the effects of artificial intelligence on research and innovation, the paper should be “withdrawn from public discourse.”

The paper in question, “Artificial Intelligence, Scientific Discovery, and Product Innovation,” was written by a doctoral student in the university’s economics program. It claimed to show that the introduction of an AI tool into a large-but-unidentified materials science lab led to the discovery of more materials and more patent filings, but at the cost of reducing researchers’ satisfaction with their work.

 

MIT economists Daron Acemoglu (who recently won the Nobel Prize) and David Autor both praised the paper last year, with Autor telling the Wall Street Journal he was “floored.” In a statement included in MIT’s announcement on Friday, Acemoglu and Autor described the paper as “already known and discussed extensively in the literature on AI and science, even though it has not been published in any refereed journal.”

However, the two economists said they now have “no confidence in the provenance, reliability or validity of the data and in the veracity of the research.”

According to the WSJ, a computer scientist with experience in materials science approached Acemoglu and Autor with concerns in January. They brought those concerns to MIT, leading to an internal review.

MIT says that due to student privacy laws, it cannot disclose the results of that review, but the paper’s author is “no longer at MIT.” And while the university’s announcement does not name the author, both a preprint version of the paper and the initial press coverage identify him as Aidan Toner-Rodgers. (TechCrunch has reached out to Toner-Rodgers for comment.)

MIT also says it has requested the paper be withdrawn from The Quarterly Journal of Economics, where it was submitted for publication, and from the preprint website arXiv. Apparently, only a paper’s authors are supposed to submit arXiv withdrawal requests, but MIT says “to date, the author has not done so.”