Analysis of approximately 75 million publications finds those employing AI are more likely to be a ‘hit paper’

From designing new drug candidates in medicine to drafting new taxation policies in social sciences, the benefits of artificial intelligence (AI) in scientific research are all around.

Just this week, two scientists known for their pioneering AI research earned the Nobel Prize in Physics, and a trio of scientists earned the Nobel Prize in Chemistry, which recognized the use of advanced technology, including AI, to predict the shape of proteins. Despite its rapid progress and broad applications, however, many researchers lack a systematic understanding of how AI may benefit their research, and skepticism remains about whether AI is capable of advancing science in every field.

A new Northwestern University study analyzing 74.6 million publications, 7.1 million patents and 4.2 million university course syllabi finds papers that employ AI exhibit a “citation impact premium.” However, the benefits of AI do not extend equitably to women and minority researchers, and, as AI plays more important roles in accelerating science, it may exacerbate existing disparities in science, with implications for building a diverse, equitable and inclusive research workforce.

The research team, led by the Kellogg School of Management’s Dashun Wang and Jian Gao, developed a measurement framework to estimate the direct use and potential benefits of AI in scientific research by applying natural language processing (NLP) techniques to these vast datasets.

Wang is a professor of management and organizations at Kellogg and of industrial engineering and management sciences at McCormick, director of Kellogg’s Center for Science of Science and Innovation (CSSI) and co-director of Kellogg’s Ryan Institute on Complexity. Gao is a research assistant professor at Kellogg CSSI.

The study, “Quantifying the Use and Potential Benefits of Artificial Intelligence in Scientific Research,” was published October 11 in the journal Nature Human Behaviour.

“These advances raise the possibility that, as AI continues to improve in accuracy, robustness and reach, it may bring even more meaningful benefits to science, propelling scientific progress across a wide range of research areas while significantly augmenting researchers’ innovation capabilities,” Gao said.

Most impactful research

The study found that the recent successes of AI, across fields, has been remarkable for research. There has been a growing use of AI in disciplinary research since 2015, proxied by the mention of AI-related terms (such as “artificial intelligence,” “deep learning” and “convolutional neural network”) in the title or abstract of publications.

From 2015 to 2019, disciplines including computer science (37%), engineering (24%), physics (24%), biology (22%), psychology (24%), economics (14%), sociology (30%) and political science (27%) have all shown notably sharp increases in direct AI use scores due to the development of new AI capabilities.

Researchers examine the number of times a paper is cited, and they define a “hit paper” as being in the top 5% by citations for papers published in the same field and year. Regardless of discipline, disciplinary papers that mention AI-related terms in their title or abstract receive more citations, being more likely to be a hit, and receive a higher fraction of citations from other disciplines.

“In addition to its expansion, the use and benefits of AI in research is pervasive across disciplines, but we found a systemic misalignment in AI education,” Gao said. “The investment in AI in higher education is not at the same pace of the AI benefit in science.”

These results suggest that the supply of AI talent and knowledge in most disciplines appears inadequate with the benefits these disciplines may extract from AI capabilities, highlighting a substantial AI use–AI training gap.

“The use of AI in scientific disciplines has raced ahead across science, while the educational focus on AI to upskill future scientists within each discipline has lagged,” Gao said.

Underrepresented groups in STEM

The study also highlights the unequal effects on women and minority researchers that the steadfast rise of AI use in scientific research may bring.

“Historically, we know that women and minorities are less represented in some fields, especially in STEM,” Gao said. “We found that as the AI use in science continues to grow, those same groups are less likely to benefit from the new technologies.”

Researchers suggest that an investment in making sure the training behind AI is equitable may have a positive impact on closing the demographic gap.

What’s next?

As AI rapidly evolves, the researchers said we need to continuously monitor and update its benefit to science.

“Women and minorities are benefiting the least, so how do we mitigate these disparities along demographic lines?” Gao said.

The research team’s analysis supports the hypothesis that collaboration between domain experts and AI researchers may represent a meaningful way to facilitate AI use across science and fill the AI use–AI training gap.

“There’s a benefit to increasing AI training across disciplines, which would likely help the disciplines to develop domain-specific AI expertise, allowing them to enjoy greater and timelier benefits from AI advances,” Gao said.