AI Drug Research Algorithm Flips to Invent 40,000 Biochemical Weapons

Artificial intelligence (AI) is frequently praised for its potential in medicine and healthcare, especially in drug research. However, a concerning new study suggests it could also become a threat. Researchers from Collaborations Pharmaceuticals, a drug discovery firm based in North Carolina, have released findings that illustrate the alarming capabilities of AI and machine learning in the creation of biochemical weapons.

By modifying a machine learning model known as MegaSyn to prioritize predictions of toxicity, researchers generated an astounding 40,000 biochemical weapons within just six hours. Disturbingly, they admitted that the risks associated with misuse had never been fully considered during the design phase of their molecules. According to the researchers, “The thought had never previously struck us. We were vaguely aware of security concerns surrounding work with pathogens or toxic chemicals, but that did not relate to us; we primarily operate in a virtual setting.”

Their focus has traditionally been on developing machine learning models to design new molecules aimed at improving human health. Surprisingly, even previous work on dangerous substances like Ebola and neurotoxins did not raise alarms regarding the potential hazards of their AI model’s capabilities.

The barriers to exploiting machine learning models such as MegaSyn for designing harmful compounds are lower than one might expect. Many open-source software tools with similar functionalities are publicly available, and the datasets used for training such models can also be accessed freely. Remarkably, the 40,000 toxins were generated using a 2015 Apple Mac laptop, with some identified as being more lethal than the notorious nerve agent VX. VX is infamous for its use in chemical warfare and works similarly to the Novichok agent implicated in the 2018 Salisbury poisonings.

It is important to note that while the generation of these potential bioweapons is relatively straightforward, synthesizing them poses a considerably larger challenge, as the necessary materials for creating VX are strictly controlled. Risks could arise if a toxin were to be discovered that did not require any regulated substances, though the researchers expressed discomfort with that potential direction.

Prior to publishing their results, Collaborations Pharmaceuticals shared their findings at the Spiez Laboratory, which is one of only five facilities worldwide certified by the Organisation for the Prohibition of Chemical Weapons (OPCW). These findings underscore the pressing need for oversight of AI models and a thorough evaluation of their implications in complex applications.

If you’re interested in expanding your knowledge of AI and big data from industry leaders, consider attending the AI & Big Data Expo. Upcoming events in the series are scheduled for Santa Clara on May 11-12, Amsterdam on September 20-21, and London on December 1-2.