Lack of transparency in AI breast cancer screening study ‘could lead to harmful clinical trials’, scientists say

In Clinical Studies News by Barbara Jacoby

By: Rhiannon Williams


A critique has claimed a January study’s lack of detail surrounding its methods and algorithm code undermined its scientific value

Scientists have raised concerns that the secrecy surrounding the methods of a recent study praising artificial intelligence’s (AI) ability to screen for breast cancer could lead to potentially harmful clinical trials.

A group of experts from Stanford University School of Medicine, Harvard Medical School and the US CUNY Graduate School of Public Health and Health Policy among others criticised the study from Imperial College London, Google Health and Google-owned British AI lab DeepMind for its lack of transparency.

The original report, which was published in January, claimed its AI system surpassed human experts in predicting the likelihood of breast cancer developing, which they said paved the way for future clinical trials to improve the accuracy and efficiency of breast cancer screening.

Lack of transparency

The critique, published in journal Nature, claimed the initial study’s lack of detail surrounding its methods and algorithm code undermined its scientific value, limiting the evidence required for others to validate and test the technologies for themselves.

Imperial College London and its tech counterparts had failed to document their research sufficiently, the group wrote, adding that mere textual descriptions of deep-learning AI systems can mask their high level of complexity.

Consequently, reproducing these complicated computational systems based on textual description alone, however detailed, is a “subjective and challenging task,” they said.

“Transparency in the form of the actual computer code used to train a model and arrive at its final set of parameters is essential for research reproducibility,” they wrote.

“If a dataset cannot be shared with the entire scientific community, because of licensing or other insurmountable issues, at a minimum a mechanism should be set so that some highly trained, independent investigators can access the data and verify the analyses.”

Imperial College London has been approached for comment.

‘Promotion of a closed technology’

Failure to provide access to the code and data published in prominent scientific journals could result in unwarranted and even potentially harmful clinical trials, the researchers concluded, adding that neglecting to share key materials transforms the study from a scientific publication open to verification and adoption by the scientific community into “promotion of a closed technology”.

“Well-funded researchers who collect patient data have little incentive to share, but they are the ones who write the informed consent asked of patients and who dictate the terms of sharing,” said associate professor Levi Waldron.

“Protecting patient privacy from even the most hypothetical of risks can become a method to keep invaluable data, and even the parameters of resulting prediction models, away from other researchers.

“I don’t think the patients volunteering for medical research are informed of the trade-offs between data privacy and utility, or are given much say in them. This is a bigger conversation that should happen above the level of individual research studies, and it needs to include patients.”

A recent study published in the BMJ cautioned that many studies claiming AI is superior or as good as human experts at interpreting medical images may exaggerate machines’ diagnoses ability and pose a potential risk to patient safety.

The report cautioned that overpromising language “leaves studies susceptible to being misinterpreted by the media and the public, and as a result the possible provision of inappropriate care that does not necessarily align with patients’ best interests”.

Other past studies have suggested AI could speed up the diagnosis of brain tumours, assess the likelihood of experiencing a heart attack and identify potential symptoms in patients who may experience problems with their eyes later in life.