The Radiological Society of North America (RSNA) has announced the official results of its latest artificial intelligence (AI) challenge. A collaboration with the Society of Thoracic Radiology (STR), the RSNA-STR Pulmonary Embolism Detection Challenge required teams to create machine-learning algorithms to detect and characterize instances of pulmonary embolism.
The RSNA-STR Pulmonary Embolism CT (RSPECT) dataset created for the challenge is comprised of more than 12,000 CT scans collected from five international research centers. The dataset was labeled with detailed clinical annotations by a group of more than 80 expert thoracic radiologists.
Pulmonary embolism is among the most fatal cardiovascular diseases, causing 60,000 to 100,000 deaths annually in the United States. Patients exhibit symptoms that are common to other diseases and rapid radiologic diagnosis is often critical to care decisions. This challenge demonstrates how machine learning can aid in effective patient management and treatment by allowing radiologists to more accurately identify pulmonary embolism cases.
“Challenges like this are like a Formula One race: an opportunity for the world’s best to come together, demonstrate their skill, learn from each other and advance the state of the art in the name of improving patient care,” says John Mongan, MD, PhD, incoming chair of the RSNA Machine Learning Steering Subcommittee. “Every year I’m impressed and inspired by what contestants achieve.”
Of the 784 teams from around the world who took part in the RSNA-STR Pulmonary Embolism Detection Challenge, the award-winning teams are:
- Guanshuo Xu
- HIGH D-DIMER
- VinBigData-Medical Imaging
- kazumax
- deepread.ai
- OsciiArt
- yuval reina
- [Aillis] Yuji + Jan + yama
- shimacha
- OrKatz
In recognition of the competition’s public value, the winning teams will share a total of $30,000 in prize money, provided by Kaggle. The winners will be recognized during the virtual RSNA annual meeting, which takes place from November 29-December 5.
The 2020 Educational Merit Award was presented to Ian Pan, MD, of team HIGH D-DIMER. The Educational Merit Award, newly created for 2020, recognizes an individual from among the top 10 teams whose entry is deemed outstanding in the clarity, completeness, organization and efficiency of its submitted code.
The top entrants on the private leader board were reviewed by a panel of judges for the purpose of awarding an Educational Merit distinction. Judges evaluated the training code, inference code and associated documentation and supporting materials, including any live websites or hosted demonstrations of the algorithm.
In reviewing entries for the Educational Merit Award, judges considered:
- Clarity and comprehensiveness of the explanation of the approach taken by the entrant
- Clarity, accessibility, organization, and architecture of the code
- Estimated effort required to run the code
- Ease of re-use and re-purposing of code
- Novelty of approach
The RSNA-STR Pulmonary Embolism Detection Challenge was conducted on a platform provided by Kaggle and was open to all AI researchers. For the first time this year, RSNA’s challenge adopted the approach of a code submission competition. The rules required competitors to submit and run their code in a standard shared environment, producing simpler, more readily usable models.
