False-Positive-Constrained Evaluation of AI Lung Nodule Detection

Purpose: False-positive findings remain a primary barrier to clinical adoption of AI lung nodule detection. This study evaluated whether an AI detector maintains high sensitivity while constraining false-positive burden at clinically relevant operating points.

Methods/Materials: An AI lung nodule detection system was trained on multicenter low-dose chest CT screening examinations and evaluated on an independent multi-reader dataset with heterogeneous reference annotations. Lesion-level sensitivity versus false positives per scan (FP/scan; unmatched model marks per examination relative to reference annotations) was assessed using free-response receiver operating characteristic (FROC) analysis. The primary operating point was ~1 FP/scan; secondary analyses evaluated alternative operating points and a nodule size threshold of ≥3 mm. The FROC score was defined as mean sensitivity at 0.125, 0.25, 0.5, 1, 2, 4, and 8 FP/scan. Ninety-five percent confidence intervals were estimated using bootstrap resampling.

Results: Evaluation included 1,009 CT examinations (330 normal, 679 abnormal) with 1,303 annotated nodules ≥5 mm. Sensitivity was 91.7% (95% CI: 89.6–93.6) at ~1 FP/scan and 94.9% (95% CI: 93.1–96.3) at 2 FP/scan; sensitivity was 86.8% at 0.5 FP/scan. Maximum sensitivity was 98.4% (95% CI: 97.6–99.1) at ~9.6 FP/scan. The overall FROC score was 0.889 (95% CI: 0.873–0.904). For nodules ≥3 mm, sensitivity was 90.8% at ~1 FP/scan.
Figure: FROC curve of lesion-level sensitivity versus false positives per scan for nodules ≥5 mm; clinical operating point (~1 FP/scan) highlighted.

Conclusions: AI-based lung nodule detection demonstrated sustained high sensitivity under workflow-aligned false-positive constraints. Reporting performance at constrained operating points provides practice-relevant insight beyond peak sensitivity and may better inform screening deployment decisions.