Superior Diagnostic Accuracy with Digital Breast Tomosynthesis White Paper

White Paper Superior Diagnostic Accuracy with Additional and Stand-Alone Digital Breast Tomosynthesis Key findings of the clinical Premarket Approval studies of Mammomat Inspiration with Tomosynthesis Option By Johannes Georg Korporaal, PhD, Nancy Nalleweg, PhD, Thomas Mertelmeier, PhD, and Axel Hebecker, PhD SIEMENS Healthineers .. siemens.com/healthineers White Paper | Superior Diagnostic Accuracy with DBT 1 Introduction Intro- duction This white paper is a summary of two clinical studies that supported the Premarket Approvals (PMA) of the “Mammomat Inspiration with Tomosynthesis Option” by the Food and Drug Administration (FDA) (1, 2)1 Both clinical studies contributed to the . assessment of the safety and effectiveness of the device for breast cancer screening and diagnosis in the US. The main difference between the two studies is how digital breast tomosynthesis (DBT) is used compared to full-field digital mammography (FFDM) alone (see Table 1). Adjunct scenario Replacement scenario 2-view FFDM plus 2-view DBT Stand-alone 2-view DBT + vs. vs. 2-view FFDM 2-view FFDM Table 1 Study setups of the two clinical studies Both studies prove the superior diagnostic accuracy of Siemens DBT compared to full-field digital mammography (FFDM) alone. In chapter 2, the study design and populations for both studies are described and the key findings are covered in chapters 3 – 5. 1 This white paper contains results from both clinical studies beyond those published in the FDA documents. 2 Superior Diagnostic Accuracy with DBT | White Paper 2 Objectives and methods Objectives The goal of investigating two distinct scenarios was 1) to prove the value of wide-angle DBT in current clinical practice and 2) to demonstrate the potential of wide-angle DBT as a stand-alone modality. In clinical practice, DBT is currently predominantly used as an additional modality to FFDM. Although the DBT slices provide more detail and can increase cancer detection (3–6), there are still clinical and traditional arguments for retaining FFDM with DBT. For example, FFDM images enable comparison with prior exams and give an overview of the whole breast. Further, reimbursement regulations may require that a 2D image is acquired together with DBT. To demonstrate the potential of wide-angle tomosynthesis systems, Siemens also investigated the use of DBT as a stand-alone modality, as this would result in a lower total examination dose by eliminating the radiation dose of the FFDM. For both scenarios, three objectives were addressed: 1. Superior diagnostic accuracy (Chapter 3) The primary objective was to show diagnostic superiority of DBT with FFDM and stand-alone DBT compared to FFDM alone, as measured by the area under the receiver operating characteristic curve (AUC ROC) at the breast level. 2. Reduced non-cancer recall rate (Chapter 4) As a secondary objective, the subject-level non-cancer recall rate was investigated. It was hypothesized that the recall rate would decrease because of higher diagnostic confidence with DBT. 3. Improved reader performance and lower interobserver variability (Chapter 5) With DBT an improvement in the diagnostic performance of each individual reader was expected as well as a reduction in interobserver variability. For both studies a multi-reader multi-case (MRMC) study design was chosen. 3 White Paper | Superior Diagnostic Accuracy with DBT Methods 2.1 Library of images Both studies were based on a library of images specifically collected for these DBT PMA studies. This case collection was performed between May 2011 and February 2014 from seven United States clinical sites: Duke University, Durham, NC; • SUNY, Stonybrook, NY; • Brigham & Women’s Hospital, Boston, MA; • Cleveland Clinic, Cleveland, OH; • NYU, New York, NY; • St. Luke’s Episcopal Hospital, Houston, TX; and • Miami Baptist Hospital, Miami, FL. • The library included FFDM images and DBT raw data as well as DBT images for presentation. All images were collected prospectively through a written informed consent process and all local institutional review boards approved the studies prior to enrollment of participants in the study. The FFDM images were acquired according to the standard of care with various commercially available FFDM systems. In addition to these clinically indicated screening mammograms, DBT images were acquired solely on Siemens Inspiration DBT systems. All breasts were imaged under craniocaudal (CC) as well as mediolateral oblique (MLO) positioning. For malignant lesions, the ground truth for the type and location of the lesions was 1) based on the mammography findings described by the radiologist at the clinical site according to the standard of care and 2) confirmed by the radiology and pathology reports after biopsy. For cases with a biopsy-confirmed benign finding, follow-up examinations were performed after 6 or 12 months. For normal cases, follow-up was performed after 12 months to confirm the non-cancer status. 4 Superior Diagnostic Accuracy with DBT | White Paper Methods 2.2 Study populations For each scenario, a subset of cases was randomly selected from the library of images (Table 2). Specific mammographic features like breast density and lesion type were taken into account to obtain a distribution of cases similar to that seen in clinical practice. Adjunct scenario Replacement scenario + vs. vs. Subjects enrolled: 300 Subjects enrolled: 330 50 with cancer 105 with cancer • • 85 with benign findings 72 with benign findings • • 165 normal 138 normal • • 15 special cases • Breasts analyzed: 490 Breasts analyzed: 548 53 biopsy-proven cancerous breasts 110 biopsy-proven cancerous breasts • • (10.8%) (20.1%) 90 biopsy-proven benign breasts (18.4%) 77 biopsy-proven benign breasts (14.1%) • • 347 normal breasts with follow-up 361 normal breasts with follow-up • • (70.8%) (65.9%) Breast density: Breast density: Almost entirely fat: 5.3% Almost entirely fat: 5.8% • • Scattered fibroglandular: 40.7% Scattered fibroglandular: 39.9% • • Heterogeneously dense: 48.7% Heterogeneously dense: 47.3% • • Extremely dense: 4.7% Extremely dense: 6.4% • • Missing data: 0.7% Missing data: 1.2% • • Lesion type: Lesion type: Mass: 65.2% Mass: 63.4% • • Calcification: 24.2% Calcification: 21.6% • • Architectural distortion: 9.1% Architectural distortion: 9.0% • • Asymmetric density: 1.5% Asymmetric density: 6.0% • • Cancer type: Cancer type: Invasive cancer: 78.8% Invasive cancer: 80.6% • • Table 2 Ductal carcinoma in situ: 21.2% Ductal carcinoma in situ: 19.4% • • Subject characteristics 5 White Paper | Superior Diagnostic Accuracy with DBT Methods 2.3 Reading and image interpretation A total of 22 MQSA-qualified radiologists (“readers”) interpreted and scored the clinical images in the adjunct scenario and 31 MQSA-qualified radiologists in the replacement scenario. All radiologists were trained in the principles of breast tomosynthesis and the interpretation of DBT images. In the adjunct scenario, the readers reviewed the cases sequentially in the following order: 1. FFDM (MLO and CC); 2. FFDM (MLO and CC) + DBT (MLO); 3. FFDM (MLO and CC) + DBT (MLO and CC). In the replacement scenario, each reader participated in two reading sessions separated by a wash-out period of at least 4 weeks. During each reading session, half of the cases were interpreted with FFDM alone (MLO and CC). The other half of the cases were interpreted with DBT MLO views first and in a second step DBT with MLO and CC views. After the wash-out period, the readers interpreted the opposite modality for each case. Cases were randomized to be read with either FFDM or DBT first and the reading order within each group was randomly assigned. Readers marked the locations of all suspected lesions on FFDM and DBT and assigned two types of scores for each lesion: a probability of malignancy confidence score on a scale from 0.5 to 100; • a BI-RADS score based on the assumption that the lesion was the only finding • (“forced” BI-RADS score) The non-cancer recall rate was based on all subjects without cancer who received a false positive result (BI-RADS 3, 4 or 5) or a BI-RADS score of 0. Scoring for all subjects was done at the breast-level. 6 Superior Diagnostic Accuracy with DBT | White Paper 3 Significant improvement in diagnostic accuracy Results The diagnostic accuracy in detecting and characterizing breast lesions can be described with the area under the receiver operating characteristic curve (AUC ROC) (see (7) for an explanation of AUC ROC). To summarize the diagnostic accuracy of all readers together, average ROC curves were calculated. For both the adjunct and the replacement scenario, the average AUC ROC was significantly higher with DBT compared to FFDM alone (Table 3). The relative increase in diagnostic accuracy at the breast level was 13.4% for the adjunct scenario and 5.3% for the replacement scenario. This demonstrates the superior diagnostic accuracy of Siemens’ 50° wide-angle DBT in addition to FFDM, as well as for its stand-alone use. Note: the AUC ROC values and the changes therein cannot be directly compared between the two studies, because the study populations had different sizes, included different cases and were read by different readers. Adjunct scenario Replacement scenario + vs. vs. Average AUC ROC: Average AUC ROC: 2-view FFDM: 0.752 2-view FFDM: 0.818 • • 2-view FFDM + 2-view DBT: 0.853 2-view DBT: 0.861 • • ΔAUC ROC: 0.101 ΔAUC ROC: 0.043 • • p-value: <0.0001 p-value: 0.011 • • Relative increase: 13.4% Relative increase: 5.3% • • 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 Sensitivity Sensitivity 0.2 0.2 2-view FFDM 2-view FFDM 2-view FFDM + 2-view DBT 2-view DBT 0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1.0 1-Specificity 1-Specificity Figure 1: Average reader breast-level ROC Figure 2: Average reader breast-level ROC curves (non-parametric) for 2-view FFDM curves (parametric) for 2-view FFDM (red) (red) and 2-view FFDM + 2-view DBT and 2-view DBT (orange). Calculation (orange). Calculation based on ratings based on ratings of 31 readers on 548 Table 3 of 22 readers on 490 breasts. breasts. Results: Significant improvement in diagnostic accuracy 7 White Paper | Superior Diagnostic Accuracy with DBT 4 Significant reduction in Results the non-cancer recall rate Typically, all women with a positive or inconclusive finding during screening will be recalled for further diagnostic work-up. This group consists of true and false positives, the latter receiving unnecessary follow-up procedures such as ultrasound or biopsy. The non-cancer recall rate describes the number of women with false positive findings as a proportion of all screened women as illustrated in Figure 3. True positive Positive test result False positive Screening population True negative Figure 3: Negative test result The non-cancer recall rate is defined as the number of false positive cases as a False negative proportion of all women screened. For both the adjunct and the replacement scenario, the readers’ average non-cancer recall rate was significantly lower with DBT compared to FFDM alone (Table 4). In the adjunct scenario, for 18 out of 22 readers their non-cancer recall rate with DBT was lower than with FFDM alone. In the replacement scenario this was the case for 28 out of 31 readers. This demonstrates the increased confidence with Siemens’ 50° wide-angle DBT in addition to FFDM, as well as for its stand-alone use. Adjunct scenario Replacement scenario + vs. vs. Readers’ average non-cancer recall rate: Readers’ average non-cancer recall rate: 2-view FFDM: 0.438 2-view FFDM: 0.479 • • 2-view FFDM + 2-view DBT: 0.355 2-view DBT: 0.386 • • Table 4: Δnon-cancer recall rate: -0.083 Δnon-cancer recall rate: -0.093 • • Results: Significant reduction p-value: 0.0009 p-value: 0.0001 • • in the non-cancer recall rate Relative decrease: 18.9% Relative decrease: 19.4% • • Note: the reported non-cancer recall rates differ in magnitude from those reported in general population screening studies due to the enriched study populations of both studies (Table 2). 8 Superior Diagnostic Accuracy with DBT | White Paper 5 Improved reader performance and lower interobserver variability Results As shown in chapter 3, the average AUC ROC increased significantly with DBT in both scenarios for all radiologists together. In addition, the performance of each individual reader is also of interest. In the adjunct scenario, all 22 readers showed improvement in their diagnostic accuracy with the addition of 2-view DBT (Figure 4) and weaker-performing readers showed more improvement compared to readers who were already stronger with 2-view FFDM alone. The differences in AUC ROC values between readers, as described with the coefficient of variation (Table 5, calculated post-hoc), were smaller with the addition of 2-view DBT, indicating lower interobserver variability. In the replacement scenario, 29 out of 31 readers showed improvement in their diagnostic accuracy with 2-view DBT alone compared to 2-view FFDM alone (Figure 5). The coefficient of variation in AUC ROC values was also smaller with 2-view DBT compared to 2-view FFDM (Table 5, calculated post-hoc), indicating lower interobserver variability. This demonstrates the improved reader performance and lower interobserver variability with Siemens’ 50° wide-angle DBT in addition to FFDM, as well as for its stand-alone use. Note: the AUC ROC values and the changes therein cannot be directly compared between the two studies, because the study populations had different sizes, included different cases and were read by different readers. Adjunct scenario Replacement scenario + vs. vs. Coefficient of variation: Coefficient of variation: 2-view FFDM: 6.4% 2-view FFDM: 5.2% • • 2-view FFDM + 2-view DBT: 3.4% 2-view DBT: 4.1% • • 0.9 0.9 0.8 0.8 AUC ROC AUC ROC 0.7 0.7 0.6 0.6 6 1 22181614 7 21 8 201713 9 5 121910 2 1115 4 3 7 1 27241226171415 9 5 3230222920132816103 111923 8 2 4 6 311821 Reader ID Reader ID Figure 4: Readers’ breast-level AUC ROC Figure 5: Readers’ breast-level AUC ROC with FFDM alone (petrol light) and with FFDM alone (petrol light) and DBT FFDM+DBT (petrol light+petrol). alone (petrol light+petrol). For readers demonstrating a lower AUC with DBT, the Table 5 AUC for FFDM is the entire bar, with the decrease with DBT shown as striped pattern. Results: Improved reader performance and lower interobserver variability 9 White Paper | Superior Diagnostic Accuracy with DBT 6 Discussion Discussion Digital breast tomosynthesis (DBT) is a promising technique that might replace FFDM as the gold standard for breast cancer screening and diagnosis. With Siemens’ wide- angle DBT, clear advantages compared to FFDM have been proven with the two PMA studies described in this white paper: superior diagnostic accuracy, • a reduced non-cancer recall rate, • improved reader performance, and • lower interobserver variability. • All of these improvements are highly relevant to the clinical implementation and are strong drivers for DBT to become the standard of care in screening and diagnostic mammography. Radiation dose is especially important in breast cancer screening, because the majority of women undergoing screening examinations are healthy. Our replacement study clearly shows that with stand-alone wide-angle DBT the acquisition and radiation dose of FFDM is not necessary to gain the advantages of superior diagnostic accuracy, a reduced non-cancer recall rate, improved reader performance and lower interobserver variability that DBT can offer. When using DBT as a stand-alone modality, a synthetic mammogram can potentially be of help for the comparison with prior exams and to get a fast overview of the entire breast. Note that synthetic mammograms were not used in either of these studies and in particular, the results of the replacement scenario were achieved without the readers referring to a 2D image when reading the DBT. Further evidence supporting the use of stand-alone DBT in breast cancer screening comes from the Malmö Breast Tomosynthesis Screening Trial (3). In this trial, Siemens’ wide-angle DBT was used as a stand-alone screening modality and examinations were performed with one view (MLO) only, to further reduce the radiation dose and reading time. The interim analysis shows an increased cancer detection rate of more than 40% and a significant increase in sensitivity. Even though the Malmö study used the Siemens wide-angle DBT system with one view only, it delivers further evidence that stand-alone DBT might be feasible as a future screening modality for the general population (8). Similar to the Malmö study, the Siemens PMA study with the adjunct scenario also included an investigation of one-view DBT (MLO) in addition to FFDM. More details can be found in a separate white paper (7) or in the FDA Summary of Safety and Effectiveness Data (1). Note that while Siemens performs and supports studies investigating the use of DBT in various protocols, these are not recommendations and a decision on the use of DBT as an additional or stand-alone modality and on the number of views must be made by the examining physician. 10 Superior Diagnostic Accuracy | White Paper 7 Summary Summary This white paper describes two clinical studies with Siemens’ wide-angle tomosynthesis system which supported the Premarket Approvals by the Food and Drug Administration (FDA). DBT was used in addition to FFDM (adjunct scenario) and as a stand-alone modality (replacement scenario) and compared with FFDM. For the adjunct as well as the replacement scenario, superior diagnostic accuracy, a reduced non-cancer recall rate, improved reader performance, and lower interobserver variability were reported compared to FFDM alone. These results show that Siemens’ wide-angle tomosynthesis system can be used with a high level of confidence in screening and diagnostic mammography. Glossary References AUC ROC Area under the ROC curve 1. Siemens Medical Solutions USA Inc. (2015) PMA P140011: FDA Summary of Safety and Effectiveness Data. BI-RADS Breast Imaging Reporting 2. Siemens Medical Solutions USA Inc. (2016) Clinical Study Report for Protocol and Data System SMS-SP09-01: Retrospective, multi-reader, multi-case study (MRMC) to demonstrate the superior accuracy of Siemens Digital Breast Tomosynthesis to CC Craniocaudal Full-Field Digital Mammography (FFDM) as a replacement for FFDM screening mammography. DBT Digital Breast Tomosynthesis 3. Lång K, Andersson I, Rosso A, Tingberg A, Timberg P, Zackrisson S (2016) Performance of one-view breast tomosynthesis as a stand-alone breast cancer FDA Food and Drug Administration screening modality: results from the Malmo Breast Tomosynthesis Screening Trial, a population-based study. European radiology 26(1):184–90. FFDM Full-Field Digital Mammography 4. Skaane P, Bandos AI, Gullien R, Eben EB, Ekseth U, Haakenaasen U et al. (2013) MLO Mediolateral oblique Comparison of digital mammography alone and digital mammography plus tomosynthesis in a population-based screening program. Radiology 267(1):47– 56. MQSA Mammography Quality Standards Act and Program 5. Ciatto S, Houssami N, Bernardi D, Caumo F, Pellegrini M, Brunelli S et al. (2013) Integration of 3D digital mammography with tomosynthesis for population PMA Premarket Approval breast-cancer screening (STORM): a prospective comparison study. The Lancet. Oncology 14(7):583–9. ROC Receiver Operating Characteristic 6. Friedewald SM, Rafferty EA, Rose SL, Durand MA, Plecha DM, Greenberg JS et al. (2014) Breast cancer screening using tomosynthesis in combination with digital mammography. JAMA 311(24):2499–507. 7. Nalleweg N, Mertelmeier T, Korporaal JG, Hebecker A (2016) Superior Diagnostic Accuracy with Additional Digital Breast Tomosynthesis: Main findings of the clinical study for Premarket Approval of Mammomat Inspiration with Tomosynthesis Option. Siemens White Paper. 8. Sardanelli F, Aase HS, Alvarez M, Azavedo E, Baarslag HJ, Balleyguier C et al. (2016) Position paper on screening for breast cancer by the European Society of Breast Imaging (EUSOBI) and 30 national breast radiology bodies from Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Israel, Lithuania, Moldova, The Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Spain, Sweden, Switzerland and Turkey. European radiology. doi: 10.1007/s00330-016-4612-z. 11 On account of certain regional limitations of sales rights Please contact your local Siemens sales representative for and service availability, we cannot guarantee that all the most current information. products included in this brochure are available through the Siemens sales organization worldwide. Availability and In the interest of complying with legal requirements packaging may vary by country and are subject to change concerning the environmental compatibility of our products without prior notice. (protection of natural resources and waste conservation), we recycle certain components. Some / All of the features and products described herein may not be available in the United States or other countries. Using the same extensive quality assurance measures as for The information in this document contains general technical factory-new components, we guarantee the quality of these descriptions of specifications and options as well as standard recycled components. and optional features that do not always have to be present Note: Any technical data contained in this document may in individual cases. vary within defined tolerances. Original images always lose Siemens reserves the right to modify the design, packaging, a certain amount of detail when reproduced. specifications and options described herein without prior notice. Order No. XXXXXXXXXXX Printed in Germany | CC XXXX 0217.X | © Siemens Healthcare GmbH, 2017