DirectORGANS

White paper DirectORGANS The world’s first contours generated by a CT simulator – Motivation and technical principles Lisa Kratzke, Dr. Nilesh Mistry, Christoph Bauer, Siemens Healthineers siemens-healthineers.com SIEMENS Healthineers ID SOMATOM go.Sim Courtesy of Leopoldina Hospital, Schweinfurt, Germany Cinematic VRT is for illustration purposes only. This feature is not part of DirectORGANS. SIEMENS Healthineers DirectORGANS · White paper Table of contents Key takeaways 3 Importance of autocontouring 4 The DirectORGANS algorithm 6 Conclusion 12 References 13 2 DirectORGANS · White paper Key takeaways to understand DirectORGANS Why a new autocontouring solution? The quality of computer generated contours is significantly impacted by the input image quality, especially in the presence of artifacts, poor image statistics (i.e. increased noise), or poor contrast. All these factors can negatively impact relevant image features and may lead to suboptimal quality of the autocontouring [1] [2]. As a result, users spend a significant amount of time editing those organs- at-risk (OAR) contours – sometimes to a point that the potential benefits of autocontouring in terms of time saving may be completely lost. How does DirectORGANS work? In order to solve the problem at hand, it is necessary to provide the autocontouring solution with optimized input. For this reason, DirectORGANS (Optimized Reconstruction based Generative Adversarial Networks) employs an optimized reconstruction that is used as a standardized input to the Deep Learning based autocontouring algorithm. Both processes, image optimization and automatic contouring, are embedded into the CT simulator enabling results as part of the image acquisition. What is the benefit? By leveraging Artificial Intelligence (AI) to generate OAR contouring directly at the CT simulator, DirectORGANS provides consistent, standardized, high quality, contoured images that are ready as an output of the CT simulation process. This solution enables time efficient OAR contouring as part of the standard CT acquisition, freeing up staff to spend more time for other tasks. 3 DirectORGANS · White paper Importance of autocontouring In the last couple of years, not only the cancer incidence rates have increased, but also the amount of patients receiving Radiation Therapy (RT). Up to two thirds of all the patients with cancer will need RT treatment during the course of their disease.[3] Increase in cancer cases Percentage of cancer patients and their costs [4] receiving radiation therapy [3] Their costs New cancer cases Up to 2/3 m m of cancer patients $290bn $458bn 13.2 22.6 receive radiation therapy. 2010 2030 Fig. 1.1 Cancer statistics and incidence predictions Each patient arriving at the RT department requires a treatment plan. Contouring the organs-at-risk is the necessary first step in the process of treatment planning. Therefore, the increase in the number of patients puts significant pressure on radiotherapy staff responsible for consistent OAR contouring results. Advances in technology and AI can help automate repetitive tasks such as OAR contouring and reduce workload. The automation may help in increasing consistency while achieving better efficiency. 4 DirectORGANS · White paper Challenges with OAR contouring In many institutions, organs-at-risk are contoured In the last decade, various autocontouring solutions manually; as a result valuable staff resources are tied up, have been introduced to address these challenges. turning OAR contouring into a cost and time intensive However, the results may not be clinically useful for task. In addition, inter-observer variability can make it the RT professionals leading to significant editing or difficult to achieve consistent contouring results and re-doing the contours. One of the reasons is that most operators need to be trained on common contouring autocontouring results have been produced on CT guidelines. Considering staffing issues such as high images optimized for human perception and may not turnover, consistent OAR contouring still is a problem be optimal for the task of automated contouring. in many institutions. However, the optimization of images is performed for a specific need in the clinic and introducing a new optimized reconstruction for the task of autocontouring may conflict with the original intent. No interobserver variability up to 1 hour/patient [5] Potential for for OAR contouring time savings Fig. 1.2 Opportunities in OAR contouring DirectORGANS supports RT professionals addressing contouring needs and workflow efficiency To overcome the challenge of clinical workflow and Additionally, we also integrate the process of optimized simultaneously enable automated contouring we introduce image reconstruction for the task of autocontouring. DirectORGANS. DirectORGANS is the first integrated Hence, in clinical routine, no adaption of the workflow solution making OAR contouring a part of the acquisition is needed. Research shows that up to one hour can task. The algorithm enables a fast and seamless workflow be saved for the contouring of the organs-at-risk [5]. not requiring manual data transfer, e.g. to a contouring – workstation. 5 DirectORGANS · White paper The DirectORGANS algorithm DirectORGANS was developed to provide contoured One of the challenges of traditional autocontouring images directly at the CT simulator. is the quality of the input images. Therefore, image Two core elements – optimized reconstruction and Deep optimization is a key step in order to provide consistent, Learning (DL) based contouring – lay the foundation for high quality contours. Figure 2.1 illustrates the differences this technology. between an image optimized for the human and an image optimized for a machine: one of the keys to obtain consistent quality contours is to provide the algorithm with as much information as possible. Artifact reduction, higher spatial resolution are examples of ways to increase the amount of relevant information for the machine, however, increased spatial resolution leads to several challenges in the clinic. For example: increased z-resolution for the same coverage means increased workload for contouring and increased in-plane spatial resolution may lead to increased noise in the image – both features that are not desirable in the clinical situation. Image designed for DirectORGANS Image designed for RT professionals Standardized contrast High contrast Highest resolution Smooth images Thin slice Fig. 2.1 Example of image designed for DirectORGANS (left) and RT professionals (right) Courtesy of Radiology Department, Hospital Particular de Viana do Castelo, Viana do Castelo, Portugal 6 6 DirectORGANS · White paper DirectORGANS is available for the most relevant cancer sites for External Beam Radiation Therapy (EBRT) such as brain, head & neck1, breast, lung, abdominal and prostate (figure 2.2). Additionally, we offer advanced packages for the heart and the lung. Cardiac substructure2 segmentation enables research in the field of cardiac toxicity. Contouring for the ribs and the lung substructures enables tailored treatment plans that minimize the risk of treatment-induced rib fractures [6]. Brain – Whole Brain Head & Neck1 – e.g. Brainstem, Parotid Glands Breast – e.g. Female Breast, Heart, Aorta3, Cardiac Chambers2,3 Lung – e.g. Lungs, Lung Lobes3, Ribs3 Abdomen – e.g. Liver, Spleen, Kidneys Pelvis – e.g. Rectum, Bladder Fig. 2.2 Examples of contours generated by DirectORGANS and DirectORGANS Advanced (Software Version VA30) 1 Atlas based 2 MSL (marginal space learning) based 3 Optional, DirectORGANS Advanced 7 DirectORGANS · White paper Input for RT professional (images) Input for algorithm (dedicated recon) Output: consistent OAR contours AI Metal artifact reduction AI-powered Deep Learning contours trained by GANS Optimized z-resolution Optimized reconstruction kV standardization ..... Streak artifact reduction Fig. 2.3 Functional steps of the DirectORGANS algorithm a) DirectORGANS in clinical routine The acquired data is reconstructed in two parallel tracks: minimized artifacts to enable consistent high-quality One to meet the requirements of human operators, i.e. contours (see figure 2.1). Leveraging Deep Learning, the with the individually preferred reconstruction parameters contours are generated based on the optimized images (“Input for RT professional” arrow in figure 2.3). The other (orange arrow in figure 2.3). In the next step, the image one to provide images that are optimized for designed for the RT professional and the contours are autocontouring by the CT simulator (“Input for algorithm” fused (figure 2.4). The resulting contoured image will be arrow in figure 2.3). Images optimized for the task of used for further treatment planning. The creation of the autocontouring have the highest possible information contours is explained in detail in the following. with the highest resolution, standardized contrast and Contoured image Image for RT professional Contours created by the DirectORGANS algorithm O Fig. 2.4 The contours created by the DirectORGANS algorithm and the image optimized for human consumption are combined. Courtesy of Radiology Department, Hospital Particular de Viana do Castelo, Viana do Castelo, Portugal 8 DirectORGANS · White paper AI Metal artifact reduction Optimized z-resolution Optimized reconstruction kV standardization ..... Streak artifact reduction Fig. 2.5 Optimized reconstruction Optimized reconstruction (OR) CT imaging is a highly accurate and quantitative imaging DirectORGANS uses a consistent slice thickness and modality that allows to obtain precise information about slice increment for the optimized image reconstruction. the tissue density distribution of the patient within a kV standardization enables departments to leverage few seconds of scanning. Nevertheless, there are sources different kV settings for different patient sizes, ages and of artifacts that make the images less quantitative than indications, while still generating consistent contours. desired. That is the reason why an optimized That means DirectORGANS is capable of handling reconstruction is performed in the background prior to different scans independent of the selected kV. The the creation of the contours (figure 2.5). One element of optimized reconstruction of DirectORGANS enables the optimized reconstruction is reducing metal artifacts. an integrated way of generating images optimized for These are caused by the presence of high density objects the contouring task without the need to change the such as implants, seeds, or fillings. Furthermore, noise and existing workflow. streak artifacts, e.g. from beam hardening, are corrected. 9 DirectORGANS · White paper Output: consistent OAR contours AI AI-powered Deep Learning contours trained by GANS Fig. 2.6 Deep Learning Contouring Deep Learning based contouring Following the reconstruction, the optimized images cropped image is used as input to create the contours. are used to create the contours (figure 2.6). This process This step is based on a Deep Image-to-Image Network is based on a two step approach as can be seen in figure (DI2IN) [8]. The DI2IN was trained to its optimal 2.7. First, the target organ region in the optimal input performance in the Siemens Healthineers AI environment. image is extracted using a Deep Reinforcement Learning The training process of the DI2IN is explained in the trained network (DRL) [7]. The result is a cropped image following section1. with the target organ region. In the second step, the Step 1: Locate target organ region Input image Deep Reinforcement Learning [6] Cropped image Step 2: Contour target organ Cropped image DI2IN [7] Segmented organ Fig. 2.7 Two step algorithm for DL based contouring 1 Please note – the algorithm is not self-learning. Your data is not used for further training. 10 DirectORGANS · White paper V CT image Generator Prediction Discriminator Human contour / Machine contour Ground truth Fig.2.8 Adversarial training scheme b) Training of the DirectORGANS algorithm The DirectORGANS algorithm was trained leveraging prediction of the first network from the ground truth Deep Learning technology. Deep Learning uses a multi- (human drawn contour). The information is then fed back layer neural network that enables unsupervised learning to the respective networks.This iterative process ensures for a specific task. Typically, the DL algorithm needs a that during the training of the networks, the machine large number of datasets to be trained. generated contours become virtually indistinguishable To perform the organ segmentation, a Deep Image-to- from the human generated contours. For algorithm Image Network is employed. It consists of a convolutional training, CT datasets were obtained for each body region encoder-decoder architecture combined with a multi-level from various radiation therapy and radiology departments feature concatenation. An adversarial network – a so called in Europe and America. Ground-truth segmentations were Generative Adversarial Network (GAN) – is selectively used manually generated on these CT datasets by a team of to regularize the training process of DI2IN by discriminating experienced annotators overseen by radiation oncologists the prediction of the DI2IN from the ground truth (figure or radiologists. For this process, a consistent annotation 2.8). The model is selected in the epoch with the best protocol was set up beforehand based on widely accepted performance on the validation set. A GAN uses two consensus guidelines such as the ones published by the networks that compete against each other during the Radiation Therapy Oncology Group (RTOG). The organ training process. The first network – the generator – tries models were then trained with pairs of CT data and the to emulate a human drawn contour while the second corresponding standardized ground-truth segmentation. network – the discriminator – tries to discriminate the 11 DirectORGANS · White paper Conclusion DirectORGANS leverages the power of an optimized additional workstation for the OAR contouring. This image reconstruction and deep learning to streamline potentially leads to less errors originating from the OAR contouring, directly at the CT simulator. This new application configuration or operation. As a result, time solution may help to reduce unwarranted variations with and resource saving can potentially be achieved as well contours that provide a consistent starting point for as user independent results. With DirectORGANS OAR radiation therapy planning. By design, DirectORGANS contouring becomes an integrated part of the standard enables a fully automated workflow requiring no CT acquisition. 12 DirectORGANS · White paper References [1] WU, X., et al. Knowledge-based auto contouring for radiation therapy: Challenges in standardizing object definitions, ground truth delineations, object quality, and image quality. International Journal of Radiation Oncology Biology Physics, 2017, 99. Jg., Nr. 2, S. E740. [2] Cheung CW, Leung KY, Lam WW, et al. Application of Model-based Iterative Reconstruction in Auto-contouring of Head and Neck Cases. Scientific Informal (Poster) Presentation at: LL-ROS-TH Radiation Oncology and Radiobiology Lunch Hour CME Posters; RSNA 2012 Nov 29; arXiv:1707.08037 [cs.CV] Chicago, IL. [3] ATUN, Rifat, et al. Expanding global access to radiotherapy. The lancet oncology, 2015, 16. Jg., Nr. 10, S. 1153-1186. [4] American Cancer Society, www.cancer.org [5] DAS, Indra J.; MOSKVIN, Vadim; JOHNSTONE, Peter A. Analysis of treatment planning time among systems and planners for intensity-modulated radiation therapy. Journal of the American College of Radiology, 2009, 6. Jg., Nr. 7, S. 514-517. [6] NAMBU, Atsushi, et al. Rib fracture after stereotactic radiotherapy for primary lung cancer: prevalence, degree of clinical symptoms, and risk factors. BMC cancer, 2013, 13. Jg., Nr. 1, S. 68. [7] GHESU, Florin-Cristian, et al. Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans. IEEE transactions on pattern analysis and machine intelligence, 2017, 41. Jg., Nr. 1, S. 176-189. [8] YANG, Dong, et al. Automatic Liver Segmentation Using Adversarial Image-to-Image Network. U.S. Patent Application Nr. 15/877,805, 2018. 13 Siemens Healthineers Headquarters Legal Manufacturer Siemens Healthcare GmbH Siemens Healthcare GmbH Henkestr. 127 Henkestr. 127 91052 Erlangen, Germany 91052 Erlangen, Germany Phone: +49 9131 84-0 siemens-healthineers.com Published by Siemens Healthcare GmbH · Online · 7871 0620 · ©Siemens Healthcare GmbH, 2020