Accelerating MR Imaging via Deep Chambolle-Pock Network

 Abstract — Com pressed sensing (CS) has been introduced to accelerate data a cquisition in MR Im aging. However, CS-MRI methods suffer from detail loss w ith large acceleration and complicated parameter selection. To address the limitations of existing CS-MRI m ethods, a model-driven MR reconstruction is proposed that trains a deep network, named CP-net, which is derived from the Cham bolle-P ock algorithm to reconstruct the in vivo MR images of human brains from highly undersam pled complex k-space data acquired on different types of MR scanners. The proposed deep network can learn the proxi mal operator and parameters among the Cham boll e-Pock algorithm. All of the experiments show that the proposed CP-net achieves more accurate MR rec onstruction results, outperforming state-of-the-art methods across va rious quantitative m etrics. I. I NTRO DUCTION Accelerating MR imaging has been an ongoing research topic since its invention in the 1970s . Among variety of acceleration tech niques, compressed sensing (CS) has become an im portant strategy using sub-Ny quist samples based on sparse prior during the past dec ades [1,2]. To recover the signals from sampled measurements, CS-based MR reconstructio n can be formulated a s an optimization problem which consists of the data consistency and the sparsity constraint. A number of itera tive al gori thms have been developed to so lve the recon struction probl em, such as FISTA [3], ADMM [ 4], Chambolle-Pock algorithm [5], etc. Among them, Chambolle-Pock algorith m is a primal dual algorithm that solves the o ptim ization problem simultaneous ly with its dual. And it has been successfully applied in medical image reconstructio n due to its general coverag e on many *Research supp orted in part by the N ational Natural Scie nce Foundation of China (61871373 , 81729003) , Natural Science Foundatio n of Guangdong Province (2018A0303130132), Guangdo ng Provincial Key Laboratory of Medical Image Processing (2017A050501026), Guangdong Provincial Key Laboratory of Magnetic Resonance and Multimodality Imaging (2014B030301 013), Shenzhen Key Laboratory for MRI (CXB20110422 0028A), Strategic Pr iority R ese arch P rogram of Chinese Academy of Sci ence s (XDB250000 00), and Shenzhen Key Laboratory of Ultrasound I maging and Therapy (ZDSYS201802 0618063147 3). H Wang, J Cheng, S Jia, Z Qiu, C Sh i, L Zou, S Su and Y Zhu are wit h Paul C. Lauterbur Res earch Ce ntre for Bi omedical I maging, Shenzhe n Institutes of Advanced Technolo gy, Chinese Aca dem y of Sciences, Shenzhen, G uangdong, China (e-mail: { hf.w ang1, jing.cheng, sen,jia, zl.qiu, cy.shi, lx.zou, sh i.su, yj,zhu}@siat.a c.cn) Y Chang i s with the Computer Science Department, University of Houston-D owntown, Houston, TX 7 7002 USA (e-mail: changy@uhd.edu). L Ying i s with the Biomedical Engineering and Electrical Engineering Departments, State Uni versity of New York, Buffalo , N Y 14260 USA (e-mail: leiying@buffalo .edu). D Liang is with Research Center a nd Medical AI, and Paul C. Lauterbur Research Centre for Biomedical Imaging, Shenzhen Institutes of A dvanced Technolo gy, Chinese Academy of Science s, Shenzhen, Guangdong, China (corresponding author to provide phone: 86-0755-863922 74; e-mail: dong.liang @ siat.ac.cn). optimizatio n formulations of interest [6]. Previously , the Chambolle-Pock algorithm has been applied by our group for acceleratin g MR imaging [7]. Although CS-based method c an achieve high performance with many theoretical gu arantees [1], it’s challeng ing to determin e the numerical uncertainties in the model such as the optimal sparse transformations, sp arse regularizer in th e transform domain, r egu larization parameters and the involve parameters of the optimization algorithm. Recently , deep learnin g has d emonstr ated tr emendou s success and h as become a grow ing trend in general data analysis [8-11]. It als o has b een a pplied successfully in medical imaging such as Ultrasound, CT, MRI, PET and so on [12-16]. Inspired by such success, deep learning has been applied in MR reconstructio n and shown potential to significantl y speed up MR acquisition and improve image quality [16-27]. Deep learning–based MR reconstructio n can be roughly divided into two categories: data-dri ven [16-22] and mo del-dr iven [23-26]. Da ta-driv en methods d irectl y le arn an end-to-end mapping between the input and the o utput with large training datasets and long training time. Model-driven networks f ormulate image recons truction as unrollin g the procedure of an iterative optimization algorithm to a network, while le arning the transforms and parameters in the model. In this work, we have proposed to incorporate the Chambolle-Pock algorithm into a network for accelerating MR imaging, which was named as CP-net , to le arn the proximal operator and parameters among the Chambolle-Pock algorithm. This is inspired b y the learned primal dual in CT reconstructio n [14] and its application in MR reconstruction [27]. Whereas in our work, the network could handle complex data and maintain the algorithm structure of Cham bolle-Pock algorithm to guarantee convergence. T he proposed method can reconstruc t images from highly undersampled complex k-space data for ac celer ating M R imaging. After many in vivo human brain e xperim ents on differen t types of MR scanners, our results have shown that the proposed method can achieve superior results compared to traditional CS method [28] a nd the current state-of-the- art model-driven method ADMM-net [23] and data-driven method D5-C5 [21]. II. M ETHOD S In gener al, the problem to reconstruct the image s from sub-Nyquist sampled k-space data can be wri tten a s min ( ) ( ) F Ap G p  (1) where ( ) F Ap is the data consistenc y term and can be expressed as 2 2 ( ) F A p A p y   , A is the encoding matrix, Accelera ting MR Imaging vi a Deep Cha mbolle-Po ck Netw ork* Haifeng Wang, J ing Cheng, Sen J ia, Zhilang Qiu, Caiyun Shi , Li xian Zou, Shi Su, Yuchou Chang, Yanjie Zhu, Leslie Ying, and Dong Liang, Senior Member, IEEE Figure 1. (a) Architecture o f the proposed CP -net . The d ual iterates are given on the first row, while the primal iterates are on the last row and the linking parts are in the green boxes. The architectures are illustrated in the corresponding enlarged b oxes. (b) The plot o f tr aining (red line) and validating errors (blue line) ver sus epoch. TABLE I. The detailed comparison on performance metrics of d if ferent reconstruction methods with different ac celer ation factor on the axial dataset. here A denotes the undersampled Fourier e ncoding , p is the image to be reconstructed, y is the sampled k-space data, and ( ) G p denotes the regularization term with respect to the image prior information. With the Chambolle-Pock algo rithm, the MR reconstructio n problem (1) ca n b e solved a s follows:         * 1 * 1 1 1 1 1 d prox F d A p n n n p prox G p A d n n n p p p p n n n n                                  (2) where d is the dual vector in k-space,  ,  and are algorithm parameters. * F is the convex conjugate o f the function F , A is the correspondin g undersampled Fourier operator and p r o x denotes the p roxim al mapping which can work directly with non-smooth objective functions. Here, a parameterized operator, where the parameters are learned by the network from training data, is used to replace the original p rox imal operator, thus the whole algorithm can be formed as:       1 * 1 1 1 1 , 1 d d A p n n n p p A d n n n p p p p y n n n n                            (3) In the CP-net, the primal proximal  , d ual proxim al  , parameters  ,  and would be learned from tr aining data. The structure of the proposed CP -net model is shown in Fig. 1(a). CP-net con sists of three parts: dual iterates, primal iterates and a linking part. The dual and p rim al iterates have the same architecture with 3 conv olutional layers i n each box. To train the network more easily , w e made it a r esidu al network. As MR data is complex, the real and imaginary parts of the d ata a re separately saved in two channels of the network. The convolutions involved are all 3×3 pixels in size, and the number of channels is 2-32-32-2 for each primal update, and 4-32-32- 2 for the d ual update. The original input undersam pled k-space data is supplied to the dual iterates. The nonlinearitie s are chosen to be Re ctified Linear Unites (ReLU) functions a nd we let the number of ite ratio ns be 1 0. Since each iteration involves 2 3-layer network s, the total depth of the CP-net is 60 convolutional layers. We trained the network using in-vivo MR datasets. Overall 200 fully sampled multi-contrast data from 8 subjects with a 3T scanner (M AGNETOM Trio, SIEMENS AG, Erlgen, Germany) were collected and inform ed consent was obtained from the imaging object in compliance with the IRB policy . The fully sampled d ata was acquired by a 12-chan nel head coil with matrix size of 256×256 and combined to single-chann el data and then retrospectively undersampled using Poisson disk sampling mask. After normalization and image augmentation, we got 1600 k-space d ata, where 1 400 for training and 200 for validation . The model was trained a nd evaluated on an Ubuntu 16.04 LTS (64-bit) o peratin g sy stem equipped with a Tesla TITAN XP Graphics Processing Unit (GPU, 12GB memory) in the open framework Tensorflow with CUDA and CUDNN Figure 2. Comparison (Axial view) o f the different reconstructio n methods with R = 6. The images in second row are the sampling mask a nd the correspondin g error maps between the reconstr uctions and the reference . The proposed achieves the highest ac cura cy reconstruction . Figure 3. Comparison (Coronal view) of reconstructio n methods w ith R=4. The proposed C P-net achieves better reconstructio n than others. support. The mean square error was chosen a s the loss function, and we trained the networks b y minimizing the loss function using the ADAM optimizer in TensorFlo w. The performan ce of training convergence is illustrated in Fig. 1(b). Three hum an brain datasets with different view of sections from the SIEMENS 3T scanner were used as the test data a nd we also have tested the proposed model on the human data acquired from a nother 3 T scanner ( uMR 790, United Imaging Healthcare, Shanghai, China). III. R ESULTS We use d the data from three differe nt views to evaluate the model and compa red the CP-net with other CS-MRI reconstructio n methods: 1) zer o-filling, the inverse Fo urier transform of the undersampled k-space data, 2) Rec_PF, traditional CSMR rec onstruction meth od, 3) ADMM -net, state-of-the- art model-driven CSMR reconstruc tion method, 4) D5C5 , an end-to -end d eep le arning meth od with data consistency. Several similar ity metric s, including MSE, SSIM and PSNR, wer e us ed to compare the re construction results of different metho ds to re ference image from fully sampled data. From q uan titative metrics shown in Table 1, on the a xial dataset with d iff erent acceleration factors, the proposed CP-net all shows significant improvements over other methods. Fig. 2 illustrates the visual c ompari son and error distributi ons with acceleration factor of 6 on the axial data. It can be seen from Fig. 2 that the proposed method can obtain better image q uality . Figure 3 shows the visual comparison and error distributi ons from coronal view with a reduction factor of 4. Figure 4. Visual comparisons (Sagittal view) for R=6. T he enclosed part is enlarged for a close- up comparison. The zoom-in visualization and the corresponding error maps show the improvement o f the proposed in detail p reser vation. Figure 5. Comparison of reconstruction methods for R=5 with data from UIH scanner. T he z oom- in visualization shows that the CP-net preserves mo re details. Reconstruction accuracy is significantly improved using CP-net, which is indicated b y both error maps and t he quantitativ e metrics. The reconstructions of different methods on sagittal data with R=6 are demonstrated on Fig. 4. The zoom-in views are on the second row and the correspon ding error maps on the last row. It can be seen that CP -net preserves more fine details while removing the undersampling artifacts. Detailed accuracy metrics are also shown in figure. Fig. 5 shows the reconstructions of data from UI H scanner, which o f zoom-in visuali zation shows that the proposed method preserves more details via the trained CP-net. Therefore, the image results presented that the p ropose d method could robust overcome the traditional CS method and current ADMM-net method via image data from differe nt types o f M R scanners. IV. C ONCLUS ION In sum, we proposed an effective deep Chambolle-Pock network for a cceler ating M R imaging. The architecture o f the proposed CP -net is defined over a data flow graph d eterm ined by the CP algorithm. Here, CP-net not only learns a ll the parameters in o riginal algorithm but also learns the p roxim al operators for MR r econs truction. The current experimental results on in vivo hum an brain data have shown the highly efficient reconstruction of the p ropose d CP-net in terms of both artifacts removal and detail preservation. In the future, the proposed CP -net method will be applied those M RI applications of nonlinear gradients [29, 3 0]. V. A CKNOWLEDGEMENT The authors will greatly thank Dr. Oz an Öktem for sharing their codes. R EFERENCES [1] D. L. Donoho, “Co mpressed sensing,” I EEE Trans. In f. Theory , vol. 52, no. 4, p p. 1289–1306, 2006. [2] M . Lustig, D. Donoho, and J. M. Paul y , “S parse mri: The application of compressed sensing for rapid mr imaging,” Magn. Reson. Med., vol. 5 8, no. 6, p . 11821195, 2007. [3] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear in verse problems,” SIAM J. Img. Sci. , vol . 2, no. 1 , pp. 1 83–202, 2009. [4] S . Boy d, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed o ptimization and statistical learning via the alternating di rection method of mu ltipliers,” Found. Trends Mach. Learn . , vol. 3, n o. 1, p p. 1–122, 2011. [5] A. Chambolle and T. Po ck, "A First-Order Primal-Dual Algorithm for Convex Problems with App lications to Imaging," J. Math. Imag ing Vis., vol. 40, no. 1 , pp. 120-145, Mar 2011. [6] E . Y. Sidky, J. H. Jorgensen, and X. C. Pan , "Convex optimization p roblem prototypin g for image reconstruction in computed tomography with th e Chambolle-Pock algorith m," Physics in Medicine a nd Biology, vol. 57, no. 1 0, p p. 3065-3091, 20 12. [7] J. Cheng, S. Jia, H. Wang, and D. Liang. "MR image reconstruction u sing the Chambolle-Pock algorithm." In Proc. of Joint Annual Meeting ISMRM-ESMRMB , Paris, France, no. 3552, 2018. [8] G. E. Hinton, and R. R. Salakhutdinov. "Reducing the dimensionality of data with neural n etworks." Science 313, no. 5786 (2 006): 504-507. [9] A. Krizhevsky, I. Su tskever, and G . E. Hinton. "Image-net classification with deep convolutional neural netw orks." In Advances i n n eural in formation p rocessing systems , pp. 1097-1105. 2012. [10] M. I. Jo rdan, and T. M. Mit chell. "Machine learnin g: Trends, perspectives, and prospects." Science 349, no. 6245 (2015): 255-260. [11] Y. LeCun, Y. Bengio, and G. E. Hinton. "Deep learning." Nature 5 21, no. 7553 (2015): 436. [12] H. Ch en, D. Ni, J. Qin, S. Li, X. Yang, T. Wang, P. A. Heng. “Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks.” IEEE J Biomed Health Inform . vol. 19, no. 5, pp. 1627-36, Sep. 2 015. [13] H. Ch en, Y. Zhang, W. Zh ang, P. Liao, K. Li, J. Zho u, G. Wang. “Low-dose CT via con volutional neural network.” B iomed Op t Express. vol .8, no. 2, p p. 679-694, Jan.. 2017. [14] J. Adl er, O. Oktem. “Learned Pri mal-Dual Reconstructi on.” IEEE Tra ns Med Imag ing. vol. 37, no. 6, pp. 1322-1332, 2018. [15] K. Gong, J. Guan, K. Kim, X. Zhang, J. Yang, Y. Seo, G. El Fakhri, J. Qi, Q. Li. “ Iterative PE T Image Reconstructio n Using Convolutional Neural Network Representation.” IEEE Trans Med Imagin g. Sep. 2 018. doi: 10.1109/TMI.2018.2869871. [16] S Wan g, Z Su, L Ying, et al. “Accelerating magnetic resonance imaging via deep learning,” ISBI2016 , p p. 514-517. [17] K. Kwon, D. Kim, and H. Park, “A p arallel MR imaging method using mult ilayer perceptron,” Medical Physics , vol. 44, no. 12, pp. 6209–6224, 2017. [18] Y. Han, J. Yoo , H. H. Kim, H. J. Shin, K. Sung, and J. C. Ye, “Deep learning with domain adap tation for accelerated projection-reconstruction MR,” Mag n Reson Med , vol. 80, no. 3, p p. 1189–1205, 2 018. [19] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Ro sen, and M . S. Rosen, “Image recon struction by domain-transform mani fold learning,” Nat ure , vol. 5 55, no. 7697, p . 487, 2018. [20] T. Eo, Y. Jun , T. Kim, J. Jang, H.-J. Lee, and D. Hwang, “KIKI-net: cross-domain con volutional neural netw orks f or reconstructing u ndersampled magnetic resonance images,” Magn Reson Med , vol. 80, no. 5, pp. 2 188-2201, 2018. [21] J. Sch lemper, J. Cab allero, J. V. Hajnal, A. N. Price, and D. Rueckert, “A deep cascade of convolutional neural networks for dynamic MR image reconst ruction,” IE EE Tra ns. Med. Imag., vol. 37, no. 2, pp. 491–503, 2018. [22] T. M. Quan, T. Nguyen-Duc, and W.-K. Jeong, “Compressed sensing M RI reconstructio n using a g enerative ad versarial network with a cyclic l oss,” IEEE Trans. Med. Imag., vol. 3 7, no. 6, pp. 1488–1497, 2 018. [23] Y. Yang, J. Sun, H. Li, and Z. Xu, "Deep ADMM-Net for Compressive Sensing MRI," presented at the 30th Con ference on Neural In formation Processing Systems , Barcelon a, Spain, 2016. [24] K. Hammernik, T. Klatzer, E. Ko bler, et al. “ Learning a Variational Network for Reconstruction of Accelerated MRI Data,” Magn Reson Med , vol. 7 9, no. 6, pp. 3055-3071, 2018. [25] C. Qin, J. V. Hajnal, D. Rueckert, J. Schlemper, J. Caballero, and A. N. Price, “ Convolutional recurrent neural networks for dynamic MR image reconstruction,” IEEE Trans. Med. Ima g., vol. 38, no. 1, p p. 280-290, 2019. [26] S. Nassirpour, P. Chang, an d A. Henning. "MultiNet PyGRAPPA: Mu ltiple neural networks for reconstru cting variable density GRAPPA (a 1 H FID MRSI stu dy)." NeuroImage , vol. 183, pp.336-345, 2018 [27] J. Cheng, H. Wang, L. Ying, and D. Liang, "Learning Primal Dual Network for Fast MR Imagin g," In th e Proc. 27th Annual Meeting of ISMRM , Montreal, QC, Canad a, no. 4775, 2019. [28] J. Yang, Y. Z hang, W . Yin, et al. “A Fast Alternating Directio n Method for T VL1-L2 Signal Reconstruction From Partial Fourier Data, ” IEEE JSTSP , vol. 4, no. 2, p p. 288-297, 2010. [29] H. Wang, L . Tam, E. Kop anoglu, D.C. Peters, R.T. Constable, G. Galiana. “E xperimental O-space turbo spin echo imaging,” Magn Reson Med. vol. 75, no. 4, pp. 1 654-61, 2016 [30] H. Wan g, L.K. Tam, E . Kopanoglu, D.C. Peters, R.T. Constable, G. Galiana. “O-space with high resol ution reado uts outperforms rad ial imaging,” Magn Reson Imaging . vol. 3 7, pp. 107-115, 2017.