Deep Learned Optical Multiplexing for Multi-Focal Plane Microscopy

1 Deep Learned Optical Multiple xing for Multi-F ocal Plane Microscopy Y i Fei Cheng, Ziad Sabry , Megan Strachan, Skyler Cornell, Jake Chanenson, Ev a-Maria S. Collins, and V idya Ganapati Abstract —T o obtain microscope images at multiple focal planes, the distance between the objective and sample can be mechanically adjusted. Images are acquir ed sequentially at each axial distance. Digital ref ocusing with a light-emitting diode (LED) array microscope allows elimination of this mechanical movement. In an LED array micr oscope, the light source of a con- ventional widefield microscope is r eplaced with a 2-dimensional LED matrix. A stack of images is acquired from the LED array microscope by sequentially illuminating each LED and capturing an image. Previous work has shown that we can achieve digital ref ocusing by post-processing this LED image stack. Though mechanical scanning is eliminated, digital refocusing with an LED array microscope has low temporal r esolution due to the acquisition of multiple images. In this work, we pr opose a new paradigm for multi-focal plane microscopy f or live imaging, utilizing an LED array microscope and deep learning . In our deep learning approach, we look for a single LED illumination pattern that allows the information from multiple focal planes to be multiplexed into a single image. W e jointly optimize this LED illumination pattern with the parameters of a post-processing deep neural network, using a training set of LED image stacks from fixed, not live, Dugesia japonica planarians. Once training is complete, we obtain multiple focal planes by inputting a single multiplexed LED image into the trained post-processing deep neural network. W e demonstrate li ve imaging of a D. japonica planarian at 5 f ocal planes with our method. I . I N T RO D U C T I O N A N D B AC K G RO U N D When looking at a thick specimen through a microscope, parts of the specimen not in the focal plane appear blurred. The blur is increased in parts of the specimen further away from the focal plane. If the axial distance between the specimen and the microscope objective is adjusted, a dif ferent axial plane will come into focus. Imaging at multiple focal planes thus allo ws us to obtain an understanding of the specimen’ s 3-dimensional structure. Generally , to refocus to a different axial plane, mechanical adjustment of the distance between the sample and optics is necessary . Often it is e xpensiv e and time-intensiv e to mechanically scan the sample. Scanning and collecting images at multiple focal planes is especially difficult for liv e imaging, as the sample might move during acquisition of an image stack. Digital refocusing refers to techniques that emulate axial mechanical scanning without moving the sample or optics. Y . F . Cheng, M. Strachan, S. Cornell, J. Chanenson and V . Ganapati are with the Engineering Department, Swarthmore College, Swarthmore, P A 19081, USA. Z. Sabry and E.-M. S. Collins are with the Biology Department, Sw arthmore College, Swarthmore, P A 19081, USA. E-mail: vganapa1@swarthmore.edu. One method to achieve digital refocusing is light-field mi- croscopy [1]. In light-field microscopy , a lenslet array is used to capture angular information at each spatial point. From the angular and spatial information collected, we can computationally post-process and determine what the image looks like at different focal planes. Howe ver , the collection of angular information comes with a tradeoff; we lose spatial resolution. Another way to capture multiple planes in a single-shot is to use a carefully engineered diffracti ve optical element [2]– [4]. W ith this approach, the pixels of the camera are split into blocks, and each block contains an image from a dif ferent focal plane. Each focal plane image has a reduced field-of-view since the image sensor pixels are split up. This method also has the disadvantage of requiring a costly dif fractiv e optical element. Phase retrie v al techniques also capture information that allows for refocusing to multiple planes. Many phase retriev al techniques use interferometry , which requires well-controlled alignment of a laser . Non-interferometric phase retriev al tech- niques generally require the acquisition of multiple images, sometimes with mechanical scanning [5]. Multi-focal plane microscopy is also possible with a light- emitting diode (LED) array microscope. In an LED array microscope, the light source of a con ventional widefield micro- scope is replaced with a 2-dimensional LED matrix. A stack of images can be acquired from the LED array microscope by se- quentially illuminating each LED and acquiring an image. The LEDs are far enough away from the sample that single LED illumination can be approximated as a plane wa ve at a certain angle. From the stack of collected images, dif ferent focal plane images can be reconstructed [6], [7]. The reconstruction of the focal stack is similar to light-field processing, except without the spatial resolution tradeoff. Though mechanical scanning is eliminated, digital refocusing with an LED array microscope still has lo w temporal resolution, due to the necessity of acquiring one image per LED. An LED array microscope has proven useful for techniques other than digital refocusing, such as high space-bandwidth reconstruction of complex objects [8]. For a thin sample, we can use an LED array microscope to collect the image stack (1 image per LED) and then computationally reconstruct both phase and amplitude. In the computational reconstruction, we achiev e resolution above the numerical aperture of the micro- scope objectiv e. The field-of-view is unchanged, but resolution is improved, so the reconstruction has a high space-bandwidth product. Howe ver , the acquisition time is long. Multiplex ed 2 LED patterns, where multiple LEDs are illuminated at once, allow the number of total images to be reduced [9], [10]. T o further reduce the number of images needed, deep learning has been applied [11]–[18]. In our pre vious work, we demonstrated the use of deep learning to allow object reconstruction with a single multiplex ed LED image [11], [12]. During training, we co-optimized the LED illumination pattern with a post-processing deep neural network. The post- processing deep neural network computationally transforms the single collected image into the complex object we would expect from iterativ e processing with a full stack of single LED images. By including the LED illumination pattern in the optimization of the neural network, we find that the mutual information between the collected image and the desired object reconstruction increases [11]. Our finding highlights the importance of “physical preprocessing:” optimizing physical parameters of the imaging system to collect the most infor- mativ e measurements. In this work, we apply the idea of physical preprocessing to multi-focal plane microscopy . Our goal is to enable single- shot collection of multiple focal plane images without sac- rificing resolution or field-of-view . W e achiev e this goal by co-optimizing a single multiplexed LED illumination pattern with a post-processing deep neural network. In this approach, our LED illumination pattern and neural network parameters are tuned for a specific sample type. W e mak e sev eral new contributions in this work. First, we demonstrate single-shot multi-focal plane microscopy without loss of resolution or field-of-view , with minimal hardware modifications to a conv entional widefield microscope. Second, we show that the mutual information between the collected measurement and desired multi-focal image reconstruction is increased with our optimized LED illumination pattern. W e show this increase in mutual information for real data, as opposed to simulated data in [11]. Finally , we demonstrate single-shot live sample imaging at 5 focal planes. I I . M E T H O D S A. LED Array Micr oscope Our experimental setup is shown in Fig. 1. A programmable matrix of 32 × 32 LEDs with a 4 mm pitch (Adafruit) is the illumination source of a widefield microscope (Nikon Eclipse TE300). In this work, only the 69 centermost LEDs of the matrix are utilized. The LED matrix is at an axial distance of 69.5 mm away from the sample, and the LED w av elength is centered around 518 nm. Images are collected using a 16 bit image sensor with 2048 × 2048 pixels, with pixel size of 6.5 × 6.5 µ m (pco.edge 4.2 L T). The microscope objectiv e has 10 × magnification and numerical aperture of 0.3 (Nikon CFI Plan Fluor). W e assume that the LEDs are far enough aw ay from the sample that the illumination from a single LED approximates a plane wa ve [8]. W e also assume that the light from the LEDs is mutually incoherent, as in [9]. T o approximate the image from a multiplexed LED illumination pattern, we can take a weighted sum of the single LED image stack: Fig. 1: In our optical setup, we replace a con ventional micro- scope light source with a 2-dimensional light-emitting diode (LED) matrix. Each LED has 8 brightness levels, from no light to maximum brightness. Multiple LEDs can be illuminated at once to create different multiplexed illumination patterns. I = n X i =1 c i I i , (1) where n is the total number of illuminated LEDs and c i is the relativ e intensity of LED i . B. Digital Refocusing with Shift-Add W ith our LED array microscope, we collect a stack of 69 images. Each image is illuminated using a single LED and captured with an exposure time of 2 seconds. From this stack of images, we can digitally refocus to different focal planes with the shift-add algorithm [6], [7]. Similar to con ventional light-field processing [1], the shift-add algorithm makes the assumptions of geometric optics. W e assume that the light from a single LED impinging on the sample all comes from a single direction. If the thick 3-dimensional sample is assumed to modulate only the intensity of the incoming light and not the phase, we assume by geometric optics that the light at the image plane remains at the same angle from the optical axis. If we simply add all the LED images together, we get the image at the focal plane. This is the same image as w ould be obtained by physically illuminating all the LEDs simultaneously . In order to digitally refocus to some axial offset ∆ z , we must shear the LED stack and add the resulting images together . The amount each LED image must be shifted depends on both ∆ z and the angle of the light rays. A steeper angle and a greater ∆ z of fset both mean a greater shift. The lateral position of the LED ( x i , y i ) determines the angle of illumination ( θ x , θ y ) : tan ( θ x ) = x i z , (2) 3 tan ( θ y ) = y i z , (3) where z denotes the distance between the sample and the LED matrix. T o refocus to an axial position z + ∆ z , each LED image must be shifted by ∆ x and ∆ y as follows: ∆ x = ∆ z tan ( θ x ) , (4) ∆ y = ∆ z tan ( θ y ) . (5) T o obtain subpixel shifts, we shift in F ourier space as follows: I shift = F − 1  F ( I ) e j 2 π ( u ∆ x + v ∆ y )  , (6) where I denotes the collected LED intensity image, F and F − 1 denote the F ourier and in verse F ourier transforms respec- tiv ely , and u and v denote the spatial frequency coordinates. T o obtain the final refocused image at z + ∆ z , we sum all the shifted images: I ∆ z = n X i =1 I shift ,i (7) where n denotes the total number of illuminated LEDs and I shift ,i is the shifted intensity image from LED i . Fig. 2 shows an example of the results of the shift-add algorithm. Fig. 2: Digital refocusing of a D. japonica planarian over 1 mm with application of the shift-add algorithm to the 69 LED image stack. C. Deep Learning The shift-add algorithm described in the previous section requires a separate image to be collected from each LED. Though we hav e eliminated mechanical scanning in this method, we still hav e to collect multiple images. In this work, we take a deep learning approach in order to obtain the same results with just a single collected image. In this deep learning approach, we first collect a training dataset using sequential LED scanning and image acquisition. F or each field-of-view in our training dataset, we collect all 69 single LED images. W e then construct a neural network graph, where the first layer outputs a linear weighted sum of the 69 input images using Eqn. 1. The weights of the layer correspond to the relati ve intensities of each LED in a multiplexed LED illumination pattern. The output of this layer is thus the image you would expect if the multiplex ed illumination pattern was used in the LED array microscope. This single image is inputted into a con volutional neural network that outputs m images, where m is the number of focal planes we are trying to reconstruct. The con volutional neural network uses residual connections and is inspired by the network architecture in [19]. The network architecture used in this w ork is similar to that in [11], [12], with the exception that we have m output images, instead of a single complex output. The LED illumination pattern and con volutional network weights and biases are optimized by training the deep neural network. Our objectiv e in training is to minimize the error between the neural network output and the results of the shift- add algorithm. W e calculate the error using a combination of mean-squared error of the images and the first differences , as in [11], [12]. As in [11], [12], in ev ery iteration of training, random Poisson noise is added to the multiplexed LED image, and image saturation is modeled. At the end of training, we normalize the LED weights and modify the exposure time accordingly , so that we obtain the lowest e xposure time. As our LED array can only accommodate 8 discrete brightness lev els, we also round the LED weights to the nearest lev el at the end of training. Our approach is summarized in Fig. 3. 1) T raining: F or the training dataset, we collect LED image stacks in 90 fields-of-view (6 sample slides, 15 field-of-views per sample). Each LED stack consists of 69 images acquired with sequential LED scanning. The biological specimens used in this work are Dugesia japonica planarians. For the training dataset, D. japonica planarians are fixed following standard procedures in [20]. Planarians are placed into 100 % glycerol for mounting on custom made tunnel slides using double-stick tape, as in [21]. The single LED image stacks are used to train the neural network graph to determine the optimal multiple xed LED illumination pattern and parameters of the post-processing con volutional neural network. 2) F ine-T uning and Evaluation: In the training of the neural network, we model the formation of a single image under a multiplex ed LED illumination pattern. T o account for errors in this model, a fine-tuning step is performed, wherein a 69 LED image stack is collected as before. In addition, a multiplexed LED image illuminated with the optimized LED pattern is also collected. The multiplexed LED image is inputted directly into the post-processing con volutional neural network, and the weights and biases are re-optimized. Again, we minimize the error between the output of the neural network and the result of the shift-add algorithm on the full 69 LED image stack. After fine-tuning is complete, we ev aluate the performance of the neural network using fields-of-view that were not pre- viously used in training or fine-tuning. T o prevent overfitting, no training or fine-tuning w as performed after the e v aluation results were obtained. For the fine-tuning and ev aluation steps, we collect images in 40 total fields-of-view (2 sample slides, 20 fields-of-vie w per sample). The ev aluation dataset consists of 8 random field- of-views selected from this dataset. Fig. 4 illustrates an ev aluation example at various points in the imaging pipeline. 4 Fig. 3: Diagram of training and ev aluation of our deep learning framew ork. (a) In the training step, 69 images are collected for each field-of-view , with each of the 69 images corresponding to illumination with a single LED. During training, we emulate the collection of a single image with a multiplex ed illumination pattern. In the illumination pattern, every LED in the matrix is allowed to take a grayscale brightness value. This emulated single image is fed into a post-processing neural network. The output of the post-processing neural network is 5 images, with each image attempting to focus at a different z distance. The illumination pattern and the parameters (weights and biases) of the post-processing neural network are co-optimized during training. The optimization objectiv e is to minimize the difference between the neural network output and a digitally refocused image stack. The digitally refocused image stack of 5 focal planes is calculated by applying the shift-add algorithm to the stack of 69 LED images. (b) In the ev aluation step, the optimized illumination pattern is programmed onto the LED matrix of the actual microscope. The single image that is collected from the microscope is then directly fed into the trained post-processing neural network. In the ideal case, the output of the neural network, obtained by a single image, should match the result obtained by collecting 69 separate LED images and applying the shift-add algorithm. 3) Live Imaging: After the neural network is trained and fine-tuned, we collect videos of li ve, unfix ed D. japonica planarians with the optimized LED illumination pattern. W e then feed each frame of the video into the post-processing neural netw ork layers to reconstruct 5 focal plane images per frame. In this step, we no longer need the full 69 LED image stack. By keeping the LED matrix statically illuminated with the optimized LED illumination pattern, we can perform single-shot, live imaging of 5 focal planes without loss in resolution or field-of-view . For liv e imaging, planarians are mounted in a drop of 1 × Instant Ocean (IO, Blacksbur g, V A, USA) and then most of the water is removed. I I I . R E S U L T S A N D D I S C U S S I O N A. Comparison of Shift-Add and Deep Learning In Fig. 5 we show a comparison of results between the shift- add algorithm (requiring the full LED stack of 69 images) and the deep learning approach proposed in this work (requiring a single multiplex ed LED image) for 3 fields-of-vie w in the ev aluation dataset. The linescans show a good match between the 2 methods. W e note that it appears that the neural network approach filters out some random noise present in the shift-add algorithm results. This noise filtering agrees with the results in [22], where the authors demonstrate that a neural network cannot learn zero-mean noise. B. Mutual Information W ith a small simulated dataset, we show in [11] that optimizing the physical parameters of data collection increases the mutual information between the measurement and the desired final output. W e call data collection with these trained parameters “optimal physical preprocessing. ” Optimal physical preprocessing allo ws for less data to be collected without harming the final reconstruction result. This lets us create faster and cheaper imaging systems with no loss in image quality . In this work, we show a mutual information increase with optimal physical preprocessing for a larger dataset of real images. W e determine the mutual information between 16 × 16 pixel patches of the multiplexed LED images and the corresponding 5 focal plane stacks of size 5 × 16 × 16 pixels. Mutual infor- mation quantifies how much the multiplexed LED illumination images “tell” us about the 5 focal plane stack. If the mutual information matches the entropy of the focal plane stacks, we should be able to perfectly reconstruct the 5 focal planes from the multiplexed LED image. Mutual information is calculated by: 5 Fig. 4: This figure illustrates an e valuation example, a field- of-view that w as not used in training or fine-tuning. The top row shows 5 of the total 69 LED images, each illuminated by a single LED of the LED matrix. The next row shows the single LED image obtained by illuminating with the optimized illumination pattern. This optimized LED pattern image is transformed by the post-processing neural network to generate the focal stack sho wn in the third row . The forth and final row is the focal stack computed by the shift-add algorithm, using all 69 single LED images. Qualitatively , we see that there is agreement between the predicted digitally refocused image stack from the neural network and the actual digitally refocused stack from the shift-add algorithm. W e note that the shift-add algorithm assumes geometric optics and doesn’t take into account wav e ef fects. Thus, the coherent wav e diffraction patterns present in the single LED images are av eraged out in the refocused image stack. X y ∈ Y X x ∈ X p ( x, y ) log 2  p ( x, y ) p ( x ) p ( y )  , (8) where p ( x ) is the marginal probability distribution of 5 focal plane stacks, p ( y ) is the marginal probability distrib ution of the multiplexed LED images, and p ( x, y ) is the joint prob- ability distribution. The Non-Parametric Entropy Estimation T oolbox (NPEET) [23] was used to estimate the probability distributions and calculate the mutual information. F or every x ∈ X , we generated 10 samples of y with random Poisson noise to approximate the distribution p ( y | X = x ) . W e show the mutual information with different multiplexed LED illumination patterns in Fig. 6, comparing mutual infor- mation among optimized, uniform, and random LED illumi- nation patterns. W e observe the highest mutual information for the optimized LED illumination pattern. Our observ ation validates the simulation result in [11] with real data and underscores the importance of optimal physical preprocessing. In our joint optimization procedure of the illumination pattern and the post-processing parameters, the LED illumination pat- tern changes so that more information about the desired 5 focal Fig. 5: Comparisons between the actual digitally refocused image stack, using 69 single LED images with the shift- add algorithm, and the predicted digitally refocused stack, using only the single image collected with the optimized LED illumination pattern and neural netw ork post-processing. The linescans show good fidelity of the neural network output to the ground truth shift-add result. plane stack is physically encoded in the single multiplex ed LED image. C. Live Imaging The deep learning approach in this work allows us to train on fixed samples but ev aluate on li ve samples. The training step requires collection of the full 69 LED image stack (1 image per LED) per field-of-view . Once training is complete, only a single image is needed for multiple focal-plane re- construction. With the trained LED illumination pattern and post-processing neural network parameters, we demonstrate liv e imaging of a D. japonica planarian. Fig. 7 shows time- lapse results with 192 ms exposure, resulting in a rate of 5.2 frames/s. W e note that with brighter LEDs we could reduce the exposure time and achieve a higher frame rate. 6 Fig. 6: Approximately 300,000 image patches of 16 × 16 pixels from the ev aluation dataset were used to calculate the mutual information of the optimized multiplex ed image and the focal stack. The higher the mutual information, the more we know about the focal stack from the optimized optical element image. If the mutual information is the same as the entropy of the focal stack, the optimized optical element image gi ves us all the information to determine the focal stack exactly . W e also calculate the mutual information using a uniform LED illumination pattern and 5 dif ferent random LED patterns. The uniform and random LED patterns are normalized such that they hav e the same av erage value as the optimized pattern. The mutual information calculated with the optimized LED pattern and the uniform pattern are shown. W e show the random LED pattern that achiev ed the highest mutual information of the 5 random patterns. In this dataset of image patches, the entropy of the focal stack is 17.84 bits. Thus, we cannot e xpect perfect reconstruction from any of the 3 LED patterns shown, b ut we can expect the best reconstruction from the optimized LED illumination pattern. I V . C O N C L U S I O N S In this work, we demonstrate proof-of-concept of a single- shot method to obtain multiple focal stacks without loss in image quality , enabling imaging of li ve, moving samples. Multiple focal plane imaging allows for understanding of the 3-dimensional structure of the sample. While the proof-of- concept we demonstrate is promising, there are a number of limitations that we plan to address in future work. One limitation of this work is the use of the shift-add algo- rithm in generating the ground truth focal stacks. The shift-add algorithm makes the assumptions of geometric optics, leading to artifacts in the refocused planes. The ground truth could be collected by mechanical axial scanning instead. Another method is to use a phase mask in Fourier space on the imaging side in order to reconstruct the refocused planes using the assumptions of scalar wave optics [24]. It should also be noted that though 3-dimensional in- formation is available from multiple focal plane stacks, 3- dimensional reconstruction cannot be directly inferred. In fluorescence microscopy , decon volution algorithms are used to extract point emitter density in 3 dimensions from focal plane stacks [25]. This work considers transmission light microscopy , where the aim is to recov er the refractive index at ev ery point in 3-dimensional space. The scattering from a 3- dimensional structure can be approximated with the first Born approximation [24], [26]–[28] or the multi-slice model [29], [30]. The deep learning approach outlined in this paper can be applied to directly obtaining the 3-dimensional structure of the sample from a single multiplexed LED image, using one of these approximate forw ard models to iterati vely solve for the ground truth 3-dimensional refractiv e index. Another limitation of this work is that we cannot change the number and spacing of the focal planes after the training step. This could be solved if we could reconstruct the entire 69 image stack from a single multiplexed LED image. Then, refocusing to any number of planes and spacings from li ve imaging data could be done at any time after data collec- tion. W e note that we were not able to achie ve full mutual information from a single multiplex ed LED image for 5 focal planes. Reconstructing the 69 single LED image stack may require 2 or more multiplexed LED images. Future work will determine ho w many multiplexed images are needed for full mutual information between the data collected and desired reconstruction. T o get single-shot imaging of multiple depth planes without loss of resolution or field-of-view , we trade-off generality of the imaging method. Our method assumes that the object imaged comes from the same probability distribution as the training set of objects. Further work is needed to answer ques- tions such as: how well does this method work, when training on objects from one probability distribution, and ev aluating on a different probability distrib ution? Finally , future work will quantify exactly how many samples are needed for training. A C K N O W L E D G M E N T S W e acknowledge funding from the Lang Center at Swarth- more College, the Surdna Foundation (Y . F . C.), the Swarth- more College Summer Research Fello wship (M. S.), and NSF CAREER Grant 1555109 (E.-M. S. C.). W e thank Leonora Blodgett for helpful comments on the manuscript. R E F E R E N C E S [1] M. Le voy , R. Ng, A. Adams, M. Footer , and M. Horowitz, “Light Field Microscopy, ” in ACM SIGGRAPH 2006 P apers , ser. SIGGRAPH ’06. New Y ork, NY , USA: ACM, 2006, pp. 924–934. [2] P . M. Blanchard and A. H. Greenaway , “Simultaneous multiplane imaging with a distorted diffraction grating, ” Applied Optics , vol. 38, no. 32, pp. 6692–6699, Nov . 1999. [3] S. Abrahamsson, R. Ilic, J. Wisnie wski, B. Mehl, L. Y u, L. Chen, M. Dav anco, L. Oudjedi, J.-B. Fiche, B. Hajj, X. Jin, J. Pulupa, C. Cho, M. Mir, M. E. Beheiry , X. Darzacq, M. Nollmann, M. Dahan, C. W u, T . Lionnet, J. A. Liddle, and C. I. Bargmann, “Multifocus microscopy with precise color multi-phase diffracti ve optics applied in functional neuronal imaging, ” Biomedical Optics Express , vol. 7, no. 3, pp. 855– 869, Mar . 2016. [4] K. He, Z. W ang, X. Huang, X. W ang, S. Y oo, P . Ruiz, I. Gdor , A. Selewa, N. J. Ferrier , N. Scherer, M. Hereld, A. K. Katsaggelos, and O. Cossairt, “Computational multifocal microscopy , ” Biomedical Optics Expr ess , vol. 9, no. 12, pp. 6477–6496, Dec. 2018. 7 Fig. 7: Our deep learning approach allows us to obtain a 5-plane focal stack with single-shot imaging. This allows for live imaging of a D. japonica planarian. T ime lapse of liv e video shown, supplementary video av ailable online . Each row depicts the planarian at one time point, at 5 different focal planes. [5] G. Popescu, Quantitative phase imaging of cells and tissues , ser. McGraw-Hill Biophotonics. New Y ork: McGraw-Hill, 2011, oCLC: ocn659763966. [6] G. Zheng, C. K olner, and C. Y ang, “Microscopy refocusing and dark- field imaging by using a simple LED array, ” Optics Letters , vol. 36, no. 20, pp. 3987–3989, Oct. 2011. [7] L. T ian, J. W ang, and L. W aller, “3d differential phase-contrast mi- croscopy with computational illumination using an LED array , ” Optics Letters , vol. 39, no. 5, p. 1326, Mar . 2014. [8] G. Zheng, R. Horstmeyer , and C. Y ang, “W ide-field, high-resolution Fourier ptychographic microscopy , ” Natur e photonics , vol. 7, no. 9, pp. 739–745, Sep. 2013. [9] L. Tian, X. Li, K. Ramchandran, and L. W aller, “Multiplexed coded illumination for Fourier Ptychography with an LED array microscope, ” Biomedical Optics Express , vol. 5, no. 7, p. 2376, Jul. 2014. [10] L. Tian, Z. Liu, L.-H. Y eh, M. Chen, J. Zhong, and L. W aller, “Com- putational illumination for high-speed in vitro Fourier ptychographic microscopy , ” Optica , vol. 2, no. 10, p. 904, Oct. 2015. [11] A. Robe y and V . Ganapati, “Optimal physical preprocessing for example-based super-resolution, ” Optics Express , vol. 26, no. 24, pp. 31 333–31 350, Nov . 2018. [12] Y . F . Cheng, M. Strachan, Z. W eiss, M. Deb, D. Carone, and V . Ganapati, “Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy, ” Optics Express , vol. 27, no. 2, pp. 644–656, Jan. 2019. [13] M. Kellman, E. Bostan, M. Chen, and L. W aller, “Data-Dri ven Design for Fourier Ptychographic Microscopy, ” arXiv:1904.04175 [cs, eess] , Apr . 2019. [14] A. Kappeler, S. Ghosh, J. Holloway , O. Cossairt, and A. Katsaggelos, “Ptychnet: CNN based fourier ptychography , ” in 2017 IEEE Interna- tional Conference on Image Processing (ICIP) , Sep. 2017, pp. 1712– 1716. [15] T . Nguyen, Y . Xue, Y . Li, L. Tian, and G. Nehmetallah, “Deep learning approach for F ourier ptychography microscopy, ” Optics Express , vol. 26, no. 20, pp. 26 470–26 484, Oct. 2018. [16] L. Boominathan, M. Maniparambil, H. Gupta, R. Bab urajan, and K. Mi- tra, “Phase retrie val for Fourier Ptychography under varying amount of measurements, ” arXiv:1805.03593 [cs] , May 2018. [17] Y . Xue, S. Cheng, Y . Li, and L. Tian, “Reliable deep-learning-based phase imaging with uncertainty quantification, ” Optica , vol. 6, no. 5, pp. 618–629, May 2019. [18] J. Zhang, T . Xu, Z. Shen, Y . Qiao, and Y . Zhang, “Fourier ptychographic microscopy reconstruction with multiscale deep residual network, ” Op- tics Expr ess , vol. 27, no. 6, p. 8612, Mar . 2019. [19] X.-J. Mao, C. Shen, and Y .-B. Y ang, “Image Restoration Us- ing Con volutional Auto-encoders with Symmetric Skip Connections, ” arXiv:1606.08921 [cs] , Jun. 2016. [20] D. Hagstrom, S. Zhang, A. Ho, E. S. Tsai, Z. Radi, A. Jahromi, K. J. Kaj, Y . He, P . T aylor, and E.-M. S. Collins, “Planarian cholinesterase: molecular and functional characterization of an evolutionarily ancient enzyme to study organophosphorus pesticide toxicity , ” Archives of T oxicology , v ol. 92, no. 3, pp. 1161–1176, Mar . 2018. [21] J. A. T albot, K. W . Currie, B. J. Pearson, and E.-M. S. Collins, “Smed- dynA-1 is a planarian nervous system specific dynamin 1 homolog required for normal locomotion, ” Biology Open , vol. 3, no. 7, pp. 627– 634, Jul. 2014. [22] J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T . Karras, M. Aittala, and T . Aila, “Noise2Noise: Learning Image Restoration without Clean Data, ” in International Conference on Machine Learning , Jul. 2018, pp. 2965–2974. [23] G. V . Steeg, “Non-parametric Entropy Estimation T oolbox, ” Oct. 2014. [Online]. A vailable: https://github .com/gregv ersteeg/NPEET 8 [24] X. Ou, J. Chung, R. Horstmeyer , and C. Y ang, “ Aperture scanning Fourier ptychographic microscopy , ” Biomedical Optics Express , vol. 7, no. 8, pp. 3140–3150, Aug. 2016. [25] J.-B. Sibarita, “Deconv olution Microscopy, ” in Microscopy T echniques , ser . Adv ances in Biochemical Engineering, J. Rietdorf, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005, pp. 201–243. [26] R. Horstmeyer , J. Chung, X. Ou, G. Zheng, and C. Y ang, “Diffraction tomography with Fourier ptychography, ” Optica , vol. 3, no. 8, pp. 827– 835, Aug. 2016. [27] R. Ling, W . T ahir, H.-Y . Lin, H. Lee, and L. T ian, “High-throughput intensity dif fraction tomography with a computational microscope, ” Biomedical Optics Express , vol. 9, no. 5, p. 2130, May 2018. [28] J. Li, A. Matlock, Y . Li, Q. Chen, C. Zuo, and L. T ian, “High-speed in vitro intensity diffraction tomography , ” arXiv:1904.06004 [physics] , Apr . 2019. [29] P . Li, D. J. Batey , T . B. Edo, and J. M. Rodenb urg, “Separation of three-dimensional scattering effects in tilt-series Fourier ptychography , ” Ultramicr oscopy , vol. 158, pp. 1–7, Nov . 2015. [30] L. Tian and L. W aller, “3d intensity and phase imaging from light field measurements in an LED array microscope, ” Optica , vol. 2, no. 2, pp. 104–111, Feb . 2015.