Machine learning applied to single-shot x-ray diagnostics in an XFEL

Mac hine learning applied to single-shot x-ra y diagnostics in an XFEL A. Sanc hez-Gonzalez, 1 P . Micaelli, 1 C. Olivier, 1 T. R. Barillot, 1 M. Ilc hen, 2, 3 A. A. Lutman, 4 A. Marinelli, 4 T. Maxw ell, 4 A. Ac hner, 3 M. Agåker, 5 N. Berrah, 6 C. Bostedt, 4, 7 J. Buck, 8 P . H. Buc ksbaum, 2, 9 S. Carron Mon tero, 4, 10 B. Co op er, 1 J. P . Cry an, 2 M. Dong, 5 R. F eifel, 11 L. J. F rasinski, 1 H. F ukuza wa, 12 A. Galler, 3 G. Hartmann, 8, 13 N. Hartmann, 4 W. Helml, 4, 14 A. S. Johnson, 1 A. Knie, 13 A. O. Lindahl, 2, 11 J. Liu, 3 K. Motomura, 12 M. Muck e, 5 C. O’Grady, 4 J-E. Rubensson, 5 E. R. Simpson, 1 R. J. Squibb, 11 C. Såthe, 15 K. Ueda, 12 M. V acher, 16, 17 D. J. W alk e, 1 V. Zhaunerch yk, 11 R. N. Coffee, 4 and J. P . Marangos 1 1 Dep artment of Physics, Imp erial Col le ge, L ondon, SW7 2AZ, Unite d K ingdom 2 Stanfor d PULSE Institute, SLA C National Ac c eler ator L ab or atory, Menlo Park, California 94025, USA 3 Eur op e an XFEL GmbH, Holzkopp el 4, 22869 Schenefeld, Germany 4 Linac Coher ent Light Sour c e, SLA C National A c c eler ator L ab or atory, Menlo Park, California 94025, USA 5 Dep artment of Physics and A str onomy, Uppsala University, Uppsala, 75120, Swe den 6 Dep artment of Physics, University of Conne cticut, 2152 Hil lside R o ad, U-3046, Storrs, CT 06269, USA 7 A r gonne National L ab oratory, L emont, IL 60439, USA 8 Deutsches Elektr onen-Synchr otr on DESY, Notkestr asse 85, Hambur g, 22607, Germany 9 Dep artment of Physics, Stanfor d University, 382 Via Pueblo Mal l, Stanford, CA 94305, USA 10 California Luther an University, 60 W Olsen Rd, Thousand Oaks, CA 91360, USA 11 Dep artment of Physics, University of Gothenbur g, Origovägen 6B, 41296 Gothenbur g, Sweden 12 Institute of Multidisciplinary R ese ar ch for A dvanc e d Materials, T ohoku University, Sendai 980-8577, Jap an 13 Institut für Physik und CINSaT, Universität K assel, Heinrich-Plett-Str. 40, 34132 K assel, Germany 14 Physics Dep artment, TU Munich, James-F r anck-Str. 1, 85748 Gar ching, Germany 15 MAX IV L ab or atory, Lund University, Box 118, SE-221 00 Lund, Swe den 16 Dep artment of Chemistry, Imp erial Col le ge L ondon, L ondon SW7 2AZ, Unite d Kingdom 17 Dep artment of Chemistry - Ångtröm, Uppsala University, Uppsala, 75120, Swe den X-ra y free-electron lasers (XFELs) are the only sources currently able to pro duce brigh t few-fs pulses with tunable photon energies from 100 e V to more than 10 ke V. Due to the stochastic SASE op erating principles and other technical issues the output pulses are sub ject to large fluctuations, making it necessary to characterize the x-ray pulses on every shot for data sorting purp oses. W e presen t a tec hnique that applies mac hine learning to ols to predict x-ray pulse prop erties using simple electron b eam and x-ray parameters as input. Using this tec hnique at the Linac Coheren t Light Source (LCLS), w e rep ort mean errors b elow 0.3 e V for the prediction of the photon energy at 530 e V and below 1.6 fs for the prediction of the dela y betw een tw o x-ray pulses. W e also demonstrate sp ectral shap e prediction with a mean agreement of 97%. This approach could p otentially b e used at the next generation of high-repetition-rate XFELs to pro vide accurate knowledge of complex x-ra y pulses at the full repetition rate. I. INTR ODUCTION X-ra y free-electron lasers (XFELs) 1–3 are emerging as one of the most versatile to ols in x-ra y research, b ecoming widely used b y the scien tific comm unit y , as well as indus- try , in many fields including physics, chemistry , biology , and material science. Their brightness, coherence, tun- abilit y , and abilit y to generate pairs of few-fs multicolor pulses for pump-probe exp eriments 4–7 mak e them ideal sources to p erform diffract-b efore-destroy imaging 8 , res- onan t x-ray sp ectroscopy 9 , and a range of time resolved measuremen ts of picosecond to few-fem tosecond dynam- ics in molecules and atoms 10–16 . A drawbac k to XFELs is their current p o or stability . XFELs are driven b y single-pass electron linear accelera- tors (LINA C) typically sev eral hundred meters in length. High densit y electron bunches are formed in an electron photoinjector, accelerated in radiofrequency (RF) ca vi- ties, compressed in magnetic chicanes, and finally driv en through one or multiple undulator sections where the electrons emit coherent x-ray pulses due to Self Ampli- fied Sp ontaneous Emission (SASE) 17 (see Figure 1) or amplify an external laser seed in High Gain Harmonic Generation (HGHG) schemes 18 . Small fluctuations in, for example, the photoinjector driv e laser, RF amplitudes or phases along the LINAC translate into fluctuations in the XFEL pulse properties. F urthermore, all existing XFEL mac hines op erating at w a v elengths shorter than 4 nm ha ve additional fluctuations due to the sto chastic c haracter of the SASE startup pro cess, and shows only partial longitudinal coherence across the XFEL pulse. As a result, even a perfectly stable electron b eam will exhibit shot-to-shot fluctuations in the fine temporal structure of the x-ray pulse. F or example, when using single-pulse SASE emission at the LINAC Coheren t Ligh t Source (LCLS), fluctuations driven primarily by the LINA C RF systems lead to photon energy jitter of 0.1% to 0.5% full width half maxim um (FWHM), dep ending on the central w av elength. The energy jitter driv es bunch compression jitter leading to pulse length fluctuations of ~5% and in- tensit y fluctuations from 1% to 10%. These num bers are exacerbated in more adv anced lasing schemes such as the t win bunc h tec hnique 5 where t w o electron bunches are 2 accelerated simultaneously to produce t w o pulses with v ariable time dela y and photon energy separation. In this case the in ter pulse delay jitter is in the order of 10 to 15 fs FWHM and the intensit y can fluctuate m uc h more widely (~20-100%). Often the only wa y around suc h instabilities is p er- forming a full x-ray characterization for eac h XFEL shot, using a v ariet y of detection metho ds such as gas detectors (total pulse energy), single-shot x-ray sp ec- trometers (wa velength, spectral shape and even p olar- ization), and transverse deflecting ca vities for the sp ent electron bunc hes (pulse shap e, pulse duration and de- la y betw een pulses). Based on these measurements, one can circumv en t issues with instability b y retaining only the even ts presen ting certain pulse characteristics, or ev en exploiting the jitter to act as an effectiv e p ow er 19 , w av elength 19,20 or dela y scan 21 b y sorting and binning the ev en ts according to those characteristics. Unfortu- nately , some diagnostics that intercept the full b eam suc h as optical spectrometers are incompatible with some ex- p erimen tal setups, requiring the x-ra ys to b e either sent to the diagnostic line or to the sample. F urthermore, man y of those essential diagnostics will not b e compat- ible with the next generation of XFELs driven by su- p erconducting LINACs op erating at MHz rates suc h as Europ ean XFEL 22 or the LCLS-II 23 . Simple shot-to-shot diagnostics such as electron bunc h monitors (b eam p o- sition, b eam energy , p eak curren t), x-ray gas detectors, or particle time-of-fligh t (TOF) detectors can in princi- ple w ork at the increased repetition rate but this will not alw ays b e the case. In general, devices that require in ter- cepting the full electron or x-ra y b eam with a solid, suc h as screens to measure the electron bunc h phase space streak ed by a transverse ca vity or the time to ol 21 for op- tical pulse synchronization, will not be able to cope with the thermal load at the high rep etition rate. Moreov er, devices that rely on a CCD array will not hav e in most cases the required bandwidth for readout and storage of the data at MHz rates. This means that any exp erimen t requiring single-shot characterization will lik ely b e lim- ited to a muc h lo w er (<1 kHz) rep etition rate. In this pap er we propose a general metho d applicable at any XFEL facility to obtain typical complex shot-to- shot diagnostics or any other v ariable at a higher repeti- tion rate than that allow ed b y the corresp onding acqui- sition devices. Using data from the LCLS op erating at 120 Hz as an example, w e found that m uc h of the infor- mation usually extracted from slo w, complex diagnostics suc h as the pump-prob e delay in the t win bunch mo de, the photon energy , or even the sp ectral shap e of the x- ra y pulses, is strongly correlated to electron bunch and x-ra y prop erties measured by fast diagnostics. While this correlations are driv en b y physical pro cesses, performing accurate direct modelling of ev ery exp erimental asp ect in mac hines as complex as XFELs is not currently possible. As an alternative w e use generic linear, quadratic and more complex but well known mac hine learning mo dels suc h as Neural Netw orks (NN) 24 or Supp ort V ector Re- gression (SVR) 25 to describ e the non-trivial hidden corre- lations b etw een the fluctuations in the simple diagnostics and the fluctuations in the complex diagnostics. Simi- lar approac hes ha ve b een successfully used for feedbac k- lo ops at particle accelerator facilities 26 . F urthermore, b y including fast gas detectors, measuring the total x-ra y energy , which is sensitive to SASE fluctuations, we can mak e predictions accounting for some of the sto c hastic jitter. These mo dels can b e fitted or tr aine d to predict the output of complex diagnostics, such as pump-prob e dela y , using a small amoun t of training data obtained for a fraction of the shots containing full diagnostics. After applying standard testing tec hniques and estimating the accuracy of the predictions, the mo dels can b e used to calculate the missing v ariables that could not b e mea- sured using complex diagnostics for all the remaining shots. This has the potential of lessening the load on the data stream requirements in existing machines, as well as pro viding full repetition rate information for ev en the most complex of the diagnostics at the new generation of XFELs. I I. METHODS A. Exp erimen tal setup Exp erimen ts w ere conducted at the LCLS 1 x-ra y free- electron laser operated in the twin bunc h mode 27 at the A tomic, Molecular and Optical Science (AMO) 28 endsta- tion in F ebruary (Expt. 1) and April (Expt. 2) of 2015. The experimental setup is depicted in Figure 1. T w o electron bunc hes were generated at 120 Hz at the photo-injector and accelerated in three differen t acceler- ator sections (L1,L2,L3), interlea v ed with tw o magnetic c hicanes used as bunc h compressors (BC), to energies near 3500 Me V and separated b y 50 Me V 1,5 . The re- sulting x-ray photon energies generated at the undulator w ere near the oxygen edge (540 e V) and separated by 15 e V. A double slotted foil (DSF) was used in the second c hicane (BC2) to partially sp oil eac h of the t wo electron bunc hes in time, limiting the emittance to a few fem- toseconds duration 29,30 . By mo difying bunc h compres- sion settings and the position of the DSF, it w as p ossible to c hange the dela y while maintaining the cen tral photon energy of eac h of the pulses. W e choose to demonstrate our technique in this mo de of op eration due to its v ersa- tilit y for ultrafast experiments, allo wing t w o color, few-fs pulses, with an adjustable dela y that can take any v alue from -100 fs and 100 fs, including zero-delay . The typi- cal energies obtained for each pulse were spread ov er the range 0 to 30 µ J for the double pulse mode, and 15 to 45 µ J in the single pulse mo de. All the data presen ted in this paper w as taken at a fixed position of the foil and compression settings, with the different v alues for the time dela y arising from fluctuations in the mac hine. In order to pro vide a larger range of photon energies, the final electron bunch energy was contin uously scanned us- 3 p h o t o n e n e r g y X - r a y G e n e r a t i o n : Ma g n e t i c U n d u l a t o r T o d i a g n o s t i c s F i r s t B u n c h C o mp r e s s o r ( B C 1 ) e n e r g y F i r s t L i n e a r A c c e l e r a t i o n ( L I N A C ) S e c t i o n ( L 1 ) tw o e lect r o n b u n ch e s a t th e ca th o d e tw o u ltr a sh o r t h igh - e n e r g y e lect r o n b u n ch e s p r o d u cing x - r a y s S e c o n d B u n c h C o mp r e s s o r ( B C 2 ) w i t h D o u b l e S l o t t e d F o i l ( D S F ) e n e r g y S e c o n d L I N A C S e c t i o n ( L 2 ) T h i r d L I N A C S e c t i o n ( L 3 ) V e r n i e r S l i t R F D e fl e c to r Ma g n e t i c D i p o l e ( D U MP ) x - r a y s e n e r g y G a s Mo n i t o r D e t e c t o r s ( G MD s ) F r o m u n d u l a t o r 2 0 0 + e l e c t r o n b u n c h a n d x - r a y f a s t mo n i t o r s 20+ v a r i a b l e s r e c o r d e d f o r e a c h s h o t T r a n s v e r s e C a v i t y ( X T C A V ) X - r a y S p e c t r o me t e r ( O p t i c a l / T O F ) FIG. 1. Schematic of the exp erimental setup. Man y fast electron beam and x-ra y detectors (orange box) are spread through all the XFEL sections. Critical single-shot data from slow er detectors (blue boxes) is recorded at a low er repetition rate to train the machine learning mo dels. ing the v ernier. An optical x-ray sp ectrometer (Expt. 1) and an elec- tron TOF x-ra y sp ectrometer 31 (Expt. 2) were used to measure single-shot sp ectra, eac h op erating at 120 Hz. The optical x-ray spectrometer was calibrated using the absorption of a mylar filter 32 at the oxygen K-edge and at the corresponding π ∗ resonance. The TOF x-ray sp ec- trometer was calibrated using CO Auger electron emis- sion at the o xygen K-edge and neon 2s and 2p photo- electrons at different photon energies. Under the applied exp erimen tal conditions, we found the signal-to-noise ra- tio of the optical sp ectrometer to b e up to 16 times better than that of the TOF sp ectrometer, how ev er, the a ver- age x-ra y p ow er in the double pulse mo de during Expt. 2 w as about three times larger, providing more signal. An x-band transv erse deflecting-mo de cavit y (XTCA V) was used to measure the single-shot sp ec- trogram image of the electron bunches (time-energy distribution) downstream from the undulator at 60 Hz. By comparing images in the lasing and non-lasing cases one can determine the lasing region for eac h of the bunc hes and measure the distance along the time axis to obtain the pump-prob e delay v alues 33–35 (Fig. 5a-b). F our gas detectors based on N 2 fluorescence 36 w ere used to measure the single-shot total x-ra y energy , recording 6 v ariables in total. Hundreds of differen t elec- tron beam parameters w ere measured on eac h shot, how- ev er, only 16 of them w ere recorded at the full rep etition rate. These included p osition monitors 37 (p osition and angle), bunch c harge monitors, and p eak curren t mon- itors at different stages (accelerators, c hicanes, undula- tors) as is indicated in Figure 1. All these diagnostics consist of fast, non-in trusive detectors, and should b e therefore scalable to the MHz regime. A dditionally nearly 300 “slo w” v ariables w ere recorded at 2 Hz b y the Exp erimen tal Ph ysics and Industrial Con- trol System (EPICS) 38 . These v ariables mainly include temp eratures of different sections or devices, pressures in the cham bers, configuration v alues suc h as voltages or field strengths, and the settings of the many slow feed- bac k lo ops that keep the FEL stable. The purp ose of these v ariables was to monitor long term drifts, which can b e useful to understand how the fluctuations evolv e o ver time. More details ab out the v ariables included in the analysis can b e found in Appendix 1. B. Computational metho ds The prop osed tec hnique for the prediction of x-ra y pulse characteristics at higher rep etition rate than mea- sured is summarized in Figure 2. It relies on a fast, high- rep etition-rate data stream containing single-shot infor- mation of simple diagnostics for all the even ts, with in- formation from complex diagnostics obtained at a lo wer rep etition rate and only for a fraction of the even ts. The set of ev ents containing correlated information from all 4 T r ain in g , v alid a t in g & t es t in g P r ed ic t F as t d a t a s tr e am Slo w c om p le x d iagn os tic s F as t simp le d iagn os tic s F as t simp le diagno s tic s Mac h ine lea rn in g mod e l F as t p r e d ic tion of c omp le x d iagn os tic s T r ain ing s e t T es t s e t V alida tion se t R an d om sp lit in t o su b se t s FIG. 2. Sc hematic of the technique based on machine learning to predict complex diagnostics at a high repetition rate based on small samples of complex diagnostics obtained at a muc h low er rep etition rate. devices can be split in three: the training, v alidation, and test sets. The training set is used to train a ma- c hine learning mo del to learn ho w to predict v ariables normally obtained with complex diagnostics, based on input v ariables from simple diagnostics. The v alidation set is used to optimize the hyp erp ar ameters . In this con- text, a h yperparameter is any parameter of the model that is no optimized by the training pro cess. Examples of hyperparameters are the maxim um degree of a polyno- mial mo del, or the n um b er of hidden lay ers in a NN. This optimization is done b y training many differen t v ersions of the same mo del using differen t sets of hyperparame- ters, and then comparing the error on the v alidation set to decide which set of h yp erparameters works best. Fi- nally , the test set is used to test the prediction accuracy of the mo del for the chosen set of hyperparameters. At this point, the mo del can be applied to predict, with a kno wn accuracy , the expected v alues from complex diag- nostics for all the remaining even ts, whic h originally did not ha v e that information. W e tested this approac h using data from LCLS ac- quired at 60 Hz. It w as implemen ted in p ython using the LCLS soft ware package psana 39 at the LCLS serv ers, and lo cally on standard consumer computers. The Scikit- learn 40 framew ork w as used for feature scaling, feature selection, Principal Component Analysis (PCA) 41 and fitting of linear, p olynomial and SVR (Gaussian kernel) mo dels. T ensorflow 42 w as used for the implemen tation of NNs. More than 300 v ariables including signals from gas de- tectors, electron b eam diagnostics, EPICS v ariables and a timestamp, were used as input v ariables or, in the com- mon terminology of mac hine learning, fe atur es for the prediction. More details ab out some of the particular v ariables included can b e found in Appendix 1. The time dela y , obtained from XTCA V, and the photon energy and sp ectral shap e (~350 spectral comp onents), mea- sured with the spectrometers, were used as the output v ariables of the mo dels. More details ab out eac h of these output v ariables can b e found in the corresp onding sub- sections for each of the prediction examples. As part of the feature selection process constant features were elim- inated, as well as features taking a small n umber (<10) of sparse discrete v alues. This normally reduced the total n umber of features to around 90. W e then gradually re- duced the num b er of features included, k eeping only the ones showing a high correlation with the v ariable to b e predicted, setting the threshold by minimizing the error of the v alidation set. Around 40 features w ere normally k ept as a result of this process. A typical dataset consisted of ab out 3 · 10 4 shots. A fil- ter w as applied to remov e all the shots where the total en- ergy w as below 5 µ J (<10% of all shots dep ending on the dataset). Shots presenting outliers in the outputs were also remov ed to av oid training on even ts where the results obtained from the complex diagnostics were potentially unreliable. W e considered as outliers all the v alues sepa- rated from the median of the distribution by more than four times the median absolute deviation. This filtering pro cess remo v ed only a small fraction of the data (<1%), except in the spectral prediction case, where the noise due to single photon spikes in the optical sp ectrometer raised this v alue to 20%. Eac h dataset was then divided randomly into 3 subsets using a common split for our dataset size, with 70% of the data used for training, 15% for v alidation and 15% for testing. The test set w as kept isolated from the rest during the training and optimiza- tion of the mo dels. Eac h of the features was normalized b y subtracting the mean v alue and dividing on the standard deviation. This w as also applied in some cases to the outputs, although w e found the latter to only b e relev an t for the NNs. W e did not find it necessary to use PCA on the features, as the total num ber of features feeded into the mo dels was lo w (~40) for machine learning standards. On the other hand, w e applied PCA to the output v ariables of the sp ec- tral shap e prediction to reduce the num ber of predicted v ariables required for the prediction of a spectrum, while 5 T ABLE I. Summary of the mean error or agreemen t of the differen t prediction examples tested using the different mo dels. The first column shows the mean error of the initial distribution. In the case of the shap e agreement, this v alue corresponds to the mean agreemen t b etw een eac h of the single-shot sp ectra and the mean spectrum. The v alues for eac h of the models correspond to the predictions on the test set, while the num b ers in brac k ets corresp ond to the training set. T est set (T rain set) Initial Distribution Linear Mo del Quadratic Mo del Supp ort V ector Regressor Neural Net work Mean error of single pulse photon energy [e V] 5.62 0.29 (0.28) 0.30 (0.24) 0.32 (0.27) 0.30 (0.29) Shap e agreement of single pulse spectrum 67% 88% (88%) 94% (95%) 95% (95%) 97% (97%) Mean error of double pulse delay [fs] 6.82 2.07 (2.04) 1.67 (1.58) 1.67 (1.57) 1.59 (1.52) Mean error of double pulse photon energy [e V] pulse 1: 1.45 pulse 2: 1.03 0.47 (0.49) 0.44 (0.44) 0.49 (0.48) 0.41 (0.39) 0.50 (0.48) 0.41 (0.39) 0.46 (0.47) 0.40 (0.40) minimizing the effects of the noise in the training with the measured sp ectra. W e found the b est results by k eep- ing only the first 20 principal comp onents out of the 350 sp ectral components measured by the sp ectrometer. W e used m ultiple mo dels to predict eac h of the output v ariables from the scaled features, and ev aluated them using the mean error, calculated as the mean distance of eac h predicted v alue to the measured v alue. The train- ing was performed to minimize the mean error on the training set. The h yperparameters of each mo del w ere mo dified to minimize the mean error on the v alidation set. Finally , the accuracy of each mo del was quoted as the mean error obtained on the test set. In the case of the sp ectral shap e prediction, w e define our accuracy by cal- culating the agreement b etw een the v ectors represen ting the measured, V m , and the predicted, V p , sp ectra using the similarit y function defined as: Agreemen t = 2 | V m · V p | | V m | 2 + | V p | 2 . (1) P olynomial mo dels w ere fit to the data using simple re- gression. Due to the large n um ber of features, it was not p ossible to use higher order models than quadratic, as the n um b er of artificial features created b y com bining all of the input features up to the required degree scales as the num b er of k-m ulticombinations of n elemen ts, where k is the polynomial order, and n the num b er of input features. In fact, the num b er of parameters to fit in the mo del can become comparable or larger than the size of the training data. In practice this limits the nonlinear- ities that can b e represented, as the order is the only h yp erparameter a v ailable to increase the complexity of p olynomial models. The optimal h yperparameters for the SVR models (C, epsilon, gamma) and the NN (num ber of hidden lay ers, n umber of cells per lay er) were found in eac h case by ap- plying a grid search. Eac h NN presen ted up to 3 hidden la yers, with b etw een 20 and 100 hidden cells per la yer. A rectified linear activ ation function w as used for the hid- den cells. The NNs w ere trained un til con v ergence using the Adagrad 43 algorithm with a batch size of 1000 sam- ples p er training step. The final hyperparameters were c hosen to minimize the error of the v alidation set, while not o verfitting the training set, to make sure the mo del w as k ept as simple as p ossible. K-neigh bors and Deci- sion T ree Regressor mo dels w ere also used, but in general ac hieved worse results for all the examples. The same technique should be applicable to mak e pre- dictions for every shot in new XFEL machines w orking at MHz, as the training, v alidation, and testing steps can still b e performed at a lo w rep etition rate (b elow 1 kHz). Nevertheless the accuracy of the predictions in this case ma y b e different from the v alues shown here, as the hidden correlations exploited by the machine learning mo dels ma y c hange in the new XFELs. I I I. RESUL TS AND DISCUSSION W e applied our tec hnique to predict the photon en- ergy , the sp ectral shap e, and the pump-prob e dela y of x-ra y pulses, whic h are the critical parameters in x-ray sp ectroscop y and time-resolv ed studies. These character- istics are predicted for both single pulse and double pulse configurations. F or each of the predictions we optimized four differen t mo dels: a linear model, a quadratic mo del, a supp ort vector regressor, and a neural netw ork. The results are summarized in T able I. A. Single pulse photon energy prediction F or the calculation of the photon energy , we used the p osition of a Gaussian fit in our calibrated optical sp ec- trometer as the v ariable to b e predicted. T wo examples of the exp erimen tal data with their corresp onding Gaus- 6 500 520 540 Photon energy [eV] 0 5 10 15 20 Spectral intensity [au] a) Sample spectrum 1 Gaussian fit Sample spectrum 2 Gaussian fit 510 515 520 525 530 Measured central photon energy [eV] 0 500 1000 1500 2000 2500 Number of events Mean error of distribution: 5.6 eV b) 510 515 520 525 530 Measured central photon energy [eV] 510 515 520 525 530 Predicted value [eV] Mean error of prediction: 0.29 eV c) FIG. 3. Photon energy prediction for a single pulse. (a) T wo samples of single-shot spectra measured with the optical sp ectrometer at t w o differen t photon energies, and the corresp onding Gaussian fits. (b) Distribution of the measured photon energies for the dataset. (c) Measured photon energies compared to the predicted photon energies for the test set using a linear mo del. sian fits are sho wn in Figure 3a. The nominal electron b eam energy w as contin uously scanned o v er a full range of 60 Me V (triangular p erio dic scan, 1 min ute p erio d, o ver 10 p erio ds) to provide a wider range of photon ener- gies for the predictions. The effect of this scan, combined with the inherent jitter, resulted in a distribution of pho- ton energies spanning a FWHM of approximately 18 e V, corresp onding to a mean error of 5.6 e V (Figure 3b). The results sho w that all 4 models are able to predict the photon energy of the test set with a mean error of near 0.3 e V when compared to the actual measured v al- ues (T able I and Figure 3c), reducing the error of the initial distribution by a factor of 20. While the error of the initial distribution w as artificially enhanced b y the electron energy scan, it is w orth noting that the model is able to automatically pick up on its own and make use of the correlations betw een the v ariables due to the scan. A dditionally , as the mean error due purely to the inher- en t jitter is near 1 e V, w e still exp ect an improv emen t factor of near 3-4 in cases where the nominal electron energy is k ept fixed. These accurate predictions are not surprising due to the well known quadratic relationship b etw een the elec- tron beam energy and the photon energy giv en b y the XFEL resonance condition. F or small v ariations in en- ergy (60 Me V c hange at 3500 Me V in our case), this leads to the linear approximation: ∆ E ph E 0,ph = 2 ∆ E e E 0,e (2) where E 0,ph and E 0,e are the cen tral photon and electron b eam energies resp ectiv ely , and ∆ E = E − E 0 , where E is the single-shot energy . In this wa y , the electron b eam energy , measured non-in v asiv ely at the LCLS by an electron b eam position monitor in the final disp ersive section, can b e used to sort data as a function of photon energy . On the other hand w e observ ed that, if w e train our mo dels using the electron beam energy as the only feature, then the mean error ac hieved is still as high as 0.7 e V, and in fact it is necessary to include at least 20 features to achiev e an error rounding to 0.30 e V. This suggests that, even in a simple case like this one, useful information about the photon energy is contained not just in the main feature, but it is also enco ded in other v ariables. Nev ertheless, most of the correlations relev an t to pre- dict the photon energy seem to b e essentially linear. As a consequence, the quadratic and the SVR mo dels tend to o verfit the data, showing a larger error for the test set than for the train set (T able I). Similarly the b est p erformance of the NN w as obtained including only t w o hidden lay ers, with 10 and 5 hidden cells respective ly , greatly limiting the degree of nonlinearities that the NN can mo del. While the degree of ov erfitting was not prob- lematic for our purposes, regularization 24 or dropout 44 tec hniques could be applied to a v oid it if necessary . B. Single pulse sp ectral shap e prediction In this case, instead of predicting the photon energy as a feature obtained from fitting the sp ectrum, w e built mo dels to directly predict the sp ectral shape by pre- dicting multiple spectral components. The histogram of agreemen ts betw een the measured and predicted spectra for the test set are shown in Figure 4a. As this problem is m uch more non-linear than the previous case, the linear mo del only achi eves a mean agreemen t of 88%, while the other three models achiev e mean agreemen ts ab o ve 94% (T able I). In particular the c hosen NN, consisting of three hid- den la yers with 50, 50 and 20 hidden cells resp ectively , allo ws the net work to find and mo del the non-linearities required for the prediction with a mean agreemen t of 7 80 85 90 95 100 Agreement (%) 0.0 0.2 0.4 0.6 0.8 1.0 Normalized counts a) Linear Quadratic SVR NN 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Spectral intensity [au] Agreement 96% b) 0.0 0.5 1.0 1.5 2.0 Agreement 97% c) 500 520 540 0.0 0.5 1.0 1.5 Agreement 98% d) 500 520 540 Photon energy [eV] 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Agreement 99% e) Measured Predicted FIG. 4. Sp ectral shap e prediction for a single pulse. (a) Histogram of agreements b etw een the predicted and the measured sp ectra for the test set using the 4 different models. (b-e) Examples of the measured and the predicted spectra using a neural net work to illustrate the accuracy for different agreement v alues. 97%. In fact, 86% of the shots in the test set show an agreemen t higher than 96%. Figures 4b-e show examples of predicted compared to measured sp ectra for increasing agreemen ts from 96% to 99%. Even the example with the lo west agreement sho ws a go o d matc h, including more details of the sp ectral shape that can be ac hiev ed with a Gaussian or Loren tzian fit. It is w orth noting that due to the non-linearity of the problem, none of the models seem to ov erfit, making this a p ossible symptom of a high-bias situation, meaning that giv en more training, more features, or more com- plex models, even b etter results could be ac hieved. Apart from p otentially solving the rep etition rate problem, this technique could also b e of interest in ab- sorption exp erimen ts, where the sp ectrum after absorp- tion through a sample has to b e measured and compared to the reference sp ectrum. Normally the reference sp ec- trum is measured b efore inserting the sample and av er- aged for man y shots, or ev en a veraged for shots sorted in differen t bins as a function of one or tw o of the features 19 . Ho wev er, this approach cannot b e used to bin with re- sp ect to more than tw o v ariables, as then the num ber of samples per bin w ould become too small. Instead a mo del could b e trained to learn ho w to predict the refer- ence sp ectrum based on training reference data obtained without an absorption sample, and then used to predict the incoming spectrum for each single-shot measurement with the sample, allowing the calculation of single-shot absorption. This approac h could be successful as long as reference data are recorded sufficiently often to account for long term drift in the mac hine. C. Double pulse time dela y prediction The delay v alues b etw een the t wo x-ra y pulses were extracted from electron time-energy distribution images recorded using the XTCA V diagnostic systems. Eac h im- age was pro cessed b y first separating the tw o bunches, and then lo cating the lasing part whic h app ears as a temp orally localized loss of electron b eam energy and increase of energy spread when compared to non-lasing references 33–35 . Figures 5a and 5b show t w o XTCA V im- ages, where the lasing slices hav e b een highlighted with a red dashed line for the high energy bunch, and an orange dashed line for the lo w energy bunc h. These tw o figures, obtained from the same dataset, already show t w o situa- tions with opp osite dela y v alues. In fact, the distribution of the dela ys due to the jitter (Figure 5c) spans a FWHM of 25 fs, yielding a mean error of 6.8 fs. After training all four mo dels using the dela y v alues from the training set, they were applied to the test set to predict the delay v alues. W e found that all the models are able to predict the delay with a mean error near or b elo w 2 fs (T able I). Considering the mean error of the initial distribution w as 7 fs, this already represents an impro vemen t factor of at least 3.5 on the accuracy of the dela y . As the physical processes that determine the final delay are complex, the non-linear mo dels show b etter results, b elo w 1.7 fs mean error. In particular, a NN with tw o hidden lay ers, with 50 and 10 hidden cells, predicts the dela y with a mean error below 1.6 fs. F rom Figure 6, w e also observe that it is the neural net work that presen ts the most symmetric deviation from the p erfect correla- tion (dashed line), as opp osed to the other mo dels where 8 20 10 0 10 20 30 Delay [fs] 0 500 1000 1500 2000 2500 Number of events Mean error of distribution: 6.8 fs c) 100 50 0 50 100 Time [fs] 3460 3480 3500 3520 3540 Electron energy [MeV] Delay: -14 fs a) 100 50 0 50 100 Time [fs] 3420 3440 3460 3480 3500 3520 Electron energy [MeV] Delay: 24.9 fs b) FIG. 5. (a-b) Examples of the XTCA V traces used to extract the delay v alues by lo cating and measuring the distance b etw een the lasing part of each bunc h. (c) Distribution of all the dela y v alues for the dataset. 10 0 10 20 Predicted delay [fs] Mean error of prediction: 2.07 fs a) Linear Model Mean error of prediction: 1.67 fs b) Quadratic Model 10 0 10 20 10 0 10 20 Mean error of prediction: 1.67 fs c) Support Vector Regressor 10 0 10 20 Measured delay [fs] Mean error of prediction: 1.59 fs d) Neural Network FIG. 6. Delay prediction errors for the test set using each of the four models. 0 5000 10000 15000 20000 Number of samples in the training set 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 Mean error [fs] Linear Quadratic SVR NN FIG. 7. Dela y prediction learning curv e showing the mean error for the v alidation set (solid line), and the training set (dashed line) for eac h of the four models as function of the n um b er of samples used for training. 9 530 535 540 545 550 Photon energy [eV] 0 2 4 6 8 Spectral intensity [au] a) Sample spectrum Double Gaussian fit 530 532 534 536 538 Measured central photon energy [eV] 530 532 534 536 538 Predicted value [eV] Mean error of distribution: 1.45 eV Mean error of prediction: 0.46 eV b) 544 546 548 Measured central photon energy [eV] 544 546 548 Predicted value [eV] Mean error of distribution: 1.03 eV Mean error of prediction: 0.40 eV c) FIG. 8. Description and results of the photon energy prediction for a double pulse mo de. (a) A sample of a double pulse sp ectrum measured with the TOF sp ectrometer and the corresponding double Gaussian fit. (b,c) Measured photon energies of eac h of the pulses compared to the predicted photon energies for the test set using a NN. there is some asymmetry in the residuals. Most of the mo dels (except the linear) seem to o ver- fit, showing larger v alues for the error of the test set, than that of the training set (T able 1.). This could b e a symptom of a high-varianc e situation where the training could b enefit from having more training data. In order to determine if this is the case w e studied the accuracy of the predictions for the training set and the v alidation set as a function of the num ber of samples used for train- ing (Figure 7). This sho ws that, except for the linear mo del, all the other mo dels hav e not fully conv erged to a v alue, so given more training data better results w ould b e obtained, and maybe even more complex mo dels could b e fitted. Nev ertheless, it is worth noting that all the non-linear models can predict the delay with a mean er- ror smaller than 1.8 fs with only 5000 even ts whic h at a rep etition rate of 200 kHz could b e recorded in only 25 seconds. While XTCA V is essential to measure some v alues of the dela y , this result shows that it is p ossible to learn ho w to create mo dels that calculate the delay from sim- pler parameters, whic h can b e measured at a higher rep- etition rate. F or an exp eriment aiming to measure few fem tosecond dynamics, requiring XTCA V, this opens the p ossibilit y of actually recording data at the full rep etition rate, not b eing limited by the XTCA V maximum rep e- tition rate. This will be critical for the next generation of high-rep etition-rate XFELs, but can also b e retroac- tiv ely applied to previous experiments at LCLS, where the XFEL and the data acquisition were w orking at 120 Hz, but XTCA V data was only recorded at 60 Hz. D. Double pulse photon energy prediction F ollo wing a similar approac h as in section I I I A, w e used the electron TOF sp ectrometer in the double pulse mo de to monitor the photon energy of each of the pulses (Figure 8a). W e scanned the electron energy ov er a range of 20 Me V, yielding a distribution of photon energies with mean errors of 1.45 e V and 1.03 e V for eac h of the pulses. In this case, as in the single pulse case, we observe that all four methods sho w similar results (T able I), with the NN (t wo hidden la yers, with 20 and 5 hidden cells) yielding the smallest mean errors of 0.46 e V and 0.40 e V resp ec- tiv ely (Figure 8b and 8c). Nev ertheless the absolute errors are still larger than the 0.3 e V mean error obtained for the single pulse. W e b eliev e the main reason for this is the lo w er signal-to- noise ratio of the TOF sp ectrometer, that we estimated to be 16 times worse than that for the optical spectrome- ter under the experimental conditions used. F urthermore the mean total x-ray energy was the same in b oth cases (~30 µ J), but in the double pulse mode eac h of the pulses carried only half the energy , providing less signal. As a consequence, the accuracy of the fits is reduced, giving less reliable v alues for the central photon energy . A dditionally , w e attempted to p erform the full spectral prediction in this case, but found that while the predicted sp ectrum matc hes w ell the p osition of the p eaks, it does not predict the correct relative in tensities b etw een the t wo pulses. The first reason for that could again b e re- lated to the low er efficiency of the TOF sp ectrometer. Another p ossible reason is that, regardless of how m uch w e learn ab out the tra jectory of the bunc hes, the sto chas- tic SASE emission cannot b e easily predicted, as it de- p ends on the microscopic structure of the bunch which is not y et possible to measure using existing diagnostics. In the single pulse mo de this is not a problem, as the gas detector directly measures the total pulse energy for ev ery single shot, how ev er in a double pulse mode the gas detector cannot tell how muc h of the energy is in eac h of the pulses. All these considerations should be tak en into accoun t to better design future XFELs, b y including sim- pler/faster diagnostics, placed strategically to ha v e some correlations with the information w e plan to predict, ev en if the correlations are not simple. 10 IV. CONCLUSION W e hav e shown that, for current existing data from LCLS, the fluctuations of the electron bunch tra jecto- ries measured with fast detectors enco de imp ortant cor- relations with man y of the required shot-to-shot x-ra y prop erties, suc h as photon energy , sp ectral shap e or time dela y . Man y of these prop erties will not b e av ailable on a shot-to-shot basis at high-repetition-rate XFELs. In man y cases, the critical prop erties for an experiment can- not b e easily measured for all shots if the design of the exp erimen t does not allow measuring downstream of the in teraction region or the diagnostics require unfeasibly high data rate in high repetition op eration. W e hav e presented a tec hnique based on machine learn- ing algorithms that allo ws man y of these k ey shot-to-shot prop erties to b e obtained, based solely on information from fast detectors recorded upstream of the in teraction region, requiring only a small amoun t of training data that can be recorded for a subset of the shots, or at a lo wer repetition rate. This ma y even b e used to automat- ically obtain shot-to-shot reference sp ectra for absorption measuremen ts. In a more general application, the same metho d could also b e applied to fill data gaps due to syn- c hronization failures through the recording of a dataset, or even to p erform online filtering of the ev en ts before the storage, whic h will b e challenging at the MHz rates. As test cases, we hav e successfully applied the pro- p osed metho d for prediction of the photon energy (mean error <0.3 e V at 540 e V), the sp ectral shape (mean agree- men t >97%), and the x-ra y pulse dela y (mean error <1.6 fs) in a t win bunch mo de, using four different machine learning models. W e presen t the results from differen t mo dels not to show a deep comparison of all 4 mo dels, but rather to prov e that when the necessary correlations exist, many machine learning mo dels can exploit them, and even non-exp ert users should b e able to apply the tec hnique using the simpler and easier-to-train models. W e b elieve that putting together XFEL science with mac hine learning op ens new opportunities, particularly for ultrafast time-resolved exp eriments, at new high- rep etition-rate XFEL facilities, but also offers a new route to reanalyze data from past experiments, includ- ing exp erimen ts inv olving XTCA V or absorption exp er- imen ts. Now that many asp ects of the next generation of XFELs are b eing defined, this w ork provides evidence that the design of the new machines should not dismiss useful and difficult-to-replace complex diagnostics that cannot work at the full rep etition rate, but should instead store as m uch full-repetition-rate single-shot information as p ossible and use the complex, low-repetition-rate di- agnostics to complemen t them. A CKNOWLEDGEMENTS W e ac knowledge the supp ort from Engineering and Ph ysical Sciences Research Council (UK) (EPSR C) gran t EP/I032517/1 and the Europ ean Researc h Coun- cil (ERC) ASTEX pro ject 290467. A.S-G. is funded b y the Science and T echnology F acilities Council (STF C). H.F., K.M. and K.U. ac kno wledge support by the X-ra y F ree Electron Laser Utilization Research Project and the X-ra y F ree Electron Laser Priority Strategy Program of the Ministry of Education, Culture, Sp orts, Science and T ec hnology of Japan. R.F., V.Z. and J-E.R. would lik e to ac knowledge the Swedish Researc h Council (VR). R.F., A.O.L and V.Z. w ould like to ackno wledge financial sup- p ort from the Knut and Alice W allenberg F oundation (KA W), Sw eden. V.Z. would lik e to ac kno wledge the Sto c kholm-Uppsala Cen ter for F ree Electron Laser Re- searc h, Sw eden. M.I. ackno wledges funding from the VW foundation within a Peter P aul Ew ald-F ellowship. W.H. ac knowledges financial supp ort from a Marie Curie Inter- national Outgoing F ellowship. Use of the Linac Coherent Ligh t Source (LCLS), SLA C National Accelerator Lab o- ratory , is supp orted by the U.S. Department of Energy , Office of Science, Office of Basic Energy Sciences under Con tract No. DE-A C02-76SF00515. 1 P . Emma, R. Akre, J. Arthur, R. Bion ta, C. Bostedt, J. Bozek, A. Brachmann, P . Bucksbaum, R. Coffee, F.-J. Dec ker, et al. , Nature Photonics 4 , 641 (2010). 2 T. Ishikaw a, H. Ao yagi, T. Asaka, Y. Asano, N. Azumi, T. Bizen, H. Ego, K. F ukami, T. F ukui, Y. F urukaw a, et al. , Nature Photonics 6 , 540 (2012). 3 E. Allaria, D. Castrono vo, P . Cinquegrana, P . Craievich, M. Dal F orno, M. Danailov, G. D’A uria, A. Demido vich, G. De Ninno, S. Di Mitri, et al. , Nature Photonics 7 , 913 (2013). 4 A. Lutman, R. Coffee, Y. Ding, Z. Huang, J. Krzywinski, T. Maxwell, M. Messersc hmidt, and H.-D. Nuhn, Ph ysical Review Letters 110 , 134801 (2013). 5 A. Marinelli, D. Ratner, A. Lutman, J. T urner, J. W elch, F.-J. Deck er, H. Lo os, C. Behrens, S. Gilevich, A. Miah- nahri, et al. , Nature Comm unications 6 (2015). 6 T. Hara, Y. In ubushi, T. Kata yama, T. Sato, H. T anaka, T. T anaka, T. T ogashi, K. T ogaw a, K. T ono, M. Y abashi, et al. , Nature Communications 4 (2013). 7 A. Lutman, T. Maxwell, J. MacArthur, M. Guetg, N. Berrah, R. Coffee, Y. Ding, Z. Huang, A. Marinelli, S. Moeller, and J. Zemella, Nature Photonics (2016), doi:10.1038/NPHOTON.2016.201. 8 H. N. Chapman, P . F romme, A. Bart y , T. A. White, R. A. Kirian, A. Aquila, M. S. Hunter, J. Sc hulz, D. P . DePon te, U. W eierstall, et al. , Nature 470 , 73 (2011). 9 W. Ga w elda, J. Szlac hetk o, and C. J. Milne, “X-ra y spec- troscop y at free electron lasers,” in X-R ay A bsorption and X-R ay Emission Sp e ctr osc opy (John Wiley & Sons, Ltd, 2016) pp. 637–669. 11 10 J. M. Glo wnia, J. Cryan, J. Andreasson, A. Belkacem, N. Berrah, C. Blaga, C. Bostedt, J. Bozek, L. DiMauro, L. F ang, et al. , Optics Express 18 , 17620 (2010). 11 B. Erk, R. Boll, S. T ripp el, D. Anielski, L. F oucar, B. R udek, S. W. Epp, R. Coffee, S. Carron, S. Schorb, et al. , Science 345 , 288 (2014). 12 J. Marangos, Con temporary Ph ysics 52 , 551 (2011). 13 C. E. Liekh us-Sc hmaltz, I. T enney , T. Osip ov, A. Sanc hez- Gonzalez, N. Berrah, R. Boll, C. Bomme, C. Bostedt, J. D. Bozek, S. Carron, et al. , Nature Comm unications 6 (2015). 14 J. Ullric h, A. Rudenk o, and R. Moshammer, Ann ual Re- view of Ph ysical Chemistry 63 , 635 (2012). 15 K. R. F erguson, M. Bucher, T. Gorkho v er, S. Boutet, H. F ukuzaw a, J. E. K oglin, Y. Kumagai, A. Lutman, A. Marinelli, M. Messerschmidt, et al. , Science Adv ances 2 , e1500837 (2016). 16 A. Picón, C. Lehmann, C. Bostedt, A. Rudenk o, A. Marinelli, T. Osip ov, D. Rolles, N. Berrah, C. Bomme, M. Bucher, et al. , Nature Communications 7 (2016). 17 R. Bonifacio, C. P ellegrini, and L. Narducci, Optics Com- m unications 50 , 373 (1984). 18 L. H. Y u, Physical Review A 44 , 5178 (1991). 19 V. Kimberg, A. Sanchez-Gonzalez, L. Mercadier, C. W eninger, A. Lutman, D. Ratner, R. N. Coffee, M. Buc her, M. Muck e, M. Agåk er, et al. , F araday Dis- cussions (2016). 20 A. Sanchez-Gonzalez, T. Barillot, R. Squibb, P . Kolorenč, M. Agaker, V. A verbukh, M. Bearpark, C. Bostedt, J. Bozek, S. Bruce, et al. , Journal of Physics B: Atomic, Molecular and Optical Ph ysics 48 , 234004 (2015). 21 M. Harmand, R. Coffee, M. Bionta, M. Chollet, D. F rench, D. Zh u, D. F ritz, H. Lemke, N. Medvedev, B. Zia ja, et al. , Nature Photonics 7 , 215 (2013). 22 M. Altarelli, R. Brinkmann, M. Chergui, W. Dec king, B. Dobson, S. Düsterer, G. Grüb el, W. Graeff, H. Graaf- sma, J. Hajdu, et al. , T ec hnical Design Report, DESY 97 , 1 (2006). 23 Linac Coher ent Light Sour c e-II (LCLS-II) pr oje ct final de- sign r ep ort , T ech. Rep. LCLSI I-1.1-DR-0251-R0 (2015). 24 B. Cheng and D. M. Titterington, Statistical Science 9 , 2 (1994). 25 A. J. Smola and B. Schölk opf, Statistics and Computing 14 , 199 (2004). 26 A. Edelen, S. Biedron, B. Chase, D. Edstrom, S. Milton, and P . Stabile, IEEE T ransactions on Nuclear Science 63 , 878 (2016). 27 A. Marinelli et al. , in Pr o c. of International Particle A c- c eler ator Confer enc e (IP A C’16), Busan, K or e a, May 8-13, 2016 , International P article A ccelerator Conference No. 7 (JA CoW, Genev a, Switzerland, 2016) pp. 1032–1036. 28 K. R. F erguson, M. Bucher, J. D. Bozek, S. Carron, J.-C. Castagna, R. Coffee, G. I. Curiel, M. Holmes, J. Krzy- winski, M. Messersc hmidt, et al. , Journal of Synchrotron Radiation 22 , 492 (2015). 29 P . Emma, K. Bane, M. Cornacc hia, Z. Huang, H. Sc hlarb, G. Stupako v, and D. W alz, Physical Review Letters 92 , 074801 (2004). 30 Y. Ding, C. Behrens, R. Coffee, F.-J. Deck er, P . Emma, C. Field, W. Helml, Z. Huang, P . Krejcik, J. Krzywinski, et al. , Applied Physics Letters 107 , 191104 (2015). 31 E. Allaria, B. Diviacco, C. Callegari, P . Finetti, B. Mahieu, J. Viefhaus, M. Zangrando, G. De Ninno, G. Lambert, E. F errari, et al. , Ph ysical Review X 4 , 041040 (2014). 32 E. Righ tor, A. Hitc hco ck, H. A de, R. Leapman, S. Urquhart, A. Smith, G. Mitchell, D. Fisc her, H. Shin, and T. W arwick, The Journal of Ph ysical Chemistry B 101 , 1950 (1997). 33 Y. Ding, C. Behrens, P . Emma, J. F risch, Z. Huang, H. Lo os, P . Krejcik, and M. W ang, Ph ysical Review Spe- cial T opics-Accelerators and Beams 14 , 120701 (2011). 34 C. Behrens, F.-J. Dec ker, Y. Ding, V. Dolgashev, J. F risch, Z. Huang, P . Krejcik, H. Lo os, A. Lutman, T. Maxwell, et al. , Nature Communications 5 (2014). 35 T. J. Maxwell, C. Behrens, Y. Ding, Z. Huang, P . Krejcik, A. Marinelli, L. Piccoli, and D. Ratner, in Pr o c e e dings SPIE 9210, X-R ay F r e e-Ele ctr on L asers: Be am Diagnos- tics, Be amline Instrumentation, and Applic ations II (2014) pp. 92100J–92100J. 36 S. Hau-Riege, R. Bionta, D. R yutov, and J. Krzywinski, Journal of Applied Ph ysics 103 , 053306 (2008). 37 S. Smith, S. Hoobler, R. G. Johnson, T. Straumann, A. Y oung, R. M. Lill, L. H. Morrison, E. Norum, N. Sereno, G. W aldsc hmidt, et al. , in Pr o c e e dings of the 2009 Particle A c c eler ator Confer enc e, V anc ouver, BC (2009) pp. 754– 756. 38 L. R. Dalesio, J. O. Hill, M. Kraimer, S. Lewis, D. Murray , S. Hunt, W. W atson, M. Clausen, and J. Dalesio, Nu- clear Instruments and Methods in Physics Research Sec- tion A: Accelerators, Sp ectrometers, Detectors and Asso- ciated Equipment 352 , 179 (1994). 39 D. Damiani, M. Dubrovin, I. Gaponenko, W. Kro eger, T. Lane, A. Mitra, C. O’Grady , A. Salniko v, A. Sanc hez- Gonzalez, D. Schneider, et al. , Journal of Applied Crystal- lograph y 49 , 672 (2016). 40 F. Pedregosa, G. V aro quaux, A. Gramfort, V. Mic hel, B. Thirion, O. Grisel, M. Blondel, P . Prettenhofer, R. W eiss, V. Dubourg, et al. , Journal of Machine Learning Researc h 12 , 2825 (2011). 41 I. Jolliffe, Princip al c omp onent analysis (Wiley Online Li- brary , 2002). 42 M. Abadi, A. Agarwal, P . Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Da vis, J. Dean, M. Devin, et al. , arXiv preprint arXiv:1603.04467 (2016). 43 J. Duchi, E. Hazan, and Y. Singer, Journal of Machine Learning Research 12 , 2121 (2011). 44 N. Sriv asta v a, G. E. Hinton, A. Krizhevsky , I. Sutskev er, and R. Salakh utdinov, Journal of Machine Learning Re- searc h 15 , 1929 (2014). 12 APPENDIX 1. DESCRIPTION OF THE V ARIABLES USED FOR THE PREDICTION A. F ast v ariables W e list here all of the fast shot-to-shot v ariables used as features for prediction, currently measured at 120 Hz at LCLS: • “ebeamCharge” and “ebeamDumpCharge”: Elec- tron beam charge measured at the accelerators, and at the electron dump. • “ebeamEnergyBC1” and “ebeamEnergyBC2”: Electron b eam energy measured at eac h of the tw o bunc h compressors. • “ebeamPkCurrBC1” and “ebeamPkCurrBC2”: Electron beam p eak current measured at each of the t w o bunc h compressors. • “ebeamL TUPosX” and “eb eamL TUP osY”: Hori- zon tal and vertical electron b eam p ositions at the Linac to Undulator (L TU) transp ort line. • “ebeamL TUAngX” and “eb eamL TUAngY”: Hori- zon tal and vertical electron b eam angles at the Linac to Undulator (L TU) transp ort line. • “ebeamL TU250” and “ebeamL TU450”: Electron b eam position in t w o dispersive regions at the L TU transp ort line. • “ebeamUndPosX” and “eb eamUndPosY”: Hori- zon tal and vertical electron b eam p ositions at the undulator. • “ebeamUndAngX” and “eb eamUndAngY”: Hori- zon tal and vertical electron b eam angles at the un- dulator. • “f_11_ENR C” and “f_12_ENRC”: Redundant x- ra y total energy measurements b efore attenuation from t w o gas detectors. • “f_21_ENR C” and “f_22_ENRC”: Redundant x- ra y total energy measurements after attenuation from t w o gas detectors. • “f_63_ENR C” and “f_64_ENRC”: Redundant x- ra y total energy measurements corrected to b e ac- curate for small signals (<0.5 mJ). B. Slo w EPICS v ariables W e list here typical slo w en vironmental prop erties recorded as EPICS v ariables measured at 2 Hz at LCLS: • P ositions of translation stages in v olv ed in the con- trol feedbac k loops. • V oltages of p o wer supplies inv olv ed in the con trol feedbac k lo ops. • Strength of magnetic fields in the magnetic chi- canes, and bending magnets. • Nominal v alues for the amplitude and phases of the radiofrequency fields. • Pressures from the v acuum systems. • T emp eratures at differen t stages. • Calibration v alues inputted manually b y operators. • Status of b eam blo ck ers.