Novelty-Driven Target-Space Discovery in Automated Electron and Scanning Probe Microscopy

1 Novelty-Driven T arget-Space Discover y in Automated Elec tron and Scann ing Pr obe Micr oscopy Utkarsh Pratiush 1* , Kamyar Ba r akati 1 , Boris N . Slautin 1 , Cathe rin e C. Bodinge r , 2 Chr istophe r D . Lowe, 2 Brandi M. Coss airt 2 , Ser gei V . Kalinin 1,3* 2 Department of Materials Science and Engineering, U nive rs ity o f Tennessee, Knoxville, TN 37996, USA 2 Department of Che mistry , University o f W ashington, Seattle, W A 98195, USA 3 Paci f ic Northwe st Na tional Labor atory , R ichland, W A 99354, USA * Corresponding aut hor Modern aut omated microscopy faces a fundamental discovery c ha lle ng e: i n m any systems, the most im por tant scientific infor mation does no t res ide in the immediately visib le image fe atures, but in the targe t space of sequentially a cquire d spectra or functiona l responses, making i t essential to develop strategies th at c an actively search for new behavi ors rather than simpl y op timize know n objec t ives. Here, we developed a deep-k ernel-learning BEACON frame work that is explicitly designed to guide discovery i n the targe t space by l earning struc ture–proper ty relationships during the experi m ent and using that evolving model to seek diverse response regime s. W e first establ ished t h e method through demonstration wo rkflows bui lt on pre -acquired ground-truth data sets, which ena bled direct be nchmarking agai nst cl assical acqu isiti on strategies and a llowed us to define a set of monitoring functions for compari ng exploration quality , tar get -space cov erage , and surrogate- model behav ior in a transparent and reproducible manner . Th is benc h marking framework provid es a practical basis for ev aluating discov ery -driven algorithms, not just optim i zation performance. W e then operationalize d and deploye d the workflow on STEM, showing that the approach c an transition from of fline validat ion to real experimental impl e mentati on. T o support adoption and ext ension by the broader community , the associated notebooks are available, allowing users to r eproduce th e workflows , test the benchmarks, and adapt th e method to t h eir own instruments and datasets. 2 I. Introduction Ele c tron microscopy 1–3 has emer ged as a found ational tool for materials discov ery , by providing direct access to structure and functionality at the length sca l es where m aterials beha v ior ulti m ately em er ges. In recent years, rapid advances in mea sure ment modalities 4,5 ranging fro m Ele c tron En er gy Loss Spectroscopy ( EELS ) 6 , off-axis EELS, and time-delay cathodoluminescence have substantially expanded what can be learned fro m a single instrument platform. Collectively , these developments enable a growing s et of con trast mecha n isms provi ding i nsight in to inner -shell ele c tronic transitions, orbital occupancies, phonon , a nd b and structure that w ere previously associat ed with mac roscop ic characterization, but are now accessible at the at o mic scale 7– 11 . In ef fect, many e lectron m icroscopy moda l ities are transferring e stablished cha racterization c oncepts into regimes where indi v idual atomic envi ronm ents ca n be interrogated wit h unprecedented specifi c ity . This c onverge n ce is ope n ing fundamental ly new oppo rtunities 12–14 t o e xp lore matter at the atomic level, not only by visualizi ng structure, but by probing local excitations and responses with inc re asing precision 15 . T o date, many of the m ost advanced STEM c apabilities have been demonstrate d f irst on idea l ize d, w ell-defined objects such as inte rf aces, topological defects, and other f eature s whose structure and lo cation c an be ant ici p ated in advance 16– 20 . These systems provide c lea r , in terpretable tar gets for method development and va lidati on 21 –23 . However , rea l m aterials are typ ically characte rized by complex, hete rogen eous microstructures in which the objects of inte rest are no t establ ished a priori 24,25 , and may not even b e readil y recogniz ab le befor e measurement. In this setti ng, the c entral challenge shifts from m easuring a known feature to di scov ering which features mat t er and where they reside within a lar ge and intricat e ato mic - or mesoscopic l andscape . A t the same time, practical constraints 22 i mpose hard limits on brute-force strate g ies. The time required for acquisi t ion, com bin ed w ith bea m dam age 26,27 considerations, preclude measurements over dense grids followed by retrospective analysis, e v en if such exhausti v e sampling woul d be conce ptu all y attracti v e. These c onsiderations necessitate t h e development of mac hine learning algorithms that can be depl oyed on automated mic roscop es to ide nt ify and prioritize regions of interest as the experi m ent proceeds 28–32 . Ra ther than r elyi ng on d ense, uniform s ampling followed by offline inte rpre tation, such algorithms are intended to guide measurement toward the most infor mative loca t ions within complex microstructures, where the relevant obj ects are not known in advance . 3 Over the last several y ears, parallel progress in machine learning methods and in instru ment - control AP Is 33–41 has made this directi on technically feasibl e, enabling the in tegration of ML - driven deci sion logic with mi croscope a utomation and there by supporting the practical impl e mentati on of automated re g ion selecti on in STEM experiments. In general , automated discove ry of structure –property relationships begins 22,31,42,43 from the s ituation in whi ch the stru ctural dat a are availa ble in ful l, and th e loc ati ons for s ubsequent spectra l measurements ar e then se lected sequentially based on the structural information and the spectra a cquire d thus far . W ithin this broad framing, a lar g e c lass of strategies operates on the structural d ata a lone: mea sure ment locations are c hosen based on a priori known objects of inte rest, statistical reweighting of the obs erved struct ures, or some combi nation of novelty s cores define d ov er the struc tural observations. In these scenarios 24,30,44–46 , navigation of the feature space, typically represented a s image patches, is effectively decoupled from the measured spec tr a: the trajectory through structural var iations is not d irectly altered by what is learned from the spectra l mea sure ments themsel ves. An a lternative set of a ppro ache s is built a round discovery in t he spectral domain. Here, the key disti nc tion is that spectral data are available o nly sequent i all y , so th e methodology must expli c itl y account for the evo lving natur e of what is known about t he stru cture–prope r ty relationship as measurements accumulate . These algo rithm s also r equire se lection of the d iscove ry tar get, typ ically a scalar functional (scalarizer) 43 of mea sured spectra via est abli shed phys ical knowledge 22 to provid e a clear discovery tar get . 43,45,60 –62 The core of th ese m ethods is therefore act iv e learning of a surrogate model connecting local structure to spectra or scalarize r that is dynami c all y updated during the experiment, and whose probabilistic predictions are used to se lect the next measurement points. T yp ical examples include varia n ts of deep kernel learning m ethods that directly construct such surrogate models 42,47–50 , as well as com binations of dime nsiona lit y reducti on and regression, where the dimensionality -reduction stage c an include linear methods such as principal component analysis (PCA ) 51 and non-linea r dim ensional i ty reduction via varia t ional au toenc oders (V AE) 52,53 , and the regression s tage can em p loy Gaussian proce ss es 54 , ensembl ed ne tworks, or other models 55 th at yield pr ediction and uncertainty required to build an acqui sit ion function for Bayesian optimization (BO) 56 . Both the feature-spa ce and spectral -space methods a lso accommodate human- in -the- loop inte rven tion 43,57–59 . In pr actice , a human operator can inf luence feature- space strategies by 4 priorit i zing the selection of par ticular ob ject s in th e fe ature space, and can influe n ce spec tral-s pace strate gi es by refining the s calarizer func tions used to define ta r gets in the s pect r al do mai n. In addit ion, in both cases the opera tor can tune 59 the ex plorati on– exploitation bal ance , shap ing how aggressivel y th e sys tem pursues nove lty v ersus consolidates around reg ions expected to yield high - value m easurements. However , to date, one of the mi ss ing constrai n ts has bee n the design of the discovery function itself. Classica l Bayesian optimization models require a combination of exploration, typic a lly de f ined through the uncertainty of prediction, and exploitation def ined through a known objec t ive function, in order to construct an acquisition funct ion. In t his framing, the uncertainty of prediction is define d over the feature s pace. In many ca ses, of inte rest is not the discovery of regions with spec ific scalarizer value s or m inimization of the predi c tion uncert a inty i n the feature space, but discovery of the possibl e behaviors in the tar get s pace irrespectively of how strongly these beh aviors are present , i.e. the algorithms that all ow to explor e nove lty 48–50 d irectly in the tar get space 63–65 . Here we demonst rate the extension of the BEACO N al gorithm 63 for novel ty discovery i mp lemente d on ST EM in the d eep kernel learning model 43,55 . 5 II. BEAC ON DKL: Framework for A utonomous Discovery Figur e 1. Schematic of the BEACON active learning loop. A t each iteration, image pa tches and thei r s calarized l ab els are used to train a deep kernel learning surrogate model. The BEACON acqui sit ion function selects the next measurement using the nearest neighbor-ba sed novelty score. The loop repeats until the measureme n t budget is exhauste d. The transition from identifying known structures to uncovering e mer gent phys ica l phenome na requires a co mputational framework that ca n navi g ate the vast, often non -l inea r relationship between atomic configuration and fun ctional respons e. In this s ection, w e detail the impl e mentati on of a discovery -driven a utono mous loop that integrates t he high-di mensional feature ex traction c apabilities of D eep Kernel Lea r ning (DKL) with the BEACON (Bayesia n Evolut ion ary Ana lysis for Cosmological Observation Networks) acq uisition protocol. By mov ing beyond traditional va lue-maximizing and unc ertainty-based strategies, this framework utilizes Thompson Sampli ng to stochasticall y explore the l atent tar ge t space , prioritizing regions of structural interest no t by their proximity to a predefined goal, bu t by their poten tial to yield stati sti cally nove l physical s ignatures as reflected in the spe ctrum of possible scalarizer va lues . This approa ch allows to explore the novelty in the feature spac e of the system, a s well as partially all ev iates the "scalarizer bottlenec k" , the requir eme n t for deep domain expertise to defin e a reward 6 function a priori, enabling the microscope to autonomously ident ify rare defects and eme rgent phases in com pl ex, heterogene ous microstructures. Whil e the origin al BEACO N frame work 63 provides a generalized mathematical approach for se arching for novelty i n black-box sys tems, our work (sche matic in Figure 1) adapts this logic into a specialized Deep Kernel L earni ng (DKL) architect ur e designe d for high -dim ensional mic roscopy da ta. The primary contribution lies in replac ing st andard pa ra meter-based inputs with a CNN -driven feature extractor that enables the dis c overy engine to pro cess raw HAADF -STEM ima ge pat ches directly . T he m ethod involves computing a novelty score by implementing a st oc hastic k-nearest neighbor (k -NN) distance metric mea sured ag ainst a dyna micall y updated elite set, specifically tuned for the "unknow n unknowns" of at omic -scale physic s. W e note that here we impl e ment the novelty discovery in the scalarizer space. In principle, the novelty can be also define d using the high-dimensional feature space of the m easured spectra, but in t his case novelty defini t ion becomes dependent on the chos en high-D metric function (distance between spectra), maki ng th e discove ry less explicit then scalarizer based definition. II .a. Deep Ker nel Learning (DKL) A rchite cture T o bridge the gap b etwee n r aw real-space imaging and spectral prediction, we employ a DKL framework. A Convolut ion al Neural Network (CN N) , acting as a feature extractor  󰇛  󰇜 , compresses high-dimensional s truct ura l patches (e. g.,     pixel s ub-ima ges) into a low- dime nsiona l latent spac e. Thi s latent representation s erves as the input for a Gauss ian Process (GP). The ke rnel function  󰇛    󰆒 󰇜 is transforme d in to a de ep kernel :   󰇛   󰆒 󰇜    󰇛   󰇜   󰇛  󰆒   󰇜  󰇛  󰇜 where  represents the w eight s of the CN N. By tr aining the CN N and GP hyperparame ters simult an eously through the maximization of the m argi nal log-l ikelihood, the model lea rns a structural embe dd ing optimized to predict t h e local physical re spons e. II . b. The BEACON A cq uisition Function The core of our discove ry engine is the BEACON acquisition function as described in Equat ion 2. Unlike standard protocols th at maximize a sc alar valu e, BEACON seeks nove lty by mea suring d istances in the target (response) spa ce. F or a set of a cquire d points   , w e def ine an "elite s et"     containi ng the top fraction of measurements based on a user-defined physical 7 crit er ia. For a candidate location  , the a cquisition v alue   󰇛  󰇜 is define d as the average distanc e to it s  -nearest neighbors in the elite set:   󰇛  󰇜           󰇛  󰇜          󰇛   󰇛  󰇜   󰇜 󰇛  󰇜 where   󰇛  󰇜 , is a sample from t he pos terior di str ibuti on at  , and   are the responses of the  most simi l ar point s in the elite set. II .c. Stochastic Explorati on via Thompson Sampli ng To robustly handle model uncertainty , we utilize Thompson Sampli ng (TS). At each s tep, instea d of using the posterior mean  󰇛  󰇜 , we draw a sampl e   󰇛  󰇜 from the fu ll posterior distribut ion :   󰇛  󰇜     󰇛  󰇜    󰇛  󰇜  󰇛  󰇜 By evaluating nove lty on these samples, the sys tem naturally balances exploration and exploi t ati on. If a reg ion has h igh uncertainty (   ), th e sample s   will vary widely , occ asionally producing "novel" val ues that trigger the microscope to expl ore that stru ctural regi m e. II .d. The BEACON-DKL Algorithm The fol lowing algorithm outline s th e aut ono mous discovery loop impl e mented in our study . 8 As illustra t ed in Algorithm 1 , the discovery process begins by initializing a search space with a smal l set of random seed points to establi sh a baseline dataset. At each it er ati on of the experi m ental loop, a Deep Kernel Learning (D KL) model is trained on the currently acquired data, optim i zing the convolutional neural network’ s feature extracti on and th e Gaussian Process hyperparamete rs simu lta n eously . Once trained, the al gorithm identifie s an elite set by ranking the acqui red points according to their physic a l response and selecting the top-p erforming fraction a s a referenc e for known "interesting" physics. This el ite s et is the n used to compute z -score normal i zation para m ete rs , the mean a nd standard de viat ion, w hich define th e stat is tical bounds of establ ished h igh-value behavi o r . T o select the next measurement point, the model g enerates predictions for all una cquire d structural patches; however , rather th an relyi ng on a dete rministic mean, it employs Thompson Sampli ng to draw stochastic realizations from the f ull poster ior distribu tion. These samples ar e normal i zed using the elite se t's statistics and then co mpared against the elite d istribution using a 9  -nearest n eighbor (  -NN) distance ca lculati on in the ta r get space. The r esulti ng BEACON novelt y score prioritizes c andidates that are predicted to yiel d physical responses furt hest from the current elite cluster . The microscope then autonomous ly acquire s a spectrum at the loc ation with the highest nov elty score, upd ate s the dataset with t his new s truct ural -property pair , and repeats the c yc le until t h e experimental budge t is exha ust ed. II .e. Discussion on Latent Mapping The power of thi s approac h li es i n the la tent m apping of the tar get space. In t rad iti onal BO, the "distance " is measured in the input space (struct ur al similarit y). However , i n ma t erials science , two s truct ures that look different might produce identical p hysics, or subtle structural changes might tr igger pha se transitions with massive spectral shift s. By us ing the Mahalanobis -l ike distanc e in the s pectral respons e space, BEACON ef fecti v ely decorrelates the discovery process from struct ural nov elt y . It fo cuses the e xperimental budget strictly on regions where t h e predi c ted physical behavior is statistically " extre m e" compared to the current el ite d istribution. Thi s allows for the discovery of rare defe cts or emer gen t phases that w ould be overlooked by m ean-seeki ng algori th ms. II . f. Implementation details The active le arning framework is implemented in P ython, leveraging GP yT orch 66 for the deep kernel learning (D KL) surrogate model and B oT orch 56 for the Bayesia n optimization loop. The surrogate m od el com bines a convol ut ional neural net work (ConvNet) feature extractor with a stochasti c variational Gaussian process (S VGP), enabling s tructure -aware uncertainty quanti fi cation directly from image pa tches. Candida te point s election is performed vi a a custom BEACON acquisiti on function inspired from 63 , which uses Thom pson s ampl ing and k -nearest- neighbor dist ances in the posterior space to promot e novelty -driven exploration of the sample. Prior to depl oy ment on a physical instrument, the ful l ac t ive lea rning pipeline is validated agai nst preacqui r ed datasets using a digital twin microscope 67 . Microscope hardware control is handled through the A utoScript TEM M icroscope Client API, wrapped within the stemOrchestrator library 33,34 , which manages HA ADF image acqu isition, beam positioning, and EDS spectrum coll e cti on across four detector channels. At each active learning s tep, the beam is directed to the selec t ed spatial coordinate, an EDS spectrum is acqui red with a fixed live- time exposure, and the 10 summed spectral counts (or element-specific peak intensities) s erve as the scalar ob jective f ed ba ck to the BO loop. The workflow is hardwa re-agnostic, running on either CPU or GPU, though GP U acc e lera t ion is rec o mmended for practical e xp erime n tal t hroughpu t. III. V al idation on SPM and S TEM T o ill ustrate t h e novelty disc overy via BEACON DKL, we first a pply this approach to pre- acqui red data s ets for electron and scanni ng probe m icroscopy , introduc e monitoring func tions that define the rates of novelty discovery/optimizatio n, and establ ish initial benchmarks. These workflows are also provided in notebooks acc ompanying the publication and can be used to deploy directly t o instru ment s onc e the APIs/clients are available. Figur e 2. Example s truct ur e–property workflows used for active l earni ng in scanning probe and ele c tron microscopy . (a) T opographic PFM image of a ferroelect r ic PTO surfac e with two example pixel locations. (b) Spe ctroscopic PF M measurements by P FM provide the information of polari z ati on sw itching mechanisms, (c) Ground-tr uth PFM scalarizer ma p derived from the spectroscopi c response across the scan area. (d) H AADF -STEM im age of pl asmonic nanoparticles with two selected probe locations. (e) Corr esponding EELS spectra from the se lected pix els highli ght ing variations in plasmoni c response within the scalarizer window . (f) G round -truth scala ri zer map d erived fro m the E ELS s ignal . These s cala r izer fields represent the targe t properties that active learning algorithms aim to discover efficie nt ly by s ampling the feature space while mini m izi ng the number of measurement s. 11 T o il lustrate the principles a nd practical i mplementation of the BEACON a lgorithm , Figur e 2 presents two r epresent a tive sys tem s for wh ich ground -truth data are availa b le. The first exam pl e comes from scanning probe microscopy , specifically a piezoresponse force microscopy (PFM) image of ferro electric domains in the lead titanate system, where th e domain pa ttern is cle ar ly resolve d. This materials system has been broadly used previously to explore and operati on alize the automated SPM workflows 31,46,68 . He re, da rk regi ons correspond to the in-plane a domains, whe reas bright regions correspond to ou t of plane c domains. The PF M c ontrast serv es as the fast- to -acquire structural im age. In the sam e experiment, the microscope c an also run spectroscopi c measurem ents by me asuring loc al hystere sis loops. F igur e 2b shows e xample loops acqui red at several locations. Here, the discovery target can be posed directly in t erms of how the hysteresis loop be hav ior varies across the sample. Under special circumstances, these hysteresis loops can be collected on a dense grid (e.g., on the order of 80 × 80 measurements), providing a ground -truth map of the loop response. Beca use direct visualization of the full hystere sis -lo op data set is cumbe r some, it is comm on to focus on a scal ar representation. For exam pl e, Figure 2c shows a ma p of the hysteresis-loop are a. In a c lassical gr id-mode experiment , loops are ac qu ired everywhere , and th e resulting maps are anal yz ed to reveal correlations between local m icrostructural features and polarization sw itc hing behavi or , including associations with dislocations. H owever , dense -grid acquisiti on is time- consuming and often ca rr ies a s ignifi c ant risk of probe and sample damage. The role of act iv e lea rning, in this setting, is the r efore t o begin from the fully available PFM ima ge and sequentially acqui re hysteresis loops at new locations s o as to lea rn the structure –property relati onship of inte rest, in th is case the re lationship between lo cal microstructure and hystere sis -loop beh avior , without re sorting to exha ust ive sam p ling. Figur e 2d-f provide an a nalogous example for a STEM- EELS data se t. Figure 2d s hows a dark-fiel d image of plasmonic nanostructures 69 , while the corresponding EELS spectra are ill ustra ted alongsid e, including the defin iti on of a s calarizer function . The latter is chosen to be dipole plasmon mode. Figure 2e then shows the ground-truth s pati al behavior of this scalarized tar get. As in the PFM case, the central active- learning problem is set by th e asymmetry b etwee n what is av ailable and what is cost ly: the s tructural i nformati on is available eve rywhere, whereas spectra l measurements are ob tained sequentially . The deci sion of w hich point to me asure next is 12 therefore driven by the overall reward function balancing discovery and optimization, while leve rag ing the available structural data to extract maximal information from a limited number of spectra . Historica l ly , active learning in these syste ms has been explore d pr imarily through the lens of opti mization: classical d eci sion functions are used to favor either exploration of t he ob ject spac e or exploitation/optimization in the fe ature space. In pract i ce, however , these strategie s often exhibi t a pronounced t endenc y to collapse onto a localized region of the image space, ef fe cti v ely conce ntr ati ng mea sure ment s in one are a and prematu rely arresting exploratory activity . T o a ddress this beha vior , we previ o usly introduced a human- in -the-l oop approach 43,57 in which the reward function can be dynam ically adjusted during the experiment, alongside several forms of novelty discovery algorithms as out line d above. Here, w e ill ustr ate exploratory behavior us ing the BEACON algorit hm, which explicitly f avors novelty discovery in the f eature s pace—that is, it is designed to se ek out r egions that exh ibit distinct behavior of the s calarizer function, ra ther th an conver ging rapidly onto a single local optimum. Figur e 3. Compa rison of active lea rn ing strate gies on P FM dat a. (a) Ground -truth PFM sca lariz er map. (b–d) Acquisition trajectories for EI (b), MU (c), and BEACON (d), with marker color encodi ng step ord er and whi te markers denoting seed points. (e) T a rge t spa ce cov erage (se e definitions in supplementary) , (f) surrogate MAE, (g) s urrogate mean, and (h) surrogate uncert a inty a s a func tion of acquisition ste p. Colors: EI (blue ), MU (pink), BEACON (orange). 13 T o compare classical deep-kernel-learning-based acti v e le arning using the expected improvement (EI) and Maximum Uncer tai n ty ( MU ) acquisiti on funct ions aga inst B EACON , Figur e 3 s hows the ground- truth reward l andscape together w ith the corresponding expl ora tion traject ori es over 300 active-learning steps. For both EI and MU as shown in Figure 3a-b, th ere is a clea r tendency for the trajectory to con cent ra te within a limited part of the im age space, in this case associated with anomalously high v alues of the acquisi t ion function. In practice, the first several steps sample the i mage r elatively broadly , bu t this early exp loratory phase is followed by a strong col lapse of the trajectory onto a na rrow s patial re gion. T h is ef fect is m ost pronounced for EI and rema ins visible, although less severe, for MU . An import ant visua l signature of this behavi or is that the colors encoding the time order of the trajectory form well -defined clusters for both EI and MU . Oper ati ona lly , this means that onc e the algorithm reaches a favorable region, it begins to sam pl e that regi on r epeatedl y ra ther t h an continuing to traverse the broade r landsca p e. This observation is part icularly important because the al gorithms do not use the explici t spatia l coordinates of the pa tch as an i nput. Instea d, t hey opera te only on the relationship between the local microstructure, repr esented by th e image patch, and th e corresponding s calarized response. Therefore, concentration of the t r ajectory in a l o calize d re gion of the  -  i mage plane i s not imposed by the model directly ; rather , it emerge s because eithe r (i) the re levant behaviors are strongly conc en trated in that part of the sample, or (ii) the acquisition policy has become ef fectively trapped in a rest r icted region of the learned feature space. During the e xperiment, th ese two poss ibil ities c anno t be cleanly separated. In either c ase, howeve r , the practical conseque n ce is the s ame: the exploratory proc ess loses diversity and ceases to probe the sample broadly . In contrast , B EACON ( Figur e 3c ) produces a much more s patially distributed trajectory , and the colors corre sponding to ac quis ition time appear substanti a lly mor e intermi x ed. This qualitative behavi or ind icates that the algorithm c on tinues to interrogate dif ferent parts of the sample surface instea d of rep eat ed ly ret urn ing to a si ng le narrow region. T arget space coverage ( Figur e 3e ) me asures the fr acti on of the ground - truth scalarizer distribut ion tha t has been v isite d as a function of acquisition step. BEACON reaches higher coverage earlier and maintains a consistent l e ad throughout the 300-step budge t, indicating that its traject ori es sample a broader range of physical respons es rathe r than repeatedly probing the same narrow value range. EI and MU conve rge toward si mil ar final cov erage values but do s o more slowly and w ith a shallower initial rise, consistent w ith the tra j ect ory collapse observed in the 14 spatia l maps. The mea n absolut e error of the surroga te model ( Figure 3f ) evo lves differently a cross the three strategies. EI exhibits pronounced fluctuations throughout the acquisi t ion, reflecting the instabi l ity int roduc ed by repeatedly concentrating measurements in a r estri c ted r egion , i.e. the model is well-c a librated locally but poorly constrained elsewhere. MU s ho ws a smoother decrease, while BE ACO N achieves a comparably low MA E wit h l ess variance, suggesti ng that its spatially distribut ed sa mpli ng leads t o a more globally acc ur ate surrog ate. The surrogate mean ( Figur e 3g ) and surrogate uncerta in ty ( Figur e 3h ) together characte rize the internal state of the model as learning progresses. For BEACON, both quantitie s stabil i ze relatively early and evolve s moothly , reflecting a model that is being upda ted with d iverse, informative measureme n ts. EI, by cont r ast, sh ows persistent l ar ge ex cursions in both mean and uncert a inty , particularly visible as s harp spikes in F igur e 3h . Th ese spikes correspond to steps where the mod el en counte rs measurements tha t are poorly represented in the current tra ining se t, which is a direc t consequence of the spatially collapsed traje c tory periodically revisiting a region that is outlying relative to the broader d istribution. MU occupi es an intermediate position, with moderate fl uc tuations that d iminish as the budge t is exhaust ed. 15 Figur e 4. E xploration in real and V AE latent and patch spac e (see definiti ons in supple mentary). (a) Beacon acquisition traject ory overl aid on the P F M im age in real space, with marker color encodi ng acquisition step order and white markers d enoting seed points. (b) 2D V AE latent space representati on (i.e., d ecoding V AE latent spac e on a grid of points) . (c) Distribution of all the patc hes in the 2D V AE l ate n t spa ce. (d) Acquisiti on trajectory mapped into the la tent space , ill ustra ting how Beacon progressively explores the manifold over 300 steps. (e) V AE latent space coverage as a funct ion of acquisition step, qu antifying the fr action of the latent manifold v i sited by each strategy . (f) Patch space coverage, measuring diversit y of acquired spectra in the h igh - dime nsiona l patch s pace. In (e–f), Be acon (orang e) achieves consistently broader coverage than EI (blue) a nd MU (pink) across the full acquisition b udget. Further insight into the behavior of the dif ferent exploration strat eg ies emer ges from anal ysis of the trajectory in the feature s pace of the system. T o construct this representation, we use a va riational autoencoder (V AE) that maps each image patch to a l ow -dimensional latent code. For ease of visualization, we us e a two -dimensional latent spac e, although in pra ctice th e l ate n t dime nsiona lit y ca n be optimized depending on the amount of avai lable data and the reconstruction qualit y required. The essential r ole of the autoencoder is to take image patches, encode them into a reduced latent r epresentation, and then d ecode that latent r epresentation b ack into the origin al 16 feature space. T h e model is trained by minimizing the combine d reconstruction loss and KL loss, so that the latent variables capture the dominant factors of variation in the data while remaining smoothly or ganized. The re al-space exploration trajectory is show n in Figur e 4a and represen ts the physical moti on of the probe over the sample. T o move beyo nd physical coordinates and instead examine what kinds of objects ar e being sampled, the full set of available patches is encoded by the autoe ncod er , and the re sult ing two-dimensional latent re presentation is shown in F igure 4b . Here, the points on a square grid in the latent plane are d ecode d ba ck into the or iginal feature space, thereby reve ali ng the correspondence bet ween the two la tent coordinates and the real mic rostruc tural mo tifs. As ill ustrated in the figure, high values of both l atent va r iables correspond to regions a ss ocia t ed with in-p lane a domains. Regions w ith high latent dimension 2 bu t low latent dime nsion 1 correspond to dense a-c domain patterns, whereas regions with hi gh latent dimension 1 but low la tent d imension 2 correspond to singl e d omai n w all s. A k ey advantage of this latent representati on is that th e autoe n coder tends to disentangle the l ate n t representations, in the sense that the major factors of variation in the data become associa t ed with s moothl y varying latent coordina t es. This discussion is ne c essarily general, but it illustrates the central principle: the l atent varia bl es provide a compact, con tinuous repr esentation of the dominant structural variati ons present i n th e image patche s. In V AE analysis, each image patch becomes a sin gle point in latent space w ith some uncert a inty(note we omit the uncertainty) and the full distr ibution of th ese points is shown in Figur e 4 c . Notably , this latent distribution has a fairly complex structure, ref lecting the fac t that some microstructural elements are statistically well r epresente d in the dataset, whereas others are much less comm on. This makes the latent space particularly usef ul for analyzing explor atory behavi or , because th e progression of the automate d experi m ent can now be visualized d irectly as a trajectory through the latent ma nifo ld, i.e., as the sequence of stru ctural motifs s electe d for mea surem ent. This trajectory i s shown i n Figur e 4 d . While some sequential ordering of coverage can be discerne d, the over all path follows a relatively specific route through th e latent s pace , illustrating that the active-learning algorithm ha s effectively zoomed in on sel ected cla ss es of mic rostructural element s associated with behaviors of intere st. In direct analogy with the real-spac e analysis, one can t hen define and compa r e mea sures of latent-s pace covera ge and patch-space coverage, shown 17 in Figures 4e and 4f , respectively . These metrics qu antify how broadly the experiment traverses the space of structural motifs ra ther than merely how far the probe moves physically . In all cases, BEACON show s superior behavior compa red with EI and MU , indicating a broader and more bala nc ed traversal of the a ccessible feature m anifold. This, in turn, is consistent with a more robust explorat ory dyn amic, in whic h the algorithm continues to sample diverse classes of objects rathe r than c ol lapsing premat ure ly onto a narrow subset of the availa b le microstruc tur e. Figur e 5. (a) Ground truth s calarizer map for the EELS data s et, (b) T ar g et space coverage as a function of acquisition step, s howing the fraction o f the ta rget value distribution expl ored over time . (c) V AE latent spa ce coverage as a fun cti on of acquisition step, quantifying th e fra cti on of the l a tent m anifol d v isite d by e ach strategy . S how are expl oratory trajectories in real space for (d) EI, (e) MU , and (f) BEACON acquisit ion functi ons. In (b-c) we s ee BEACON (orange) achi eves consistent ly bro ader c ov erage t h an EI (blue) and MU (pink) ac ross the full ac qu isiti on budge t. A similar pa tte rn is observed for the STEM- EELS dataset, as illustrated in Figure 5 . In this case, th e image and the corre sponding ground -truth behavior of the acqui si tion tar get are bo th compa ra tively simpler . The response is dominated by a relativel y clear t wo -level c ontrast b etwee n the pl asmonic nanoparticles and the regi ons between them, together with a more gradual de cay of the plasmonic r esponse along the outer edge of the dense pa rticle assembl y . Thi s simpler structure 18 provides a useful complementary case, s ince it allow s the behavior of th e dif ferent acquisition functions t o be examined i n a setting where t he und erlying tar get landscape is less intricate. The evolut ion of the ta rget-space coverage and the V AE la tent-space coverage for BEACON, expect ed i mprovem ent, and MU is s how n in Figure s 5b -c . A not able feature of this data set is that the t ar get-spa ce cov erage appears bro adly comparable across all three acqui si tion functions. In ot her words, when judged only by the ra nge of scala ri zed re sponses that are discovere d, the methods perform si milarly . H owever , the distinction becomes much cl eare r in the feature -spac e coverage, w here BEACON shows a cl ear advantage. Thi s ind icates that even when the algor ithm s reach similar levels of tar get -space discove ry , they do so through very different modes of trave rsa l of the under lying structural manifold. This dif ference bec om es eve n more appare n t in the rea l -space trajectories shown in Figur es 5d-f . For expected i mprovement, the algorithm cl earl y conver ges onto two localized regions of the sa mple surface. This is espec ially evident from the color gradi ent, where the transit ion from blu e to y ell ow remains tightly confined to t hose same r egions, indic ating t h at once the algorit h m identified these areas, it re mai n ed ef fe ctively locked there and continued to explore them sys tem a ticall y . T h e MU traje ctory is more e xpl oratory , but the sam pling is still conc en trated prima ri ly in the central r egion, w ith on ly modestl y broader coverage. In contrast, BEACON exhibi ts an almost uniform explorati on of the available object spac e, with t he trajectory d istributed much m ore bro adly across the sampl e. As in the previous example, these r eal-space trajectories provide an ind epende n t m easure of explorati on quality . This is beca use the algorithm uses the structural conte nt of the patch, but not the explicit s patial coordinates from which that patch was ext racted. Therefore, any spatial conce ntr ati on observed in the traject ory is an eme rgent property of the explor ati on dynamics rath er than a direct consequence of the mod el input. In this s ense, the bro ader spatial traversal produced by BEACON again indi cates that it sustai ns more diverse exploration, even in a sys tem where the tar get-space coverage alone mi ght suggest only modes t dif ferences between acquisition stra tegies. 19 IV . Implementation on STEM Figur e 5. BEACON applied to li v e STEM -EDX a cquisition on a R andom libra ry . (a) H AADF ima ge with three representative measurement locatio ns marke d. (b) Normal ized EDX spect ra from the t hre e locations, with Se L, K α, and K β emission lines indicated. (c) Pat ch space coverage as a function of acquisition step for EI, M U, and BEAC ON. (e –g) Acqui sition trajectories for EI (e), MU (f), and BEACON (g) overlaid on the HA ADF im age, with marker color encoding step order and white squares deno ting see d po ints. T o operationalize the nove lty disco very in experimental settings , we deploye d BEACON in a live aut ono mous STEM-EDX experiment on a mi xed nanoparticle sampl e containing CdS nanorods (~6 × 18 nm and ~6 × 80 nm), CdSe nan oplatelets, and several quantum dot species ( Figur e 5 ). The scalarizer w as define d from the integrated Se EDX signal , directi ng th e active lea rning ag ent toward Se-rich nanostructures. Cons istent w ith the pre - acqui r ed dataset results, BEACON achie ves substantia l ly higher pat ch space c overage over 100 acquisi tion s teps compared to both EI and MU ( Figur e 5 c ), confir ming that the exploration advantage of the DKL -based diversit y objective transfers directly to rea l experimental conditions. The a cquisition traject or ies 20 ( Figur e 5e-g ) further mirror th e trends observed on PFM data: EI and MU trajectori es show spatial cluste ring, w hile BEACON distribute s measurements more uniforml y across structurally disti nc t regions of the samp le. Figur e 6. Computational timing per acquisition step. (a) Hardware and Model training time. (b) Acquisiti on func tion c o mputation time for EI, MU, and Beacon. All three strategies impose neg ligi b le computational overhead relative to the hardware acqui sit ion time. As shown in Figure 6, th e hardware s tep dom inates at ~3 s per point, w ith Surrogate model training a round ~2.4 s. Acquisition function evaluation is fast for all methods (Figure 5b), with EI and MU completing in ~0.02 s per step. Beacon requi r es mar gin all y more comput e (~0.05 s) due to Thompson sampl ing and c o mputi ng the acquisition fun cti on. The timing will de pend on the dataset si ze; currently , we use a 64*64 overview image. V . Summary Overall , we developed th e deep -kernel-learning BE ACON approac h, imple mented it on pre-acqui r ed dataset s, and provide the associated not ebooks so that ot hers ca n expe riment with t he workflow and deploy it on their ow n tools. W e fu rther operationa l ize d i t on the S pectra 300 mic roscope, illustrating how these idea s can tran siti on from benchmarking on ground - truth data sets to execution in a real experi m ental s etting. The resul ts plac e B EACON in the broader conte xt of a rap idly expanding e cosystem of active-l earning and d eci sion- making algorithms for 21 autom a ted experiments and, correspondingly , sharpen the need for clear , transpar ent ways to vali da te, c omp are, a nd ben chma rk their performance across representative da t aset s and use cases. T aken together , these developments po int to an e mer ging “open era” of automated discovery , where algorithmic de cisions become a first -class part of the measurement process rather than an afte rthough t applied onl y in post-processing. Historica l ly , exploration w ith imaging probes , w hether in electron microscopy , scanning probe microscopy , nanoindenters, and related platforms , has been guided by a combina tion of pri or human knowledge about which objects m ight matter and the dist inctly human impulse to pursue unexpec t ed beha viors. In practi c e, human dec isio n-making i s naturally anchore d in what is imm ed iately visible in the image spac e, because those features are available for dire ct inspection. Y et many discovery problems are def ined not by w hat is apparent in the structural image, but by what emer ges in the t ar get spac e - often a spectral res ponse that is only revealed sequentially and is not access ible to the hu man operator without substantial effort. T raditionally , the on ly reliable way to inte rroga te tar g et-space beh avior h as been exhaustive mapping on a rectangular gr id followed by post-ac qu isition analysi s, oft en using uns upervise d methods after the fact. Deep kernel le arning–type approa ches change th is operating mode by enabling the correlati on between structure and prop erties to be lea rned on t h e fly , during the exp eriment, rather than retrospectively . Withi n thi s fra ming, B EACON provides a con crete imp l ementation th at moves beyond purely optimi zation-dr iven policies and inste ad supports systema tic discovery of distinc t beh aviors in the tar ge t space. Rather th an colla psing quick ly into a narrow s ubset of the sample once a locally favorable reg ion is id entif ied, BEACON is designed to continue samp ling in a w ay th at pro motes the identification of diverse target behav iors i.e., to find r epresentative exam pl es of the be h avioral r egimes present i n the system. At the same time, the r esults e mphasiz e that “discovery” is not a singl e universal objective: it requi r es explicit choices about what const itutes novelty (in feature space, tar get space, or both), how discovery criteria are encoded, and w hich monitoring m etrics are used to evalua te progress. In this study , the un iformity and m ixing of explora tion tr ajectories in image space served as a parti cu larl y useful i nd epende nt indic a tor that B EACON sustaine d exploration rather than becom ing trapped, even though such tr ajectory-based measures could themselves be elevated to optim i zation ta rgets depending on the intended experimental goal. 22 Data and C o de availabili ty: All the da ta a nd code are available at b elow GitHub repository : https:/ /g ithub.c o m/utkarshp1 161/Act ive-learning- in - mic roscopy/tree/beacon- dkl/not ebooks/b eac on-dk l Acknowledgements This work (BEACON DK L development) was sup ported ( K.B., U.P ., B.S., S.V .K., C.B., C.L., B.C. ) by the U.S. Departme n t of En er gy , Offi ce of S cie nce, Office of Basi c Energy Sciences as part of the Ener gy Frontier Research Centers program: CS SAS -The Center for the S cie n ce of Synthesis A cross Scal es under award nu mber D E-S C0019288. The work was p artially s upported (U.P .) by A I T ennessee Initiative a t U niversit y of T ennessee K noxville (U TK). W e gra tefully acknowle dge Kevin Roccapriore (ORNL) and Y ongta o Liu (ORNL) for making th eir data publicly avai l able for the benc h marking analysis. 23 Refer ences (1) Pennycook, S. J. , N. P . D. Scanning T rans mission Elec tr on Micr oscop y ; Pennycook, S. J., Nelli st, P . D., Eds.; Spring er New Y ork: New Y ork, NY , 201 1. htt ps:/ /doi.or g/10.1007/978 - 1-4419-7200-2. (2) W illiams, D. B.; Carter , C. B. The T ransmission E lectron Microsc ope. In T ransm ission Ele c tr on Micr oscopy ; Springer US: Boston, MA, 1996; pp 3–17. https:/ /do i.or g/10.1007/978-1-4757-2519-3_1. (3) Crewe, A. V . Scanning T ransmission Electron Micro scopy*. J. Mic r osc. 1974 , 100 (3), 247–259. https:/ /do i.or g/10.1 1 1 1/j.1365-2818.1974. tb03937.x. (4) Krivanek, O. L.; Chishol m, M. F .; Nicol osi, V .; Penn ycook, T . J .; Corbin, G. J.; Dell by , N.; Murfitt, M. F .; Own, C. S.; Szilagyi , Z. S.; Oxley , M. P .; Pantelides, S. T .; Pennycook, S. J. Atom-by-Atom Struct ur al a nd Ch emical Analysis by A nnular Dark-Field Electron Microscopy . Natur e 2010 , 464 (7288), 571 –574. https://doi.or g/10.1038/nature08879. (5) Muller , D. A.; Kourkoutis, L. F .; Murfi tt, M.; Song, J. H.; Hwang, H. Y .; Silc ox, J.; Dellby , N.; Krivanek, O. L. Atomic-Sca le Che m ical Ima g ing of Composition and Bonding by Aberrati on-Correc ted Mic roscopy . S cience (1979). 2008 , 319 (5866), 1073 –1076. https:/ /do i.or g/10.1 126 /science.1 148820. (6) Egert on, R. F . Electr on Ener gy-Los s Spectr oscopy in the Elect r on Mi cr os cope , T h ird edit ion. ; Springer: New Y ork, NY , 201 1. (7) Noircle r , G.; Lebreton, F .; Drahi, E.; de Coux, P .; W arot -Fonrose, B. STEM-EELS Investiga t ion of c-Si/a-AlO Interface for Solar Cell Applica t ions. Micr on 2021 , 145 , 103032. https:/ /do i.or g/10.1016/j.micron.2021.10303 2. (8) Qu, J.; Sui, M.; Li, R. Rec en t Advances in In -Situ T r ansmission El e ctron Mic roscopy T echniques for Hete rog eneous Catalysis. iScience 20 23 , 26 (7), 107072. https:/ /do i.or g/10.1016/j.isci.2023.107072. (9) Y u, L.; Li, M.; W en, J.; Amin e, K.; Lu, J. (S)TEM-EELS as an Advanced Characterization T echnique for Lithium-Ion Batteries. Mater . Ch em. F r ont. 2021 , 5 (14), 5186 –5193. https:/ /do i.or g/10.1039/D1QM00275A. (10) Nakama e, K. E lec tron Microscopy in Semiconductor Inspection. Meas. Sci. T echnol. 2021 , 32 (5), 052003. https:/ /doi.org/10.1088/ 1361 -6 501/abd96d. (1 1) Schofiel d, S. R.; J F isher , A.; Ginossar , E.; L yding, J. W .; Silve r , R.; Fei, F .; Namboodiri, P .; W yr ick, J.; Ma stegh in, M. G.; Cox, D. C.; Murdin, B. N.; Clowes, S. K.; G Keize r , J.; Y Simmons, M.; Stem p, H. G.; Morello, A .; V o isin, B.; Rogge, S.; A W olkow , R.; Liva daru, L.; Pitters, J.; J Z Stoc k, T .; J Curson, N.; Butera, R. E . ; V Pavlova , T .; Jakob, A . M.; Spemann, D.; Rä ck e, P .; Sch midt-Kaler , F .; Jamieson, D . N.; Pratiush, U.; Duscher , 24 G.; V Kalini n, S.; Kazazis, D.; Constantinou, P . ; Aep pli, G.; Eki n ci, Y .; Owen, J. H. G.; Fowler , E.; Moheimani, S. O. R.; Randall, J.; Misra, S.; A Ivie, J.; Al lemang, C. R.; Anderson, E. M.; Bussmann, E.; Ca mpbell, Q.; Gao, X .; Lu, T . -M.; Schmucker , S. W . Roadma p on Ato mic-Scale Semiconductor Devices. Nano Futur es 2025 , 9 (1), 012001. https:/ /do i.or g/10.1088/2399-1984/ada901. (12) Dyck, O.; Zia td inov , M.; Lingerfelt, D. B.; Unocic, R. R.; Hudak, B. M.; Lupini, A . R.; Jesse, S.; Kalinin, S. V . Author Corr ection: Atom -by- Atom Fabric a tion wi th Electron Beam s. Nat. R ev . Mater . 2020 , 6 (7), 640 –640. https://doi.or g/10.1038 /s41578-020-0188- y. (13) Boebinge r , M. G.; Brea, C.; Ding, L.; Misra, S.; Olunloyo, O.; Y u, Y .; Xiao, K.; Lupini, A. R.; Ding, F .; Hu, G.; Ganesh, P .; Jess e, S.; Unoci c, R . R. The A tomic Drill Bit: Preci sion Controll ed A tomic Fabricati on of 2D Mat eria ls. Adv anced Mat eria ls 2023 , 35 (14). https:/ /do i.or g/10.1002/adma.2022101 16. (14) Zhou, M.; Zha ng, W .; Sun, J .; Chu, F .; Dong, G.; Nie, M.; Xu, T .; Sun, L. A tomi c Fabrica t ion of 2D Materials Using El ectron Beams Ins ide a n Electron Microscope. Nanomaterials 2024 , 14 (21), 1718. htt ps:/ /doi.or g/10.3390/nano1421 1718. (15) Batson, P . E.; Dellby , N.; Krivanek, O. L. Sub-Ångs trom Resoluti on Us ing Aberr ation Correcte d Electron Opti cs. Nature 2002 , 418 (6898), 617–620. https:/ /do i.or g/10.1038/nature00972. (16) Gui, C.; Zha ng, Z. ; Li , Z.; Luo, C.; Xia, J.; W u, X.; Chu, J. Deep Lea rning A nalysis on T ransmission Ele c tron Mic roscope Imaging of Ato mic Defects in T w o -Dimensional Materi a ls. iScience 2023 , 26 (10), 107982. https:/ /do i.or g/10.1016/j . isci.2023.107982. (17) Y ang, Z .; Y in, L.; Lee, J.; Ren, W .; Cheng, H.; Y e, H. ; Pantelides, S. T .; Pennycook, S. J.; Chisholm, M. F . Direct Obs ervation of Atom ic Dynamic s and Sil icon Doping at a T opological Defect in Graphene. Angewandte Chemie International Edition 2014 , 53 (34), 8908–8912. https:/ /do i.or g/10.1002/anie.201403382. (18) Rocca prior e, K. M.; T orsi, R.; Robinson, J.; Kali n in, S.; Ziatdinov , M . Dynamic ST EM - EEL S for S ingle-Atom and Defe c t Measurement during Electron Beam T ransformations. Sci. Adv . 2024 , 10 (29). https://doi.org/10.1 126 /sciadv .adn5899. (19) Shen, M.; Li, G.; W u, D.; Y aguchi, Y .; Haley , J. C.; Field, K. G.; Mor gan, D. A D eep Lea rning B ased Autom atic Defe ct A nalysis Frame w ork for In -Situ TEM Ion Irradiations. Comput. Mater . Sc i. 2021 , 197 , 1 10560. ht tps:// do i.org/10.1016/j .co mmatsci .2021.1 10560. (20) Groschner , C. K .; Choi, C.; Scott, M. C. Machine Le arning Pipe l ine for Se gm entation a nd Defect Iden tifi c ati on fro m High-Resolution T ransmi ssion Ele ctron Microscopy Data. Micr oscopy and Micr oana lysis 2021 , 27 (3), 549 –55 6. https:/ /do i.or g/10.1017/S1431927621000386. 25 (21) V olpe, G.; Wä hlby , C.; T ian, L .; Hecht, M.; Y akimovich, A.; Monakhova, K.; W aller , L.; Sbalza rin i, I. F .; Metzler , C. A.; Xi e, M.; Z hang, K.; Lent on, I. C. D.; Rubinsztein -Dunlop, H.; Brunner , D.; Bai, B.; Ozcan, A .; Midtvedt, D.; W ang, H.; Sladoj e, N.; Lindbl ad, J.; Smith, J. T .; Ochoa, M.; Barroso, M.; Intes, X.; Qiu, T .; Y u, L . -Y . ; Y ou, S.; Liu, Y .; Zia td inov , M. A .; Kali n in, S. V ; Sheridan, A .; Manor , U.; Nehme, E .; Gold enber g, O.; Shechtm an, Y .; Moberg, H. K.; Langhammer , C.; Šp ačková, B.; Helgadot tir , S.; Midtvedt, B.; Argun, A .; Thalheim, T .; Cichos, F .; Bo, S.; Hubatsch, L.; Pineda, J.; Manzo, C.; Bachi m anchi , H.; Se lander , E.; Hom s -Corbera, A.; Fränzl, M.; de Haan, K.; Rivenson, Y .; Korczak, Z .; A diel s, C. B.; Mi jal kov , M.; V eréb, D.; Chang, Y . -W .; Pereira, J. B.; Matuszewski, D.; Kylber g, G.; Sintorn, I.-M.; Caicedo, J. C.; Cimini, B. A.; Bell, M. A. L.; Sara iv a, B. M.; Jacquemet , G.; Henriques, R.; Ouyang, W .; Le, T . ; Gómez-de- Mariscal , E. ; Sage, D.; Muñoz-Barrutia, A.; Lindqvi s t, E. J.; Ber gm an, J. Roadma p on Deep Le arn ing for Microscopy . 2023. https://arxiv .org/abs/2303.03793. (22) Kalini n, S. V ; Barakati, K.; Slautin, B. N.; Houston, A. C.; Pratiush, U.; Duscher , G.; Cao, C.; Sgourali s, I.; Spurgeon, S. R.; Y ang, C.; Channin g, G. Do W e Re ally Need A ll T hat Data: From Dat a to Agency in Auto mated Microscopy . February 15, 2026. https:/ /do i.or g/10.26434/chemrxiv .15000070/v1. (23) Alldrit t, B. ; Hapala, P .; Oinon en, N.; Urtev , F .; Krejc i, O.; Federi c i Canova , F .; Kann ala , J.; Schulz, F .; Lilj ero th, P . ; Foster , A . S. Automated Structure Discovery in A tomic Force Microscopy . Sci. A dv . 2020 , 6 (9). https://doi.org/10.1 126/sci adv .aay6913. (24) Zia td inov , M.; Liu, Y .; Ka lini n, S. V . A cti v e Learning in Open Experimental Environm en ts: Selecti ng the Ri gh t Information Channel(s) Based on Predicta bi lit y in Deep Kernel Learning. 2022. https://arxiv .org/abs/2203.10181. (25) Liu, Y . ; Ziatdinov , M.; Kalinin, S. V . Exploring Causal Physical Me chani sms vi a Non - Gaussian Line ar Mode ls and Dee p Kernel Learning: Applic ations for F erroel ectric Domain Struc tur es. ACS Nano 2022 , 16 (1), 1250 –1259. https:/ /do i.or g/10.1021/acsnano.1c09059. (26) Thac h, R. E. ; Thach, S. S. Damage to Biological Samples Ca used by th e Elect ron B eam during El e ctron Mic roscopy . B iophys. J. 1971 , 11 (2) , 204–210. https:/ /do i.or g/10.1016/S0006-3495(71)86208-2. (27) Karuppasamy , M.; Karimi Nejadasl, F .; V ulovic, M.; K oster , A. J .; Rave lli, R. B. G. Radia t ion Dama g e in Single-Particle Cryo-Electron M icroscopy: Ef fe cts of Dose and Dose Rate. J. Sy n chr otr on Radiat. 201 1 , 18 (3), 398 – 412. https:/ /do i.or g/10.1 107 /S090904951 100820X. (28) Liu, Y . ; Pratiush, U.; Ba r akati, K.; Funakubo, H.; Lin, C.-C.; Kim, J.; Martin, L. W .; Kalini n, S. V . Domain Switching on the Pareto Front: Multi-O bjective Deep Kerne l 26 Lea rning in Autom ate d Pi ezoresponse Force Microsc opy . 2025. https:/ / arxiv .org/abs/2506.08073. (29) Liu, Y . ; Kalinin, S. V . Pare to-Optimal Experime n tation: Human-G uided Mul t i-Objective Bayesia n Opti mizati on in Scanni ng Probe Mi croscopy . Nano Lett. 2026 , 26 (1), 441 –447. https:/ /do i.or g/10.1021/acs.nanolett.5c05373. (30) Narasimha , G.; Kong, D .; Regmi, P .; Jin, R. ; Gai, Z . ; V asudevan, R.; Ziatdinov , M . Multisca l e Struct ur e-Property Discovery via Act ive Learni ng in Scanni ng T unneling Microscopy . 2024. https://arxiv .org/abs/2404.07074. (31) Liu, Y . ; Kelley , K. P .; V asudevan, R. K.; Funakubo, H .; Zi atdinov , M. A.; Kalinin, S. V . Experi m ental Discovery of Struc ture –Property Relationships in Ferroelectric Mate rials via Active L earni ng. Na t. Mach. Intell. 2022 , 4 (4), 341 –350. https://doi.or g/10.1038/s42256- 022-00460-0. (32) Pratiush, U. Utkarshp1 161/Active- Lea rn ing- in -Micros copy: V1.0.0. Zenodo Dec e mber 2024. https:/ /do i.or g/10.5281/zenodo.14562670. (33) Pratiush, U.; Aus tin Houston; Paol o Longo; Remco Geurts; Ser gei Kalinin; Gerd Duscher . StemOrche stra tor: E n abling Seaml ess H ardware Control a nd High -Throughpu t W orkflows on Ele c tron Microscope s. January 3, 2026. https://doi.org/10.31224/ 4645. (34) Pratiush, U.; Duscher , G. S tem Orchestrator . GitHub 2025. https:/ /g ithub.c o m/pycroscopy/pyAutoMic/tree/main/TEM/stemOrchestrator . (35) Houston, A.; Pratiush, U.; Pycroscopy Devel opers. A syncroscopy: Asynchronous Orchestrat ion Fra mework for Automated Electron Microscopy . G itHub 2025. https:/ /g ithub.c o m/pycroscopy/asyncroscopy . (36) Liu, Y . ; Roccapriore, K.; Checa , M.; V alleti, S. M .; Y ang, J.; Jesse, S.; V asudevan, R. K. AEcroscopy: A Software–Hardware Fra mework Empowering Microscopy T oward Automat ed and Autonomous Exp eriment a tion. Small Methods 2024 . https:/ /do i.or g/10.1002/smtd.202301740. (37) Pratiush, U.; Houston, A .; Kali nin, S. V .; Duscher , G. Realizing Smart Scanning T ransmission Ele c tron Mic roscopy Us ing High Performa nce Computing. Review of Scie nt ific Instruments 2024 , 95 (10). https://doi.or g/10.1063/5.0225401. (38) Liu, Y . ; Pratiush, U.; Bemis, J.; Proksch, R.; Emery , R.; Rack, P . D.; Liu, Y .-C.; Y ang, J.- C.; Udovenko, S.; T rolier -McKinstry , S.; Kalinin, S. V . Integration of Scanning Probe Microscope wit h High-P erformance Com pu ting: F ix ed-Policy a nd R eward-Driven W orkflows Imple m entation. 2024 . (39) Therm o Fisher Sci enti f ic. AutoScr ipt TEM Software. 2025. 27 (40) de la Cruz, M. J.; Martynowyc z, M. W .; Hatt n e, J.; Gonen, T . MicroED Data Col lecti on with Seria l EM. Ultramicr os copy 2019 , 201 , 77–80. https:/ /do i.or g/10.1016/j.ultramic.2019.03.009. (41) Pycroscopy . P yAutoMic: Digi talMicrograph TE M S cript s. 2025. (42) Slauti n, B. N.; Prat iush, U.; Ivanov , I. N.; Liu, Y .; Pa nt, R.; Z hang, X.; T akeuc h i, I.; Zia td inov , M. A .; Kali n in, S. V . Co -Orchestration of Multiple Instruments to Unc ov er Structure–Propert y R elationships in Com b inatorial Libraries. Digi tal Discovery 2024 . https:/ /do i.or g/10.1039/D4DD00109E. (43) Pratiush, U.; Rocc apr iore, K. M.; Li u, Y .; Duscher , G .; Ziatdi nov , M.; Kal inin, S. V . Buildi ng W orkflows for Interactive Human in the Loop Automated Experiment (HAE) in STEM-EEL S. 2024 . (44) Liu, Y . ; Y ang, J.; V asudevan, R. K.; Kelley , K. P .; Ziatdinov , M.; Kalinin, S. V .; A hmadi , M. Exploring t he R elationship of Mic rostru cture and Conducti vi ty in Me t al Halide Perovskites via Ac tive Lea rn ing-Driven Automated Scanning Probe Microscopy . J. Phys. Chem. Lett. 2023 , 14 (13), 3352 –3359. htt ps: //doi .org/10.1021/acs.jpclett.3c00223. (45) Zia td inov , M.; Liu, Y .; Ke lle y , K.; V asudevan, R.; Kalinin, S. V . B ayesia n Active Lea rn ing for Scanning Probe Mic roscopy: From Gaussian Pro cesses to Hypothe sis Learning. ACS Nano 2022 , 16 (9), 13492 –13512. https:// do i.or g/10.1021/acsnano.2c05303. (46) Slauti n, B. N.; L iu, Y . ; Funakubo, H.; Kalinin, S. V . Unraveling t h e Impact of Initial Choice s and In-Loop In terventions on Lea rn ing Dynami cs in A utonomous Sca nning Probe Microscopy . J. Appl . Phys. 2024 , 135 (15). https://doi.or g/10.1063/5.0198316. (47) Um, M.; Sanchez , S. L.; Song, H.; Lawrie, B. J.; Ah n, H.; Kalini n, S. V .; Liu, Y .; Cho i, H.; Y ang, J.; Ahmadi, M. T ailoring Molecular Space t o Navigate Phase Co mple x ity in Cs‐ Based Quasi‐2D Perovskites vi a Gated‐Gauss ian‐Dri ven High‐Throughput Discovery . Adv . Ener gy Mater . 2025 , 15 (16). https://doi.or g/10.1002/aenm.202404655. (48) Biswas, A.; V asudevan, R.; Pant, R.; T akeuchi, I.; Fu nakubo, H.; Li u, Y . SANE: Strate g ic Autonomous Non-Smooth Exploration for Multi p le Optima Discove ry in Multi-Modal and Non-Diffe rentiable Black-Box Functions. Digital Discovery 2025 , 4 (3), 853 –867. https:/ /do i.or g/10.1039/D4DD00299G. (49) Pratiush, U.; Funakubo, H.; V asudevan, R.; Kalinin, S . V .; Liu, Y . Scientific Exploration with Expe rt Knowledg e (SEEK) i n Autono mous Scanning Probe Microscopy with Ac tive Lea rning. Dig ital Di sco very 2025 , 4 (1), 252 –263. https://doi.or g/10.1039/D4DD00277F . (50) Bulana di, R. ; Chowdhury , J .; Funakubo, H.; Zi atdino v , M.; V asudevan, R.; Biswas, A .; Liu, Y . B eyond Optimization: Exploring Novelty Discovery in Autono mous Experiments. 28 ACS Nanoscie nc e Au 2026 , 6 (1), 86 –94. https:/ /do i.or g/10.1021/acsnanoscienceau.5c00106. (51) Jollif fe, I. T .; C adima, J. Principal Compone n t A nalysis: A Review and Recent Developm en ts. Philosophical T rans acti ons of the Royal Societ y A: Mathematical, Physic al and Engineering Science s 2016 , 374 (2065), 20150202. https:/ /do i.or g/10.1098/rsta.2015.0202. (52) Kingma, D. P .; W elling, M. A uto-Encoding V ariational Bayes. 2022. https:/ / arxiv .org/abs/1312.61 14. (53) Slauti n, B. N.; Prat iush, U.; Lupasc u, D. C.; Z iat d inov , M. A .; Kali nin, S. V . Integrating Predict iv e and Gene ra tive C apabilities by Latent Spa ce Design vi a the DKL-V AE Model. 2025. https:/ / arxiv .org/abs/2503.02978. (54) Rasmussen, C. E. Gaussian Processes in Mac hine Le arning; 2004; pp 63 –71. https:/ /do i.or g/10.1007/978-3-540-28650-9_4. (55) W ilson, A. G.; Hu, Z.; Salakhutdinov , R. ; Xing, E. P . Deep Kernel Learning. In Pr oce edings of the 19th International Confer ence on Arti ficial Intelligence and Statistics ; Gretton, A., Robert, C. C., Eds.; Proceedings of M achine L e arning Re se arch; PMLR: Cadiz , Spain, 2016 ; V o l. 51, pp 370–378. (56) Bala nda t, M.; Karrer , B.; Jiang, D. R.; Daulton, S.; L etha m, B. ; W i lson, A. G.; Bakshy , E. BoT orch: Programmable Bayesian Optimizati on in PyT orch. ArXiv 2019 , abs/1910.06403 . (57) Kalini n, S. V ; Liu, Y .; Biswas, A .; Duscher , G.; Prati ush, U.; Roccapriore , K.; Ziatdinov , M.; V asudevan, R. Human- in -the-Loop: The Future of Machine L earni ng in Automated Ele c tron Microscopy . Micr os. T oday 2024 , 32 (1), 35 –41. https:/ /do i.or g/10.1093/mictod/qaad096. (58) W u, X .; Xia o, L.; Sun, Y .; Zh ang, J.; Ma, T .; He, L. A S urvey of Human- in -the- Loop for Machine L earni ng. Futur e Gene rat ion Computer Systems 2022 , 135 , 364–381. https:/ /do i.or g/10.1016/j.future.2022.05.014. (59) Pratiush, U.; Duscher , G.; Kalinin, S. Human- in -the-Loop Interface for Autom ate d Experi m ents in Electron Microsc opy , Autom ate d Ch aracteri zation. In AI for Accele rat ed Material s D esign - NeurIPS 2024 ; 2024. (60) V alleti, M.; V asudevan, R. K.; Ziatdinov , M. A .; Kalinin, S. V . Deep Kernel Methods Lea rn Be tte r : From Cards to Process Optimiza t ion. Mach. Learn. Sci. T echnol. 2024 , 5 (1), 015012. https:/ /doi.org/10.1088/ 2632 -2153 /ad1a4f. (61) Pratiush, U.; Houston, A .; Li u, R.; Duscher , G .; Kalinin, S. T owards Self -Optimizing Ele c tron Microscope: Robust T uning of A berration Coeffi cients via Physics -A ware Multi- Object iv e Baye si an Optimiza t ion. 2026. htt ps ://arxiv .org/abs/ 2601.18972. 29 (62) Liu, Y . ; Ziatdinov , M. A.; V asudevan, R. K.; Kalinin, S . V . Explainability and Human Intervention i n Autono mous Scanning Probe Microscopy . Patterns 2023 , 4 (1 1), 100858. https:/ /do i.or g/10.1016/j.patter .2023.100858. (63) T ang, W .- T .; Chakrabarty , A .; Paul son, J . A . BEACON: A Bayesia n Opt imization Strategy for Novelty Sea rch in Expe nsive Blac k-Box S ystems. 2025. https:/ / arxiv .org/abs/2406.03616. (64) T ang, W .- T .; Chakrabarty , A .; Paul son, J . A . TR-BE ACON: Shedding Light on E fficie n t Behavi or Discovery in High-Dimensional Space s wit h Bayesia n Novelty Search over T rust Regions. In Pr oce edings of the NeurIPS 2024 W or kshop on Baye sian Decision- making and Uncertai nt y ; 2024. (65) T erayama, K.; Sumita, M.; T amura, R.; Payne, D. T .; Chahal, M. K.; Ishihara , S.; T suda, K. Pushing Property Limi ts in Materials Discovery via Boundless Objective-Free Exploration. Chem. Sci . 2020 , 11 (23), 5959 –5968. h ttps:// do i.or g/10.1039/D0SC00982B. (66) Gardner , J . R.; Ple iss, G .; Binde l, D.; W einber ger , K . Q .; W ilson, A. G. G PyT orch: Blac kbox Matr ix-Matrix Gaussia n Process Inferen ce with GPU Acceleration. In Pr oce edings of the 32nd International Confer ence on Neural Information Pr oc essing Systems ; NIPS’18; Curran A ssociate s Inc.: R ed Hook, NY , USA, 2018; pp 7587 –7597. (67) Pycroscopy Develope rs. D TMic roscope: Digital T wi n Microscope . GitHub 2025. https:/ /g ithub.c o m/pycroscopy/DTMicroscope . (68) Liu, Y . ; Kelley , K. P .; V asudevan, R. K.; Zhu, W .; Ha yden, J.; Maria, J.; Funakubo, H.; Zia td inov , M. A .; T rolier‐McKinstry , S.; Kalinin, S. V . Automated Experiments of Local Non‐Linea r Beh avior i n Ferroe lectric Ma terials. Small 2022 , 18 (48). https:/ /do i.or g/10.1002/smll.202204130. (69) Rocca prior e, K. M.; Ziatdinov , M.; Cho, S. H.; Hachtel, J. A.; Kalinin, S. V . Predictability of Loca l ize d Pl asmonic Responses in Nanoparti cle A ssemblies. Sm all 2021 , 17 (21). https:/ /do i.or g/10.1002/smll.202100181. 30 Supp leme ntary T o quantify behavior of active learning trajectory , we introduce se v eral c o mplementary metric s that ch aracteri z e how the surrogate model evolves (discussed in S1) during the experiment and how they explore (d iscussed in S2.) the patch, feature and ta rget space. Imple m entation e x amples using pre-acquired data are provided as Jupyter notebooks in t h e acc omp anying c od e repository . S1. Definition of lear ning curves: Purpose of these is to m oni tor and diagnose t he surrogate model during active learning. Has been also discussed in i n th is work 43 . S1. a. Mean absolut e error of surrogate v/s steps Mean absolut e error (MAE) of surrogate, which eva luates how accurat ely the surrogate model r eproduce s the ground- truth sc alarize r map at a giv en active -learning st ep. If   󰇛  󰇜 i s the surrogate prediction at location   afte r  measurements, a nd  󰇛  󰇜 i s the corresponding ground- truth scala ri zer va lu e, the n ov er a benchma rk d ataset containing  candidate locations one may defi n e          󰇛  󰇜  󰇛  󰇜      Lower values of   indic ate a more acc ur ate global s urrogate . This metric captures a dif ferent aspect of performance than t arget-space coverage: an algorithm may reduc e MA E rapidly by focusing on a n arrow but information -rich r egion yet still f ail to explore th e full diversity of behavi ors present in th e sample. Conversely , a method that pr ioritize s discov ery may maintain broader tar get coverage while reduc ing MA E more gradually , especiall y in earl y stages. Thus, MAE quanti fi es the fidel ity of th e learne d structure–p roperty model, while coverage quantifies the diversit y of discov ered be hav iors. S1. b. Surrogate me an v /s steps 31 The surrogat e pr edic t ive mean re f lects the ove r all level of the property being ma pped as estim a ted by t h e model at a give n a cti v e-learning step. A veragi ng the pointwise predictions across all  candi da te l oc ati ons giv es a single sc ala r su mmary,             󰇛  󰇜 A rising    over steps indi c ate s th at t h e acqui si tion function is prefe r entiall y d irecting mea surem ents toward hi gh-va lue regions, consistent with expl oi tation-dominated behavior. A flat or slowly evo lving    , by contrast , suggests tha t the algorithm is samp ling broa d ly across the property range ra ther t h an conve rg ing on a specific target. This metric is most informative whe n read al ongsid e surrogate unc ertainty   and ta rge t-space coverage: rapi d growth in    paired with decl in ing cove r age is a signature of premat ur e expl o itation, whereas a stable    combi ned with expandi ng cov erage ref lects healthy expl or atory be h avior. S1. c. Surrogate unc er tai n ty v/s steps Surrogate predicti v e unce rt aint y , whi ch reflect s how uncertain the model remains across the c and idate me asure ment spa ce. If   󰇛  󰇜 denotes the pr edictive standard deviation of the surrogate a t location   , then a natural global summary is            󰇛  󰇜 A decreasi ng   indic ates that the model is beco ming more conf ident as mea sure ments acc umu late. However , the interpre t ati on of this trend requi res ca re. A rapi d drop in uncertainty can be benefici a l if it reflects genuine learning over t he full f eature space, bu t it can a lso signal premat ur e overconfidence caused by repeated s ampling of a restricted region. Conversely , persistent ly high uncertainty may indicate tha t the algorithm con tinues to explore broadly , but it may also imp ly that the surrogate h as not yet consol idated a stable globa l model. For this reason, uncert a inty must be interpreted jointly with tar g et- space coverage ( S2 c.) and M AE: the most desirabl e behavior is not simpl y minimal uncertaint y , but a balanced regime in which unc ertainty decrea ses whil e the a lgorithm continues to expand coverage and improve pre d ictive accuracy . 32 S2. Definition of cover age: The purpose of t hese met r ics is to qua nt ify the nov elty and dive rsity of the acti ve learning traject ory a cross three c omp lementa ry spa ces: t h e raw image patch spac e, the learne d f eature space, a nd th e ta rge t (scalarizer) spac e. Toget her they provide a mu lti-sc ale p icture of exploration t ha t no single metric ca n capt ur e alone. We de fin e pat ch spa ce a s th e space of r aw local ima g e windows extrac t ed from t he scan, feature space a s the low-dimensional latent r epresentati on learned by a variational a u toenc od er (VAE) traine d on those patches, and t arg et spa c e as the sc alar property va lues assigned t o each location by the sca l arizer funct ion. Cov erage in each space is compute d as the fra c tion of di scr ete r egions define d by  -means clusters or histogram bi ns , that have been visited by at lea s t one ac qu ired point up t o step  . S2 a. Patch spac e covera ge : Patch spac e covera ge mea sures th e struct ur al diversity of the image windows selected by t he act iv e learning a lgor ithm , wi thout a ny learned representation. Each patc h      (where      for a window of height  and width  ) is first mean- and variance-normalize d, then projec t ed to a lower-dimensional space via a random line ar pro jection, a nd fin all y assigned to one of  cluste rs obt ained by  -means on all  patches. Letti ng 󰇛󰇜  󰇝    󰇞 denote the cluste r l abel of p atch  , patc h-spac e cove r age at step  is defined as   patch   󰇝 󰇛󰇜     󰇞    where   denote s th e set of ac qu ired indices up to step  . A value of   patch   , indicates tha t at lea st one p atc h fro m every structural cluste r h as been measured. This met r ic ca n b e evaluated on 33 both pre-ac qu ired dataset s and in r eal-time during live instrument operation, since it requires no ground-truth propert y labels. S2 b. Feature space ( V AE latent space) cove r age: Fe a ture space cove rag e measures diversi ty in the learned latent repr esenta t ion of image patc hes. A two-dimensional v ariational aut oen coder specifi c ally, the invariant VAE (iVAE) impl e mentati on in pyroVED(https://github.com/ziatdinovmax/pyroVED), is tr aine d on all  patc hes simu lta n eously. Each pat ch is encode d to a late nt mea n v ector      , and al l latent vect ors are cluste red into  groups using  -means. F eatu re-space c over age at ste p  is then   feat   󰇝   󰇛󰇜     󰇞    where   󰇛󰇜 is the cluster label of patch  in latent space. Compared to patch-space coverage, this met ri c is sensiti v e to semantically me aningful struct ural variati on capture d by the VAE rather than pi xe l-level diffe r ence s, m aking it more robust to noise and imaging artifacts. Like p atch- space c overag e, it can be eval u ated on pre-acquired data sets and during live experiments, as it depends only on t he ima g e data and not on measured property val u es. S2 c. T ar get space cove r age – Only on pre-acquired dataset Target spa ce c over age qua n tifies how much of the o bservable prope r ty range ha s be en sampled by the a c tive learni ng traject ory. The ground-tru th scalarizer map is discre tized into  equal- width bins spanning    󰇛  󰇜    󰇛  󰇜 , and only bins containing at least one ground- truth poi nt are c onsid ered reachable. Ta rg et-space co verage at step  is   target           s.t. 󰇛  󰇜  bin      34 where   is the num b er of non-empty bins. Unlike patch and feature space coverage, this metric requires acce ss to the ful l ground-truth scalarizer map and is therefore only applic ab le t o pr e- acqui red b enchm ark d ata se ts rathe r than live instrument experiments. T aken together , th ese six metric s provide a com pr ehensive and multi-scale diagnostic framework for eva luating active learning trajectories in scanni ng m icroscopy . T h e lea rn ing curve met ri cs — MAE, surrogate me an, and surrogate un certainty , characterize the internal state of the probabili st ic m od el and answer the que stion : is the s urrogate learni ng e f fectively? T h e cove rag e met ri cs, in patch, feat ur e, and tar get space, ch aracterize t he external behavior of the acquisition strate gy and answer the c omp lement ary qu estion: is t he al gorithm exploring meaningfully? Crucia l ly , no single metric is suffi c ient on its own. An algorit h m may achieve low MAE by repeatedl y samp ling a narrow but information-rich region while failing to discover the full diversit y of struc tural behaviors present in the sa mple. Conversely , broad coverage in patch or feature spa ce doe s not guar antee accurate propert y pr edic t ion if t h e acquired point s ar e not informative for th e surrogate . The most desirable active learning behavior is one in which MAE decrea ses steadily , uncertainty reduces without premature collapse, and coverage expands consistent ly across all three spac es simultane ously . B y reporting all six metrics in pa rallel, one can di stingu ish betwee n g enuinely expl ora tory strategies and those th at merel y appear diverse in one space whil e rem a ining c onf ined i n another , a distinction that is especially consequential when deployi ng a cti v e lea rn ing on a physical i nstru ment whe re eac h m easurement carrie s a r eal experi m ental cost.