A Review of Physics-Informed Machine Learning Methods with Applications to Condition Monitoring and Anomaly Detection

A Review of Ph ysics-Informed Mac hine Learning Metho ds with Applications to Condition Monitoring and Anomaly Detection Y uandi W u a (wuy187@mcmaster.ca), Brett Sicard a (sicardb@mcmaster.ca), Stephen Andrew Gadsden a (gadsden@mcmaster.ca) a In telligent and Cognitive Engineering Laboratory , McMaster Universit y , Hamilton, On tario, Canada Corresp onding Author: S. Andrew Gadsden 1280 Main Street W est, Hamilton, ON L8S 4L8, Canada T el: (905) 525-9140 Email: gadsden@mcmaster.ca List of T ables 1 Literature compiled for feature formation or augmen tation us- ing physics-based or ph ysics-informed means, for use in mac hine learning algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . 10 2 Literature compiled for studies emplo ying transfer learning algo- rithms to learn ph ysically relev an t features. . . . . . . . . . . . . 20 3 Literature compiled lev eraging data-driv en mo dels w orking in tandem with ph ysics-based mo dels . . . . . . . . . . . . . . . . . 26 4 Literature Compiled for ph ysics-guided or ph ysics-informed reg- ularisation tec hnique employ ed . . . . . . . . . . . . . . . . . . . 30 5 A summary of literature compiled for the design of ph ysics-informed arc hitecture, with inno v ations to the feed-forw ard neural net work arc hitecture primarily . . . . . . . . . . . . . . . . . . . . . . . . . 46 6 A summary of literature compiled for the design of ph ysics-informed arc hitecture, with inno v ations to the conv olutional neural net- w ork architecture primarily . . . . . . . . . . . . . . . . . . . . . . 51 7 A summary of literature compiled for the design of ph ysics-informed arc hitecture, with inno v ations to the recurrent neural netw ork ar- c hitecture and its v arian ts. . . . . . . . . . . . . . . . . . . . . . . 60 8 A summary of literature compiled for the applications of the ph ysics-informed graph neural netw ork arc hitecture. . . . . . . . 65 9 A summary of literature compiled for the applications of the ph ysics-informed generative adversarial net w ork arc hitectures. . . 67 List of Figures 1 T allied num ber of literary w orks discussed in this review, with resp ect to their year of publication. Note: Literature w orks re- view ed in 2023 w ere limited up until the time of writing of this surv ey (June 2023). . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 General outline for the pro cess of the generation of synthetic data via ph ysics-based metho ds. . . . . . . . . . . . . . . . . . . . . . 11 3 Finite Element mo del of the structure monitored constructed to pro vide simulated training data for Neural Net w ork, as demon- strated in the w ork of Seven tekidis et al. (2020). . . . . . . . . . 13 4 Data augmen tation employ ed to incorp orate simulated fault and op eration data for the training pro cess of a machine learning fault classiﬁcation algorithm, adapted from Hop woo d et al. (2022). . . 15 5 The h ybrid model, featuring physics-based modeling as a basis to map the observ able parameters to unobserv able parameters, for input to the mac hine learning algorithm. . . . . . . . . . . . . . . 16 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 7 Illustration depicting the principles and functioning of T ransfer Learning in mac hine learning, a tec hnique in mac hine learning that leverages knowledge gained from one task to improv e p er- formance on another related task. . . . . . . . . . . . . . . . . . . 21 8 Represen tation of the application of T ransfer Learning in the con- text of Digital Twins, a virtual representation of a ph ysical en- tit y or system, show casing the transfer of knowledge from a pre- existing Digital Twin. . . . . . . . . . . . . . . . . . . . . . . . . 23 9 General outline of correction to physics-based mo deling via data- driv en solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 10 Ph ysics-informed Neural Netw ork structure . . . . . . . . . . . . 37 11 Neural net w ork arc hitectures for the solutions of unknown v ari- ables (A) for a uniﬁed neural netw ork, (B) for indep endent net- w orks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 12 In tegration of physics-based and data-driven domains through feature fusion: The CNN architecture is employ ed as a feature extractor. Adapted from Huang et al. (2022) and Yin et al. (2023) 42 2 13 Cross Data-Physics F usion, as presented by W ang et al. (2020) predictions based on information from b oth the data domain (comprised of features derived from lab eled monitoring data), and physics domain (comprised of features derived from unla- b eled data) are sim ultaneously mapp ed to a shared space, and concatenated. Both are pro cessed through a regression la y er for the ﬁnal prediction. . . . . . . . . . . . . . . . . . . . . . . . . . . 43 14 Incorp oration of physically interpretable feature extraction for use in conjunction with the Extreme Learning Machine: adapted from W ang et al. (2022a) . . . . . . . . . . . . . . . . . . . . . . 47 15 In tegration of physics-based and conv en tional sigmoid activ ation functions in neural net works: adapted from Chen et al. (2023) . . 48 16 Design of Physics-Informed la y ers for CNN net w orks, including example arc hitectures adapted from: (A) Sadoughi & Hu (2019) who emplo y ed a ph ysics-based k ernel generation sc heme to gen- erate conv olv ed ﬁlters for ph ysics-informed conv olutions, (B)Li et al. (2021a) utilizing a con v olutional lay er to process Contin u- ous W a velet T ransform. (C) Lu et al. (2023) emplo ying a ph ysics- informed feature selection la yer. . . . . . . . . . . . . . . . . . . . 53 17 Design of a multi-branc h CNN, for individual mo deling of dis- placemen t form strain and acceleration measuremen ts respec- tiv ely; adapted from Ni et al. (2022) . . . . . . . . . . . . . . . . 54 18 An illustration of (A) the general Recurrent Neural Net w ork ar- c hitecture, and (B) the inner computational pro cesses within each RNN cell. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 19 Incorp oration of physics within an RNN cell, via ph ysics-informed Deep Residual Recurren t Neural Net w ork: adapted from Y u et al. (2020b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 20 Incorp oration of physics within a recurrent neural netw ork cell via Euler in tegration: adapted from Viana et al. (2021) . . . . . 62 3 21 A depiction of a graph neural netw ork (GNN) architecture, sho w- casing its k ey comp onents and information ﬂo w: GNNs op erate on graph-structured data, enabling eﬀective analysis, inference, and learning tasks within complex relational datasets. . . . . . . 64 22 Distribution of Publications Review ed on Ph ysics-Informed Ma- c hine Learning for Condition Monitoring . . . . . . . . . . . . . . 70 4 Highligh ts 1. Surv eyed recen tly published literature on ph ysics-informed mac hine learn- ing. 2. Establishes a bridge betw een physics-based mo deling and data-driven ap- proac hes. 3. Emphasizes the imp ortance of integrating domain knowledge into the PIML framew ork. 4. Sho wcases real-world case studies and industrial applications. 5. Pro vides a foundational survey for future research w ork in the ﬁeld. 1 A Review of Ph ysics-Informed Mac hine Learning Metho ds with Applications to Condition Monitoring and Anomaly Detection Y uandi W u a , Brett Sicard a , Stephen Andrew Gadsden* a a McMaster University, 1280 Main Str e et West, Hamilton, ON L8S 4L8, Canada Abstract Condition monitoring plays a vital role in ensuring the reliabilit y and optimal p erformance of v arious engineering systems. T raditional methods for condi- tion monitoring rely on physics-based mo dels and statistical analysis techniques. Ho wev er, these approac hes often face challenges in dealing with complex systems and the limited av ailability of accurate ph ysical mo dels. In recent years, physics- informed machine learning (PIML) has emerged as a promising approach for condition monitoring, combining the strengths of physics-based mo deling and data-driv en mac hine learning. This study presen ts a comprehensiv e o v erview of PIML tec hniques in the context of condition monitoring. The cen tral con- cept driving PIML is the incorp oration of known ph ysical laws and constrain ts in to mac hine learning algorithms, enabling them to learn from av ailable data while remaining consistent with physical principles. Through fusing domain kno wledge with data-driv en learning, PIML metho ds oﬀer enhanced accuracy and interpretabilit y in comparison to purely data-driv en approac hes. In this comprehensiv e survey , detailed examinations are performed with regard to the metho dology by which kno wn physical principles are in tegrated within mac hine learning frameworks, as well as their suitability for sp eciﬁc tasks within condi- tion monitoring. Incorporation of ph ysical kno wledge in to the ML mo del ma y b e realized in a v ariet y of methods, with each having its unique adv antages and dra wbacks. The distinct adv antages and limitations of eac h metho dology for the integration of physics within data-driven mo dels are detailed, considering factors such as computational eﬃciency , mo del interpretabilit y , and generaliz- abilit y to diﬀeren t systems in condition monitoring and fault detection. Several case studies and w orks of literature utilizing this emerging concept are presented to demonstrate the eﬃcacy of PIML in condition monitoring applications. F rom the literature review ed, the versatilit y and potential of PIML in condition moni- toring may b e demonstrated. Nov el PIML metho ds oﬀer an inno v ative solution for addressing the complexities of condition monitoring and asso ciated chal- ∗ Corresponding author. Email addresses: wuy187@mcmaster.ca (Y uandi W u), sicardb@mcmaster.ca (Brett Sicard), gadsden@mcmaster.ca (Stephen Andrew Gadsden*) Pr eprint submitte d to Exp ert Systems with Applic ations January 23, 2024 lenges. This comprehensive survey helps form the foundation for future work in the ﬁeld. As the tec hnology con tinues to adv ance, PIML is expected to play a crucial role in enhancing main tenance strategies, system reliabilit y , and o verall op erational eﬃciency in engineering systems. Keywor ds: Mac hine learning, deep learning, physics-informed mac hine learning, condition monitoring, anomaly detection 1. In tro duction Throughout the last decade, Mac hine learning (ML) algorithms hav e wit- nessed rapid developmen t in a v ariety of industries for their eﬃcacy and ability to extrapolate patterns from data. Purely through av ailable data, ML models are capable of accurately representing the relation b et ween a given set of in- puts and outputs with minimal h uman interference. This prop erty made ML mo dels ideal for the representation of complex systems in whic h the relation and parameters go verning b ehavior are not easily obtained. Despite their many adv an tages, how ever, ML mo dels are not without their drawbac ks. In general, ML algorithms are a data-driven pro cess that seeks to deriv e the relationship betw een a given input and its resp ective output. This pro cess is generally performed in accordance with some deﬁned optimization algorithm, in whic h predictions made by the mo del are ev aluated and contin uously adjusted to better represen t the data given. As can be exp ected, the p erformance of ML models are heavily reliant on the data by which they are optimized up on. Indeed, restrictions to data quality and av ailabilit y are amongst the main con- cerns when choosing to work with ML (L’heureux et al., 2017). F or man y engineering applications, the collection of suﬃcient quantities of data to build a reliable mo del ma y be challenging, costly , and/or not feasible due to time and resource constrain ts. A considerable amount of clean, represen tative, and non-sparse data is required to properly form ulate the mo del (L’heureux et al., 2017). Insuﬃcient quan tities and/or non-represen tative data often lead to a sk ewed representation of system behavior that is inconsisten t with the true un- derlying ph ysical relationship, ultimately resulting in misleading conclusions. F urthermore, ML mo dels are considered to be ”blac k b o x” models, in whic h in termediary information b etw een input and output is not relev ant nor required in pro ducing a correlation betw een some input and output. That is to say , the underlying mec hanism of a system is often not considered in the dev elopment of these models, and while eﬀectiv e in represen ting a system, ma y not further con tribute to our understanding of said system (Rudin, 2019). With resp ect to the represen tation of systems based on prior knowledge, ph ysics-based modeling has also b een traditionally emplo yed. How ever, models dev elop ed purely on the understanding of the system see limited use in mo d- eling real-w orld systems, due to the man y c hallenges to its applicability . First 3 and foremost, ph ysical mo dels are often computationally expensive to model Jia et al. (2019). Due to the computational complexit y of most real-w orld phys- ical systems, and the v ariety of gov erning equations in v olved for each speciﬁc ph ysical agent or phenomenon, the cost required to fully model said systems is considerable. F urthermore, ph ysical models often represent an imp erfect in- terpretation of the system, due to missing or incomplete understanding of the system. Naturally , researchers ha v e come to the realization that the com bination of ph ysical and data-driv en models w as the next step in the prediction and mod- eling of system b eha vior. This paradigm of PIML w as initially conceptualized b y (Karpatne et al., 2017) formally in their study of theory-guided data science, in which they outlined v arious av enues of integration b etw een domain knowl- edge and data-driven solutions. Through this uniﬁcation, new ph ysics-informed mo dels are capable of beneﬁting from both ph ysics-based and data-driv en meth- o ds concurren tly . Since their publication, a plethora of studies regarding the PIML paradigm has b een conducted. V arious authors, most notably Raissi et al. (2019), further adv anced the in tegration b etw een theory and data science with the in tro duction of Physics-Informed Neural Netw orks (PINNs), whereby ph ysi- cal laws in the form of go verning equations are enco ded within neural net works. The neural net w ork architecture and properties made it esp ecially suitable in their use case, for appro ximating the solutions of P artial Diﬀeren tial Equations (PDEs). Raissi et al. (2019) made use of the neural net work architecture in their demonstration of a systematic metho dology for solving non-linear partial diﬀeren tial equations. Karniadakis et al. (2021) review ed p opular metho dolo- gies by which the integration of ph ysics and data-driven techniques takes place, as w ell as presented their insigh ts on limitations and p otential applications of the tec hnique. Meng et al. (2022) also survey ed a v ariety of work in the area of PIML, and presented a summary of core motiv ations b ehind their developmen t, p opular physical gov erning equations emplo yed in v arious applications, as w ell as metho ds of in tegration. F rom literature, it is eviden t that despite their nov- elt y , applications of PIML hav e b een prominen t in a v ariety of ﬁelds. F or this surv ey , the applications of PIML metho ds within the context of condition monitoring in v arious engineering applications are examined. Con- dition monitoring is an essen tial aspect of the engineering industry as it pla ys a vital role in ensuring the reliabilit y , safet y , and eﬃciency of assets. Imple- men tations of PIML in this area inv olve the con tinuous monitoring of v arious parameters suc h as vibration, temp erature, pressure, and other critical factors that can indicate the health state of the asset monitored. Through contin uous sampling of these parameters, engineers may identify p otential problems b efore they o ccur, and take correctiv e actions to prev ent costly and unplanned down- time, equipment failure, or ev en catastrophic acciden ts Surucu et al. (2023). Recen t dev elopments in PIML and information capabilities hav e led to a wide v ariet y of inno v ative methodologies for the in tegration of ph ysical knowledge for applications in condition monitoring. In the surv ey by Xu et al. (2022), the au- 4 thors hav e already outlined extensively , the speciﬁc applications of PIML with condition monitoring. As suc h, rather than fo cusing on the speciﬁc applications, this survey aims to provide readers with an ov erview of recent metho dologies of integration betw een the integration of ph ysics-based knowledge with mac hine learning methods. The o verall ob jectiv e of this paper is thus to provide read- ers with a foundation for comprehending its speciﬁc applications, and a deeper understanding of the underlying principles and mec hanisms of PIML. As will be discussed in the bo dy of this surv ey , PIML learning approac hes oﬀer distinct adv antages ov er con ven tional m ac hine learning techniques due to their abilit y to incorp orate fundamen tal ph ysical la ws and principles into the learning pro cess. PIML eﬀectively combines the in terpretive capabilities of ma- c hine learning algorithms with the foundational understanding of ph ysics, lever- aging prior knowledge to guide the learning pro cess. Often, this learning pro cess results in a more accurate and interpretable model. F urthermore, PIML meth- o ds b eneﬁts from reduced reliance on v ast amoun ts of lab eled training data, as ph ysics-based guidelines for optimization can constrain the solution space and provide insigh ts, even in data-scarce scenarios. In all, ph ysics-informed metho ds enable b etter generalization, robustness, and interpretabilit y , making them sup erior to con ven tional ML approaches in man y scien tiﬁc and engineering applications. F urthermore, these metho ds oﬀer b etter explainabilit y to the end user in the con text of explainable artiﬁcial in telligence (xAI), whic h is a gro wing consideration for the wide-adoption of AI tec hniques. The literature survey is organized as follo ws: Section 2 provides an outline of the search methodology in determining articles for review. Section 3 pro vides a detailed explanation of the metho dologies by whic h physics ma y b e integrated in to data-driv en solutions. F urthermore, the section also details the background of p opular architectures within the machine learning communit y , as w ell as how authors in v arious ﬁelds seek to incorp orate prior ph ysical kno wledge within these mo dels. Section 4 pro vides a summary and interpretations of recen t trends, with a fo cus on discussion p ertaining to the adv antages and limitations of the metho dologies surv eyed. Finally , the survey is concluded and summarized in Section 5. 2. Literature Review Methodology This surv ey reviewed recen t developmen ts for in tegration b etw een physics- based mo deling and ML with applications in condition monitoring and anomaly detection. A total of 105 pap ers published were selected after screening. Literary w orks co vered are tallied with resp ect to their year of publication, and presen ted in the visual format in ﬁgure 1. F rom the ﬁgure, it is eviden t that the concept of PIML has b een rapidly gaining p opularity within the researc h communit y . In this surv ey , search methodologies inv olve ﬁlter k eywords suc h as “physics- informed”, “physics guided”, “physics-based”, “Machine learning”, “condition monitoring”, ”fault detection”, ”anomaly detection”, and et cetera. Searc hes w ere performed on platforms suc h as Go ogle Sc holar, IEEE Xplore, Science- 5 Direct, and ACM Digital Library . Results were ﬁltered based on relev ancy , y ear, and citations. 2015 2016 2017 2018 2019 2020 2021 2022 2023 Y ear P ublished 0 4 8 12 16 20 24 28 32 36 Number of P ublications 1 1 0 0 8 11 25 38 21 Publications Reviewed Number of publications per year on the topic of physics-infor med machine lear ning for condition monitoring Figure 1: T allied num b er of literary w orks discussed in this review, with respect to their year of publication. Note: Literature works reviewed in 2023 were limited up until the time of writing of this survey (June 2023). 3. Ph ysics-Informed Mac hine Learning This section details the bac kground of PIML models, as w ell as in tro duces sev eral metho ds b y whic h physical meaning ma y b e embedded within data- driv en solutions. Implementation of PIML v aries greatly dep ending on the ﬁeld of application, and a div erse set of implementation methodologies exists. In general, integration betw een physics-based mo deling with ML is t ypically accomplished through the follo wing frameworks and may be summarized: 1. Ph ysics Embedded in F eature Space 2. Data-Enhanced Reﬁnemen t of Physical Mo dels 3. Ph ysics-Informed Regularization 4. Ph ysics-Guided Design of Architectures Details of these will b e discussed in the follo wing sections. 3.1. Physics Emb e dde d in F e atur e Sp ac e P erhaps the most straigh tforward metho d of integration b et w een physical principles with ML methods is the developmen t of the feature space of an ML mo del with ph ysical mo deling. Augmentations or alterations to the fea- ture space do not directly aﬀect mo del architecture, and the resultant mo del 6 is still considered to be a blac k-b ox mo del, that is, a model capable of pro- ducing relev an t results without revealing information regarding the mec hanisms b y which the results are deriv ed Karniadakis et al. (2021). By leveraging the fundamen tal understanding of the underlying ph ysics, ho wev er, these methods shap e the feature space of a machine learning algorithm in a manner consis- ten t with the physical laws. This integration oﬀers several adv antages ov er traditional machine learning approac hes and leads to a more robust, and data- eﬃcien t framew ork. Through this in tegration, ML algorithms may b e designed to exploit prior kno wledge of physical relationships to be more accurately and eﬃcien tly applied to a v ariety of engineering applications. As describ ed by Karniadakis et al. (2021), this form of integration primarily concerns with the in tro duction of observ ational biases to enhance the performance of ML mo dels. Here, observ ational biases refer to the sp eciﬁc measuremen ts or features that em b o dy the underlying physics or prior knowledge ab out the system under con- sideration. Through the incorp oration of prior knowledge, the in tro duction of observ ational biases through v arious input augmen tation pro cedures serves as a guide in constraining algorithm predictions to b e ph ysically plausible. V ari- ous studies hav e demonstrated that algorithms are more capable of identifying relev an t features in comparison to purely data-driven methods, leading to im- pro ved mo deling capabilities and reduced data limitations (Leturiondo et al., 2017; Gitzel et al., 2021; Deng et al., 2022). Within the con text of applications in condition monitoring, it is often critical to ha ve engineered features within the ML mo del that are sensitive to c hanges in the condition of the asset and is capable of prop erly diﬀerentiating nominal operational conditions from fault conditions. Several approaches to this incorp oration ma y b e seen in literature. F or example, physically generated parameters and v ariables may be employ ed as additional inputs within the feature space. The addition of ph ysics-informed features may b e done either directly in the form of an additional augmen ted dataset parsed through the ML pip eline, or indirectly through metho ds suc h as transfer learning, whereb y the features from a physics-informed source domain are captured via the ML algorithm and re-purposed. Subsequent subsections will discuss these metho ds of feature manipulation, with examples. 3.1.1. Physics-Guide d Input F e atur e A ugmentation The ﬁeld of machine learning has exp erienced tremendous gro wth in re- cen t y ears, and this gro wth has b een fueled in part by the a v ailability of large datasets for the expressive and representativ e training of ML models (L’heureux et al., 2017). Ho w ever, in the context of complex engineering tasks, collecting and lab eling large quantities of data may b e expensive, time-consuming, and in some cases, impractical or imp ossible. Moreov er, due to the blac k-b ox nature of ML mo dels, it is diﬃcult to adjust the b ehaviors of the ML model purely from adjustmen ts to datasets, even when information regarding the system is known b eforehand. A prev ailing solution in literature has b een to use synthetically generated features from system mo dels to supplement or replace real-w orld data, with 7 the main adv antage being that it allows for the creation of large datasets with a high degree of v ariability , while sim ultaneously adhering to gov erning ph ysi- cal principles. This prop ert y is v alued in man y such engineering applications, where small quan tities of observ ational data a v ailable may not accurately re- ﬂect the full range of op erating conditions of a system or piece of equipment (Hop woo d et al., 2022; Gardner et al., 2021). F or example, observ ational data regarding sp eciﬁc fault conditions are rare and impractical to curate in many suc h applications. F urthermore, due to the rarit y and impracticality of induc- ing sp eciﬁc system faults, av ailable datasets are often im balanced and severely sk ewed (Hop woo d et al., 2022). This p oses signiﬁcant issues for a v ailable ML algorithms and their performance, as standard classiﬁers tend to ov erly fo cus on larger classes. As suc h, the syn thesis of ph ysically relev ant features or data represen ts an eﬀectiv e metho dology for obtaining clean, balanced datasets in these scenarios. Limitations that this approac h encoun ters are often with resp ect to the accu- rate replication of the complexit y of real-world operating conditions, and the risk that the generated data will not accurately reﬂect the b eha vior of the equipment or system in question due to incomplete or false prior physics knowledge Serre (2019). Despite this, many authors hav e nevertheless elected to resolve this issue through the generation of ph ysically consistent syn thetic features or data through known ph ysics regarding the system. In this fashion, the generative mo del forms or supplemen ts existing feature space with tailored observ ational biases. The ov erall ob jectiv e of this integration is the detection of potential issues with a higher degree of accuracy , with lesser requiremen ts with resp ect to real-world data collection, and improv ed ov erall adherence to the exp ected b eha vior of the system with resp ect to ph ysical principles. A summary of recent w orks implementing this framework is pro vided in T able 1. Article Title Citation Description Application Motor b earing fault detec- tion using spectral kurtosis- based feature extraction coupled with k-nearest neighbor distance analysis Tian et al. (2015) F eature engineering with spectral kurtosis, with classification using the k- nearest neighbor algorithm Machinery fault diagnosis with b earings Hybrid Mo del-Based and Data-Driven F ault Detec- tion and Diagnostics for Commercial Buildings F rank et al. (2016) F eature engineering using first principles and empir- ical analysis, classification with variet y of classical machine learning algorithms Anomalous b ehavior de- tection in building energy consumption Physics-guided logistic classification for tool life modeling and pro cess pa- rameter optimization in machining Karandik ar et al. (2021) T aylor tool life relation with cutting sp eed applied to form input feature space through logarithmic trans- formations, in conjunction with a linear logistic classi- fier Remaining useful life esti- mation and state of health monitoring for machine tools A physics-informed machine learning approach for notch fatigue evaluation of alloys used in aerospace Hao et al. (2023) Physic-driv en parameters for augmenting input fea- ture space, regression using Support V ector Regres- sion, Random F orest, and XGBoost F atigue life estimations in poly-crystalline alloys 8 Article Title Citation Description Application Structural Health Moni- toring using deep learning with optimal finite element model generated data Seven tekidis et al. (2020) Finite element simulation generated structural data, using classification with conv olutional neural net- works Structural health monitor- ing A hybrid physics-assisted machine-learning-based damage detection using Lamb wa ve Rai & Mitra (2021) Finite element mo dels to form input feature space comprised of damage sp e- cific features, for training a neural network Structural health monitor- ing A p ersonalized diagnosis method to detect faults in gears using numerical simulation and extreme learning machine Liu et al. (2020) Finite element simulation generated fault data, for use with Extreme Machine Learning classification Machine Condition Monitor- ing for gearboxes Physics-informed machine learning mo del for battery state of health prognos- tics using partial charging segments Kohtz et al. (2022) Finite element simulation of dominant degradation mode, Gaussian pro cess regression for learning relation b etw een voltage curve and solid electrolyte buildup State of health monitoring and remaining useful life estimation for lithium-ion batteries Physics-informed machine learning assisted uncertaint y quantification for the corro- sion of dissimilar material joints Bansal et al. (2022) Finite element corrosion model to simulate the cor- rosion pro cess, generated data employed to train a Gaussian Pro cess mo del Structural health monitor- ing for corrosion damage estimation Hybrid deep fault detection and isolation: Com bining deep neural netw orks and system p erformance mo dels Chao et al. (2019) Calibration-Based system performance models, in- formed feature selection for v ariational auto encoder and artificial neural netw ork classification Machinery fault diagnosis in turbine engines F using physics-based and deep learning models for prognostics Chao et al. (2022) Parameter estimation with physics-based models, clas- sification with artificial neural networks Machinery fault diagno- sis and remaining useful life estimation in turbine engines Physics-informed neural netw orks for electro de-level state estimation in lithium- ion batteries Li et al. (2021b) Electrochemical-thermal model for the generation of synthetic data, for use with an artificial neural netw ork for estimation of electrochemical state at different spatial p ositions Remaining useful life esti- mation and state of health monitoring of lithium-ion batteries A physics-informed ma- chine learning mo del for porosity analysis in laser powder b ed fusion additive manufacturing Liu et al. (2021) F eature engineering with deriv ation of physical effects using machine op erating pa- rameters, for use as feature space in a supp ort vector regressor Monitoring for porosity buildup in components during the additive manu- facturing pro cess Physics-informed Cyber- Attac k Detection in Wind F arms Alotibi & Tipper (2022) Physics-based power in- equalities as an indicator of deviations from nominal operations, classification with the isolation forest algorithm Anomalous b ehavior de- tection and monitoring of cyber-physical assets Physics-Based Method for Generating F ully Synthetic IV Curve T raining Datasets for Machine Learning Clas- sification of PV F ailures Hopw o od et al. (2022) Av alanche breakdown mo del simulations of string-level current-v oltage curves, de- tection with 1-dimensional conv olutional neural net- work F ault detection and diagno- sis in photov oltaic cells Hybrid mo del of a physics- based mo del and machine learning for real-time es- timation of unmeasurable parts: Mapping from mea- surable to unmeasurable v ariables Kaneko et al. (2022) Multiple mass-spring- damper models for the generation of labeled time series data, gated recurrent unit recurrent neural net- work for the prediction of parameters Estimation of parameters and anomalous behavior detection in offshore drilling systems Physics-informed deep learning for track er fault detection in photov oltaic power plants Zgraggen et al. (2022) Generation of fault data through physics-informed corruption of operational data, classification with a 1-dimensional convolutional neural network F ault detection and diag- nosis in photov oltaic p ow er plants 9 Article Title Citation Description Application A Combined Machine Learn- ing and Physics-Based T ool for Anomaly Identification in Propulsion Systems Darr et al. (2023) Automatic simulation of anomalies in fluid networks with real-time fault detec- tion and classification using long short-term memory recurrent neural network Anomalous b ehavior detec- tion in propulsion systems, Automation of Simulation of Anomalies Physics-informed long short- term memory netw orks for resp onse prediction of a wind-excited flexible structure Tsai & Alipour (2023) Data generation through mathematical mo del op- timized aero dynamic and aeroelastic parameters for the resp onse of the struc- ture, with a long short-term memory prediction frame- work Structural Health Monitor- ing A novel scalable method for machine degradation assessment using deep con- volutional neural network Li et al. (2020) Establishment of health indicators via high-fidelity physics-based methods. Conv olutional Neural Net- work employ ed to map mon- itored low-fidelity data to established health indicators Machinery degradation modeling and remaining useful life estimation Real-Time F aulted Line Localization and PMU Placement in Pow er Systems Through Convolutional Neural Networks Li et al. (2019a) F eature engineering based on substitution theory , con- volutional neural network based classifier for fault localization F ault diagnosis and lo caliza- tion in electrical grids Comparative Study b etw een Physics-Informed CNN and PCA in Induction Motor Broken Bars MCSA Detection Boushaba et al. (2022) Extraction of fault cor- related features in the frequency domain through F ourier transforms and pro- cessing in the frequency domain for physically rel- ev ant features, detection via convolutional neural netw orks Anomalous b ehavior detec- tion in induction motors Physics-informed machine learning for sensor fault detection with flight test data De Silva et al. (2020) Dynamic mo de decomp osi- tion with control to extract dominant features, clas- sifications with decision tree Anomalous b ehavior de- tection for sensor faults in commercial flight test data Physics-Informed Machine Learning for Degradation Modeling of an Electro- Hydrostatic Actuator Sys- tem Ma et al. (2023) F eatures and model hy- perparameters selection through failure mechanism of system, classification with a long short-term memory network State of health monitoring for an electro-hydrostatic actuator system Roll W ear Prediction in Strip Cold Rolling with Physics-Informed Autoen- coder and Counterfactual Explanations Jakubowski et al. (2022) Generation of new physics- driven features correlated to physical wear of asset, for use in training an autoen- coder for wear prediction Machinery health mon- itoring for degradation prediction Physics-Informed F eature Space Evaluation for Diag- nostic Pow er Monitoring Green et al. (2022) F eature selection through ev aluation of relevance through time, feature sep- arability chec k using geo- metric overlap with respect to hyper-ellipsoidal regions, ev aluated through SVM and neural network. Condition monitoring and p ow er monitoring om electro-mechanical system T able 1: Literature compiled for feature formation or augmentation using physics-based or physics-informed means, for use in machine learning algorithms. Ph ysics-based models ma y be used to simulate a wide range of ph ysical sys- tems. Through augmentation of feature space from said mo dels, ML algorithms ma y b e trained to accurately predict the b ehavior of these systems based on grounded, albeit p otentially incomplete physical principles. This approach is preferred due to the ease of generation of large quan tities of generally reliable 10 data, as well as its capabilit y to circumv ent man y practical and ethical concerns (de Melo et al., 2021). F or example, additional features may b e extracted or generated through knowledge of the system itself, forming an augmented feature space 2(A). Alternativ ely , large quantities of labeled data may be obtained from parsing unlabeled inputs through a ph ysical or numerical simulation mo del, for a ph ysically generated output. Thereafter, the labels and generated outputs ma y b e used in the training pro cess, as illustrated b y Figure 2(B). Subtractiv e feature engineering inv olves mainly feature selection: a tech- nique commonly emplo y ed in ML algorithms to select features that are rele- v an t and meaningful to the problem. Lev eraging physics-based constraints, a ph ysics-informed feature selection strategy ma y aim to identify and retain the most critical features for accurate and interpretable predictions. In addition to the plethora of implementations men tioned ab ov e, the action of generating syn thetic data has also been semi-automated through deep learning structures kno wn as generative adversarial net works. In these structures, a generator and a discriminator neural netw ork are trained sim ultaneously via physics-informed regularization to pro duce physically consisten t synthetic data. More information regarding the net works in particular, as w ell as several examples of implemen- tations in literature ma y b e found in Section 3.4.4: Generativ e Deep Learning Net works. (A) Feature Augmentation + Available Features Physical Model + Extended Physical Features Machine Learning Model Optimization Predicted Outputs Physically Augmented Feature Space (B) Synthetic Data Generation Unlabeled Inputs Physical Model Correlating Inputs with Labelled Outputs Predicted Physical Outputs + Physically Generated Data Machine Learning Model Optimization Predicted Outputs + Observed / Measured Data Physically Augmented Feature Space Figure 2: General outline for the pro cess of the generation of synthetic data via ph ysics- based methods. Prior to the p opularization of the ph ysic-informed mac hine learning paradigm, early studies ha ve already made use of the v arious aforemen tioned adv antages and prop erties of physics-guided syn thetic data generation to generate physically consisten t results on a large scale for use in the training pro cess of data-driven mo dels. Rather than deﬁning the data-driv en mo del from scratch, the a-priori parameters or v ariables deﬁned in or by the physics-based mo dels are used to full eﬀect. F or example, early works by Tian et al. (2015) and F rank et al. (2016) made use of informed data pre-pro cessing techniques and physical mo dels to generate or supplement the input feature spaces of their resp ectiv e ML mo dels. In the w ork of Tian et al. (2015), the authors explored an informed strategy for feature extraction with applications in the monitoring and diagnosing of 11 b earing faults within electrical motors. Known frequency domain fault features w ere extracted via spectral kurtosis, and w ere subsequently utilized to train a semi-sup ervised K-Nearest Neigh b our (kNN) algorithm. F rank et al. (2016) prop osed a hybrid model for fault diagnostics and anomaly detection in building energy usage. The authors employ ed a high-ﬁdelit y system mo del to supplement a v ailable data for use in data-driv en models. Data generated comprises of the system in b oth the health y and fault y state and served to supplement av ailable historical data from a statistical model, and observed data. A v ariety of classi- ﬁcation algorithms, such as the Support V ector Mac hine (SVM), and Random F orest (RF) are presented to classify anomalous b ehaviors from data. In more recen t times, Karandik ar et al. (2021) prop osed a logistic classiﬁcation sc heme to model the degradation of machine to ols making use of kno wn ph ysical la ws as constrain ts to the mo del. In their study , the non-linear ph ysical relationship b et ween cutting sp eed and to ol life is em b edded through a logarithmic manip- ulation of the input parameters. T ransforms of input v ariables suc h as cutting sp eed and time are used as the input feature space for a logistic classiﬁer model to ensure ph ysical consistency with the T a ylor tool life mo del, enforcing linearity in the logarithmic space. F ollowing this, the logistic clas siﬁer outputs the prob- abilit y of degradation state in the to ol. Similarly , Li et al. (2020) proposed a deep CNN-based surrogate mo del for to ol w ear monitoring. The mo del employs high-ﬁdelit y information from sensors, informed via physics-based metho ds suc h as vibration modal analysis or ﬁnite elemen t analysis. Ph ysics-based methods are employ ed to not only optimize the data collection pro cedure by determining sensor placemen ts but also as a feature engineering mechanism for the construc- tion of health indicators. A machine learning mo del is subsequen tly trained to learn the relationship b et ween lo w-ﬁdelity signals and established health indi- cators. Hao et al. (2023) in tro duced a framework for the estimation of notch fatigue degradation in p oly-crystalline allo ys through the embedding of v ari- ous ph ysical parameters in the input feature space. Employing a sensitivity analysis, key parameters gov erning the b ehavior are identiﬁed: Physics-driv en parameters introduced in volv e un-notc hed sp ecimen reference life, derived from the Basquin model, stress state and stress ratio at the notch root, from Neu- b er’s rule, and energy-type damage parameter from the Smith-W atson-T opp er mo del. In all, the Latin h yp ercubic sampling-based PIML mo del in tro duced w as shown to hav e sup erior generalizability and predictiv e capabilities. A common theme in existing literature, for applications inv olving solid struc- tures suc h as structural health or machinery health monitoring, is to emplo y ﬁ- nite elemen t models to generate ph ysical data. With their inheren t v ersatile and robust nature in sim ulating complex real-w orld systems, ﬁnite element mo dels pro vide a systematic approach for predicting and analyzing v arious ph ysical b e- ha viors through the discretization of complex geometries in to smaller elemen ts. More sp eciﬁcally , each element is modeled using mathematical equations that describ e the ph ysics gov erning the b ehavior of the particular elemen t. In this format, go verning or constitutiv e equations representing the physics of the sys- tem may b e embedded within the feature space of the ML model itself; equations 12 go verning physical phenomena suc h as the la ws of conserv ation of mass, momen- tum, and energy , as well as material properties and b oundary conditions may be represen ted and loosely enforced. A v ariety of studies establishes the physical mo del through ﬁnite elemen t sim ulations in whic h the physics of the system is incorp orated via mathematical formulations. F or example, Sev entekidis et al. (2020) utilized the ﬁnite elemen t mo del as a source of sim ulation data to train an ML mo del for damage identiﬁcation problems for structural health monitoring applications, with the pro cedure emplo y ed illustrated in ﬁgure 3. Experimental Domain Vibrational Data Observed Physically Simulated Domain Structure Finite Element model construction Train Simulated Vibrational Data from physical models Neural Network Prediction Deployment for Fault Classifiaction Figure 3: Finite Elemen t mo del of the structure monitored constructed to provide sim ulated training data for Neural Netwo rk, as demonstrated in the work of Sev entekidis et al. (2020). The health state classiﬁcation model is trained solely on lab eled structural resp onse vibrational data generated through a ﬁnite element mo del in v arious loading conditions. The resultan t CNN-based classiﬁer was applied to a b ench- mark linear b eam structure with go od accuracy in determining damage states. Rai & Mitra (2021) emplo yed an artiﬁcial neural netw ork for damage localiza- tion and detection under the lamb w a ve response in an aluminum sample. In their work, v arious ﬁnite element sim ulations are employ ed for the construc- tion of damage-sp eciﬁc features, in a system which the authors hav e termed the damage parameter database. Subsequently , the database is used as the input la yer in the training pro cess of an artiﬁcial neural net work, whereby parameters are updated using the robust Leven b erg–Marquardt algorithm. Liu et al. (2020) similarly emplo yed ﬁnite element methods for the sim ulation of fault data. In their work, the authors introduced a gearbox fault diagnostics pip eline whereby ﬁnite elemen t metho ds w ere employ ed to numerically sim ulate fault samples during gearb o x op erations. Signals obtained are separated into the time and time-frequency domains for use in the generation of fault samples in training an extreme learning machine model. Koh tz et al. (2022) employ ed a Gaus- sian pro cess regression for prognostics and estimating the remaining useful life of a lithium-ion battery . The eﬀect of the dominant degradation pro cess, the build-up of solid electrolyte interface, is mo deled from a physical ﬁnite element sim ulation. Subsequen tly , results from the ph ysical mo del are used in combi- nation with exp erimental data to train a co-kriging-based multi-ﬁdelit y mo del. Through the mo del, an empirical relation b etw een measured voltage curves and 13 the state of health of the lithium-ion battery is deriv ed. Bansal et al. (2022) studied the eﬀect of galv anic corrosion on joints comprised of diﬀering materi- als. The authors prop osed a framew ork whereby feature selection is p erformed based on results from physical sim ulation. More sp eciﬁcally , a ﬁnite element mo del was emplo yed to sim ulate material loss due to galv anic corrosion, while taking in to account en vironmental factors. Subsequently , based on the results of a sensitivit y analysis, parameters most correlated to material loss are selected as features for use in PIML-based surrogate mo deling of the join ts. Incorp oration of synthetically generated data or features ma y prov e in v alu- able in systems where data collection remains a limiting factor. PIML mo dels are commonly emplo yed to estimate diﬃcult-to-observ e v ariables in a v ariety of applications. Leveraging physical constraints, models are capable of providing insigh ts into the b eha vior of complex systems, ev en when direct measuremen ts are limited or una v ailable. F or instance, in the work of Chao et al. (2019), the authors explore a h ybrid approac h for fault detection and isolation in engines. In their study , a ph ysical mo del of an engine is constructed and non-observ able pro cess v ariables are inferred with the Unscented Kalman Filter. Through this pro cess, the authors eﬀectiv ely enhance the feature space of the t wo data-driv en diagnostics models explored, based on Artiﬁcial Neural Net works (ANNs) and V ariational Auto-Encoders (V AEs) resp ectively . Using this study as a basis, the authors further expanded on this mo del with their proposed hybrid framew ork for prognostics and Remaining Useful Life (R UL) estimation in a ﬂeet of engine systems. In another study b y the s ame author, a physical mo del of the system w as employ ed to estimate diﬃcult-to-measure parameters of the system relat- ing to component health. In combination with observed data, the estimated parameters are fed in as data to a neural netw ork, forming a physics-augmen ted feature space (Chao et al., 2022). F urther examples include the w ork of Darr et al. (2023), who sough t to detect and alleviate issues asso ciated with anomalies in propulsion systems during launc h. The group proposed a nov el data genera- tion sc heme that automates the process of ph ysical sim ulations for the creation of anomalous data. The group utilized an LSTM net work for the detection of anomalous behaviors and even ts. Alotibi & Tipp er (2022) created a framework for the detection of false data injection attacks on the op eration of wind tur- bines. Monitored parameters suc h as p ow er output from the physical asset are parsed through a physics-based mo del, whereby based on the law of kinetic en- ergy , augments the av ailable feature space for ML. A ph ysics-informed Isolation F orest is emplo y ed to p erform the anomaly detection. The algorithm combines historical temp oral data from measuremen ts with feature augmentations from the ph ysics-based model to create an ensemble of Random F orests for anomaly detection. The authors demonstrated the increase in anomaly detection accu- racy of the in tegration of ph ysics in their proposed framew ork b y applying the framew ork to a real-world dataset. With resp ect to monitoring the state of health in electro chemical applica- tions, Li et al. (2021b) emplo yed a high-ﬁdelit y electro c hemical-thermal ph ysical 14 mo del for the generation of non-observ able data regarding the electro chemical states in batteries. V ariables generated suc h as lithium-ion concentrations and electric potentials were used in the training pro cess of a neural netw ork whic h learns the nonlinear relationship betw een observ able data and data which can- not be measured physically . In another study b y Hop w o o d et al. (2022), the authors primarily employ ed ph ysical mo deling to ov ercome cost issues asso ci- ated with the high-ﬁdelity condition monitoring of photov oltaic arra ys. The group prop osed a fully syn thetic training dataset based on ph ysical sim ulations of photov oltaic arrays in the health y state, partial soiling fault state, and cell crac k fault state, whereby the framework is illustrated in ﬁgure 4. Data gener- Training Data from Simulation Experimental Data Operational Condition Data Fault Condition Data (Partial Soiling) Measured Operational Dondition Data Machine Learning Model Deployment for Fault Classifiaction Prediction Figure 4: Data augmen tation employ ed to incorp orate simulated fault and op eration data for the training process of a machine learning fault classiﬁcation algorithm, adapted from Hopw o od et al. (2022). ated w ere employ ed to train a 1-dimensional CNN for the classiﬁcation of fault states, and the eﬀectiv eness of this approach w as v alidated with observ ational data. F rom exp erimentation, the accuracy of the ML mo del trained on the syn thetic dataset was identical to that of the observed data. A similar strategy is emplo yed b y Zgraggen et al. (2022), utilizing synthetically generated data to supplemen t a v ailable lab eled fault data. Due to the scarcit y of labeled data for fault scenarios, the authors proposed a fault generation strategy via ph ysics- informed corruption of a v ailable normal operational data, based on a model of the correlating irradiance and p o wer produced given the tilt angle of trac king sensors. Through the ph ysical mo del, the group augmented training data for a CNN mo del in diagnosing anomalous conditions of tracking sensors in a ﬂeet of solar panels in a photo voltaic p ow er plant. With resp ect to applications in health monitoring in structural comp onen ts, 15 Tsai & Alipour (2023) further automated the process of data generation through their proposed LSTM for monitoring and response prediction of a structure sub- jected to excitation by wind. The authors employ ed a mathematical mo del based on optimized aero dynamic and aeroelastic parameters to generate synthetic data on the resp onse of the structure. T o further facilitate data generation and av oid the computational cost that is asso ciated with the mathematical mo dels, the mathematical mo del was emplo yed to train an intermediary LSTM netw ork to automate the generation of large quan tities of data while main taining relative adherence to physical principles of structural response. Data generated from the sim ulated resp onse w as further employ ed to train an LSTM classiﬁer, in con- junction with monitoring data to predict structural response. Similarly , Kaneko et al. (2022) employ ed a physics-informed data generation scheme for the esti- mation of non-observ able parameters in oﬀshore drilling systems. Input data for the model are generated through a ph ysical mo del of the system, whereby v arious input parameters are fed in to the system to obtain the measurable data and identify the unmeasurable data. The general pro cess of whic h is illustrated in ﬁgure 5. Available Data Deployment Known Parameters Physical Model Simulated Unobservable Data Observable Data Machine Learning Model Train T rained Model Experimental Input Data Predictions Figure 5: The h ybrid mo del, featuring ph ysics-based modeling as a basis to map the observ- able parameters to unobserv able parameters, for input to the mac hine learning algorithm. Subsequen tly , a Gated Recurren t Unit (GR U) type recurrent neural net w ork (RNN) is trained to derive the relation b etw een the v arious inputs, outputs, pa- rameters, and measurable data from the ph ysical model, and the unmeasurable data. Liu et al. (2021) prop osed a no vel generalizable physics-informed model for the monitoring and prediction of porosity during the additive manufactur- ing pro cess. Rather than directly correlating machine op erating parameters to p orosit y buildup within the part, the authors instead deriv ed the direct physi- cal eﬀects of mac hine op erating parameters suc h as energy density and pressure distribution. Using physical in terpretations as the input feature rather than the 16 mac hine parameters allows for a generalizable, mac hine-indep endent diagnostics framew ork yielding sup erior predictiv e capabilities. In addition to augmen ting the input feature space, physics-guided methods ha ve also b een emplo yed for feature selection and feature engineering. Through the integration of physical constraints, equations, or relationships into the fea- ture selection algorithm, practitioners are b etter capable of iden tifying essential features that align with underlying physical mec hanisms, pro viding a more ro- bust and interpretable model for data analysis, prediction, and decision-making. In the w ork by Li et al. (2019a), the authors prop osed a feature vector with phys- ical interpretations based on the principles of substitution theorem for fault lo calization in a p ow er grid system. The feature v ector was parsed through a CNN to drastically low er the required net work complexit y for eﬀectiv e fault lo calization. Another example of this is apparent in the work of Boushaba et al. (2022), whereb y the authors compared the eﬀectiveness of a ph ysics-informed CNN approac h for the detection of faults within induction motors. Of note in their study , prior to classiﬁcation with the netw ork devised, measuremen ts from the motor curren t signature analysis were pre-pro cessed in the frequency domain through F ourier transforms to form the input to the net work, as illustrated in ﬁgure 6. Physics-Based Pre-Processing Convolutional Neural Network: Comprised of convolution layers, pooling and fully-connected layers Input Fast Fourier Transform Squared Envelope Final Prediction Figure 6 Here, the pre-pro cessing step mainly serv es as a method for feature selection, extracting certain sub-bands from the signal spectrum correlating to fault y com- p onen ts. De Silv a et al. (2020) automated the process of sensor fault detection in a system with m ultiple fault classes. Due to the complexit y and high dimen- sionalit y of the system, Dynamic mo de decomp osition with control (DMDc), as deﬁned b y Proctor et al. (2016), was employ ed to iden tify a linear time-in v ariant mo del of sensor readings with resp ect to time. Although DMDc is data-driven, the methodology itself allows practitioners to identify and extract underlying coheren t structures, or modes, from complex data. F rom this, DMDc ma y re- v eal the dominant patterns of b eha vior in a system and pro vide insigh ts into its underlying ph ysics. The mo del is applied with a Kalman observ er, which pro vides an estimate of sensor measuremen t v ariables of the healthy state in real- 17 time. F or the classiﬁcation of anomalies, features are in part deriv ed from the DMDc procedure. During v alidation, features expected b y the decision tree ma y b e computed with the linear-time inv ariant system, and measuremen t anoma- lies are classiﬁed. Ma et al. (2023) in vestigated the degradation mec hanism of an electro-h ydro-static Actuator system b y employing a physics-informed Long Short T erm Memory (LSTM) net work. Due to the complexity of the degrada- tion mec hanism, the authors p erformed a physics-informed selection of features, and model hyperparameters w ere performed based on the failure mechanism of the system. In their study , the physical state of the system is represented by a physical parameter indicator: the rise time. Based on the ph ysical state of the system, the monitoring dataset is selected and split into a training and test dataset, which is emplo yed to train and ev aluate an LSTM netw ork. Ev aluation of netw ork p erformance with diﬀerent h yp erparameters is performed through the selected dataset, and the parameters corresp onding to the most accurate predictions are selected. Finally , in the work of Jakubowski et al. (2022), the authors prop osed a ph ysics-informed auto encoder mo del for the estimation of roll wear in equipmen t during the pro cess of cold-rolling. Similar to the ab ov e cases, input space augmen tation was performed employing ph ysics-based simu- lation mo dels. In this case, information for parameters relev ant to wear from cold-rolling, suc h as friction coeﬃcients and forw ard slip, was generated with the prior knowledge av ailable. The roll w ear prediction is performed with an autoen- co der, whereb y data extracted from early stages of degradation, in conjunction with physically derived features, were utilized to train the autoenco der. Pre- dictions for roll wear w ere p erformed based on deviations from the established nominal state. F urthermore, through counterfactual explanations metho ds, the authors sough t to impro ve the interpretabilit y of predictions from the netw ork. Authors ha ve also prop osed methodologies for the selection of feature space sub ject to evolution ov er time. Green et al. (2022) presen ts a strategy for a ph ysics-informed feature space ev aluation in monitoring of electro-mechanical loads. F eatures were curated based on a load separability veriﬁcation, in which the reliabilit y of past training data for future classiﬁcations is ev aluated. The underlying physics of the deviation ov er time is represented via the geometry of h yp er-ellipsoidal regions generated by principle comp onent analysis. Through this, the authors addressed the issue of separabilit y in a system under a multi- load scenario sub ject to op erational or degradation drift. The authors hav e demonstrated the eﬀectiv eness of their approac h through both linear and non- linear classiﬁers, namely an SVM and a neural netw ork. Ov erall, feature augmentation b y means of previously kno wn physical prin- ciples represents an easy-to-implemen t approach to enforce soft constrain ts to mac hine learning algorithms. By tailoring the feature space in which the algo- rithm sp eciﬁcally is consistent with physics, the predictiv e capabilities of the algorithms are more likely to fall within the domain of physical feasibilit y . Con- v ersely , while the mo dels may b e built up on physically consistent training data, the method b y which they arriv e at their predictions remains an enigma to prac- titioners. F urthermore, the lo ose constraints to the feature space of the mo del, 18 rather than the model itself makes these t yp e of algorithms especially prone to o ccasional predictions that are inconsisten t with ph ysical laws. 3.1.2. T r ansfer L e arning Another metho d of integration for ML algorithms may b e through the T rans- fer learning (TL) pro cedure. TL is a technique commonly emplo yed in mac hine learning and deep learning applications, whereby a mo del trained to p erform a certain task is adapted to perform alternate tasks sharing similarities to the original. It has become prominen t due to its abilit y to impro ve p erformance and reduce training requirements and has seen a great deal of use in applications suc h as image analysis, natural language pro cessing, and sp eech recognition for its time and data eﬃciency . With transfer learning, the pre-trained model eﬀec- tiv ely acts as a v essel for feature extraction, leveraging learned features from the source domain and re-purp osing for the target domain. Through this pro cess, training time and resources required are drastically reduced, making TL suit- able for mitigating the cost of complex deep learning architectures. A summary of compiled w orks may b e found in T able 2. Article Title Citation Description Application F ault Cause Assignment with Physics Informed T ransfer Learning Guc & Chen (2021) Dynamic mo de decom- position with control ex- tracts features representing physics of dynamics, con- tinuous wa velet transforms represents modes in the time-frequency domain, classification with pre- trained Go ogLeNet CNN F ault diagnosis in fault source separation in sensor- actuator system Sensor F ault Diagnostics Using Physics-Informed T ransfer Learning F rame- work Guc & Chen (2022) Dynamic Mo de Decom- position with control ex- tracts features representing physics of dynamics, Con- tinuous W av elet T ransforms represents modes in the time-frequency domain, classification with pre- trained Go ogLeNet CNN F ault diagnosis and fault source separation in sensor- actuator system A physics-informed trans- fer learning approach for anomaly detection of aerospace cmg with limited telemetry data Gong et al. (2021) Neural network established to represent the system in the healthy state, based on power balance equations, parameters of the network defined as degradation fea- tures fixed for a healthy state, with fine-tuning to account for degradation conditions. Anomaly de- tection via kernel density estimation Anomalous b ehaviour detec- tion in aerospace control Physics-guided, data- refined mo deling of gran- ular material-filled particle dampers by deep transfer learning Y e et al. (2022) Artificial neural netw ork trained on physical model based on gov erning and constitutive equations of particle dampers, re- calibrated on high-fidelity observ ational data State of health monitoring for particle dampers Using T ransfer Learning to Build Physics-Informed Machine Learning Mo dels for Improved Wind F arm Monitoring Schr¨ oder et al. (2022) Artificial neural netw ork pre-trained on Monte-Carlo simulation database of turbines op eration data, re-calibration with av ailable data Anomalous b ehavior detec- tion in wind turbine sensor data 19 Article Title Citation Description Application Multi-fidelity physics- informed machine learning for probabilistic damage diagnosis Miele et al. (2023) Artificial neural netw ork trained on low-fidelit y fi- nite element simulation, transference of low fi- delity trained layers and re-calibration with high- fidelity finite element simu- lation data Structural health monitor- ing in concrete structures Intelligen t fault diagnosis of machinery using digital twin-assisted deep transfer learning Xia et al. (2021) Sparse de-noising autoen- coder trained on fault conditions pro duced by a digital twin of asset F ault detection and diagno- sis in pump system Digital-twin assisted: F ault diagnosis using deep trans- fer learning for machining tool condition Deebak & Al-T urjman (2022) Stack ed sparse auto enco der trained on simulated dataset Condition monitoring for machine tools Structural damage detection based on transfer learning strategy using digital twins of bridges T eng et al. (2023) Conv olutional neural net- work trained on a digital twin of asset Structural health monitor- ing of bridge Structures Digital twin-driven intel- ligent assessment of gear surface degradation F eng et al. (2023) Conv olutional neural net- work trained on digital twin of asset Condition monitoring for gear surface degradation T able 2: Literature compiled for studies emplo ying transfer learning algorithms to learn physically relevan t features. According to the deﬁnition b y Pan & Y ang (2010), the transfer learning framew ork op erates on the source domain D whereby D = {X , P ( X ) } , deﬁned b y input features space X and marginal probabilit y P ( X ). Here, X represents a sample data, comprised of vectors from the feature space: X = { x 1 , ..., x n } , x i ∈ X . Similarly , a lab el space may be deﬁned for the data as Y . Th us, for a deﬁned domain, a task T may be deﬁned as T = {Y , P ( Y | X ) } = {Y , η } , Y = { y 1 , ...y n } , y i ∈ Y , whereby the predictiv e function η is learned from labeled data pairing of ( x i , y i ), suc h that η ( x i ) = y i . F or a given target domain D t and unknown learning task T t , the ob jectiv e of a transfer learning framework is to employ a learned predictiv e function η based on latent knowledge gained from source domain D s and known learning task T t . Currently , TL frameworks ha ve been used extensively in deep learning applications. Due to the universal appro ximation capabilities of neural net works, the predictive function ma y easily b e approximated b y the non-linear feed-forward function. A general sc heme of the op erations in a t ypical TL framew ork may b e seen in Figure 7. In literature, there are t wo main methodologies by whic h transfer learning ma y b e incorporated in to the PIML framework: Leveraging the source domain, the trained mo del may be transferred to the target domain in v arious engineering applications. Source domains may b e mo deled based on physical mo dels, or are deﬁned such that the mo del is physically sound and consistent with ph ysical principles. In traditional mac hine learning approac hes, models are trained from scratc h on large datasets, whic h may b e a resource-intensiv e and time-consuming pro cess. Physics-based mo dels can provide a more accurate representation of the underlying system dynamics than purely data-driven mo dels, which can b e limited by the quality and quantit y of a v ailable training data. By incorp orating prior knowledge regarding underlying system dynamics in the form of ph ysics- 20 T ransfer Learning Source Domain T arget domain Knowledge T ransfer Model Weights and Bias Known Source Domain Input Known Source Domain Output ML Model Optimization T arget Domain Input Data Predictions ML Model Figure 7: Illustration depicting the principles and functioning of T ransfer Learning in ma- chine learning, a technique in machine learning that lev erages knowledge gained from one task to impro ve performance on another related task. based mo dels, transfer learning can reduce the computational complexity of the mo del and enable more eﬃcien t training and inference (T orrey & Sha vlik, 2010; Zh uang et al., 2020). Alternatively , physics-based or physics-informed data may b e parsed in as the target domain training data. The mo del is ﬁne-tuned using a smaller dataset speciﬁc to the target problem, containing features related to the target problem in the ph ysics domain. By initializing the model with the pre-trained parameters, the mo del already has a degree of kno wledge regarding features to b e learned, enabling faster con vergence during ﬁne-tuning. Fine- tuning allows the model to adapt its representations to the sp eciﬁc features of the target problem, th us customizing the pre-trained model for the new task. In this approac h, the source domain acts more as a supporting library of learned features, allo wing the TL framework to lev erage said features to signiﬁcantly relax target domain data requiremen ts and exp edite the training pro cess. Examples of the TL methodology ma y be seen in the w ork of Guc & Chen (2021), who prop osed a metho d of fault source identiﬁcation of complex and dynamic systems through their physics-informed CNN. Through dynamic mo de decomp osition, a physical representation of the system ma y b e constructed in the form of linear reduced-order spatial-temp oral mo des. Dynamic mode decom- p osition modes are then formulated in to images in the time-frequency domain b y means of contin uous w av elet transforms. F ault conditions are classiﬁed via a CNN image classiﬁer, leveraging a pre-trained net work structure kno wn as Go ogLeNet (Szegedy et al., 2015) to take adv antage of learned features from other domains. The Googlenet architecture is comp osed of 22 main la yers and 21 emplo ys the inception arc hitecture with w eighted Gab or ﬁlters. The authors later extended this proposed framework to p erform diagnostics on the v arious faults that are prev alent in sensors. The eﬀectiv eness of their proposed frame- w ork is demonstrated exp erimen tally with the real-time velocity control of the target system (Guc & Chen, 2022). Through pre-trained mo dels that hav e already learned relev ant features, transfer learning can reduce the amount of labeled data required for training. Instead of training a mo del from scratch on the new dataset, the pre-trained mo del can be adapted to the new task b y up dating select lay ers in the netw ork. Sp eciﬁc to the ﬁeld of condition monitoring and anomaly detection, physics- based models ha ve b een emplo y ed to alleviate the issue of limited labeled data. In man y cases, labeled data for a sp eciﬁc machine or failure mode may b e scarce, making it challenging to train accurate mo dels. By generating synthetic data using ph ysics-based models, practitioners can augmen t the training dataset and improv e the model’s abilit y to generalize to new data. F or instance, Gong et al. (2021) facilitated the process of anomaly detection on an aerospace con- trol momen t gyro through a physics-informed transfer learning neural netw ork. Through this framework, they w ere able to o vercome the limitations in data with regard to the telemetry signals monitored. The non-linear relationships b etw een telemetry signals are captured with an ANN approximating the pow er consump- tion b ehavior. Subsequen tly , the degradation of the system is captured through a transfer learning approac h, whereby the last lay er of the neural netw ork mo del for the health y state is ﬁne-tuned to represen t the new degradation state. A p erformance index was constructed based on the Mahalanobis distance, and anomaly detection was performed with the kernel density estimation approach. Similarly , Y e et al. (2022) employ ed a multi-ﬁdelit y framew ork, physics-based lo w-ﬁdelity data generated is employ ed to pre-train a neural net work, such that when used with the limited amoun t of high-ﬁdelit y exp erimental data av ailable, the net work demonstrated robust c haracterization of gran ular material-ﬁlled particle damp ers. Sc hr¨ oder et al. (2022) applied the transfer learning paradigm for anomaly detection based on the operating b ehavior of a wind turbine through a physics-constrained artiﬁcial neural netw ork. The net work w as pre-trained on data generated b y a physics-based Monte-Carlo sim ulation, and the transfer learning of data from the ph ysical sim ulation w as v alidated in the detection of anomalies in turbine blade angles through monitoring data. With limited data, the model demonstrated sup erior capabilities in b oth prediction accuracy and robustness due to the incorporation of physical constraints from the pre-trained net work. More recen tly , Miele et al. (2023) proposed a transfer learning-inspired neural net work framew ork for structural health monitoring applications. Due to computational restrictions of high-ﬁdelity models, the authors elected to train the net work initially on lo w-ﬁdelity physical mo del deriv ed from a 2-dimensional ﬁnite elemen t simulation. Mo del w eights are held constant, and an additional la yer is added to the neural netw ork structure to re-calibrate the mo del with high-ﬁdelit y 3-dimensional ﬁnite element sim ulation. The resultan t mo del is v alidated in pro ducing the probabilistic classiﬁcation in a sample concrete sp ec- 22 imen. A common form of physically constrained data of the source domain comes from digital t wins (DTs). DTs are virtual replicas of physical systems or pro- cesses that mimic their real-w orld behavior in a digital environmen t. They are created b y using a combination of sensor data, ph ysics-based models, and ML algorithms to create a digital representation of the system or asset. DTs ha ve b een extensively utilized in applications such as predictiv e maintenance and condition monitoring, and ha ve gained signiﬁcan t attention in recen t years due to their ability to impro ve the eﬃciency and eﬀectiveness of v arious engineer- ing tasks (Liu et al., 2022). A generalized form ulation for emplo ying the DT framew ork in conjunction with machine learning may be seen in ﬁgure 8. Digital T wins for Generation of Physically Consistent Data for Machine Learning Deployment T raining Phase Parameter / knowledge transfer Machinery Data from Sensors Physical Model of Asset Feedback & parameter updates Digital T win of Asset Physical Asset User Inputs Pre-train Source Domain Data Machine Learning Algorithm Machine Learning Algorithm Fine-Tune Observed Validation / T est Data Inputs (T arget Domain) Prediction Figure 8: Represen tation of the application of T ransfer Learning in the con text of Digital Twins, a virtual represen tation of a physical entit y or system, sho wcasing the transfer of knowledge from a pre-existing Digital Twin. One of the k ey b eneﬁts of DTs is that they allo w for real-time monitor- ing, analysis, and optimization of physical systems, enabling users to iden tify p oten tial problems and make informed decisions to improv e p erformance and eﬃciency . In literature, there has been gro wing in terest in using DTs in conjunc- tion with machine learning algorithms to create PIML framew orks. The idea b ehind this approac h is to use data from DTs to train mac hine learning mo dels that can then b e applied to real-world systems to predict their behavior and op- timize their p erformance. One c hallenge in using DTs for condition monitoring is that they often require signiﬁcant computational resources to simulate the 23 ph ysical system accurately . This is where transfer learning, with its capabilities for domain adaptation, prov es v aluable. In the context of DTs for condition monitoring, transfer learning can be used to build a framew ork that lev erages pre-existing DTs mo dels to accelerate the developmen t of PIML models. In general, the framew ork for deplo ying DTs in conjunction with TL is as follo ws: 1. A high-ﬁdelity digital t win mo del of the ph ysical system or pro cess of in terest is dev elop ed, capable of sim ulating the system’s b eha vior under v arious conditions. 2. Through the DT model, a large dataset of simulated data may be gen- erated b y v arying the system’s input parameters and monitoring the sys- tem’s output v ariables. 3. This dataset is then used to train an ML model to predict the system’s b e- ha vior. Kno wledge is transferred from the source domain, the DT model, to the target domain of a sp eciﬁc condition monitoring task with transfer learning. 4. Subsequen tly , models are ﬁne-tuned on a smaller amoun t of real-world data to impro ve their p erformance on the target system. Real-w orld data is used to adjust the mo del’s parameters to b etter ﬁt the sp eciﬁc system’s b eha vior. Once trained, the adapted machine learning model ma y b e deploy ed to predict the system’s behavior and or to detect anomalies or deviations from normal operation. Sev eral examples of the ab o ve framework ha ve been utilized for v arious engineering tasks, for example: Xia et al. (2021) prop osed a transfer learning framework for diagnosing faults of a triplex pump system. A Digital twin of the physical asset w as constructed to generate data that is consistent with underlying physical constraints on the system. Along with this, the authors also prop osed a no vel deep de-noising auto-encoder. In conjunction with the health y state data generated b y the digital t win, the au- to encoder is pre-trained. Subsequently , the arc hitecture may then b e emplo yed for anomaly detection in the ph ysical machine. On the same topic, Deebak & Al-T urjman (2022) prop osed a similar transfer learning framew ork featuring DT-assisted fault diagnosis, fo cusing mainly on condition monitoring for ma- c hine to ols and equipmen t. The authors resolv ed the issue with the lac k of real-w orld data through their prop osed stack ed sparse auto enco der structure, reducing the amoun t of ph ysical data required for accurate predictions by the net work, and improving the ov erall robustness of the mo del. T eng et al. (2023) applied a digital twin for the diagnosis of structural fault in bridges, whereby generated data is employ ed in the training pro cess of a CNN. Knowledge transfer from the sim ulated results w as prov en to b e eﬀective, as the mo del demonstrated sup erior con vergence rate and accuracy , in comparison to ph ysically naive trans- fer learning classiﬁcation tec hniques. F eng et al. (2023) applied the framework to gear surface degradation monitoring for predictive main tenance. Digital twin mo dels were dev elop ed and ﬁned tuned based on the gov erning equations for the dynamic and degradation b ehavior of their spur gearbox system. A CNN is established and trained on data from the DT model to assess surface pitting and to oth proﬁle c hange. 24 Through the eﬀective transfer of domain kno wledge, the T L algorithms dis- cussed ab o v e were capable of eﬀectively utilizing ph ysically relev ant knowledge to aid in the predictive capabilities of automated learning. Through this pro- cess, several adv antages presen t themselv es. In addition to the reduced training time and data requirements discussed ab ov e, TL algorithms are also capable of impro ved generalization to data, dep enden t on the training dataset emplo yed. F urthermore, the pre-trained model allows for impro ved in terpretability within the ov erall predictive process of the mo del, through insigh ts into the learned represen tations and the features that inﬂuence the model’s decision. By their nature, TL algorithms are also designed with a particular aim to be ﬁne-tuned to adapt to sp eciﬁc tasks. This prop erty allo ws practitioners an added la yer of ﬂexibilit y in devising the ﬁnal learning pip eline and its constituent comp onen ts, whether those comp onen ts are more-so ph ysically derived, or data-driven. 3.2. Data-Enhanc e d R eﬁnement of Physic al Mo dels Another archet yp e common in literature is the use of ML mo dels as cor- rectional mec hanisms for kno wn inaccuracies or deﬁciencies betw een predicted and observed data. In current applications, ph ysical models are based on sim- pliﬁcations and assumptions that may not accurately capture the complexit y of real-world phenomena. As a result, physical mo dels produce errors or in- accuracies in their predictions. Several works of literature fo cus on developing data-driv en mo dels to address these errors b y learning to accoun t for observ ed deviations, and subsequently , using ph ysics-based and ML mo dels in conjunc- tion for the resultan t predictions. In the works discussed in this section, ML mo dels ha ve been shown to work concurrently with physics-based models to ﬁne-tune results based on outputs from b oth models. A summary of compiled w orks employing this strategy of integration is presen ted in T able 3. Article Title Citation Description Application Battery health man- agement using physics- informed machine learn- ing: Online degradation modeling and remaining useful life prediction Shi et al. (2022) Recurrent neural net- work to mo del the de- viation from physics- based aging model and observed aging State of health moni- toring and degradation modeling for batteries Probabilistic physics- informed machine learn- ing for dynamic systems Subramanian & Ma- hadev an (2023) Augmentation of physics-based mo del with a machine learning model, Bay esian state estimation of model form error is learned by probabilistic ML structure Prognosis and structural response prediction under dynamic loads 25 Article Title Citation Description Application F using physics-inferred information from stochastic mo del with machine learning ap- proaches for degradation prediction Li et al. (2023) Bi-directional LSTM to model residual b etw een observed and sto c hastic degradation mo del Structural health moni- toring T able 3: Literature compiled lev eraging data-driv en models working in tandem with physics-based mo dels In this approac h, a physical mo del is used to generate initial predictions, whic h are adjusted in tandem emplo ying the predictive capabilities of an ML algorithm. The algorithm learns from the set of training data that includes b oth the input features used by the ph ysical mo del and the corresp onding ground- truth outcomes and applies this learning to generate corrections to the ph ysical mo del’s predictions. In literature, this strategy has often b een referred to as h ybrid-mo deling or residual mo deling. The general process b y whic h this in te- gration tak es place is illustrated in Figure 9. Data-Driven Correction Mechanism for Physical Models Predicted Physical Outputs Physical Model Available Data Inputs Predictions ML Model Optimize to model inconsistencies in results Observed / Expected Output - + Gap between Physically Simulated and Expected Outputs Minimize Deviation Figure 9: General outline of correction to physics-based mo deling via data-driven solutions Examples of this implementation are illustrated in v arious works, such as Shi et al. (2022) combined a physics-based degradation mo del with a deep learning netw ork to estimate the state of health in lithium-ion batteries. The ph ysics-informed deep learning mo del is a combination of the physics-based cal- endar and cycle aging model and a Long Short-T erm Memory (LSTM) mo del. Through parameters go verning stress during operation, an initial estimate of calendar aging and cyclic aging of the battery is calculated. Thereafter, the LSTM learns the deviation b et ween observed conditions and the predictions of the physics-based aging mo del ov er time. In conjunction, the physics-informed 26 LSTM mo del was capable of accurately capturing the ov erall degradation trend of the asset.Subramanian & Mahadev an (2023) prop osed a data-driv en correc- tion mec hanism for a structure sub jected to dynamic loading. Error resulting from the physics-based mo del were determined via Bay esian state estimation, whereb y a probabilistic ML structure learns the discrepancies. In conjunction, the combined pip eline demonstrated robust predictions for linear and nonlinear systems with both Gaussian and non-Gaussian noise. Finally , Li et al. (2023) emplo yed a Bi-directional LSTM to estimate the residuals betw een the observ ed degradation b eha vior and the degradation tendency from a tw o-stage sto chastic degradation mo del. The estimated residual is used in conjunction with the out- puts of the physics-driv en sto chastic degradation model to predict degradation in a bridge dec k rebar structure. Though the in tro duction of residual learners has seen success in the ab o ve cases, limitations incurred b y this arc hitecture render it diﬃcult to provide in- sigh tful and in terpretive predictions. As the ML mo del learns to model the discrepancy , rather than the system itself. While studies discussed ab o v e ha ve had success in utilizing this com bination of ph ysics-based modeling and machine learning, this key drawbac k sev erely limits its use cases, as w ell as its explain- abilit y and interpretabilit y , in comparison to other architectures. 3.3. Physics-Informe d R e gularization Regularization techniques ha v e b een fundamental in training ML mo dels since their inception. Conv entional regularization, suc h as Lasso (L1) or Ridge (L2) regularizations, incorporates an additional p enalt y term to reduce the mo del’s capacity to ov erﬁt to data that may not b e reﬂective of the general b eha vior of the system, resulting in simpler and more robust solutions. While this has b een used extensiv ely , a new trend inv olves the usage of physics-based regularization with machine learning. This approac h seeks to com bine the ad- v an tages of physics-based mo dels to enhance the accuracy , interpretabilit y , and robustness of conv entional data-driven solutions. Prior knowledge regarding the ph ysical system is in tegrated as a part of the learning pro cess, either as constrain ts or regularizers, eﬀectively encoding the ph ysical constraints to aid in guiding the optimization process in producing physically meaningful solu- tions. Past implemen tations of physics-based regularization in volv ed solving the ph ysical equations and incorp orating them as constrain ts in the optimiza- tion problem (Ruhnau et al., 2007; Ow are et al., 2013). How ever, this approac h is computationally exp ensive and limited to ph ysical systems that are mostly w ell-understo o d. With the recen t adv ancements in deep learning and the a v ail- abilit y of large amoun ts of data, new techniques hav e b een developed that com- bine ph ysics-based mo deling and ML to b e more eﬃcien t and scalable. F or instance, in recent studies such as the w ork of Raissi et al. (2019), a no vel reg- ularization approach was prop osed that leverages the structure of the ph ysical system to learn more eﬃcien t representations. The prop osed metho d, termed ph ysics-informed neural netw orks (PINNs), incorp orates the gov erning equa- 27 tions of the physical system as regularizers in the loss function. A summary of compiled literature emplo ying this technique is provided in T able 5. Article Title Citation Description Application Microcrack Defect Quan- tification Using a F o cusing High-Order SH Guided W av e EMA T: The Physics- Informed Deep Neural Netw ork GuwNet Sun et al. (2021) Quantification of micro crack defects with hybrid physics- informed architecture design based on v arious deep learn- ing frameworks, regularized via network structure and hybrid feed-forward and back-propagation loss structural health monitoring for detection of micro-crack defects Physics-informed turbulence intensit y infusion: A new hybrid approach for marine current turbine rotor blade fault detection F reeman et al. (2022) F eature extraction via con- tinuous wa velet transform from vibrational data. The classification was p erformed with a neural network, with physics-informed loss func- tion to obtain turbulence intensit y features Anomalous b ehavior detec- tion and fault diagnosis in turbine rotor blades A physics-informed neural netw ork for creep-fatigue life prediction of comp o- nents at elevated tempera- tures Zhang et al. (2021) Neural network regularized via physics-informed loss function, p enalizing the model for unrealistic predic- tions (negative or extreme v alues) of fatigue life Structural health monitor- ing for creep-fatigue life in steel sp ecimen Data-driven prognostics with low-fidelity physical information for digital twin: physics-informed neural netw ork Kim et al. (2022a) Physics-informed loss func- tion p enalizing deviations from exp ected values, de- termined by low-fidelity physical model Structural health monitor- ing for crack propagation Long-term fatigue esti- mation on offshore wind turbines interface loads through loss function physics-guided learning of neural netw orks De Santos et al. (2023) F eatures selected through recursive feature elimination from sensors and moni- toring data. Estimation of fatigue via neural net- work regularized by nov el physics-informed loss func- tion, reflective of priority given to long-term estima- tion Structural health monitor- ing for wind turbines fatigue life Physics-informed meta learning for machining tool wear prediction Li et al. (2022) Parameters of dynamic relationships governing tool wear used to establish input space for individual models at different stages of degradation via cross physics-data fusion. Meta- learning mo del is employ ed to learn the exp eriences of ML mo dels and optimized via physics-informed loss T ool life predictions A physics-informed deep learning framework for inv ersion and surrogate modeling in solid mechanics Haghighat et al. (2021) Physics-informed neural net- work for solving differential equations governing linear elasticity and non-linear von Mises elastoplasticity Elastostatics mo delling in solid mechanics Identification of Material Parameters from F ull-Field Displacement Data Using Physics-Informed Neural Netw orks Anton & W essels (2021) Material parameter estima- tion via solution of momen- tum equation and governing equations of linear elasticity Structural health monitor- ing Inferring vortex induced vibrations of flexible cylin- ders using physics-informed neural networks Kharazmi et al. (2021) Approximation of the linear beam-string equations via PINN for simulation of a cylindrical structure in uniform flow Structural health monitor- ing 28 Article Title Citation Description Application Physics-Informed Machine Learning and Uncertaint y Quantification for Me- chanics of Heterogeneous Materials Bharadwaja et al. (2022) Solution of PDE governing momentum balance and constitutive equations of elasticity , optimized via physics-informed loss func- tion p enalizing deviations from PDE and b oundary conditions Surrogate mo deling of elastic deformations Simulation of guided wav es for structural health mon- itoring using physics- informed neural netw orks (Rautela et al., 2021) Solving PDEs gov erning wa ve propagation with PINNs, regularized by physics-informed loss func- tion based on deviations from PDEs and b oundary conditions Structural health monitor- ing in aerospace structures A physically consistent framework for fatigue life prediction using probabilis- tic physics-informed neural netw ork Zhou et al. (2023b) Probabilistic PINN opti- mized via hybrid loss func- tion based on fatigue life distributions with respect to stress exp erienced State of health monitoring and fatigue life estimation A robust physics-informed neural network approach for predicting structural instability Mai et al. (2023) F eed-forward PINN opti- mized based on deviation from data, instability in- formation, and boundary conditions Structural health moni- toring via estimation of structural instability Machine F ault Classification using Hamiltonian Neural Netw orks Shen et al. (2023) PINN enco ding the laws of Hamiltonian mechanics to learn operating state of system from vibrational data, machinery state iden- tification using netw ork parameters as features Machinery fault diagnosis for rotating machinery Physics-informed machine learning for surrogate mo d- eling of wind pressure and optimization of pressure sensor placement Zhu et al. (2022a) Finite element based com- putational fluid dynamics model for the generation of input features, PINN employ ed for the solution to Navier–Stokes equations of incompressible flows, with Dirichlet and Neumann boundary conditions Structural health monitor- ing in buildings Physics informed neural net- work for health monitoring of an air preheater Jadhav et al. (2022) Stack ed PINNs for solving non-denationalized gov- erning equations for heat transfer b etw een the fluids and metal interface, regu- larized by physics-informed loss function based on devi- ation from PDEs, b oundary and interface conditions Condition monitoring and health monitoring in air heating system Robust Regression with Highly Corrupted Data via Physics Informed Neural Netw orks Peng et al. (2022) F eed-forward PINN based on the least absolute devia- tion metho d to reconstruct PDE solutions and param- eters from highly corrupt sensor data Corrupt data and parameter reconstruction A generic physics-informed neural network-based frame- work for reliability assess- ment of multi-state systems Zhou et al. (2023c) F eed-F orward PINN regu- larized by deviations from ODE of system state transi- tion and initial conditions. Individual element of the loss parse through pro ject- ing conflicting gradients to establish contin uous la- tent function for reliability assessment Reliability assessment Physics-guided conv o- lutional neural netw ork (PhyCNN) for data-driven seismic resp onse mo deling Zhang et al. (2020) Physics-Informed Loss (Dynamic System with Ground Excitation) Structural Health Monitor- ing A physics-informed deep learning approach for b ear- ing fault detection Shen et al. (2021) Physics-Informed Loss (Deviation from Physics- Based Threshold Model Penalized) Machinery fault detection and diagnosis in b earings 29 Article Title Citation Description Application Physics-guided deep neural netw ork for structural damage identification Huang et al. (2022) CNN employed as feature extraction for both the physics and data domain. The network was regularized in accordance to lab elled data as well as the ob jective of minimizing feature dis- crepancy b etw een domains Structural health monitor- ing in bridge structures Bridge damage identifi- cation under the moving vehicle loads based on the method of physics-guided deep neural netw orks Yin et al. (2023) Physics-informed loss func- tion for feature fusion be- tw een the physics-based numerical model and data- driven model) Structural health monitor- ing in bridge structures A physics-informed convolu- tional neural netw ork with custom loss functions for porosity prediction in laser metal dep osition McGow an et al. (2022) Physics-informed CNN with loss function penalizing deviations from ideal simu- lated parameters) Process monitoring in ad- ditive manufacturing for porosity buildup) Physics-Informed Learning for High Impedance F aults Detection Li & Dek a (2021a) Physics-informed conv olu- tional auto encoder, with physics-informed regular- ization based on elliptical relation characteristics of voltage and current plots F ault detection in p ow er grids Physics-informed deep learning for signal com- pression and reconstruction of big data in industrial condition monitoring (Russell & W ang, 2022) Physics-informed conv olu- tional auto encoder, featur- ing loss term incorp orating auto-correlation and F ast F ourier T ransform metrics Data compression for col- lected monitoring signatures in machinery fault detection and diagnosis Physics guided neural net- work for machining tool wear prediction W ang et al. (2020) Cross physics-data fusion for the integration of physi- cal parameters within mo del input. Physics-informed loss function employ ed to enforce relationship between tool degradation with re- spect to op eration progress Condition monitoring for tool wear A Novel Physics-Informed F ramework for Real-Time Adaptive Monitoring of Offshore Structures Liu et al. (2023) Employ ed a physics- informed RNN for solution to governing equations of eigensystem, representative of the modal identification process Structural health monitor- ing Physics-Informed LSTM hyperparameters selection for gearb ox fault detection Chen et al. (2022b) Maximization of Maha- lanobis distance between healthy state and estab- lished physics-informed fault state for LSTM opti- mization pro cess Machinery fault diagnosis in gear b oxes T able 4: Literature Compiled for ph ysics-guided or physics-informed regularisation technique employed Ph ysics-guided regularizations consist primarily of tailoring constrain ts that directly alter the data-driv en mo del in the training phase to fav or predictions that are consisten t with underlying physics. Constraints of this t yp e are also kno wn as learning biases, as characterized by Karniadakis et al. (2021), and implemen ted through physics-informed loss functions. These loss functions pe- nalize deviations from ph ysical la ws, making the mo del more lik ely to pro duce ph ysically plausible solutions. Conv en tionally , the loss function emplo y ed in ML algorithms is a measure of the empirical diﬀerence b etw een the model prediction and ground truth, with the ob jectiv e of minimizing the loss function through an iterative pro cess. Mo del loss is optimized by adjusting the parameters of the mo del to reduce the aforemen tioned diﬀerence in mo del predictiv e capabilities v ersus ground truth. In con trast, a physics-informed loss function incorporates 30 additional information ab out the system b eing mo deled, suc h as ph ysical con- strain ts, conserv ation la ws, and other kno wn properties of the system in tandem with the p enalization of deviations from ground-truth observ ations. Through this framew ork, the ML algorithm may more eﬀectiv ely constrain the prediction space to a void violations of physical principles. Algorithms in tro duced in this format aim to simultaneously minimize er- rors to b oth the labeled data and ph ysical constrain ts. This is reﬂected in the structure of the loss functions implemen ted, whereby the physics-informed loss function is comprised of a data-driv en loss term and a ph ysics-based loss term. The data-driv en loss term measures the error b etw een the predicted output of the model and the ground truth, or observ ational data. In con trast, the physics- based loss term ensures that the solution satisﬁes the underlying physics of the problem through adherence to gov erning equations sp eciﬁc to the problem. Con ven tionally , compliance with observed data (data-driv en loss) is achiev ed by minimizing the residual b et ween predictions of the net work and true state and is p erformed with a v ariety of distance ev aluators suc h as the mean squared error (MSE) or cross-entrop y error (CSE). Compliance with known ph ysical laws is case sp eciﬁc and v aries in implemen tation, ho wev er, the aforementioned meth- o ds for ev aluation ha v e seen man y implementations in literature. The general form of the loss function then, ma y b e represen ted as: Loss total = λ 1 Loss empirical ( Y prediction , Y targ et ) + λ 2 Loss phy sical ( Y prediction ) (1) Where the parameter λ 1 and λ 2 is the regularization factor to adjust loss terms to b est-ﬁt system c haracteristics. Thus in this format, authors ha ve intro- duced a methodology for the incorp oration of go verning equations to inﬂuence the direction of loss minimization in net works. In literature, physics-informed regularization has b een employ ed to incorp orate knowledge of the exp ected fault signatures of the system under diﬀerent failure mo des, in an eﬀort to ensure that the mo del is able to accurately detect and classify faults, ev en in the presence of noise or other confounding factors. F or instance: (Sun et al., 2021) prop osed a metho dology for the non-destructive detection and quan tiﬁca- tion of micro-crac k defects, a framework based on the electromagnetic acoustic transducer, whic h functions by exciting guided wa ves for crac k detection. The group develops a nov el ph ysics-informed arc hitecture that they hav e termed GuwNet . The proposed net work seeks to employ v arious deep le arning mo dules suc h as conv olutional la yers, dense lay ers, and GR U lay ers in conjunction with the introduction of physical parameters for the approximation of v ariables of crac k propagation. The physical pro cess is represented through v arious connec- tions within the data-driv en and physics-based lay ers and parameters within the netw ork. The net work is optimized by h ybrid feed-forw ard and feedbac k loss functions, comprised of empirical and physics-informed error terms to in- tegrate the physics of ultrasonic w a ve testing in to the training process of the net work. Ph ysics-informed terms are deriv ed from the relationship of defect 31 depth, and quantiﬁed b y transmitted w av e in tensity and reﬂected wa ve in ten- sit y of the ultrasonic guided w av e nondestructive testing method. The metho d demonstrated great promise in the detection of length, depth, and direction of crac k propagation, and w as sho wn to hav e signiﬁcant improv ements in accu- racy in comparison to conv entional deep learning approaches. F reeman et al. (2022) proposed a hybrid approach for anomaly and fault detection in turbine rotor blades, whereby fault features acquired from turbine p ow er signals are com bined with en vironmental data to ensure conformit y to the dynamics of the h ydro-kinematic rotor. The framework extracts statistical features b y means of con tinuous wa velet transforms, and categorized via m ulti-nomial regression. The time domain features selected were pro ven b y the authors to b e physi- cally signiﬁcant, accurately reﬂecting the high-frequency ﬂuctuation b ehavior in signals with resp ect to turbulence intensit y . T urbulence in tensity is classiﬁed with a neural net w ork, based on time-domain features extracted from the re- duced feature space and ph ysically constrained through a hybrid loss function, whereb y deviation from the dynamics of turbulence intensit y is p enalized. Regularization has also b een applied with resp ect to applications in fatigue stress and life monitoring. Zhang et al. (2021) constrained the pro cess of creep- fatigue life estimation in a stainless steel sp ecimen with physics-augmen ted feature engineering and ph ysics-informed regularization. The developed feed- forw ard model introduces t wo physics-informed loss terms that tak e into ac- coun t and penalize physical violations with regard to fatigue life. F rom the exp ected behavior of creep-fatigue in the specimen, the authors added ph ysical constrain ts in the form of p enalization for negative v alues, as well as extreme v alues of creep-fatigue life within the loss function. The mo del constructed b oasted superior p erformance when compared with b enchmark empirical and purely data-driven methods. Kim et al. (2022a) adopted a data-driven prognos- tics model that incorp orates lo w-ﬁdelit y ph ysical features in the optimization pro cess. The authors presented an innov ative metho dology for obtaining train- ing parameters for unlab elled extrap olation data. In general, the process for obtaining the extrapolated region, that is, the target of the prognostics frame- w ork, inv olves the ph ysics-based regularization term that p enalizes deviation from the lo w-ﬁdelity physical mo del. T o this eﬀect, the mo del is optimized to minimize in terp olation error with a v ailable data, as w ell as extrap olation error, from the embedded physical mo del. The authors v alidated their approac h with their v eriﬁcation of fatigue crac k gro wth with resp ect to Paris’s law. De San tos et al. (2023) built up on conv entional frameworks for monitoring the progres- sion of fatigue on oﬀ-shore wind turbines by extending the monitoring time p eriod. Conv entional ev aluation of damage monitoring models is based up on the mo del’s ability in ten-minute fatigue damage estimations, whereas Santos et al. ha v e extended this metho dology for monitoring long-term fatigue accu- m ulation. The PINN mo del prop osed fo cuses on minimizing the Mink owski logarithmic error, pro viding a more conserv ativ e estimation of fatigue damage in the form of damage estimation momen ts. The loss function was derived such that accuracy b etw een the model’s abilit y to predict short-term and long-term 32 damage is not compromised. (Li et al., 2022) further extended the ph ysics-informed loss function to meta- learning, in their prop osed strategy for estimating tool wear. The method in- tegrates b oth ph ysically deriv ed model inputs, as well as ph ysics-informed loss terms with data-driv en models ov er a series of ML models for the purposes of meta-learning. Meta-learning is deﬁned as the systematic observ ation and learn- ing of learning from meta-data or the observed exp erience accrued by ML mo dels and their performance on v arious tasks. Meta-learning ma y b e classiﬁed as a sub-ﬁeld of machine learning, whereby artiﬁcial intelligence models are trained to automatically solve tasks or problems more eﬃciently and eﬀectiv ely . In their w ork, the inherent principles of to ol w ear are learned for applications in to ol w ear predictions under v arying to ol w ear rates. Through the v arious parameters deriv ed from the dynamic relationships go verning tool w ear, the authors deriv ed the input feature space of the v arious deep learning and machine learning al- gorithms tested, for enhanced interpretabilit y and robustness. Individual ML mo dels are constructed with the basis of physics-informed data-driv en mo deling with cr oss physics-data fusion . Initially conceptualized by W ang et al. (2020), the mo del represen ts a methodology to fuse data from both the physics and data-driv en features. The meta-learning mo del is emplo yed to learn the experi- ences of three mac hine learning models and their predictions of the degradation state of the asset at diﬀerent stages of wear. The algorithms tested were op- timized via the ph ysics-informed loss function, whereby constraints to the tool w ear rate are imposed based on inherent attributes of tool wear and relations go verning to ol wear and cutting force. 3.3.1. Physics-Informe d Neur al Networks Ph ysics-informed neural netw orks (PINNs) are a rapidly gro wing ﬁeld that lev erages the pow er of neural net works to learn complex patterns and rela- tionships from data, while also incorp orating the underlying physical princi- ples such as partial diﬀeren tial equations (PDEs) or ordinary diﬀeren tial equa- tions (ODEs) that gov ern the system. This sp eciﬁc implementation of ph ysics- informed regularization enables the developmen t of predictiv e mo dels that can not only mak e accurate predictions but also pro vide physical insigh ts into the system’s behavior. PINNs are referred to as ph ysics-informed in that they incor- p orate ph ysics-based knowledge or constrain ts into the mo del training pro cess, whereb y the neural net work is employ ed to mak e predictions on the solutions space of gov erning PDEs. Through the in tro duction of learning biases, PINN signiﬁcan tly relaxes restrictions in terms of the quan tity of data required to prop erly train deep learning algorithms (Xu et al., 2023). PINNs are kno wn for their ability to generate accurate predictions with small amoun ts of data, whic h is especially imp ortan t in cases where data acquisition is exp ensive or c hallenging. F urthermore, PINNs are designed in accordance with the ph ysical la ws and constraints of the system and pro duce predictions that boast superior accuracy and that are ph ysically meaningful. These factors make PINNs partic- ularly well-suited for applications in which the underlying ph ysics of the system 33 is w ell-understo o d. The concept of leveraging the computational capabilities of neural net w orks for solutions to diﬀerential equations was initially presented by Lagaris et al. (1998), ho wev er, its reac h w as limited due to limitations of computational pow er at the time. More recen tly , Raissi et al. (2019) p opularized the concept through their study , where they demonstrated the eﬀectiveness of PINNs in solving for- w ard and in verse problems p ertaining to gov erning diﬀerential equations of a ph ysical system. The eﬀectiveness of PINNs, as deﬁned in the w ork of Raissi et al. (2019), is deriv ed, in part, from their usage of the univ ersal approximation capabilit y of neural net works (Hornik et al., 1989), whic h states that a neural net work with a single-lay ered feed-forw ard net work with an activ ation function ma y approximate any function, provided that it is comprised of a suﬃcient n umber of neurons. Naturally , researc hers ha ve extended this prop erty to the solution com complex, non-linear diﬀeren tial equations, in which n umerical or empirical solutions are diﬃcult or impossible. In these scenarios, PINNs ha ve b een lev eraged to learn the mapping b etw een the input data and the output v ariables while enforcing the ph ysical constrain ts of the system. In addition to their abilit y to incorp orate prior kno wledge, PINNs are capable of learning the solution to ODES or PDEs from incomplete data or data with noise, while sim ultaneously satisfying the gov erning equations of the system, making them particularly useful for applications in which data is scarce or exp ensive to collect (Raissi et al., 2019). Through this framework, researchers can build accurate mo dels that provide insigh ts in to the underlying physical pro cesses, making them a v aluable to ol in many scien tiﬁc and engineering applications (Ra ymond & Camarillo, 2021). The original PINN arc hitecture by Raissi et al. (2019) is based on the feed- forw ard structure, and employ ed to solv e the ﬁrst-order non-linear PDE. V arious names exist for this structure in literature suc h as F eed-F orw ard Neural Net- w orks, Artiﬁcial Neural Netw orks, Multi-lay er P erceptron Neural Net works, and Deep Neural Netw orks. The feed-forw ard neural netw ork is a type of artiﬁcial neural net work that consists of m ultiple lay ers of in terconnected no des, or neu- rons, that transmit information through weigh ted connections. In the context of PINNs, the input la y er of the net work corresponds to the physical domain, while the output lay er represen ts the solution to the problem of interest. The in termediate la yers, also known as hidden lay ers, pro vide the necessary compu- tational p o wer to map the input to the output. An artiﬁcial neural netw ork may b e describ ed as a series of non-linear trans- formations. In terms of a mathematical deﬁnition of the netw ork: F or a giv en input lay er of N neurons, and may b e denoted as X = { x 1 , ..., x n ], whereby x i represen ts a feature within the input space X . The net work ma y be deﬁned to host H hidden la yers, with eac h la yer containing M neurons. F rom this, the out- put of the I -th hidden lay er, i ∈ [1 , H ] may b e represented as A I = { a I 1 , ..., a I m } , where a I j represen ts the j -th neuron in the I -th hidden lay er. F or eac h hidden 34 la yer, the output A I is computed through an elemen t-wise applications of non- linear activ ation function Θ i to the w eighted sum of inputs from the prior lay er I − 1, which ma y b e written as: z I j = X  w I j i ∗ a I − 1 i  + b l j (2) Where w j i represen ts the weigh t connecting the i -th neuron in the prior lay er I − 1 to the j -th neuron in the curren t lay er I , a I − 1 i represen t the output of the i -th neuron in the prior lay er, and B i represen ting the bias term associated with the j -th neuron in the I -th hidden lay er. The output of the i -th hidden la yer is computed as: a I j = Θ I  z I j  (3) The output la yer is comprised of K neurons, with predicted output denoted as as Y = { y 1 , ..., y k ). Thus, the output of the neural net work ma y b e computed as: z H +1 j = X ( w H +1 j i ∗ a H i ) + b H +1 j (4) Where w H +1 j i represen ts the w eight connecting the i -th neuron in the H-th hidden lay er to the j -th neuron in the output la yer, a H i is the output of the i -th neuron in the H -th hidden la yer, and b H +1 j is the bias term asso ciated with the j -th neuron in the output lay er. Collectively , this may b e referred to as: z H +1 = w H +1 ∗ a H + b H +1 (5) The PINN emplo ys this existing framework to b e an approximator of the solution to the PDE. In the general case, the non-linear PDE parameterized by γ , as well as its initial and boundary conditions ma y be represented by the form: F  x, t, u, ∇ u, ... ; δ u δ t ... ; γ  = 0 , x ∈ Ω , t ∈ [0 , t ] (6) u ( x, t = t 0 ) = g ( x ) , x ∈ Ω (7) u ( x, t ) = h ( x, t ) , x ∈ δ Ω , t ∈ [0 , t ] (8) Deﬁned in the domain Ω, where Ω ∈ R d with b oundaries δ Ω. F represen ts the non-linear function that deﬁnes the relationship b etw een unkno wn function u , its deriv atives, and its parameters. The PDE deﬁned has hidden solution u ( x 1 ...x n , t ), with input space that ma y be composed of spatial v ariables x and temp oral v ariables t . F or some subsequen t literary works discussed, the system in question ma y b e time-indep enden t, therefore, terms in the ab o ve equations p ertaining to time w ould not b e relev ant. The PDE has initial conditions g and b oundary conditions h . The neural netw ork seeks to mak e a computa- tional appro ximation of the solution u N N from input space (Raissi et al., 2019; Karandik ar et al., 2021). The approximation of solution space by the neural 35 net work is denoted as: u N N ( x 1 ...x n , t ) ≈ z H +1 (9) The deriv atives of this approximation may then b e calculated b y automatic diﬀeren tiation, emplo ying the chain rule of calculus to compute the exact deriv a- tiv es of a function with respect to its input v ariables (Baydin et al., 2018). Utilizing the predicted solution u N N and its deriv ativ es, it is p ossible to then reconstruct the PDE and its initial and b oundary conditions. This reconstruc- tion is then ev aluated with resp ect to an y lab eled data provided, the residual to the diﬀeren tial equation itself, and any boundary or initial condition provided for deviations to an y of the aforementioned terms, represented as: Loss total = λ 1 Loss Data + λ 2 Loss P DE + λ 3 Loss B C + λ 4 Loss I C (10) With parameters λ 1 , λ 2 , λ 3 , λ 4 represen ting w eights for the adjustment of eac h loss term. Deviations, typically ev aluated as mean squared error (MSE) are minimized during the back-propagation pro cess, whereby neural netw ork parameters, suc h as w eight and biases, are adjusted accordingly in accordance with the gov erning equations, as represented in 20. Minimization of the total deviation through the optimization algorithms suc h as gradient descent allows the netw ork to learn the mapping b etw een the input and output space, while sim ultaneously complying with known physical la ws and constraints. In the context of condition monitoring, PINNs allo w for accurate predic- tions by incorporating both data-driven and physics-based approac hes. PINNs can handle sparse and noisy data, extrapolate b eyond training data (Kim et al., 2022a), and provide in terpretable results. They also enable early fault detection, reduce false alarms, and can be used for online monitoring. Since their initial p opularization b y Raissi et al. (2019), a plethor a of subsequen t implementations that follo wed their publication ha ve em plo yed the same feed-forw ard arc hitec- ture. How ever, exp erimen tation with other p opular deep learning architectures, suc h as the CNN, RNN and its v ariants, enco der and decoder net works, as w ell as graph neural netw orks hav e b een deplo yed in literature. The follo wing sec- tions will detail the integration of ph ysics-based regularization with a v ariety of neural net work architectures. 3.3.2. Data-Driven Solutions to Diﬀer ential Equations V arious curren t applications of the PINN framework hav e remained faith- ful to the initial PINN arc hitecture, via the solution to gov erning diﬀeren tial equations of ph ysical systems. Applications of such methods v ary greatly across industries, and ha ve b een applied to numerous areas in which gov erning dif- feren tial equations are kno wn b eforehand. F or instance, within the domain of solid mechanics, PDEs of physical parameters suc h as elasticit y , deformation, and structural response are determined with the purpose of contin ued struc- tural health monitoring. One suc h example is eviden t in the w ork of Haghighat 36 Physics Informed Neural Networks Neural Network Automatic Differentiation x 1 x n t u NN 𝛿 x 1 𝛿 xn 𝛿 t I Physics-Informed Loss Differential Equation Loss: ℒ DE Initial Condition Loss: ℒ IC Minimize: ℒ T otal = λ 1 ℒ Data + λ 2 ℒ PDE + λ 3 ℒ IC +λ 4 ℒ BC Boundary Condition Loss: ℒ BC Data Loss: ℒ Data Figure 10: Ph ysics-informed Neural Netw ork structure et al. (2021), who dev elop ed a method for surrogate modeling and mo del in ver- sion with resp ect to b eha vior in structures deﬁned by the principles of linear elasticit y . This is p erformed through the incorp oration of go verning PDEs and v arious constitutive equations into a PINN for parameter estimations. Through their experimentation, the authors demonstrated the proof of concept through a mo del of the displacement ﬁeld under elastic plane-strain conditions. F or their use case, the authors compared the eﬀects of a collective net work with shared hidden la yers 11 (A), as opposed to utilizing the PINN framew ork to solv e for individual outputs irrespective of the others 11 (B), with eac h output being solv ed b y a PINN dra wing data from a collectiv e input space. The authors ha ve concluded that, while in principle, a wider netw ork will allo w individual asso ciations to b e made betw een sections of the netw ork and output, it w as more eﬀectiv e for eac h v ariable of the solution to b e calculated separately . The authors attributed this to the h yp erbolic tangen t activ ation function used, be- ing incapable of accurately representing the cross-dependencies of the net work outputs in a manner faithful to kinematic relations. An ton & W essels (2021) applied the PINN framew ork for material p arameter estimation with inputs in the form of full-ﬁeld displacemen t data. With resp ect to structural health monitoring on existing infrastructure, the estimation of ma- terial parameters of structural comp onents ma y b e a metho d of ev aluation of 37 (B) Branched Network for Solving Multiple V ariables (A) Single Unified Network for Solving Multiple V ariables Network Solving V ariable 1 Unified Neural Network Network Solving Variable 1 Output Variable 2 Output Variable 1 Output Variables Network Solving Variable 2 Input Variables Input Variables Figure 11: Neural net work arc hitectures for the solutions of unknown v ariables (A) for a uniﬁed neural net work, (B) for independent networks. degradation. T o that eﬀect, the authors derived the solutions to the momen- tum balance equation, as well as the constitutive equations for linear-elastic materials with the classic ph ysics-informed neural net work architecture. Phys- ical regularization w as implemen ted with resp ect to the PDE established, as w ell as lab eled data av ailable for b oundary conditions and observ ed deforma- tion. Similarly , Kharazmi et al. (2021) estimated the structural parameters of a ﬂexible cylinder structure sub jected to v ortex-induced vibrations from the hy- dro dynamic force, with the ob jective of ev aluating structural damage due to fa- tigue. Utilizing the PINN framework, the authors solved the linear beam-string equation, which go verns the motion of the cylindrical structure in question. Bharadw a ja et al. (2022) utilizes a PINN to mo del and quan tify uncertaint y in the elastic deformation of heterogeneous solids. More sp eciﬁcally , isotropic linear elastic b ehavior is assumed to solv e the gov erning diﬀerential equation for the appro ximation of momen tum balance and constitutive equations gov- erning elasticit y . The prop osed PINN is optimized via the ph ysics-based loss function, represen ting mo del error to gov erning diﬀerential equation, as well as the Dirichlet, Neumann b oundary conditions, the boundary conditions as- so ciated with ﬁb ers and voids, and initial conditions. F rom their analysis, the proposed ph ysics-informed metho dology returned results that are similar to that of the Monte Carlo ﬁnite elemen t sim ulation mo del, designated as the b enc hmark mo del in this scenario. As another example: Rautela et al. (2021) sim ulated guided w av es for monitoring structural health with applications in aerospace applications. The framew ork rev olves around using a PINN to solv e go verning PDEs asso ciated with wa ve propagation. In their study , the one- dimensional w av e equation with Dirichlet b oundary conditions is form ulated as the target of the loss function, and predictions b y the PINN are contin uously optimized by the loss function to more accurately reﬂect the ph ysical go verning PDE. Zhou et al. (2023b) prop osed a metho dology for fatigue life estimation, ph ysically constrained by a h ybrid loss function within a probabilistic PINN framew ork. Through the feed-forw ard model, the stress-life relationship is ap- 38 pro ximated. Physical violations are determined through the ev aluation of select collo cation p oints, whereby the ground truths are appro ximated b y the prob- abilit y distribution out-putted b y the feed-forward mo del. Finally , Mai et al. (2023) emplo yed the PINN architecture in predicting structural instability in truss structures. The net work outlined is a represen tation of the displacemen t ﬁeld of the structure, and analysis of parameters allows for the lo cation of crit- ical points susceptible, giv en the input load factors. Optimization is performed via the minimization of the physics-informed loss function, whic h represen ts, ph ysically , the residual load and stiﬀness characteristics of the structure. In all, the metho d yields sup erior accuracy through the v arious example v alidations on sev eral truss structures. With applications to machinery fault detection and classiﬁcation, Shen et al. (2023) proposed a no vel machine fault classiﬁcation framew ork employing a unique PINN framew ork based on Hamiltonian mechanics, whereb y the model is trained to represent the energy conserv ation of the system in healthy and ab- normal states. Hamiltonian systems are those that ob ey Hamilton’s equations of motion, which describ e the time evolution of a system’s state v ariables in terms of its energy . Based on the principle of Hamiltonian mec hanics, the ev olution of a physical system is describ ed via the energy of the system as a function of its p osition and momen tum. This net work is termed Hamiltonian Neur al Network (HNN) and ma y b e considered a class PINNs sp eciﬁcally tailored tow ards the mo deling dynamical systems gov erned by Hamiltonian equations. This incor- p oration allo ws netw orks to predict the evolution of a system ov er time (Grey- dan us et al., 2019). In their work, Shen et al. (2023) applied this concept for the classiﬁcation of faults in rotating mac hinery . Estimations of system energy signatures are deriv ed from observed sensor measurements through the HNN. Subsequen tly , parameters of the HNN are extracted to form the total energy function, whic h is used as the input features for the classiﬁcation algorithm based on the con ven tional RF algorithm. An abundance of studies has also b een performed in optimizing or comple- men ting the av ailable data from sensors for monitoring applications. Through optimization, the ob jective of designed systems is to maximize the relev ant and informativ e data for monitoring the system. An example of this optimization pro cess with PINNs may b e seen in the work of Zh u et al. (2022a), who op- timized sensor placement lo cations for the monitoring of low-rise buildings in resp onse to wind pressure. The ML mo del is trained on data generated from a ph ysical simulation b y means of a high-ﬁdelit y ﬁnite elemen t computational ﬂuid dynamics model. F rom the data pro vided, the ML model seeks to construct a surrogate model of pressure-ﬁeld in real time. This surrogate mo del is further em b edded within a neural netw ork for the optimization of sensor placemen t lo cations. F or inference of non-observ able sensor data, Jadha v et al. (2022) p er- formed condition monitoring of fouling conditions on system health with resp ect to an air pre-heating system in thermal pow er plan ts. Issues arising from the lac k of av ailable sensors on the interior of the system are resolved with the pro- 39 p osed PINN arc hitecture based on the non-dimensionalized gov erning equations for heat transfer for ﬂuid and metal in terfaces. The authors emplo y ed a series of m ultiple PINNs in parallel, op erating from the same set of input features to re- solv e a plethora of equations gov erning heat transfer. PINNs are regularized via the physics-informed loss function, comp osed of the loss comp onents of the gov- erning equations, b oundary conditions, and in terface conditions. F rom the v ar- ious applications listed, the accuracy of sensor data is critical for the collection of data faithful to the system. Decisions based on inaccurate or incomplete in- formation may lead to sub-optimal outcomes or catastrophic consequences, and as suc h, one direction of this arc hitecture has been the reconstruction of corrupt sensory data to allow users a holistic view of system op erations. In particular, in the work by P eng et al. (2022), the authors proposed a PINN structure to reconstruct data with signiﬁcan t corruption from sensor errors. The net works prop osed are based up on the Least Absolute Deviation and median absolute deviation, whereby the PINN architecture is contin uously optimized b y mini- mizing the residual b et ween data-driv en and physical models. The design of the arc hitecture w as v alidated on sev eral classical problems in volving PDEs, such as the Na vier-Stok es equation, Poisson’s equation, and wa ve equations, whereby the algorithm w as capable of accurately reco v ering go verning equations from corrupted observ ation data. In other av enues of researc h, PINNs hav e b een applied for the mo deling of dynamic systems, as demonstrated in the w ork of Zhou et al. (2023c). The authors applied the PINN framew ork for the ev aluation of reliability in multi- state systems. Given that the gov erning equations for Mark ov processes take the form of diﬀerential equations, the computational eﬃciency of PINNs is lev er- aged. The group utilized the gradient surgery method for multi-task learning as outlined by Y u et al. (2020a) to improv e the PINN’s precision in approximating solutions to diﬀerential equations b y alleviating issues with imbalanced gradi- en ts during training phases. F or m ulti-state system reliabilit y ev aluation, the PINN solves for the state estimates of systems with the input of time instant. As with the traditional PINN, the netw ork is p enalized based on loss with resp ect to b oundary conditions, and with resp ect to appro ximation of gov erning equa- tions. In addition to the PINN architecture based on ANN, v arious w orks hav e incorp orated alternate deep learning architectures to b est optimize the netw ork for the data structure of particular applications, whic h will b e detailed b elo w. 3.3.3. Physics-Informe d R e gularization in T andem with Other De ep L e arning A r chite ctur es A plethora of literary works employs the inheren t symmetries and inv ari- ances enco ded b y v arious conv entional deep-learning arc hitectures in compliance with the philosophy of physics-guided regularizations. Literary works presented in this section mainly utilize physics-informed regularizations as the primary metho dology to enco de ph ysical knowledge in to the system. Leveraging the unique computational eﬃcacy and eﬃciency of certain architectures for speciﬁc data types, researchers ha ve drastically innov ated up on the structure of the 40 original PINN and emplo yed the framework in their own ﬁelds of sp ecialization. F or instance, with resp ect to the CNN architecture, their unique con volu- tional la yers are v alued for their capabilities in automatically extracting fea- tures without the need for manual feature engineering, making them inv aluable in complex applications whereb y the relev ant features are diﬃcult to under- stand or quan tify . Studies emplo ying the CNN arc hitecture can b e seen in the w orks of McGow an et al. (2022), who monitored the p orosity during the additive man ufacturing process with their introduction of a set of loss functions. The regularization of the net w ork comprises standard cross-entrop y data loss, as w ell as losses informed b y ph ysical parameters that p enalize deviations from ideal sim ulated melt po ol temp erature and length-to-width ratio and relative error prior to normalization. As another example: Zhang et al. (2020) established a surrogate mo del for the estimation of structural seismic response, informed via equations of motion representing a dynamic system sub jected to ground excita- tion. Sev eral instances of literature attempt to emplo y the physics-informed loss function as a metho dology to minimize deviations b etw een established ph ysical and data-driv en domains. F or example: Shen et al. (2021) adopted a hybrid approac h in their dev elopment of a ph ysics-informed CNN model for fault de- tection in bearings under v arying rotational sp eeds. The proposed CNN mo del and the physics-based threshold model op erated co-curren tly to ev aluate the health class of b earings. The threshold mo del is established based on known limits with regard to the amplitude of env elop e sp ectra of healthy and damaged b earings. Subsequently , a customized physics-informed loss function is imple- men ted, which serves to p enalize the model for predictions that deviates from kno wn ph ysics, as represented by the threshold model. Through this format, ho wev er, the authors hav e made the simplifying assumption that predictions of physics-based mo dels are correct, or rather the probability of predictions b eing correct is v ery high, due to the extreme thresholds set. Huang et al. (2022) explored a similar approach for the combination of the ph ysical and data domains. The authors trained a CNN emplo ying a ﬁnite elemen t mo del for ap- plications in structural health monitoring. Through their designed framework, the authors sought to incorp orate predictions from both the ph ysics-based ﬁ- nite element model and data-driven methods. The CNN proposed functions as a set of feature extractors that op erates simultaneously based on inputs from the ﬁnite element mo del-driv en physics domain, and the data domain. Ph ys- ical constraints are enco ded in a classiﬁer through a no vel cross-physics-data domain loss function, whereby predictions of the classiﬁer are ev aluated with resp ect to the labeled data, as well as the discrepancy of features b et w een the ph ysical domain and the data domain. On a similar note, Yin et al. (2023) monitored structural damage localization in bridge structures due to loads ap- plied b y v ehicles. The authors developed a n umerical sim ulation of the structure and, using the ph ysics-informed loss function sought to fuse features from the ph ysics and data domains. Pro cessed data from b oth domains are fed through 41 the Visual Geometry Group 16 arc hitecture (Simony an & Zisserman, 2014), whereb y damage features are extracted from the time-frequency map of accel- eration signals. The optimization w as carried out with a h ybrid loss function comprised of data-driv en cross-entrop y loss and physics-informed loss penaliz- ing deviations from the physical domain established via numerical sim ulations. Eﬀectiv ely , the netw ork seeks to minimize discrepancies betw een the physical and n umerical mo dels. Feature Fusion through Minimization of Discrepancy between Domains Data-driven domain Physics-Based domain CNN Feature extractor CNN Feature extractor Weight Sharing Feature Fusions Physical Domain Inputs Data Domain Inputs Minimize: ℒ T otal = ℒ CSE + λ*[Discrepancy(X phys, X Data )] Figure 12: In tegration of physics-based and data-driven domains through feature fusion: The CNN arc hitecture is employ ed as a feature extractor. Adapted from Huang et al. (2022) and Yin et al. (2023) Another implementation of ph ysics-informed regularization is with struc- tures in v olving the enco der-deco der st yle netw orks, or auto enco ders. The struc- ture of netw orks of this style may be described as t wo comp onents working in tandem: an enco der and a deco der net work. Through the enco der netw ork, in- put data is compressed through m ultiple transformations to a lo w-dimensional represen tation. This represen tation is subsequen tly decompressed and trans- formed back into the original representation through v arious transforms in the deco der, with the ob jective of accurate reconstruction of input data. Interme- diate lay ers typically consist of lo wer quan tities of neurons, which in eﬀect force the net work to learn a compressed represen tation. In general, AEs are partic- ularly well-suited for condition monitoring tasks as they are able to learn the represen tations of the normal op erating state of a system and detect anomalies 42 or deviations from that state Zhou & Paﬀenroth (2017). Implementations of the autoenco der learn to identify these changes by encoding the normal b ehav- ior of the system into a low er-dimensional represen tation, and then detecting anomalies in the reconstruction error when the system deviates from this nor- mal b eha vior. This strategy has b een employ ed in subsequen t literary works for the eﬀective detection of devian t b ehavior without the need for additional lab eled data. F or example; Li & Dek a (2021a) designed a physics-informed con volutional auto enco der for the detection of high imp edance faults in pow er distribution grids to ov ercome the issue of the lack of lab eled data from con v en- tional approaches. The ph ysics hybrid physics-informed loss term featured in the net work serves to regularize the prediction of v oltage, taking adv antage of the ph ysical relationship, the elliptical tra jectory b etw een measured v oltage and curren t. As another example: Russell & W ang (2022) prop osed a framework for signal compression and reconstruction of large quantities of data in the setting of industrial condition monitoring through a physics-informed deep conv olutional auto encoder. A hybrid loss function was developed comprised of the traditional MSE, Pearson’s correlation coeﬃcient loss, and a ph ysics-informed loss term. As the primary ob jective of an autoenco der is to reconstruct a given signal, domi- nan t frequencies in the signals m ust b e preserved p ost-reconstruction. This fact is leveraged by the authors to impose a physical constraint on the data-driven solution through a loss term sensitive to frequency . The authors also selected to learn laten t representations of operating conditions individually , eﬀectively isolating the compressed represen tations, with the ob jective of optimal repre- sen tation for individual faults. Cross Physics Data fusion Shared Feature space Data-Driven Domain Prediction Process Physical Domain Prediction Process + Physics- Guided Input V ector Known Empirical Equations Unlabeled Monitoring Data Labeled Monitoring Data Local Features Feature Extraction Physical Mapper Data-driven model + Physical Prediction Data Prediction Regression Layer Final Prediction Figure 13: Cross Data-Ph ysics F usion, as presen ted by W ang et al. (2020) predictions based on information from b oth the data domain (comprised of features derived from la- beled monitoring data), and physics domain (comprised of features derived from unlab eled data) are sim ultaneously mapp ed to a shared space, and concatenated. Both are pro cessed through a regression la yer for the ﬁnal prediction. 43 Sev eral examples in literature also tak e adv an tage of the RNNs’ ability to ex- tract temp orally in v ariant data, for use in applications in volving time-domain monitoring. F or example, W ang et al. (2020) fused features from the data- driv en and physics domain through their applications of the cr oss physics-data fusion , with application in modeling damage accumulation in to ols. F eatures from the data domain and ph ysics domain are extracted separately , and subse- quen tly mapp ed to a shared feature space, represen ting tool w ear. Predictions from b oth domains are concatenated, and ev aluated in the ﬁnal regression lay er of the netw ork whereb y a physics-informed loss function is emplo yed to mini- mize discrepancies b et w een the data-driven Bi-directional Gated Recurrent Unit mo del and empirical equations. Liu et al. (2023) prop osed a physics-informed RNN for oﬀshore structural monitoring. The methodology prop osed emplo ys an optimal singular v alue decomp osition procedure for modal identiﬁcation of the structure. Through their study , the authors formulated the ph ysics-informed mo dal identiﬁcation pro cess in to an eigensystem and emplo yed an RNN for the solution of the gov erning diﬀeren tial equations of the eigensystem. Through their proposed framew ork, the authors improv ed up on conv entional monitoring metho ds to devise an eﬃcien t strategy for mo dal iden tiﬁcation and monitoring in real-time, and under dynamic en vironmental conditions. Researc hers ha v e also inno v ated upon the metho dology by which the loss is ev aluated. T raditionally , the v ast ma jority of literature explores the minimiza- tion of deviations from a target v alue. Chen et al. (2022b) instead proposed an LSTM diﬀeren tiation strategy for the state of health focusing on maximizing deviations betw een kno wn states. In their dev elop ed strategy for the selection of LSTM hyperparameters in the detection of gearb ox faults, rather than the con ven tional minimization of mean squared error of the lab eled data, the se- lection strategy proposed maximizes the discrepancy , in this case, ev aluated by the Mahalanobis distance, betw een healthy and physics-informed fault y states. Data of vibration signatures correlating to the fault state are generated based on prior kno wledge of the system and used to establish the target of ev aluation. In all, ph ysics-informed regularization tec hniques represen t a pow erful to ol for the in tro duction of constrain ts with in the training pro cess of deep learning net works. Unlike the previously detailed models, physics-informed regulariza- tion presen ts a guided process by whic h the algorithm is able to acclimate to the domain of physical feasibility , as illustrated in the n umerous w orks discussed in this particular section. Though eﬀective, the main limitations of this approach are primarily regarding the increased complexit y of the loss landscap e, and diﬃ- culties in ac hieving generalization. V arious authors ha ve devised methodologies to circumv ent this issue, with several further exploring the idea of physical con- strain ts to net work optimization, through v arious alterations to the arc hitecture itself, as will b e discussed in the follo wing section. 44 3.4. Physics-Guide d Design of A r chite ctur es In addition to the loss function, the arc hitecture of the ML algorithm itself can be designed to incorporate ph ysics-based constrain ts. F rom the literature, this area of developmen t primarily fo cuses on the design of appropriate neural net work architectures that can eﬃcien tly encode biases and learn the underlying ph ysics of a system. A n umber of sp ecialized neural netw ork arc hitectures ha ve b een proposed to tackle the unique c hallenges in engineering applications. One suc h approac h is to lev erage the information a v ailable to enco de some ph ysical meaning to hidden v alues within the blac k-b o x structure. Particularly with deep learning architectures, physical meaning may b e assigned to interme- diary no des or outputs to facilitate ph ysically-guided and interpretable infor- mation ﬂo w throughout the net work. Depending on the application, through sp ecialized op erations and/or transformations of data retained in intermediary no des in the form of net work lay ers and connections, the ph ysical relev ance of the no de may be propagated. Another approach commonly employ ed is to ascrib e ph ysical signiﬁcance to the connections b et w een no des. Through this no de connection, a ﬁxed ph ysical op eration or transformation ma y b e speciﬁed b et ween la yers of the net work, also accomplishing the task of the preserv ation of ph ysical principles within information ﬂow, alb eit with a diﬀerent metho dology . Subsequen t subsections will detail some applications of the aforementioned arc hitecture design, with resp ect to a selection of popular deep learning frame- w orks. In addition, this section will detail the w orkings of conv entional deep learning architectures within the frame of physics-informed architecture design, with details regarding their arc hitecture and their suitabilit y for speciﬁc appli- cations p ertaining to data t yp e and ph ysics enco ded. F e e d-F orwar d Neur al Networks. V arious examples of this adjustmen t to archi- tecture exist in literature. As the feed-forw ard structure has already b een dis- cussed in section 3.3.1: Physics-Informed Neural Net works, this section will not feature the description of the netw ork itself. Despite recen t innov ations in ar- c hitectures, feed-forw ard neural net works are still commonly emplo yed for their simplicit y , relatively eﬃcient computation, and capabilities for universal ap- pro ximation of con tinuous functions. Their structure itself makes feed-forw ard net works comparativ ely easier to analyze, and subsequently enco de ph ysical relev ance to sections of the netw ork. As suc h man y authors ha ve taken to the dev elopment of interpretable and physics-informed architectures based on the feed-forw ard structure. T able 5 pro vides a brief summary of literary works compiled for the em b edding of ph ysics within the feed-forward architecture: 45 Article Title Citation Description Application Probabilistic physics-guided machine learning for fatigue data analysis Chen & Liu (2021) Probabilistic feed-forward neural network with phys- ically constrained weigh ts and or bias optimization to model fatigue life curve Condition monitoring and fatigue life estimation Integration of a novel knowledge-guided loss func- tion with an architecturally explainable network for ma- chine degradation mo deling Y an et al. (2022) feed-forward netw ork with physically interpretable lay- ers based on signal process- ing techniques, optimized via knowledge-guided loss function Machine condition monitor- ing for bearings F ully interpretable neural netw ork for lo cating reso- nance frequency bands for machine condition monitor- ing W ang et al. (2022a) Extreme learning machines, with physically interpretable signal pro cessing algorithms and physical feature extrac- tion enco ded as additional lay ers in the network Machine condition monitor- ing A physics-informed neural netw ork approach to fatigue life prediction using small quantit y of samples Chen et al. (2023) F eed-forward netw ork, with physical meaning ascrib ed to certain nodes, enforced by physics-based activ ation functions based on the W alker mean stress model and Basquin relation mo del F atigue Life Estimation T able 5: A summary of literature c ompiled for the design of physics-informed architecture, with inno v ations to the feed-forward neural netw ork arc hitecture primarily . Muc h of the literature developed in this section sought to provide in ter- pretabilit y and explainabilit y to the neural netw ork model by imp osing ph ysi- cal constraints on the feed-forward and bac k-propagation process of the neural net work itself. One such example of assigning ph ysical parameters as no des to enforce information ﬂo w consistent with underlying ph ysics ma y b e found in the w ork of Chen & Liu (2021), who proposed a probabilistic approach, whereb y a feed-forw ard mo del is emplo yed to learn the mean and standard deviations for stress to fatigue life distribution relation. Prior knowledge is imposed through a constrained optimization process, whereby ph ysical parameters suc h as the fatigue stress applied, fatigue life, and an index indicating if the sample failed or is sustained through the trial are assigned as input no des. Output no des in volv e parameters to deﬁne the probabilistic distribution of fatigue life, with mean and standard deviation. The net work is constrained via its weigh ts and/or bias restrictions based on kno wn ph ysical relations b et ween parameters, enforc- ing the intermediary v alues to b e consistent in terms of the sign. As another example Y an et al. (2022) employ ed physics-based signal processing techniques in conjunction with ph ysics-informed regularization for a fully arc hitecturally in terpretable neural netw ork. The resultan t feed-forw ard neural net work devel- op ed was designed with three hidden lay ers, represen tative of a data-driv en for- m ulation of signal pro cessing tec hniques suc h as the Hilbert transform, squared en velope, and F ourier transform resp ectiv ely . The net work was regularized by a hybrid loss function, whereb y desired characteristics of the health indicator constructed, suc h as the sensitivit y of early fault detection, are optimized. The authors applied this framework to directly construct health indicators from vi- brational signals for applications in degradation mo deling in mac hines. Similar to the ab ov e w ork, W ang et al. (2022a) developed an in terpretable 46 Extreme Learning Machines Feature Processing Network Extreme Learning Machine Input Interpretable Layers within Network Output Figure 14: Incorporation of physically interpretable feature extraction for use in conjunction with the Extreme Learning Machine: adapted from W ang et al. (2022a) framew ork through the assignment of appropriate physical meanings to lay- ers within the netw ork. The authors applied their prop osed extreme learning mac hine framew ork for applications in machine health monitoring. Extreme learning machine may b e deﬁned as a subset of conv entional neural net works that emphasizes the use of simple models to enable eﬃcien t and scalable learn- ing. Initially in tro duced b y Huang et al. (2006), rather than the multiple hidden la yers of conv entional neural netw orks, an extreme learning mac hine framework is t ypically comp osed of a single hidden la yer that maps inputs to outputs based on a set of ﬁxed weigh ts. These mo dels are m uch easier to train and require m uch less data and computational resources than standard neural nets. T o com- p ensate for the simplicit y of the mo dels, extreme learning mac hines emphasize the use of adv anced techniques for feature extraction, data pre-processing, and data fusion to enable the mo dels to learn complex patterns in the data. Suc h is the case in this study , whereb y the authors employ ed additional feed-forward la yers for the purp oses of applying the w av elet transform, square en velope and F ourier transform to the sampled input features as illustrated in 14, similar to the work of Y an et al. (2022). T raditionally , hidden no des within the ex- treme learning machines are randomly initialized, with random input w eigh ts and random biases. Due to this structure, extreme learning machine mo dels only require the accurate learning of the output la yer, thereb y directly bypass- ing muc h of the time and computational required in comparison to a traditional 47 bac k-propagation optimization approach. W ang et al. (2022a) further inno v ated up on this structure by in tro ducing speciﬁc sparsit y measures as a replacemen t for the randomly initialized hidden lay ers, greatly increasing the in terpretabilit y of the net work. No v el transformations and indices of ev aluation employ ed b y the authors include the Gini index, kurtosis, smo othness index, and negative en tropy . Physics-Informed Activation Functions in Network Linear Activation Sigmoid Activations No Activations Physics-Based Activations Standard Connections Identity Mapping Figure 15: In tegration of physics-based and conven tional sigmoid activ ation functions in neural net works: adapted from Chen et al. (2023) In contrast to the ab ov e w orks, Chen et al. (2023) prop osed an alternate approac h in the integration of physics through arc hitecture, with applications in fatigue life estimation. The author emplo yed a m ulti-ﬁdelity mo del, whereb y ph ysics gov erning fatigue life is embedded in the system through a com bination of data-driven and no v el physics-informed neurons. Interestingly , the authors c hose to apply ph ysics-based activ ation functions to certain no des within the mo del, based on purely ph ysical models suc h as the W alker mean stress mo del and Basquin relation model. The resultan t mo del structure features certain ph ysical neurons operating in conjunction with data-driv en neurons, as illus- trated in 15. This, in eﬀect, enforce the ph ysical relev ance of the node itself via its relations with other no des in the net w ork. Due to their simplicity , there exists a wide v ariety of researc h a v ailable for the application of this particular architecture. F eed-forw ard neural net w orks 48 ha ve b een emplo yed to great eﬀect in a v ariety of no vel alterations, as seen in the w orks discussed abov e. In the past y ears how ever, n umerous research in this area hav e impro ved up on the base neural net work structure to b e more suitable and sp ecialized for the sp eciﬁc data types and structures, whic h will b e detailed in the follo wing sections. 3.4.1. Convolutional Neur al Networks In addition to direct feed-forward models, CNNs ha ve also enjo yed great p opularit y in the research comm unity . Through their innate arc hitecture, CNNs ha ve the ability to enco de certain in v ariances or symmetries that are inherent in the data they are trained on, making them useful in encoding certain bi- ases based on prior knowledge. By design, CNNs innately tak e in to accoun t spatial in v ariance through their use of conv olutional la yers and p ooling lay ers. The unique conv olutional lay ers of the CNN oﬀer an eﬀectiv e and automatic metho dology for the extraction of physical meaning from data. These la y ers serv e to extract spatial features from input data, and are employ ed in tandem with physics-informed lay ers, whic h impose ph ysical constrain ts on the predic- tions. More speciﬁcally , regarding the working of the conv olutional lay er: Within the conv olutional lay er, the netw ork applies a set of ﬁlters to the input data, with eac h ﬁlter detecting a particular feature or pattern in the input, thereb y allo wing the netw ork to detect local patterns in diﬀeren t regions of the input regardless of their lo cation in data. In each conv olutional lay er, the ﬁlter is con volv ed across the en tire range of input data in accordance with stride size. The output of this action is known as feature maps, tensors of lo cally weigh ted sum. F or a t ypical 2-D conv olution op eration, the action ma y b e giv en mathematically as: S ( i, j ) = ( I ∗ K ) ( i, j ) = X m X n I ( m, n ) K ( i − m, k − n ) (11) Whereb y the input data I is conv olved with ﬁlter k ernel K . F rom this con volution action, the CNN is capable of accoun ting for local connectivity , al- lo wing for the capability to detect features in v ariant of lo cation LeCun et al. (1998, 2015). A nonlinear activ ation function is t ypically applied after conv o- lutions to introduce non-linearities to the system. Pooling la y ers are generally inserted b et ween con volutional la yers to reduce dimensionalit y while maintain- ing descriptions of features. In the p o oling lay er, a sub-sample of eac h region in the resultant feature map is taken, Instead of the precise feature lo cations outputted b y the conv olutional la y er, subsequent op erations are p erformed on the summarized features from the p o oling lay er, allowing for the netw ork to b e more robust to v ariations in feature lo cations. P o oling la yers also help to in tro duce spatial inv ariance in to the netw ork by reducing the spatial resolution of the input, t ypically by taking the maximum or av erage v alue in eac h local re- gion. This has the eﬀect of making the netw ork more robust to small v ariations in the input, suc h as translations or distortions. Other inv ariances that may 49 b e represen ted may b e rotational, scale, or p ermutation in v ariances, dep ending on the application. This prop erty makes CNN an imp ortan t asset in condi- tion monitoring tasks where fault signatures may v ary . A summary of compiled studies using the ph ysics-informed CNN framework is presented in T able 6. Article Title Citation Description Application Physics-based convolu- tional neural netw ork for fault diagnosis of rolling element b earings Sadoughi & Hu (2019) Spectral kurtosis and env elop e analysis em- bedded within layers of CNN for informed feature extraction Machinery fault detec- tion and diagnosis in bearings W aveletKernelNet: An interpretable deep neu- ral network for indus- trial intelligent diagno- sis Li et al. (2021a) Contin uous w av elet con- volutional layer as the initial layer for eﬀective extraction of bearing fault features Machinery fault detec- tion and diagnosis in bearings A health-adaptive time- scale representation (HTSR) embedded con- volutional neural net- work for gearb o x fault diagnostics Kim et al. (2022c) Input signals mapped to health adaptive time scale representation as initial feature map of CNN Machinery fault detec- tion and diagnosis in gearboxes F ault Diagnosis of Rolling Element Bear- ings on Low-Cost and Scalable I IoT Platform LU et al. (2019) Physics-based feature weigh ting based on fault characteristic frequen- cies for ev aluation of fault information car- ried by features Machinery fault diagno- sis in bearings A physics-informed fea- ture weighting metho d for b earing fault diag- nostics Lu et al. (2023) F eature weighing layer for evaluation of dis- crepancy b etw een monitored signals and physics of fault, for construction of input feature map of CNN classiﬁer Machinery fault diagno- sis in bearings Fleet-based early fault detection of wind tur- bine gearb oxes using physics-informed deep learning based on cyclic spectral coherence Perez- Sanjines et al. (2023) Spectral coherence map established based on vibration signals. Con- volutional auto enco der employ ed for fault de- tection based on sp ec- tral coherence maps of fault conditions Machinery fault detec- tion and diagnosis in gearboxes Physics-informed light weigh t T emp oral Conv olution Net works for fault prognostics associated to b earing stiﬀness degradation Deng et al. (2022) Developed temp oral CNN based on the relationship b etw een stiﬀness and vibration amplitudes to construct physics-informed health indicator State of health monitor- ing for bearing stiﬀness 50 Article Title Citation Description Application T raﬃc-induced bridge displacement recon- struction using a physics-informed convo- lutional neural netw ork Ni et al. (2022) Branched network design based on sep- arate analysis from acceleration-based and strain-based metho ds, optimized via physics- informed-loss function Prediction of displace- ment in infrastructure for structural health monitoring On-line chatter detec- tion in milling with hybrid machine learn- ing and physics-based model Rahimi et al. (2021) Energy-based chatter detection mo del, supple- mented by data-driven estimation of the op era- tional state of machine anomaly detection dur- ing pro cess monitoring for milling T able 6: A summary of literature c ompiled for the design of physics-informed architecture, with inno v ations to the conv olutional neural netw ork architecture primarily . The use of sp ecially designed la yers or arc hitectures enables the net works to capture the underlying ph ysics while still leveraging the p o w er of deep learning to make accurate predictions. A common approach emplo y ed in current litera- ture is to incorporate physics-inspired la yers suc h as F ourier features, tailored to the physical problem b eing addressed, with the ov erall architecture of the CNN itself (Jing et al., 2017). The fundamental concept b ehind the net work design is to in tegrate physics-based tec hniques such as signal pro cessing within the net w ork lay ers, allo wing for the visualization of fault features related to ph ysics, and pro viding a physical p ersp ective on the impact of physics-based, in terpretable features in the decision-making pro cess. In many suc h studies, (LU et al., 2019; Li et al., 2021a; Kim et al., 2022c; Lu et al., 2023), la yers within a ph ysics-informed CNN can b e sp eciﬁcally designed to prompt the netw ork to extract features that are related to the sp eciﬁc fault types of in terest. These la yers pro duce a physically relev ant feature, map which may then be propa- gated through v arious abstractions within the CNN architecture. Through this constrain t, subsequen t lay ers are more capable of fo cusing on more complex feature extraction and classiﬁcation, impro ving the accuracy and robustness of the monitoring system, as demonstrated in the works of W ang et al. (2022b) and Li et al. (2019b). Ph ysics-informed CNN architectures ha ve seen prominen t use in analyzing time-frequency t yp e data due to their inherent structure and the symmetries and in v ariances enco ded. In man y such applications, the metric b y whic h the state of the system is ev aluated is often the vibrations of the asset in op eration. Deviations from the standard op eration may b e determined based on the ev alu- ation of pro cessed op erational vibration signals through either one-dimensional CNN for vibration signals or t wo-dimensional CNN for images mapped in the time-frequency domain. Authors such as Sadoughi & Hu (2019) hav e also taken to representing ph ysical pro cesses within the CNN through modiﬁcations to con volutional ﬁlters, or kernels. In their work, a physics-informed CNN frame- 51 w ork is established for the diagnostics of faults in rolling elemen t b earings. T o pro cess signals from the frequency domain, the authors mo diﬁed the conv en- tional CNN classiﬁcation scheme, whereb y additional pro cesses are included to enhance fault features. Additional la yers consist of a sp ectral kurtosis la yer, an en velope analysis lay er for pre-pro cessing information, as well as a F ast F ourier T ransform la yer for the post-pro cessing transformation of the predicted feature map to the frequency domain. F or the net work itself, the k ernels con volv ed are generated with reference to the shaft rotation sp eed and characteristic frequen- cies of the bearing. The architecture may b e seen in ﬁgure 16 (A). The authors noted the eﬃcacy of this approach, which may b e attributed to its non-reliance on hyper-parameters due to the physics-based nature of kernels. Through this metho dology , the authors hav e sho wn that the framework is capable of con- straining the faults consistently with higher accuracy than conv entional deep learning approaches. F urther examples inv olving the use of signal pro cessing tec hniques em b edded within la y ers of the net work are apparent in the work of Li et al. (2021a), who introduced a nov el physics-informed CNN arc hitecture that they ha ve termed W av eletKernelNet, as illustrated in ﬁgure 16 (B). The authors presented modiﬁcation to the conv entional CNN architecture through a no vel con tinuous wa velet con volutional lay er, allo wing the netw ork to more ef- fectiv ely extract impulses em b edded in vibrational signals represen ting b earing faults. Similarly , A similar approach was taken b y Kim et al. (2022c), who devel- op ed a health-adaptiv e time-scale representation mo del, ph ysically informed by c haracteristic time and frequency domain fault signatures, and embedded within a CNN for analysis of time-frequency images. The authors adapted the physics- informed CNN framew ork in tro duced for the monitoring of gearb o x faults from vibrational signals with a similar structure as sp eciﬁed in the abov e w ork, em- plo ying a health-adaptive time-scale representativ e mo dule for the generation of indicators. As an extension of Sadoughi & Hu (2019)’s work, LU et al. (2019) con- structed a ph ysics-informed CNN based on their prop osed ph ysics-based fea- ture weigh ting mechanism, whereby prior knowledge regarding c haracteristic fault frequencies are employ ed in weighing vibrational features of rolling ele- men t b earings, as seen in ﬁgure 16 (C). Inspired b y the ab o ve works, in Lu et al. (2023), the authors further built up on their initial mo del with the introduction of a ph ysics-informed CNN framew ork, whereby prior to classiﬁcation with the CNN, the features are pre-processed in accordance to an initial feature w eigh- ing lay er and signal pro cessing lay ers. The prop osed lay ers function to assign greater imp ortance to features with minimal discrepancy to the bearing fault c haracteristic frequencies. In comparison to Sadoughi & Hu (2019)’s w ork, Lu et al. (2023) has elected to directly op erate in the frequency domain when con- structing the input space of the CNN classiﬁer, with notably lo wer requiremen ts in terms of computational complexit y and similar accuracy . P erez-Sanjines et al. (2023) presented an alternate metho d for vibrational signal pro cessing based on cyclo-stationary analysis. Ph ysical information from vibration signals obtained via a 2-dimensional cyclic sp ectral coherence map is incorporated with ML for anomaly detection. Through the cyclic sp ectral coherence maps, ph ysical in- 52 (B) (C) (A) Convolutional Neural Network: Comprised of convolution layers, pooling and fully- connected layers Input Signals from Time Domain Physics-Based Wavelet Convolution layers Softmax activation for final predictions Convolutional Neural Network: Comprised of convolution layers, pooling and fully- connected layers Input Signals from Time Domain Spectral Kurtosis Softmax activation for final predictions Envelope Analysis Physics Based Convolution Prior Physical Knowledge from System Generated Physics- Based Kernels Fast Fourier Transform Convolutional Neural Network: Comprised of convolution layers, pooling and fully- connected layers Softmax activation for final predictions Input Signals from Time Domain Envelope Analysis Fast Fourier Transform Prior Physical Knowledge from System Fault Characteristacs Physics-informed Fault weighing lyaer Figure 16: Design of Physics-Informed la yers for CNN netw orks, including example arc hi- tectures adapted from: (A) Sadoughi & Hu (2019) who employ ed a physics-based kernel generation sc heme to generate con volved ﬁlters for physics-informed con volutions, (B)Li et al. (2021a) utilizing a conv olutional la yer to pro cess Contin uous W av elet T ransform. (C) Lu et al. (2023) employing a ph ysics-informed feature selection lay er. sigh ts are indirectly integrated through the assumption of the vibration mo del. A con volutional auto enco der is leveraged for its abilit y to pro cess spatial data, and employ ed to reconstruct cyclic sp ectral coherence maps based on machine data collected in the health y state. Ev aluation of anomalies is performed on ph ysical comp onents sub ject to rotary motion, with the ev aluation creation b e- ing the motion pro ducing or exacerbating the cyclo-stationary signal if deviating from nominal operation behaviors. Another implemen tation of physically rele- v an t lay e rs is demonstrated in the w ork of Deng et al. (2022), who prop osed a series of ph ysics-informed temp oral CNN for the estimation of bearing stiﬀness degradation. The authors ha v e presen ted several frameworks implemen ting the CNN with physics-informed in tegration, as discussed in prior sections. These strategies inv olve a ph ysics-augmented input feature space, a ph ysics-informed loss function, and netw ork arc hitecture design based on ph ysical principles. Of note, the authors sought to emulate the mapping betw een the remaining useful 53 life of the b earing with resp ect to features extracted from vibrational signals through a custom physics-informed la yer in the netw ork. The lay er is designed to ensure that the pro cess of neural netw ork computations adheres to that dic- tated b y prior physical knowledge. Data Fusion Quasi-static response: derived from strain measurements Convolutional Layers Acceleration and Strain Inputs Dynamic response: derived from acceleration measurements Residual Encoder Decoder Convolutional Layers Residual Encoder Decoder Convolutional Layers Residual Encoder Decoder + + Displacement Output Minimize: ℒ T otal = ℒ Data + λ ℒ Physics Displacement Output Minimize: ℒ T otal = ℒ Data + λ ℒ Physics Residual Encoder Decoder Convolutional Layers Figure 17: Design of a multi-branc h CNN, for individual mo deling of displacement form strain and acceleration measuremen ts resp ectively; adapted from Ni et al. (2022) An alternate implemen tation of physically inspired arc hitecture design is demonstrated in the w ork of Ni et al. (2022), who implemen ted a m ulti-branch structure of the CNN for the monitoring of deﬂection in bridge structures. Through the architecture illustrated in 17, the authors fuse analysis approac hes for displacement reconstruction with respect to strain-based and acceleration- based methods. Due to the shortcomings of each metho d: in that acceleration- based metho ds are less capable of reconstructing quasi-static displacement, and pure strain-based metho ds are inaccurate with resp ect to the reconstruction of dynamic components in displacement, the authors proposed a tw o-branch CNN to construct individual comp onen ts of the exp ected displacements. In this fashion, relations b etw een eac h comp onen t with respect to displacement ma y b e learned indep endently of the other. Similar to the feed-forward net work prop osed b y Haghighat et al. (2021), the individualistic mo deling of physical parameters within the net work is more eﬃcient with regard to optimization. F eature maps, in this scenario, are also indep enden t of each other, al lowing for eac h branch of the netw ork to exclusiv ely fo cus on deﬁning features charac- teristic of quasi-static, or dynamic resp onse, with minimal ”false” or spurious in terference from feature maps depicting another t yp e of b ehavior. A further residual enco der-deco der block w as employ ed follo wing con volution lay ers for enhanced information transmission. Comp onents are aggregated, and further pro cesses through con volution la yers and residual encoder-deco der la y ers for en- hanced accuracy and robustness to noise. The pro cess is also sup ervised b y a 54 ph ysics-informed loss function based on the minimization of the residual be- t ween predicted displacemen ts through time states, formulated as acceleration term from calculus, and observ ed acceleration. A similar idea is illustrated b y Rahimi et al. (2021), who introduced a decision-making algorithm capable of alerting op erators to abnormal conditions suc h as c hatter in the milling pro cess. The framew ork com bines results from a physics-based vibration analysis, as w ell as sp ectral features from a CNN to determine probabilistically , the presence of c hatter during operations. Through this design, the authors circum v ented the issues with existing ph ysics-based monitoring metho ds, in whic h false alarms are often produced due to the transien t vibrations from the excitation of the mac hine under dynamic op erating conditions. Based on the energy-based c hat- ter detection mo del, the hybrid framew ork trains a CNN in parallel during the mac hining pro cess to ascertain the speciﬁc state of operation, with assistance from the physics-based model. In conjunction with the physics-based mo del, the probability of c hatter is up dated with the op erating state for an accurate and robust prediction. 3.4.2. R e curr ent Neur al Networks Another popular deep-learning archi tecture popular within the comm unity is the RNN. RNNs hav e b een prev alent since their inception due to their ca- pabilities in processing sequen tial data: taking in to accoun t the context of the previous inputs in a sequence. Information from the previous time state is parsed as the inputs to a new time state, along with the conv entional input data, al- lo wing the netw ork to incorp orate information from previous inputs in to its curren t pro cessing. As a direct result, RNNs are inherently designed to encode temp oral inv ariance and hav e b een prov en to b e inv aluable in tasks that inv olve understanding temp oral dynamics and relationships. Con ven tional RNNs maps some input x ( t ) at time t to an output y ( t ) through p ossessing information form both the input space x ( t ) , and prior time state h ( t − 1) , also kno wn as the hidden state. An illustration of the RNN architecture ma y b e seen in ﬁgure 18 (A). The mathematical representation of an RNN may b e written as follows, for the given input and prior hidden state, the hidden state of a cell ma y b e represen ted as: z ( t ) = W hh h ( t − 1) + W hx x ( t ) + b h (12) Where W hh and W hx represen ts the w eight matrix associated with the prior temp oral state, and current input state resp ectively , and b h represen ts the as- so ciated bias for the current hidden state. A non-linear activ ation function g ( . ) is applied elemen t-wise to pro duce the hidden state of the cell: h ( t ) = g  z ( t )  (13) F ollo wing this, the output at time t , y ( t ) , ma y b e represen ted as: y ( t ) = g ( W hy h ( t ) + b y ) (14) 55 (A) (B) RNN Cell RNN Cell Input: (x) Output: (y) h (t) RNN Cell x (t) y (t) h (t-1) RNN Cell x (t-1) y (t-1) RNN Cell x (t+1) y (t+1) y (t) W hx + x (t) W hy + Activation Function (g) + b h Activation Function (g) + by W hh h (t-1) h (t) Figure 18: An illustration of (A) the general Recurrent Neural Netw ork architecture, and (B) the inner computational pro cesses within each RNN cell. Whereb y W hy and b y represen ts the associated weigh ts and biases respectively . The activ ation function g ( . ), typically the soft-max or sigmoid function, is ap- plied to a linear transformation of the hidden cell state to pro duce the ﬁnal cell state output. The ab ov e computational pro cess is visually represented in ﬁg- ure 18 (B). F rom their feedback connections, RNNs are capable of main taining hidden cell states that capture the information from prior time states, grant- ing the ability to process sequen tial data and capture temp oral dep endencies. Additionally , unlik e other structures lik e the CNN, RNNs and their v ariants ha ve the ﬂexibility in processing and outputting sequences of v arying lengths, allo wing them to be applied to pro cesses inv olving data with dynamic lengths, a common prop ert y in real-w orld monitoring applications. Long Short-T erm Memory (LSTM) and Gated Recurrent Unit (GRU) are t wo popular v ariants of RNNs dev elop ed to address the problem of v anishing 56 gradien ts, a prev alent issue in the training pro cess of traditional RNNs. LSTMs and GRUs both use gating mec hanisms to selectiv ely store or discard informa- tion within the internal memory . These mec hanisms enable LSTMs and GR Us to capture long-term dep endencies in the data, while sim ultaneously alleviating the issue with v anishing gradien ts. LSTMs, in tro duced in the work of Hochreiter & Sc hmidhuber (1997), has since b ecome one of the most widely used v ariants of RNNs. LSTMs main tain an additional in ternal cell state represen ting long-term memory and employ three gating mechanisms to regulate the ﬂow of informa- tion. The input gate selectiv ely up dates the memory cell with new information from the input of the cell net work, while preven ting irrelev ant information from b eing added to the existing memory state. The forget gate allows for the selec- tiv e remov al of irrelev ant information from the memory cell. Finally , the output gate selectiv ely passes relev ant information from the memory to the next hid- den state and output, eﬀectiv ely controlling the ﬂow of information through the net work. A more recent v ariant of the RNN, the GRU, w as introduced in the w ork of Ch ung et al. (2014), and is a simpler v ariant of LSTMs that use tw o gating mechanisms: the up date gate and the reset gate. The update gate deter- mines ho w m uch of the new input should b e stored in the memory cell, while the reset gate determines ho w muc h of the previous memory should b e discarded. In addition to these v ariants, the in tro duction of bi-directionalit y in the RNN arc hitecture has also been w ell-studied, whereb y at the cost of increased com- putational resources, the hidden states of tw o RNNs pro cessing information in forw ard and bac kward time steps are com bined, allowing the netw ork to capture information from b oth past and future con texts. In spite of their popularity , RNNs and v ariants of the RNN mo del hav e ma jor limitations in terms of their computational eﬃciency . This limitation arises due to the sequen tial nature of the RNN computation (Kolen & Kremer). F or se- quen tial data pro cessing tasks, the ineﬃciency of RNNs for parallel computation ma y b e a ma jor limitation, especially when dealing with large-scale datasets. Due to the nature of their computations inv olving sequen tial dependencies and hidden states, RNNs require a signiﬁcan t amount of time and computational resources to process each data point, especially for long sequences or deep ar- c hitecture. This sequen tial dependency also makes it c hallenging to parallelize the computations across time steps, as the hidden states need to be computed in a sequen tial manner, severely limiting the abilit y of RNNs to take adv antage of parallel pro cessing architectures, suc h as GPUs or TPUs, and leading to further dela ys and ineﬃciencies in the monitoring process. In T able 7, an o verview of the literature review ed is provided. Article Title Citation Description Application Structural dynamics simula- tion using a novel physics- guided machine learning method Y u et al. (2020b) Embedded residual block within RNN cell as a rep- resentation of prediction consistency with physics, it- eratively optimized through a deep residual-based RNN Structural health moni- toring through dynamic simu- lations 57 Article Title Citation Description Application Physics-Informed Deep Neural Network for Bearing Prognosis with Multisensory Signals Chen et al. (2022a) Physical knowledge regard- ing monotonic degradation behavior integrated within LSTM cell, regularized by physics-informed loss func- tion based on observed degradation Prognosis and remaining use- ful life estima- tion in bearings Fleet prognosis with physics-informed recurrent neural networks Nascimento & Viana (2019) Paris’ law governing crack growth embedded within RNN cell as a physics-based module to capture cumula- tive damage employing the RNN architecture Prognosis with respect to fa- tigue crack propagation in aircraft Cumulativ e damage mo d- eling with recurrent neural netw orks Nascimento & Viana (2020) Paris’ law governing crack growth embedded within RNN cell as a physics-based module to capture cumula- tive damage employing the RNN architecture Prognosis with respect to fa- tigue crack propagation in aircraft Wind T urbine Main Bear- ing F atigue Life Estimation with Physics informed Neu- ral Networks Y ucesan & Viana (2019) Data driven metho d to ev al- uate grease degradation. Utilizing parameters of characterized grease degra- dation, as well as physical modeling to characterize bearing fatigue, embedded within RNN cell Prognosis with respect to bearing un- der fatigue and grease degrada- tion A hybrid mo del for main bearing fatigue prognosis based on physics and ma- chine learning Y ucesan & Viana (2021b) Modiﬁed RNN cell for ev al- uation of grease degradation and b earing fatigue simulta- neously Prognosis with respect to bearing un- der fatigue and grease degrada- tion A hybrid physics-informed neural network for main bearing fatigue prognosis under grease quality varia- tion Y ucesan & Viana (2022) Physics of degradation em- bedded within RNN cell, with fo cus on a probabilis- tic metho dology for the identiﬁcation of grease qual- ity and v ariation Prognosis with respect to bearing un- der fatigue and grease degrada- tion Hybrid physics-informed neural networks for main bearing fatigue prognosis with visual grease insp ec- tion Y ucesan & Viana (2020a) Modiﬁed RNN cell for ev al- uation of grease degradation and b earing fatigue simul- taneously . A nov el ordinal classiﬁer that aids in cal- ibrating mo del for grease degradation Prognosis with respect to bearing un- der fatigue and grease degrada- tion A Hybrid Model for Wind T urbine Main Bearing F a- tigue with Uncertaint y in Grease Observations Y ucesan & Viana (2020b) Modiﬁed RNN cell for ev al- uation of grease degradation and b earing fatigue simul- taneously . A nov el ordinal classiﬁer that aids ins cal- ibrating mo del for grease degradation Prognosis with respect to bearing un- der fatigue and grease degrada- tion 58 Article Title Citation Description Application A Probabilistic Hybrid Model for Main Bearing F atigue Considering Uncer- taint y in Grease Quality Y ucesan & Viana (2021a) Graph implementation of physics-informed and data- driven comp onents within RNN cell, for estimation of fatigue damage accumula- tion with consideration to bearing fatigue and grease degradation. Prognosis with respect to bearing un- der fatigue and grease degrada- tion Estimating mo del inade- quacy in ordinary diﬀeren- tial equations with physics- informed neural netw orks Viana et al. (2021) Utilised physics-based RNN as a method of n umerical integration for the solu- tion of ordinate diﬀerential equations. A data-driven term is introduced to cor- rect for discrepancies in physics through embedding within the physics-based RNN Prognosis in v arious models sub ject to com- plex degrada- tion mechanism Physics-Informed Neural Netw orks for Corrosion- F atigue Prognosis Dourado & Viana (2019) Integration of W alker’s equation governing fatigue crack growth within RNN cell architecture to model corrosion fatigue stress. P a- rameters of the equation are solved via data-driven or physics-based comp onents within the RNN cell. Prognosis with respect to cor- rosion damage and fatigue in aircrafts Physics-informed neural netw orks for bias compensa- tion in corrosion-fatigue Dourado & Viana (2020) RNN with modiﬁed physics- based layers incorp orates W alker’s mean stress mo del for fatigue crack propaga- tion. Employed data-driven lay ers to comp ensate for additional corrosion degra- dation Prognosis with respect to cor- rosion damage and fatigue in aircrafts Hybrid physics-informed neural networks for lithium- ion battery modeling and prognosis Nascimento et al. (2021a) Integration of the Nernst and Butler-V olmer equa- tions within RNN cell to represent battery dis- charge at each time state. Data-driven neural netw ork module within RNN com- pensates between known physics and observed degra- dation b ehavior Prognosis of degradation of charge and aging within batteries Li-ion Battery Aging with Hybrid Physics-Informed Neural Networks and Fleet- wide Data Nascimento et al. (2021b) Fleet-wide prognosis with modiﬁed RNN cell struc- ture, based on physical degradation b ehavior gov- erned by the Nernst and Butler-V olner equations Prognosis of degradation of charge and aging within batteries 59 Article Title Citation Description Application Nov el informed deep learning-based prognos- tics framework for on- board health monitoring of lithium-ion batteries Giorgiani do Nascimento et al. (2023) Cumulativ e damage mo del via mo diﬁcations to the RNN cell, comprised of physics-informed mo dules and data-driven neural net- work mo dule, for prediction of charge state. Neural net- work embedded further regularized via Monte-Carlo dropout State of Heath Monitoring and prognosis of degradation of charge and aging within batteries Nov el informed deep learning-based prognos- tics framework for on- board health monitoring of lithium-ion batteries Kim et al. (2022b) Cumulativ e damage mo del via mo diﬁcations to the RNN cell, comprised of physics-informed mo dules and data-driven neural net- work mo dule, for prediction of charge state. Neural net- work embedded further regularized via Monte-Carlo dropout State of Heath Monitoring and prognosis of degradation of charge and aging within batteries T able 7: A summary of literature compil ed for the design of physics-informed architecture, with inno v ations to the recurrent neural netw ork arc hitecture and its v ariants. An approach prev alent in literature is the incorp oration of physics-based constrain ts directly into the RNN architecture, whereby the neural net w ork ar- c hitecture is designed to incorporate physical mo dels as an integral part of the mo del’s arc hitecture. This can b e achiev ed b y including ph ysical equations or constrain ts as additional la yers in the neural net work, which are trained along- side the traditional neural net work la yers. F or example, Y u et al. (2020b) aug- men ted the structure of RNNs b y embedding ph ysics-informed residual blocks within certain RNN cells for structural dynamic simulation. Residual v alues represen t the deviation of predictions with the known physics. Within the con- text of their w ork, the residual blo ck seeks to model exactly the inconsistencies in the dynamic system b etw een each time state and is iterativ ely minimized through the proposed RNN, as illustrated in 19. Chen et al. (2022a) prop osed an architecture in volving the LSTM for the detection of faults and prognosis for b earings. The proposed method has been referenced as a degradation-consistent RNN. The netw ork is physically informed through the in tegration of the mono- tonic degradation b eha vior of mec hanical comp onents. The authors enforce the irreversible nature of the degradation b ehavior of b earings through the in- tro duction of an intermediary v ariable within the netw ork. The v ariable is a represen tation of degradation in time and is embedded within the cell of an LSTM netw ork. The authors also implemen ted a physics-informed loss func- tion whereby the p erformance of the training phase is ev aluated against lab eled data. A physics-informed term ev aluates the observed degradation at any state, with the in termediary v ariable representing degradation, further re-enforcing the underlying ph ysics represented by the LSTM. A recent p opular methodology in literature comes in the form of cumulativ e damage mo deling based on RNN. Initially emplo yed b y Nascimen to & Viana 60 Physics-Guided Computations in RNN Cell Physics-Informed Deep Recurrent Residual RNN Cell Predicted Structural Response: h t Predicted Structural Response: h t-1 + + + Physics-Based Residuals Residual of Predictions (h t-1, h t ) Activation Function Activation Function Residual of Predictions (h t-1, h t ) Activation Function Residual of Predictions (h t-1, h t ) Data Driven Feed-Foward Neural Network System Input Observable output: y t Figure 19: Incorporation of physics within an RNN cell, via ph ysics-informed Deep Residual Recurrent Neural Netw ork: adapted from Y u et al. (2020b) (2019), an RNN to mo del w as employ ed to capture the temp oral dynamics of a mac hine ﬂeet. The authors incorp orated domain knowledge regarding the ph ysics of the mac hines in question into the mo del through the inclusion of ph ysics-based mo del elemen ts directly within the RNN architecture in a format that they ha v e termed the Euler Inte gr ation Cel l , as seen in ﬁgure 20. Em- plo ying Euler’s forward metho d, the authors formulated the discretized system state as a function of the previous system state and the input vector. In this particular instance, based on Paris’ la w gov erning crac k gro wth, a nov el RNN arc hitecture was dev elop ed whereby a physics-informed la yer was incorporated within the cell of a con ven tional RNN architecture to mo del mechanical factors aﬀecting crac k propagation. W orking in tandem with the ph ysics-based mo del, the traditional data-driven mo del estimates the stress intensit y factor range. The combination of these t wo mo dels with the RNN cell yields an accurate es- timation of temp oral dynamics and cum ulativ e damage in the sp ecimen. F ollo wing this publication, the authors also emplo yed the same mo del to es- timate fatigue crac k length growth in aircraft with limited observ ations Nasci- men to & Viana (2020). Subsequent works by other authors are based upon the mo diﬁcation and tailoring of the framework in alternativ e applications. F or instance, studies by Y ucesan & Viana (2019, 2021b, 2022) applied a mo diﬁed v ersion of this framework to model bearing fatigue in wind turbines, whereby the data-driv en mo del w as employ ed in combination with kno wn physics to es- timate the unknown eﬀects of lubrication on failure. Through a combination of data-driv en elemen ts within the cell, as well as physics-based lay ers suc h as 61 Physics-Based Alterations to RNN cell Modified Hybrid Physics-Informed Cell Physics-Based Module Data-Driven Module Physics-Based Module + h t + h t h t-1 + x t Figure 20: Incorporation of physics within a recurrent neural netw ork cell via Euler integra- tion: adapted from Viana et al. (2021) the P almgren-Miner’s rule, the authors sough t to c haracterize the relationship b et ween b earing fatigue and grease degradation through the combined net w ork. T o this eﬀect, the structure of the netw ork was designed to take in to account parameters from both degradation behaviors, and accurately c haracterize each form of degradation with respect to the other. The authors further innov ated on this mo del b y extending its applications to used cases whereb y elemen ts of uncertain ty are introduced to the grease degradation pro cess (Y ucesan & Viana, 2020a,b, 2021a). Viana et al. (2021) presented a metho d for estimation of missing ph ysics utilizing the model, whereby a data-driven la y er is emplo yed to approximate the uncertain behavior of the physical mo del. In terestingly , Viana et al. (2021) c ho oses to employ the RNN arc hitecture as a purely ph ysics-based solution to ordinary diﬀeren tial equations, with the addition of a data-driv en node to quan- tify the discrepancy b etw een known ph ysics and observ ed results. The authors ha ve veriﬁed the approac h with v arious case studies such as the modeling of fatigue gov erned by established physics-based models suc h as Paris’ law for fa- tigue crack propagation, W alker’s equation for fatigue crack propagation, and P almer-Miner’s rule for fatigue life estimation. Another av enue for the application of this hybrid RNN architecture w as explored b y Dourado & Viana (2019) who employ ed a similar framework for the estimation of corrosion eﬀects system cum ulativ e damage. In their w ork, the structure of the RNN was designed to represen t Paris’s equation, with stress 62 in tensity factors b eing determined physically , and the rest of the parameters b eing determined by data-driv en feed-forward modules within the cell. The authors later expanded on their work with the introduction of a data-driven comp ensator to correct for W alker’s mo del for crac k propagation, whereb y data- driv en la yers are emplo yed to model the bias in damage accum ulation as a result of corrosion (Dourado & Viana, 2020). The cum ulative damage mo del has also seen m uch use in mo deling degradation behaviors in lithium-ion batteries: for instance, based on their previous w ork, Nascimento et al. (2021a) mo diﬁed the existing framework to be consisten t with the Nernst and Butler–V olmer equations, with a m ulti-lay er p erceptron mo dule within the cell to characterize the model-form uncertaint y . The approac h fo cuses on building a reduced-order mo del based on Nernst and Butler-V olmer equations. F ollowing a similar idea as Viana et al. (2021), the authors employ ed m ultiple data-driv en mo dules within their mo diﬁed RNN cell structure to compensate for deviations betw een kno wn ph ysics, and observed degradation in the asset. The authors further expanded up on their work to extend the range of applications. In the work: (Nascimento et al., 2021b) to a ﬂeet-wide dataset, allo wing for the identiﬁcation of assets deviating from ﬂeet norms established. While in Giorgiani do Nascimento et al. (2023), the authors further extended the mo del for use with incomplete historical usage of assets through a Ba yesian update strategy of revising the probabilit y of a hypothesis or b elief based on new evidence or information. Kim et al. (2022b) applied the cumulativ e damage framework to the estimation of lithium- ion battery state in a model that they ha ve termed the knowledge-infused RNN. In their mo del, the recurren t cell is further mo diﬁed via the addition of physics- informed mo dules based on a double-exponential mo del of battery capacit y . F urthermore, the authors also introduce a Mon te-Carlo dropout within the data- driv en feed-forward netw ork em b edded wit hin the RNN cell to secure robust and reliable probabilistic estimations of p erformance. 3.4.3. Gr aph Neur al Networks Another example of ph ysics-informed architecture comes from the structural comp osition of Graph Neural Net works (GNNs). GNNs are a class of deep learn- ing mo dels capable of pro cessing graph-structured data, initially conceptualized b y Scarselli et al. (2008). GNNs are comprised of no des and edges, as deﬁned in the w ork of Scarselli et al. (2008). In this representation, no des within the net- w ork represent entities and edges represen t the connection or relation b etw een en tities. An illustration of this arc hitecture is shown in ﬁgure 21. F or graph G = ( V , E ) with no des (also kno wn as v ertices) V and edges E , eac h no de ma y b e represen ted as v ∈ V with feature vector h v . The op eration of the GNNs may then b e deﬁned as the iterativ e pro cess of up dating the node feature vector representations b y aggregating information from their neighbor- ing nodes and then using these updated representations to make predictions or classiﬁcations. Emplo ying a message-passing mec hanism, nodes exc hange in- formation with neigh b oring nodes, enabling them to up date the feature vector based on the information received. This operation is reminiscent of the c on vo- lution op eration applied for CNNs, in the sense that b oth operations eﬀectiv ely 63 Graph Neural Network Structure Hidden Layer Hidden Layer Hidden Layer Input Layer Output Figure 21: A depiction of a graph neural netw ork (GNN) architecture, show casing its k ey components and information ﬂo w: GNNs op erate on graph-structured data, enabling eﬀec- tive analysis, inference, and learning tasks within complex relational datasets. aggregate and process neigh b oring entities to update the v alue of the entit y in question. Eac h node aggregates information from its neigh b ors using a learnable function that takes in to accoun t b oth the no de features and the edge w eigh ts. This information is then passed to each no de’s neighbors. This may b e un- dersto od as follo ws: for eac h no de v , compute the message v ector m l v through aggregating information from its neigh b ors N ( v ) using a learnable function f e : m l v = X u ∈ N ( v ) f l e ( h l − 1 v , h l − 1 u , e l v ,u ) (15) Where e l v ,u is the edge feature vector b et ween no des v and u , f ( l ) e is the learnable function that maps the features inputted of prior lay er l − 1 to the resultan t message at la yer l . Subsequently , each node up dates its represen tation through the com bination of information it receiv ed from its neighbors in the prior pro cedure, with its own original represen tation. This may be represented as the computation of the new feature vector h l v for each node v b y combining the previous feature v ector with the aggregated messages: h t +1 v = f l n ( h l v , m l v ) (16) The message passing and updating steps outlined ab ov e are repeated for a ﬁxed n umber of lay ers un til a ﬁnal represen tation is obtained for eac h no de in the graph. The ﬁnal no de features can then be used for do wnstream tasks such as no de classiﬁcation or link prediction. The functions f e and f n can b e an y 64 diﬀeren tiable functions and v aries with resp ect to the application. T ypically in a GNN, this is appro ximated with a deep learning structure such as the feed- forw ard neural net work, or graph conv olutional net works, and can b e learned through bac k-propagation during training. Through this connected architec- ture, GNNs are capable of capturing complex relationships betw een en tities in the graph, suc h as the lo cal and global structure of the graph, enabling them to mak e accurate predictions and p erform v arious tasks on graph-structured data. A summary of the literature compiled is presented in T able 8. Article Title Citation Description Application Physics-informed geo- metric deep learning for inference tasks in p ow er systems de Jongh et al. (2022) GNN with physics- informed loss function based on power ﬂow State estimation and anomalous b ehaviour detection p ow er systems PPGN: Physics- Preserved Graph Net- works for Real-Time F ault Location in Dis- tribution Systems with Limited Observation and Lab els Li & Dek a (2021b) Physics of p o wer grid embedded in graph structure to train a GNN F ault detection and localization in p ow er systems Data-Driven T rans- mission Line F ault Location with Single- Ended Measurements and Knowledge-Aware Graph Neural Netw ork Xing et al. (2022) Inherent relationship between observable parameters and fault location embedded in graph structure F ault detection and localization in p ow er systems T able 8: A summary of literature c ompiled for the applications of the physics-informed graph neural net work arc hitecture. The inheren t structure of GNNs, which allows them to op erate on graph- structured data, mak es them suitable for applications with v arious real-world systems, in whic h the b ehavior of the system is determined by complex in terac- tions b etw een v arious components and can be naturally represented as a graph. In particular, GNNs hav e emerged as a p ow erful approach in mo deling p o wer systems, for applications such as pow er system state estimation, load forecast- ing, fault detection and diagnosis, and optimal p ow er ﬂo w estimation (Liao et al., 2021; Y u et al., 2022; Gao et al., 2020; Zhu et al., 2022b). The p opular- it y of graph neural netw orks in modeling p o wer systems may b e attributed to the structure of p ow er systems being inheren tly graph-lik e as well, consisting of in terconnected no des (suc h as p o w er generators, transformers, and loads) and edges (suc h as transmission lines and cables) that represen t the ﬂow of pow er and information. F or example, through a ph ysics-informed GNN, (de Jongh et al., 2022) monitored and p erformed state estimations in their study . Po wer systems exhibit an underlying, irregular structure in the form of grid top ology , whic h can be represen ted mathematically as a graph. Due to this structure, 65 geometric deep learning metho ds such as GNNs are suitable due to their in- heren t structure. The group prop osed a generic framework that uses geometric deep learning techniques and a physics-informed loss function to solv e p ow er ﬂo w calculation and state estimation tasks in p ow er systems. The framework is sho wn to p erform w ell on simulated medium v oltage grid top ologies with v arious sensor penetrations. (Li & Dek a, 2021b) further proposed a physics-preserv ed graph net work for the estimation of the lo cation of faults in a pow er grid system. The t wo-stage framework pro vided an accurate estimation of node fault location with limited data. Through a nov el adjustable adjacency matrix b y which sparse fault currents are aggregated, the ﬁrst stage of the framew ork approximates the top ology of the structure. Whereas the second stage of the framew ork learns the correlation betw een observed, and non-observ able data samples. Finally , Xing et al. (2022) adapted the physics-informed GNN framework to improv e fault lo- cation in transmission lines. The authors incorp orated prior ph ysical knowledge through their establishmen t of a graph structure of known fault t yp es, whereb y the inherent relation betw een fault types and lo cations is incorp orated within measured mo de v oltages and measured mo de curren ts. 3.4.4. Gener ative De ep L e arning Networks Generativ e adv ersarial netw orks (GANs) are a class of ML mo dels designed to automatically discov er and learn regularities from training data, such that the mo del ma y b e able to generate realistic samples of data that plausibly could ha ve belonged to the dataset pro vided. GANs consist of tw o neural netw orks that are trained collectively in a comp etitive setting. A generator netw ork pri- marily learns to generate samples that resemble the training data provided, and a discriminator netw ork learns to distinguish betw een the generated samples and the real training data (W ang et al., 2017). The generator netw ork takes random noise or a latent v ector as input and generates syn thetic data samples. As the pro cess of training progresses, the generator netw ork learns to generate increasingly realistic samples that resemble the training data distribution. The discriminator is represen ted as a binary classiﬁer that seeks to distinguish b e- t ween real and synthetic sample data, with inputs from both real data samples from the training set and synthetic samples from the generator. As the genera- tor net work learns to generate more realistic samples, the discriminator net work b ecomes b etter at distinguishing b etw een the generated and real data and thus pro vides more informativ e feedbac k to the generator net work. The training ob jective of GANs can b e framed as a min-max game b et ween the generator and the discriminator. The generator aims to minimize the discriminator’s abil- it y to distinguish b etw een real and fake samples , while the discriminator aims to maximize its discriminative accuracy . This ob jectiv e is typically expressed as the minimization of the Jensen-Shannon div ergence or the W asserstein dis- tance betw een the real and generated data distributions. This iterative process con tinues un til the generator netw ork is able to pro duce samples that are indis- tinguishable from the real training data (Go o dfello w et al., 2020). A summary of compiled literature is pro vided in T able 9. 66 Article Title Citation Description Application PhyMD AN: Ph ysics- informed knowledge transfer b etw een build- ings for seismic damage diagnosis through ad- versarial learning Xu & Noh (2021) Multiple source domain adaptation framework, with physics-guided loss function based on similarities in domains Structural health moni- toring in buildings A new cyclical gen- erative adversarial netw ork-based data augmentation metho d for multi-axial fatigue life prediction Sun et al. (2022) Dynamic Time W arping equation to eliminate generated samples in- consistent with physical knowledge F atigue life estimation for sp ecimen under multi-axial loading Deep convolutional generative adversarial netw ork with semi- supervised learning enabled physics elucida- tion for extended gear fault diagnosis under data limitations Zhou et al. (2023a) Deep Convolutional Generative Adversarial Netw ork to establish implicit physical corre- lation b etw een known and new faults Machinery health moni- toring in gear transmis- sions Adversarial uncer- taint y quan tiﬁcation in physics-informed neu- ral networks Y ang & Perdik aris (2019) Generative adversarial netw ork for construction of surrogate models to physical systems, regularized via physics- informed loss function Uncertaint y quan tiﬁca- tion and propagation in non-linear systems Physics-informed deep learning: A promising technique for system reliability assessment Zhou et al. (2022) Generator network con- strained by domain knowledge via physics- informed loss function, trained in an adversarial setting with discrimi- nator to produce prob- abilistic estimates of system state and reli- ability for Marko vian systems Reliability assessment and degradation moni- toring T able 9: A summary of literature c ompiled for the applications of the physics-informed gen- erative adversarial net work architectures. Lev eraging the underlying physics of the system, ph ysics-informed GANs ha ve been employ ed to generate synthetic data that may b e used to supplement the a v ailable observ ational or measured data, th us enabling more accurate mo d- eling and prediction. Core to its functionality , ph ysics-informed GANs constrain the generated samples through the application of ph ysical laws. Some instances of this exist in works b y Xu & Noh (2021), who introduced a framew ork that they ha ve termed Physics-Informed Multi-source Domain Adversarial Netw orks for the unsupervised identiﬁcation of structural damage in buildings. The proposed metho d employs a m ultiple-source domain adaptation framew ork that seeks to extract domain-inv ariant features from a v ariety of source domains. The au- thors proposed a no vel loss function whereby additional consideration is giv en 67 to similarities b etw een source and target domains, and the knowledge transfer from similar source domains are prioritized. Similarly , Sun et al. (2022) pre- sen ted a cyclical GAN mo del that embeds the physics of the h ysteretic b ehavior within the netw ork for the augmentation of a v ailable data. More sp eciﬁcally , the authors aimed to capture the relation betw een cyclic h ysteresis loops of the half-life cycle of a specimen under m ulti-axial loading and corresponding fatigue life. The authors enabled the generation of synthetic data that ob eys the dis- tribution characteristics of real fatigue b ehavior, constrained by ph ysical la ws through v arious F ourier transforms and semi-empirical equations. Through the Dynamic Time W arping algorithm and v arious semi-empirical equations repre- sen ting the relation b et w een fatigue life and loading, strain loading, and stress resp onse, samples with deviations from ph ysical principles are eliminated. With the augmentation, multi-axial fatigue life data of a test sp ecimen was employ ed to train sev eral well-kno wn ML mo dels, including feed-forw ard netw orks, Ran- dom F orest, SVMs, and extreme gradien t b o osting algorithms, demonstrating a signiﬁcan t impro vemen t in accuracy . Similarly , for applications in gear fault diagnosis, Zhou et al. (2023a) prop osed a conv olutional GAN mo del to extend a v ailable training data in the gear fault diagnosis pro cess, due to the high-cost limitations of lab eled fault data for speciﬁc gear fault failure modes. Through this framew ork, the authors leveraged fault features from large quan tities of unlab elled training data to b e representativ e of new fault data, with resp ect to lab eled training data, eﬀectiv ely extending the prediction space of the deep con- v olutional GAN. Through this process, the ph ysical correlation betw een kno wn and unseen faults ma y b e deriv ed. V arious authors hav e also emplo yed the framework for uncertaint y quan tiﬁ- cation. This is typically p erformed b y training the net works on data with kno wn uncertain ties, eﬀectively allo wing GANs to eﬀectiv ely generate synthetic data samples with associated uncertain ties. The generated sample may b e used to estimate and quantify the uncertain t y in predictions made b y mac hine learning mo dels. GANs can also b e used to generate diverse data samples that span the en tire range of p ossible uncertain ties, helping to impro ve the robustness and reliabilit y of uncertain ty quantiﬁcation methods. Applications of the GAN arc hitecture may b e seen in works suc h as (Y ang & P erdik aris, 2019), which emplo yed the framew ork for the quan tiﬁcation and propagation of uncertaint y p ertaining to the non-linear PDEs in physics-informed neural netw orks. Due to limitations in data acquisition, the authors sought to produce a metho d of uncertain ty propagation based on a-priori kno wledge b y means of gov erning diﬀeren tial equations. Through latent v ariable models, the probabilistic repre- sen tations of the system states w ere pro cured. In a laten t v ariable mo del, the observ ed v ariables are typically considered to be inﬂuenced b y one or more la- ten t v ariables. These latent v ariables are not directly observed or measured but are assumed to underlie the relationships among the observed v ariables. The ob jective of the laten t v ariable mo del is to estimate the v alues of the latent v ariables and understand their impact on the observed v ariables. An adv ersar- ial inference pro cedure was prop osed for the training of mo dels with resp ect to a v ailable data. The incorporation of physical constrain ts in the form of the 68 ph ysics-informed loss function during optimization phases of the deep adversar- ial generative netw ork allo ws for training utilizing smaller datasets. Appro xi- mation of the solution w as p erformed b y the minimization of error with respect to minimizing the rev erse Kullbac k-Leibler divergence. In this fashion, pre- dictions are constrained to b e consisten t with kno wn physics. Employing the ph ysics-informed constrain ts as regularization mec hanisms, the authors trained a deep generativ e mo del for the generation of surrogates for physical systems, eﬀectiv ely circum ven ting the issue with data acquisition through the c haracter- ization of uncertaint y within the ph ysical system outputs. This metho dology has been v alidated with a series of experiments demonstrating uncertaint y prop- agation in systems gov erned by non-linear PDEs. As another example, Zhou et al. (2022) incorp orated ph ysics-informed GAN in their framework for sys- tem reliability analysis. The netw ork conﬁguration is mo deled based on system state probabilit y and enco des the gov erning equations of the reliabilit y ev olution mo del. The authors characterized the system p erformance at each time state problematically via deriv ations from the forward Kolmogoro v equations, and subsequen tly , the system reliabilit y as an aggregate of state probability where the system is considered to b e functional. The authors further prop osed a GAN net work for uncertain ty quantiﬁcation with respect to reliability assessmen t, whereb y the generator seeks to pro duce synthetic data based on the deriv ative of system state probability , or state transition deﬁned. The generator model is also constrained b y an y observ ed data from initial conditions or the con tinued op eration of the system, whereas the discriminator seeks to pro duce conﬁdence estimates of the data. The t wo are regularized b y competing loss functions and trained in an adv ersarial setting. In particular, ph ysics-based regularization is emplo yed for the generator as an additional loss term, in accordance with do- main knowledge. The authors demonstrated the eﬀectiveness of the prop osed metho dology with a v ariety of numerical examples. In all, the prop osed metho d yielded similar results to that of conv entional Runge-Kutta and Mon te Carlo sim ulations. 4. Discussion In total, a sample size of 105 literary works w as explored in the survey , with the ov erall ob jective of discussing and summarizing popular implemen tations of PIML learning framew orks, with applications to the monitoring of assets for anomalous behavior and or op erating conditions. Of the w orks of literature surv eyed, the methods of integration b etw een ph ysics-based methods and data- driv en mo dels w ere sub divided into four distinct categories, as discussed in section 3. 4.1. Summary of Compile d Liter atur e and Interpr etations An illustration of the distribution of literature reviewed may b e seen in ﬁgure 22. The pie c hart illustrates the distribution of publications reviewed, highligh t- ing the diﬀeren t areas within the ﬁeld of PIML for condition monitoring. 69 Physics Embedded in F eature Space 36.19% Physics-Informed Regularization 26.67% Data-Enhanced Refinement of Physical Models 2.86% Physics-Guided Design of Architectures 34.29% Generative Deep L ear ning Models 4.76% Graph Neural Networks 2.86% R ecur r ent Neural Networks 16.19% Convolutional Neural Network 8.57% F eed-F orwar d Neural Networks 3.81% Physics-Infor med R egularization with Deep L ear ning Ar chitectur es 17.14% Data-Driven Solutions to Differ ential Equations 9.52% Physics - Guided Cor r ection Mechanism 2.86% T ransfer L ear ning with Physically Consistent Domain 9.52% Physics- Guided F eatur e Augmentations 24.76% Distribution of Publications Reviewed P er centages of publications r eviewed within each section, on the topic of physics-infor med machine lear ning for condition monitoring. Figure 22: Distribution of Publications Reviewed on Physics-Informed Mac hine Learning for Condition Monitoring Of the literature survey ed, a large sample of w orks (36) iden tiﬁed emplo yed ph ysics-based techniques to mo dify the input feature space of the ML model, in tro ducing ph ysical knowledge through observ ational biases. Alterations to the input space indirectly allow mo dels to learn ph ysically consisten t relationships through restricting mappings that are not adheren t to physical principles. This implemen tation has witnessed great popularity whic h may b e attributed to its simplicit y and ease of implementation. F rom literature, this type of integration dealt primarily with direct ph ysical- mo del-driv en generation of input data or augmentation of feature space: with (21) studies review ed aiming to generate syn thetic data or used ph ysics-based metho ds to create new physics-based features, and (5) studies using ph ysics- 70 based metho ds to select discriminating features. Despite the v aried approaches, a commonalit y with the metho ds mentioned abov e is with resp ect to the cus- tom tailoring of feature space for use with con ven tional ML and deep learning mo dels. Incorp oration of physics within the feature space of machine learning has also been p erformed in sev eral works (10), utilizing the philosophy of trans- fer learning, whereb y the model is pre-trained on a known source domain, and subsequen tly re-calibrated for a target domain. Almost all w orks in this area ha ve designated the source domain as the kno wn ph ysical domain, and emplo yed kno wledge transfer to capture known physics to b e re-purp osed; Exceptions to this trend were with the w ork of Guc & Chen (2021) and Guc & Chen (2022) who instead relied on the pre-trained features of the source domain, and incor- p orated ph ysics through the ﬁne-tuning phase. V arious authors ha ve employ ed this framework for the supplemen tation of av ailable data and enhancemen t of ML learning space for impro ved p erformance and robustness. With resp ect to the limitations of this tec hnique, despite the ease of imple- men tation and apparent eﬃcacy , this t yp e of implementation do es not directly incorp orate any ph ysical constraints during the learning process, resulting in a naiv e blac k-b ox mo del, with minimal in terpretability . While feature engineering ma y indirectly restrict the mo del’s capabilities for ph ysical violations, no con- strain ts are enforced during the learning pro cess. F urthermore, there exists a degree of dependency on the completeness and reliability of the ph ysical model for a true-to-life or authen tic generation of features. As suc h, the outlined ap- proac h ma y not be suitable for complex systems where the underlying physics is not well understo od, due to the diﬃculties in capturing the intricacies and n uances of real-world phenomena in a set of predeﬁned features. Another form ula for the incorp oration of ph ysical knowledge within ML mo dels is the applications of data-driven modules in tandem with physics-based mo dels, suc h that the data-driv en mo del acts as a correctional mechanism to complemen t the decisions made based solely on ph ysical principles. (3) of the w orks sampled hav e emplo yed this format for their applications. While the tec h- nique has demonstrated some success as demonstrated in the abov e literature, the action of utilizing ML as a correction mechanism for physical models is not without limitations. As with most purely data-driv en models, a ma jor limitation in this strat- egy is its inabilit y to capture b eha viors not present in the domain on which they are trained. In this format, ML models op erate independently from the ph ysics-based mo dels, and as a direct result, in the case that the training data do es not accurately capture the true physics of the system as is c haracteristic of the error of the ph ysical system, the ML algorithm may learn to correct the errors in the physical mo del, but may not be able to accurately capture the un- derlying ph ysica l phenomena. With resp ect to the integration of physics with data-driv en mo dels, another ma jor limitation results from the target learning space of the ML mo del. Due to the ML mo del learning the error of the system, 71 rather than the system itself, it is diﬃcult to ensure that the resulting correc- tions are physically meaningful. In some cases, machine learning algorithms ma y iden tify patterns or relationships in data pro vided that are not related to the underlying ph ysics, leading to incorrect or spurious corrections. More recently , through several deﬁning contributions, physical knowledge of this system has been emplo yed in conjunction with the pow erful appro xima- tiv e capabilities of neural netw orks. T raditional metho ds in training a neural net work in volv e an initial prediction made by the neural net work, and its sub- sequen t optimization in accordance with some form of distance ev aluation of predictions of the neural net work and prior kno wledge, in the form of a loss function. Optimization in sup ervised learning metho ds has b een carried out with respect to lab eled data; the established methodology has remained un- c hanged since its inception. Recen tly , several authors hav e made innov ations to this process through the in tro duction of physics-informed regularizations. Con ven tional regularization suc h as L1 or L2 regularization has b een used ex- tensiv ely with ML mo dels as a methodology for machine learning and statistical mo deling to address o verﬁtting and improv e the generalization capabilit y of the mo del. This eﬀectively balances the trade-oﬀ betw een ﬁtting the training data w ell and limiting the model complexity with additional p enalt y terms in the loss function. With ph ysics-informed regularization, rather than limiting mo del complex- it y , models are p enalized based on their deviations from physical principles through the introduction of a physics-based loss term. (38) sampled w orks applied this format of regularization for their prop osed metho dologies. As char- acterized by Karniadakis et al. (2021), physics-based regularizations ha ve been kno wn to introduce knowledge of the underlying physical system through learn- ing biases. Predictions from deep learning arc hitectures a b e iteratively guided via the loss functions o ver sev eral optimization cycles to b e consistent with kno wn ph ysics. In addition, (32) of the studies emplo ying this metho dology emplo y physics-based regularizations for the solution of go verning ordinary or partial diﬀerential equations. Through indep endent v ariable inputs, the neural net work seeks to predict the unknown v ariable. Lev eraging automatic diﬀer- en tiation, the predictions of v ariables from the base neural netw ork ma y b e emplo yed to reconstruct the diﬀeren tial equations, as well as initial or bound- ary conditions. These reconstructions are subsequently ev aluated in the form of the loss function, with some studies electing to include loss with resp ect to lab eled data as well. Initial works in this area by the likes of Raissi et al. (2019) made use of con ven tional feed-forward net w orks, although the general framew ork of ph ysics-based regularization has been quic kly expanded to lev er- age other deep learning arc hitectures as w ell as demonstrated in a sample of (6) works. Architectures suc h as con volutional and recurrent neural netw orks ha ve been employ ed for their capabilities in capturing spatial and temp orally in v ariant features resp ectiv ely , and autoenco ders for their unsup ervised learning capabilities. Several adv antages of this approach are apparen t, as demonstrated 72 b y the ab o ve works. The p opularization of this format represents an eﬀective metho dology for the incorp oration of prior kno wledge of physical principles within the optimiza- tion pro cess of neural netw orks, and its sup eriority o v er conv en tional ”naive” metho ds has been demonstrated in v arious works (Raissi et al., 2019; Haghighat et al., 2021). Mo dels constructed are also reliant to a lesser degree, on the av ail- able data for learning, enabling authors to reduce data requiremen ts to train a deep learning arc hitecture and impro ve the robustness of mo dels to noisy or incomplete data. In fact, some studies (8) hav e employ ed purely physical loss terms in the optimization pro cess. T raining a mo del in this format may prov e adv an tageous when limited data is a v ailable. In addition, it reduces sensitivit y to noisy or inaccurate data due to the absence of dep endence. In general, a data-driv en loss term ma y also increase con vergence and stabilit y , as well as generalization to unseen data via additional guidance during training. While the implemen tation of physics-informed loss functions has b een prov en to b e eﬀective in solving issues p ertaining to condition monitoring as has b een sho wn by man y of the studies listed ab ov e, there exists some limitations to this approac h. One suc h limitation is with respect to the metho d b y which the ph ysical constraints are imposed in the netw ork. Through the physics-informed loss functions, physics-based loss terms act as a penalization for the net work in the case of violations, how ever, they are not enforced as hard constrain ts. This ma y prov e an issue in hybrid loss functions inv olving penalization terms with resp ect to labeled data in particular, as inaccuracies in the data ma y cause the corresp onding loss term to dominate within the h ybrid loss function. T o a lesser exten t, with resp ect to ph ysics-based regularization and PINNs in general, as the physical loss is not strictly enforced, ph ysical violations or deviations from exp ected physical b ehaviors may still b e pro duced by the net work. Another limitation of note is that the addition of a ph ysical regularizer, dep ending on the problem b eing solved, ma y in tro duce additional degrees of complexity to the loss function ov erall. Current metho ds of optimization rely primarily on gra- dien t descent and its v ariants, in which the net work adjusts its parameters in steps to ward the direction of minimal error with r esp ect to loss. The increased complexit y of the loss function landscap e ma y further complicate or hinder the pro cess of optimizations through the introduction of lo cal minima, for example. This aspect of physics-based regularization has been noted in the w ork of Krish- napriy an et al. (2021), whereby the characteristic increase in mo del complexit y has b een noted with the in tro duction of soft regularization terms. Alternativ ely , authors ha v e also attempted to incorporate hard constrain ts through the design of neural netw ork architecture. A total of (34) of the works sampled provided inno v ative solutions to incorp orate physical principles as part of the computational processes of deep learning arc hitectures itself. Ov erall, this format oﬀers improv ed interpretabilit y , as the computation pro cess is de- signed within the framework of deep learning netw orks. The learned parame- 73 ters and mo del outputs can be directly related to physical quan tities , making it easier to understand and v alidate the predictions, allowing practitioners a deep er understanding of the pro cess by which algorithms predict and ascer- tain a predicted result. Inno v ations ha ve b een made with resp ect to several p opular net work arc hitectures such as the conv entional feed-forw ard neural net- w ork (4), the CNN (9), RNN (17), GNN (3), and GANs (5). With respect to arc hitecture design, the ma jorit y of studies examined either assigned ph ysical meaning with resp ect to intermediary no des or lay ers or alternativ ely , to the connections b etw een nodes themselv es in the form of constrained optimization (Chen & Liu, 2021). Of whic h, in addition to employing physics-informed la y- ers, man y such studies also employ a physics-informed regularisation as well, for additional guidance during the optimization pro cess. Some studies, suc h as the work of Chen et al. (2023), alternatively emplo yed informed activ ation functions for eac h no de within the net work. With respect to the feed-forw ard net works, several authors ha ve prop osed interpretable lay ers within the netw orks to elucidate the computational pro cesses of data-driven models, with physical meaning being assigned to la yers. This methodology represen ts an alternate form of physics-based feature extraction. With regards to applications with vibrational data from structural and machinery health monitoring sp eciﬁcally , con ven tional signal pro cessing tec hniques suc h as F ourier transforms, env elop e analysis, and w av elet transforms are embedded within neural net work la yers as a form of ph ysics-informed feature extraction and pro cessing. A similar tec h- nique is emplo y ed with CNNs, whereb y la yers within CNNs p erform adv anced feature selection or extraction with regard to a deﬁned computational process that is adheren t to known physics. While the framew ork outlined ab o ve shares man y similarities to simply tai- loring the input feature space, as discussed in section 3.1, there exist several k ey adv an tages of incorp orating the pre-processing stage within the netw ork itself. F or one, the framew ork outlined is eﬀectiv ely an end-to-end learning pip eline, whereb y the en tire netw ork, including the pre-pro cessing stage, is incorporated within the learning pro cess. The adv antages of this design lie in the fact that the netw ork can adapt and optimize b oth the pre-processing and subsequen t feature extractions sim ultaneously and eliminates the need for manual feature engineering. In addition, the resultan t net work architecture em b eds physical kno wledge, and is therefore more in terpretable, due to the netw ork’s b eha vior b eing enforced to align with kno wn ph ysical principles. By explicitly mo deling and accoun ting for factors that may be ph ysically mo deled during the feature extraction pro cess, the netw ork can learn to extract more reliable and in v ariant features, resulting in impro ved p erformance under challenging conditions. The design of net work lay out has also been explored, as tabulated in the w ork of Ni et al. (2022), whereby the branched netw ork w as introduced to solve for m ultiple pre-determinate physics-based relations indep enden t of each other. As noted in b oth Ni et al. (2022) and Haghighat et al. (2021), while tec hnically p ossible to solve for multiple physical v ariables with a wide enough net w ork 74 la yer, in the case where the relations may b e mo deled indep endent of the other, it is often more eﬃcient in terms of computational resources, and more accurate to mo del each v ariable individually through separate branc h netw orks. Several studies also focus on the RNN structure, with the primary form of physical infor- mation b eing embedded in computational pro cedures within the RNN cell. Key con tributors within this area include the w orks of Nascimento & Viana (2019), who initially made use of the Euler Integration cell to embed the ph ysics of crac k propagation within the RNN cell, as a represen tation of cumulativ e dam- age modeling. This model is later extended to v arious other applications in mo deling the propagation of damage through time, as w ell as mo del form un- certain ty (Viana et al., 2021; Y ucesan & Viana, 2020b, 2021a). Of the works co vered, (14) made use of this format of in tegration. Other works such as the study b y Y u et al. (2020b) also made alterations through the incorporation of the Deep Residual Recurren t Neural Netw ork, as initially prop osed b y Kani & Elsheikh (2017). Utilizing the em b edded physical dynamics of the system, practitioners were able to better c apture dep endencies and impro ve the mo del’s abilit y to make accurate predictions ov er longer time horizons. (3) studies em- plo yed the graph neural net work, in whic h the inherent structure of the net w ork is lev eraged to b etter mo del and pro cess graph-structured data, with extensive applications in p o wer systems. In contrast to con ven tional neural netw orks, GNNs are capable of handling non-Euclidean data via graph representations, whereb y no des in the graph structure represent en tities and connections repre- sen t the relationships betw een them. Unique to their structure, GNNs do not assume spatial locality . This assumption is commonly used in CNNs, which are designed to op erate on grid-like data, such as images. This prop erty of GNNs allo ws for op eration on data structures of arbitrary sizes and complex topolo- gies. (5) samples of literature reviewed dealt mainly with the optimization of GANs, of which, (2) studies implemen t the net work as an automatic framework for the syn thetic generation of physically plausible data, while the remainder (3) emplo yed the netw ork to characterize and quantify the uncertaint y in pre- dictions made b y machine learning mo dels. In all, through the embedding of physical models within netw ork architec- ture, ph ysical principles are enforced, leading to theory-adherent communication through the arc hitecture itself. Ho wev er, as with all learning algorithms, there exists a trade-oﬀ b etw een the detail in which the mo del is designed to inter- pret, and computational demands of the mo del. In addition to the domain kno wledge required, in tegrating ph ysical principles within deep learning mo dels increases their complexity . Dep ending on the implementation, physics-informed arc hitectures ma y require more computational resources than conv entional deep learning models, whic h could be a limiting factor in some applications in which computational speed is a requiremen t. Viana et al. (2021) has also noted this limitation in their study , wherein the complexity of the ph ysical mo dels em b ed- ded ma y prov e unwieldy . By extension, an a v enue of potential further research ma y be the adaptation of said complex mo dels through guided simpliﬁcations or reduced-order mo dels. The introduction of inductive biases through this format 75 ma y also restrict the learning model, as it imp oses strong assumptions on the data and learning pro cess. While rigid constrain ts imp osed by biases may b e able to enhance eﬃciency through explicit guidance to the mo del, they ma y also serv e to limit the mo del’s ﬂexibility to capture the underlying complexity of the data and the ability to generalize. Th us, the suitability design of the architec- ture with resp ect to its application must be carefully ev aluated and tailored to ensure the eﬃcacy of the algorithm. 4.2. Outlo ok Despite some limitations outlined abov e, the combination of ph ysics-guided arc hitecture design, in tandem with physics-informed regularisation tec hniques for optimization remains some of the cutting-edge and most sophisticated metho d- ologies for in tegration of physical knowledge with data-driven techniques. Through a com bination of hard and soft constrain ts, researchers hav e b een able to tailor curren t mac hine learning algorithms to suit the need of sev eral real-world condi- tion monitoring applications. Current studies hav e already demonstrated great promise with regard to ev aluation metrics such as accuracy , reliance on data, and robustness to noise and or incomplete data. With ongoing adv ancements in computational p ow er, researc hers can tac kle more complex and realistic ph ysi- cal problems. The increased com putational resources enable the exploration of larger and more comprehensive datasets, facilitating the discov ery of intricate relationships and patterns that migh t hav e otherwise remained hidden. Addi- tionally , higher computational capabilities allo w for more sophisticated mo del- ing techniques, enabling the consideration of complex physical phenomena and nonlinear dynamics that w ere previously challenging to capture accurately . 4.3. Limitations A limitation of the ﬁndings in this survey w as with regard to the sample siz e of literary w orks examined. Although the paradigm of PIML has b een rapidly expanding since its inception, instances of literature implementing the PIML for applications within condition monitoring systems remain relativ ely lo w in com- parison to other areas of developmen t. T rends and literature outlined b y this surv ey ma y b e sk ewed tow ards authors or metho dologies, and may not accu- rately capture the underlying trend of the technology , with resp ect to condition monitoring applications. 5. Concluding Remarks Ph ysics-informed machine learning (PIML) methods oﬀer a promising a v- en ue for improving predictiv e mo deling in physical systems, whereb y the un- derlying ph ysics-based constrain ts can b e lev eraged to further enhance conv en- tional data-driv en metho ds. By in tegrating the go verning la ws of ph ysics in to the learning algorithm, PIML is capable of eﬀectively determining a non-naiv e, ph ysically consisten t represen tation of the system, thereby enabling accurate predictions and extrap olations b eyond the training data. F urthermore, PIML 76 metho ds facilitate data-eﬃcient learning by guiding the learning algorithm to prioritize regions of in terest and reduce the need for large training datasets. The incorp oration of physics also enhances generalization capability , as the mo dels can naturally handle extrap olation and capture the behavior under diﬀeren t conditions or p erturbations. This w ork serves to provide an ov erview of such metho ds, with a focus on the metho dology by which ph ysical knowledge is in tegrated in to con ven tional machine naiv e learning frameworks to form ulate predictiv e models with a higher level of understanding and sophistication in re- lation to the underlying ph ysical principles of the system. A total of 105 literary w orks ha ve b een sampled, with applications of PIML for condition monitoring in v arious ﬁelds of engineering. In the con text of condition monitoring and fault detection, PIML metho ds leverage underlying kno wn ph ysical principles and domain knowledge to dev elop mo dels capable of accurately predicting sys- tem b eha vior, detecting anomalies, and assessing the health status of critical comp onen ts. Through this incorp oration, models are able to more eﬀectively capture the complex in teractions betw een v arious system v ariables, enabling the identiﬁcation of incipien t faults and abnormalities with high sensitivity and sp eciﬁcit y . A detailed exploration of curren t methodologies for the in tegration of kno wn physics with machine learning methods is provided in this context, whic h is classiﬁed in to primary categories with respect to the metho dology b y whic h ph ysical knowledge of the system is integrated. F urthermore, this sur- v ey provides a generalized ov erview of some of the most popular deep learning algorithms emplo yed, with brief explanations regarding their workings, their inheren t adv antages as well as limitations. Leveraging the initial understand- ing provided, the w ork seeks to detail recent innov ations in the incorp oration of physical kno wledge by v arious authors in their resp ectiv e studies. In all, sev eral av enues of researc h w ere iden tiﬁed, including physics-guided augmen- tation or feature space, data-driven correctional mechanism, physics-informed regularization, and ﬁnally , ph ysic-guided design of deep learning architectures. An interpretation of the v arious strengths, weaknesses, and limitations of eac h a ven ue of research is provided, and recommendations are made regarding nur- turing areas of research with respe ct to the integration of the PIML paradigm with applications to condition monitoring of assets. CRediT authorship con tribution statemen t Y. W u: Conceptualization, Metho dology , In vestigation, W riting - Original Draft, Visualization B. Sicard: W riting - Review and Editing S. A. Gadsden: Metho dology , Sup ervision, F unding Acquisition, W riting - Review and Editing 77 Declaration of Comp eting In terest The authors declare that they ha ve no known comp eting ﬁnancial interests or p ersonal relationships that could ha ve appeared to inﬂuence the w ork rep orted in this pap er. Ac knowledgemen ts The authors w ould like to giv e their sincerest thanks and appreciation to the p ostdoctoral fello ws, researc h asso ciates, graduate students, and undergraduate researc hers in the Intelligen t and Cognitiv e Engineering (ICE) Lab oratory at McMaster Univ ersity for their contin ued supp ort and encouragement. References Alotibi, F., & Tipp er, D. (2022). Ph ysics-informed cyb er-attack detection in wind farms. In GLOBECOM 2022-2022 IEEE Glob al Communic ations Confer enc e (pp. 1–6). IEEE. An ton, D., & W essels, H. (2021). Iden tiﬁcation of material parameters from full-ﬁeld displacement data using ph ysics-informed neural net works. R e- se ar chgate Pr eprint , . Bansal, P ., Zheng, Z., Shao, C., Li, J., Ban u, M., Carlson, B. E., & Li, Y. (2022). Ph ysics-informed mac hine learning assisted uncertain ty quantiﬁcation for the corrosion of dissimilar material join ts. R eliability Engine ering & System Safety , 227 , 108711. Ba ydin, A. G., Pearlm utter, B. A., Radul, A. A., & Siskind, J. M. (2018). Au- tomatic diﬀeren tiation in mac hine learning: a surv ey . Journal of Mar chine L e arning R ese ar ch , 18 , 1–43. Bharadw a ja, B., Nabian, M. A., Sharma, B., Choudhry , S., & Alank ar, A. (2022). Physics-informed machine learning and uncertain ty quantiﬁcation for mechanics of heterogeneous materials. Inte gr ating Materials and Man- ufacturing Innovation , (pp. 1–21). Boushaba, A., Cauet, S., Chamro o, A., Etien, E., & Rambault, L. (2022). Comparativ e study betw een ph ysics-informed cnn and pca in induction motor brok en bars mcsa detection. Sensors , 22 , 9494. Chao, M. A., Kulk arni, C., Goeb el, K., & Fink, O. (2019). Hybrid deep fault detection and isolation: Com bining deep neural netw orks and system p er- formance mo dels. arXiv pr eprint arXiv:1908.01529 , . Chao, M. A., Kulk arni, C., Go eb el, K., & Fink, O. (2022). F using physics-based and deep learning models for prognostics. R eliability Engine ering & System Safety , 217 , 107961. 78 Chen, D., Li, Y., Liu, K., & Li, Y. (2023). A physics-informed neural net- w ork approac h to fatigue life prediction using small quantit y of samples. International Journal of F atigue , 166 , 107270. Chen, J., & Liu, Y. (2021). Probabilistic physics-guided mac hine learning for fatigue data analysis. Exp ert Systems with Applic ations , 168 , 114316. Chen, X., Ma, M., Zhao, Z., Zhai, Z., & Mao, Z. (2022a). Physics-informed deep neural net work for b earing prognosis with multisensory signals. Journal of Dynamics, Monitoring and Diagnostics , (pp. 200–207). Chen, Y., Rao, M., F eng, K., & Zuo, M. J. (2022b). Physics-informed lstm h yp erparameters selection for gearb ox fault detection. Me chanic al Systems and Signal Pr o c essing , 171 , 108907. Ch ung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical ev aluation of gated recurrent neural net works on sequence modeling. arXiv pr eprint arXiv:1412.3555 , . Darr, S. R., Engelstad, S., Knighton, T., T aliaferro, M. E., & Go yal, V. K. (2023). A com bined mac hine learning and ph ysics-based tool for anomaly iden tiﬁcation in propulsion systems. In AIAA SCITECH 2023 F orum (p. 2375). De Santos, F., D’Antuono, P ., Robbelein, K., Noppe, N., W eijtjens, W., & Devriendt, C. (2023). Long-term fatigue estimation on oﬀshore wind tur- bines in terface loads through loss function ph ysics-guided learning of neural net works. R enewable Ener gy , 205 , 461–474. De Silv a, B. M., Callaham, J., Jonk er, J., Goeb el, N., Klemisch, J., McDonald, D., Hic ks, N., Kutz, J. N., Brun ton, S. L., & Aravkin, A. Y. (2020). Physics- informed mac hine learning for sensor fault detection with ﬂight test data. arXiv pr eprint arXiv:2006.13380 , . Deebak, B., & Al-T urjman, F. (2022). Digital-twin assisted: F ault diagnosis using deep transfer learning for mac hining to ol condition. International Journal of Intel ligent Systems , 37 , 10289–10316. Deng, W., Nguyen, K. T., Gogu, C., Morio, J., & Medjaher, K. (2022). Ph ysics- informed ligh t weigh t temp oral con volution netw orks for fault prognostics asso ciated to b earing stiﬀness degradation. In PHM So ciety Eur op e an Con- fer enc e (pp. 118–125). v olume 7. Dourado, A., & Viana, F. A. (2019). Physics-informed neural net works for corrosion-fatigue prognosis. In Annual Confer enc e of the PHM So ciety . v olume 11. Dourado, A. D., & Viana, F. (2020). Physics-informed neural net works for bias comp ensation in corrosion-fatigue. In Aiaa scite ch 2020 forum (p. 1149). 79 F eng, K., Ji, J., Zhang, Y., Ni, Q., Liu, Z., & Beer, M. (2023). Digital twin- driv en intelligen t assessmen t of gear surface degradation. Me chanic al Sys- tems and Signal Pr o c essing , 186 , 109896. F rank, S., Heaney , M., Jin, X., Rob ertson, J., Cheung, H., Elmore, R., & Henze, G. (2016). Hybrid mo del-b ase d and data-driven fault dete ction and diagnos- tics for c ommer cial buildings . T echnical Rep ort National Renewable Energy Lab.(NREL), Golden, CO (United States). F reeman, B., T ang, Y., Huang, Y., & V anZwieten, J. (2022). Ph ysics-informed turbulence intensit y infusion: A new h ybrid approach for marine current turbine rotor blade fault detection. Oc e an Engine ering , 254 , 111299. Gao, X., Huang, A., T rask, N., & Reza, S. (2020). Physics-informed graph neural netw ork for circuit compact mo del developmen t. In 2020 Interna- tional Confer enc e on Simulation of Semic onductor Pr o c esses and Devic es (SISP AD) (pp. 359–362). IEEE. Gardner, P ., Bull, L., Dervilis, N., & W orden, K. (2021). Overcoming the problem of repair in structural health monitoring: Metric-informed transfer learning. Journal of Sound and Vibr ation , 510 , 116245. Gitzel, R., Kotriw ala, A., F echner, T., & Schiefer, M. (2021). Using syn thetic data to train a visual condition monitoring system for leak detection. In 2021 IEEE Seventh International Confer enc e on Big Data Computing Ser- vic e and Applic ations (BigDataServic e) (pp. 196–200). IEEE. Gong, M., T ang, D., Y u, J., & Tian, L. (2021). A ph ysics-informed trans- fer learning approac h for anomaly detection of aerospace cmg with limited telemetry data. In 2021 Glob al R eliability and Pr o gnostics and He alth Man- agement (PHM-Nanjing) (pp. 1–5). IEEE. Go odfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., W arde-F arley , D., Ozair, S., Courville, A., & Bengio, Y. (2020). Generative adversarial netw orks. Communic ations of the A CM , 63 , 139–144. Green, D. H., Langham, A. W., Agustin, R. A., Quinn, D. W., & Leeb, S. B. (2022). Physics-informed feature space ev aluation for diagnostic p ow er monitoring. IEEE T r ansactions on Industrial Informatics , . Greydan us, S., Dzam ba, M., & Y osinski, J. (2019). Hamiltonian neural net- w orks. A dvanc es in neur al information pr o c essing systems , 32 . Guc, F., & Chen, Y. (2021). F ault cause assignment with physics informed transfer learning. IF A C-Pap ersOnLine , 54 , 53–58. Guc, F., & Chen, Y. (2022). Sensor fault diagnostics using physics-informed transfer learning framew ork. Sensors , 22 , 2913. 80 Haghighat, E., Raissi, M., Moure, A., Gomez, H., & Juanes, R. (2021). A ph ysics-informed deep learning framew ork for in version and surrogate mod- eling in solid mechanics. Computer Metho ds in Applie d Me chanics and Engine ering , 379 , 113741. Hao, W., T an, L., Y ang, X., Shi, D., W ang, M., Miao, G., & F an, Y. (2023). A physics-informed mac hine learning approac h for notch fatigue ev aluation of allo ys used in aerospace. International Journal of F atigue , 170 , 107536. Ho c hreiter, S., & Schmidh ub er, J. (1997). Long short-term memory . Neur al c omputation , 9 , 1735–1780. Hop woo d, M. W., Stein, J. S., Braid, J. L., & Seigneur, H. P . (2022). Ph ysics- based method for generating fully syn thetic iv curv e training datasets for mac hine learning classiﬁcation of pv failures. Ener gies , 15 , 5085. Hornik, K., Stinchcom b e, M., & White, H. (1989). Multila yer feedforw ard net- w orks are universal approximators. Neur al networks , 2 , 359–366. Huang, G.-B., Zh u, Q.-Y., & Siew, C.-K. (2006). Extreme learning machine: theory and applications. Neur o c omputing , 70 , 489–501. Huang, Z., Yin, X., & Liu, Y. (2022). Ph ysics-guided deep neural net work for structural damage iden tiﬁcation. Oc e an Engine ering , 260 , 112073. Jadha v, V., Deodhar, A., Gupta, A., & Runk ana, V. (2022). Ph ysics informed neural netw ork for health monitoring of an air preheater. In PHM So ciety Eur op e an Confer enc e (pp. 219–230). volume 7. Jakub o wski, J., Stanisz, P ., Bob ek, S., & Nalepa, G. J. (2022). Roll wear predic- tion in strip cold rolling with physics-informed auto enco der and coun ter- factual explanations. In 2022 IEEE 9th International Confer enc e on Data Scienc e and A dvanc e d A nalytics (DSAA) (pp. 1–10). IEEE. Jia, X., Willard, J., Karpatne, A., Read, J., Zwart, J., Steinbac h, M., & Kumar, V. (2019). Physics guided rnns for mo deling dynamical systems: A case study in simulating lak e temp erature proﬁles. In Pr o c e e dings of the 2019 SIAM International Confer enc e on Data Mining (pp. 558–566). SIAM. Jing, L., Zhao, M., Li, P ., & Xu, X. (2017). A con volutional neural netw ork based feature learning and fault diagnosis method for the condition moni- toring of gearb o x. Me asur ement , 111 , 1–10. de Jongh, S., Gielnik, F., Mueller, F., Schmit, L., Suriyah, M., & Leibfried, T. (2022). Physics-informed geometric deep learning for inference tasks in p o wer systems. Ele ctric Power Systems R ese ar ch , 211 , 108362. Kanek o, T., W ada, R., Ozaki, M., & Inoue, T. (2022). Hybrid mo del of a ph ysics-based model and mac hine learning for real-time estimation of un- measurable parts: Mapping from measurable to unmeasurable v ariables. Oc e an Engine ering , 261 , 112123. 81 Kani, J. N., & Elsheikh, A. H. (2017). Dr-rnn: A deep residual recurrent neural net work for mo del reduction. arXiv pr eprint arXiv:1709.00939 , . Karandik ar, J., Schmitz, T., & Smith, S. (2021). Physics-guided logistic clas- siﬁcation for to ol life mo deling and pro cess parameter optimization in ma- c hining. Journal of Manufacturing Systems , 59 , 522–534. Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdik aris, P ., W ang, S., & Y ang, L. (2021). Physics-informed machine learning. Natur e R eviews Physics , 3 , 422–440. Karpatne, A., Atluri, G., F aghmous, J. H., Stein bach, M., Banerjee, A., Gan- guly , A., Shekhar, S., Samato v a, N., & Kumar, V. (2017). Theory-guided data science: A new paradigm for scien tiﬁc disco very from data. IEEE T r ansactions on know le dge and data engine ering , 29 , 2318–2331. Kharazmi, E., F an, D., W ang, Z., & T riantafyllou, M. S. (2021). Inferring v ortex induced vibrations of ﬂexible cylinders using ph ysics-informed neural net works. Journal of Fluids and Structur es , 107 , 103367. Kim, S., Choi, J.-H., & Kim, N. H. (2022a). Data-driven prognostics with lo w-ﬁdelity ph ysical information for digital t win: physics-informed neural net work. Structur al and Multidisciplinary Optimization , 65 , 255. Kim, S. W., Oh, K.-Y., & Lee, S. (2022b). No vel informed deep learning- based prognostics framework for on-b oard health monitoring of lithium-ion batteries. Applie d Ener gy , 315 , 119011. Kim, Y., Na, K., & Y oun, B. D. (2022c). A health-adaptiv e time-scale repre- sen tation (h tsr) embedded conv olutional neural net work for gearbox fault diagnostics. Me chanic al Systems and Signal Pr o c essing , 167 , 108575. Koh tz, S., Xu, Y., Zheng, Z., & W ang, P . (2022). Ph ysics-informed machine learning model for battery state of health prognostics using partial charging segmen ts. Me chanic al Systems and Signal Pr o c essing , 172 , 109002. Kolen, J., & Kremer, S. (). Gradien t ﬂo w in recurren t nets: the diﬃculty of learning longterm dep endencies. a ﬁeld guide to dynamical recurrent net works: Ieee; 2001. Krishnapriy an, A., Gholami, A., Zhe, S., Kirb y , R., & Mahoney , M. W. (2021). Characterizing p ossible failure mo des in physics-informed neural net works. A dvanc es in Neur al Information Pr o c essing Systems , 34 , 26548–26560. Lagaris, I. E., Lik as, A., & F otiadis, D. I. (1998). Artiﬁcial neural netw orks for solving ordinary and partial diﬀeren tial equations. IEEE tr ansactions on neur al networks , 9 , 987–1000. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. natur e , 521 , 436–444. 82 LeCun, Y., Bottou, L., Bengio, Y., & Haﬀner, P . (1998). Gradien t-based learn- ing applied to document recognition. Pr o c e e dings of the IEEE , 86 , 2278– 2324. Leturiondo, U., Salgado, O., & Galar, D. (2017). V alidation of a physics-based mo del of a rotating machine for syn thetic data generation in h ybrid diag- nosis. Structur al He alth Monitoring , 16 , 458–470. Li, P ., Jia, X., F eng, J., Zhu, F., Miller, M., Chen, L.-Y., & Lee, J. (2020). A nov el scalable method for mac hine degradation assessmen t using deep con volutional neural netw ork. Me asur ement , 151 , 107106. Li, T., Zhao, Z., Sun, C., Cheng, L., Chen, X., Y an, R., & Gao, R. X. (2021a). W a veletk ernelnet: An interpretable deep neural net work for industrial in- telligen t diagnosis. IEEE T r ansactions on Systems, Man, and Cyb ernetics: Systems , 52 , 2302–2312. Li, W., & Dek a, D. (2021a). Ph ysics-informed learning for high imp edance faults detection. In 2021 IEEE Madrid PowerT e ch (pp. 1–6). IEEE. Li, W., & Dek a, D. (2021b). Ppgn: Physics-preserv ed graph net works for real- time fault location in distribution systems with limited observ ation and lab els. arXiv pr eprint arXiv:2107.02275 , . Li, W., Dek a, D., Chertko v, M., & W ang, M. (2019a). Real-time faulted line lo calization and pm u placemen t in p ow er systems through con volutional neural net works. IEEE T r ansactions on Power Systems , 34 , 4640–4651. Li, W., Zhang, J., Ringb eck, F., J¨ ost, D., Zhang, L., W ei, Z., & Sauer, D. U. (2021b). Ph ysics-informed neural net works for electro de-level state estima- tion in lithium-ion batteries. Journal of Power Sour c es , 506 , 230034. Li, X., Zhang, W., & Ding, Q. (2019b). Understanding and impro ving deep learning-based rolling bearing fault diagnosis with atten tion mec hanism. Signal pr o c essing , 161 , 136–154. Li, Y., W ang, J., Huang, Z., & Gao, R. X. (2022). Physics-informed meta learning for machining tool wear prediction. Journal of Manufacturing Systems , 62 , 17–27. Li, Z., Zhou, J., Nassif, H., Coit, D., & Bae, J. (2023). F using physics-inferred information from sto chastic mo del with machine learning approaches for degradation prediction. R eliability Engine ering & System Safety , 232 , 109078. Liao, W., Bak-Jensen, B., Pillai, J. R., W ang, Y., & W ang, Y. (2021). A review of graph neural netw orks and their applications in pow er systems. Journal of Mo dern Power Systems and Cle an Ener gy , 10 , 345–360. 83 Liu, F., Y u, Q., Song, H., Li, X., Liu, L., & Liu, D. (2023). A nov el physics- informed framework for real-time adaptive monitoring of oﬀshore struc- tures. A vailable at SSRN 4349124 , . Liu, H., Xia, M., Williams, D., Sun, J., & Y an, H. (2022). Digital twin-driv en mac hine condition monitoring: A literature review. Journal of Sensors , 2022 . Liu, R., Liu, S., & Zhang, X. (2021). A ph ysics-informed machine learning mo del for p orosity analysis in laser p owder bed fusion additive man ufacturing. The International Journal of A dvanc e d Manufacturing T e chnolo gy , 113 , 1943– 1958. Liu, X., Huang, H., & Xiang, J. (2020). A p ersonalized diagnosis metho d to detect faults in gears using n umerical simulation and extreme learning mac hine. Know le dge-Base d Systems , 195 , 105653. Lu, H., Nemani, V. P ., Barzegar, V., Allen, C., Hu, C., Laﬂamme, S., Sark ar, S., & Zimmerman, A. T. (2023). A ph ysics-informed feature w eighting method for b earing fault diagnostics. Me chanic al Systems and Signal Pr o c essing , 191 , 110171. LU, H., SADOUGHI, M., ZIMMERMAN, A. T., & HU, C. (2019). F ault di- agnosis of rolling element bearings on lo w-cost and scalable iiot platform. Structur al He alth Monitoring 2019 , . L’heureux, A., Grolinger, K., Elyaman y , H. F., & Capretz, M. A. (2017). Ma- c hine learning with big data: Challenges and approac hes. Ie e e A c c ess , 5 , 7776–7797. Ma, Z., Liao, H., Gao, J., Nie, S., & Geng, Y. (2023). Physics-informed ma- c hine learning for degradation mo deling of an electro-h ydrostatic actuator system. R eliability Engine ering & System Safety , 229 , 108898. Mai, H. T., T ruong, T. T., Kang, J., Mai, D. D., & Lee, J. (2023). A robust ph ysics-informed neural net work approac h for predicting structural insta- bilit y . Finite Elements in Analysis and Design , 216 , 103893. McGo wan, E., Ga wade, V., & Guo, W. (2022). A physics-informed conv olutional neural net work with custom loss functions for porosity prediction in laser metal dep osition. Sensors , 22 , 494. de Melo, C. M., T orralba, A., Guibas, L., DiCarlo, J., Chellappa, R., & Ho dgins, J. (2021). Next-generation deep learning based on sim ulators and synthetic data. T r ends in c o gnitive scienc es , . Meng, C., Seo, S., Cao, D., Griesemer, S., & Liu, Y. (2022). When ph ysics meets machine learning: A survey of ph ysics-informed machine learning. arXiv pr eprint arXiv:2203.16797 , . 84 Miele, S., Karve, P ., & Mahadev an, S. (2023). Multi-ﬁdelity ph ysics-informed mac hine learning for probabilistic damage diagnosis. R eliability Engine ering & System Safety , (p. 109243). Giorgiani do Nascimen to, R., Viana, F., Corb etta, M., & Kulk arni, C. S. (2023). Quan tifying uncertaint y in li-ion battery aging due to unknown usage with h ybrid physics-informed neural netw orks. In AIAA SCITECH 2023 F orum (p. 0536). Nascimen to, R. G., Corbetta, M., Kulk arni, C. S., & Viana, F. A. (2021a). Hybrid ph ysics-informed neural net works for lithium-ion battery mo deling and prognosis. Journal of Power Sour c es , 513 , 230526. Nascimen to, R. G., Corbetta, M., Kulk arni, C. S., & Viana, F. A. (2021b). Li-ion battery aging with h ybrid ph ysics-informed neural netw orks and ﬂeet-wide data. In A nnual Confer enc e of the PHM So ciety . v olume 13. Nascimen to, R. G., & Viana, F. A. (2019). Fleet prognosis with physics-informed recurren t neural netw orks. arXiv pr eprint arXiv:1901.05512 , . Nascimen to, R. G., & Viana, F. A. (2020). Cumulativ e damage modeling with recurren t neural netw orks. AIAA Journal , 58 , 5459–5471. Ni, P ., Li, Y., Sun, L., & W ang, A. (2022). T raﬃc-induced bridge displace- men t reconstruction using a physics-informed conv olutional neural net work. Computers & Structur es , 271 , 106863. Ow are, E., Moysey , S., & Khan, T. (2013). Ph ysically based regularization of h ydrogeophysical in verse problems for impro ved imaging of pro cess-driven systems. Water R esour c es R ese ar ch , 49 , 6238–6247. P an, S. J., & Y ang, Q. (2010). A surv ey on transfer learning. IEEE T r ansactions on know le dge and data engine ering , 22 , 1345–1359. P eng, W., Y ao, W., Zhou, W., Zhang, X., & Y ao, W. (2022). Robust regression with highly corrupted data via physics informed neural netw orks. arXiv pr eprint arXiv:2210.10646 , . P erez-Sanjines, F., Peeters, C., V erstraeten, T., Antoni, J., Now ´ e, A., & Helsen, J. (2023). Fleet-based early fault detection of wind turbine gearb oxes using ph ysics-informed deep learning based on cyclic spectral coherence. Me chan- ic al Systems and Signal Pr o c essing , 185 , 109760. Pro ctor, J. L., Brunton, S. L., & Kutz, J. N. (2016). Dynamic mode decom- p osition with control. SIAM Journal on Applie d Dynamic a l Systems , 15 , 142–161. Rahimi, M. H., Huynh, H. N., & Altintas, Y. (2021). On-line chatter detection in milling with hybrid machine learning and physics-based model. CIRP Journal of Manufacturing Scienc e and T e chnolo gy , 35 , 25–40. 85 Rai, A., & Mitra, M. (2021). A hybrid physics-assisted mac hine-learning-based damage detection using lam b wa ve. S¯ adhan¯ a , 46 , 64. Raissi, M., P erdik aris, P ., & Karniadakis, G. E. (2019). Ph ysics-informed neu- ral netw orks: A deep learning framework for solving forward and inv erse problems inv olving nonlinear partial diﬀeren tial equations. Journal of Com- putational physics , 378 , 686–707. Rautela, M., RA UT, M., & Gopalakrishnan, S. (2021). Sim ulation of guided w av es for structural health monitoring using physics-informed neural net- w orks. STR UCTURAL HEAL TH MONITORING 2021 , . Ra ymond, S. J., & Camarillo, D. B. (2021). Applying ph ysics-based loss func- tions to neural netw orks for improv ed generalizability in mec hanics prob- lems. arXiv pr eprint arXiv:2105.00075 , . Rudin, C. (2019). Stop explaining blac k b ox mac hine learning models for high stak es decisions and use in terpretable mo dels instead. Natur e machine intel ligenc e , 1 , 206–215. Ruhnau, P ., Stahl, A., & Sc hn¨ orr, C. (2007). V ariational estimation of ex- p erimen tal ﬂuid ﬂows with physics-based spatio-temporal regularization. Me asur ement Scienc e and T e chnolo gy , 18 , 755. Russell, M., & W ang, P . (2022). Ph ysics-informed deep learning for signal com- pression and reconstruction of big data in industrial condition monitoring. Me chanic al Systems and Signal Pr o c essing , 168 , 108709. Sadoughi, M., & Hu, C. (2019). Ph ysics-based con volutional neural net work for fault diagnosis of rolling elemen t b earings. IEEE Sensors Journal , 19 , 4181–4192. Scarselli, F., Gori, M., Tsoi, A. C., Hagen buc hner, M., & Monfardini, G. (2008). The graph neural net work mo del. IEEE tr ansactions on neur al n etworks , 20 , 61–80. Sc hr¨ oder, L., Dimitrov, N. K., V erelst, D. R., & Sørensen, J. A. (2022). Using transfer learning to build physics-informed machine learning models for impro ved wind farm monitoring. Ener gies , 15 , 558. Serre, T. (2019). Deep learning: the go o d, the bad, and the ugly . A nnual r eview of vision scienc e , 5 , 399–426. Sev entekidis, P ., Giagop oulos, D., Arailopoulos, A., & Mark ogiannaki, O. (2020). Structural health monitoring using deep learning with optimal ﬁnite elemen t mo del generated data. Me chanic al Systems and Signal Pr o c essing , 145 , 106972. Shen, J., Chowdh ury , J., Banerjee, S., & T erejanu, G. (2023). Machine fault classiﬁcation using hamiltonian neural netw orks. arXiv pr eprint arXiv:2301.02243 , . 86 Shen, S., Lu, H., Sadoughi, M., Hu, C., Nemani, V., Thelen, A., W ebster, K., Darr, M., Sidon, J., & Kenn y , S. (2021). A ph ysics-informed deep learning approac h for bearing fault detection. Engine ering Applic ations of A rtiﬁcial Intel ligenc e , 103 , 104295. Shi, J., Rivera, A., & W u, D. (2022). Battery health management using physics- informed machine learning: Online degradation mo deling and remaining useful life prediction. Me chanic al Systems and Signal Pr o c essing , 179 , 109347. Simon yan, K., & Zisserman, A. (2014). V ery deep conv olutional netw orks for large-scale image recognition. arXiv pr eprint arXiv:1409.1556 , . Subramanian, A., & Mahadev an, S. (2023). Probabilistic physics-informed mac hine learning for dynamic systems. R eliability Engine ering & System Safety , 230 , 108899. Sun, H., P eng, L., Lin, J., W ang, S., Zhao, W., & Huang, S. (2021). Micro c- rac k defect quantiﬁcation using a fo cusing high-order sh guided wa ve emat: The ph ysics-informed deep neural net work guwnet. IEEE T r ansactions on Industrial Informatics , 18 , 3235–3247. Sun, X., Zhou, K., Shi, S., Song, K., & Chen, X. (2022). A new cyclical gener- ativ e adv ersarial net work based data augmen tation metho d for multiaxial fatigue life prediction. International Journal of F atigue , 162 , 106996. Surucu, O., Gadsden, S. A., & Y awney , J. (2023). Condition monitoring using mac hine learning: A review of theory , applications, and recent adv ances. Exp ert Systems with Applic ations , 221 , 119738. Szegedy , C., Liu, W., Jia, Y., Sermanet, P ., Reed, S., Anguelov, D., Erhan, D., V anhouck e, V., & Rabinovic h, A. (2015). Going deeper with con v o- lutions. In Pr o c e e dings of the IEEE c onfer enc e on c omputer vision and p attern r e c o gnition (pp. 1–9). T eng, S., Chen, X., Chen, G., & Cheng, L. (2023). Structural damage de- tection based on transfer learning strategy using digital twins of bridges. Me chanic al Systems and Signal Pr o c essing , 191 , 110160. Tian, J., Morillo, C., Azarian, M. H., & P ech t, M. (2015). Motor b earing fault detection using spectral kurtosis-based feature extraction coupled with k- nearest neigh b or distance analysis. IEEE T r ansactions on Industrial Ele c- tr onics , 63 , 1793–1803. T orrey , L., & Sha vlik, J. (2010). T ransfer learning. In Handb o ok of r ese ar ch on machine le arning applic ations and tr ends: algorithms, metho ds, and te chniques (pp. 242–264). IGI global. 87 Tsai, L.-W., & Alip our, A. (2023). Physics-informed long short-term mem- ory net works for resp onse prediction of a wind-excited ﬂexible structure. Engine ering Structur es , 275 , 114968. Viana, F. A., Nascimento, R. G., Dourado, A., & Y ucesan, Y. A. (2021). Es- timating model inadequacy in ordinary diﬀeren tial equations with physics- informed neural net works. Computers & Structur es , 245 , 106458. W ang, D., Chen, Y., Shen, C., Zhong, J., Peng, Z., & Li, C. (2022a). F ully in terpretable neural netw ork for lo cating resonance frequency bands for mac hine condition monitoring. Me chanic al Systems and Signal Pr o c essing , 168 , 108673. W ang, H., Liu, Z., P eng, D., & Cheng, Z. (2022b). A ttention-guided joint learning cnn with noise robustness for b earing fault diagnosis and vibration signal denoising. ISA tr ansactions , 128 , 470–484. W ang, J., Li, Y., Zhao, R., & Gao, R. X. (2020). Physics guided neural netw ork for mac hining to ol wear prediction. Journal of Manufacturing Systems , 57 , 298–310. W ang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., & W ang, F.-Y. (2017). Generativ e adv ersarial net works: in tro duction and outlo ok. IEEE/CAA Journal of A utomatic a Sinic a , 4 , 588–598. Xia, M., Shao, H., Williams, D., Lu, S., Shu, L., & de Silv a, C. W. (2021). Intel- ligen t fault diagnosis of mac hinery using digital twin-assisted deep transfer learning. R eliability Engine ering & System Safety , 215 , 107938. Xing, Y., Liu, Y., Nie, Y., Li, R., & He, X. (2022). Data-driven transmission line fault lo cation with single-ended measuremen ts and knowledge-a ware graph neural netw ork. In 2022 IEEE Power & Ener gy So ciety Gener al Me eting (PESGM) (pp. 1–5). IEEE. Xu, S., & Noh, H. Y. (2021). Phymdan: Ph ysics-informed knowledge transfer b et ween buildings for seismic damage diagnosis through adversarial learn- ing. Me chanic al Systems and Signal Pr o c essing , 151 , 107374. Xu, S., Sun, Z., Huang, R., Guo, D., Y ang, G., & Ju, S. (2023). A practical ap- proac h to ﬂo w ﬁeld reconstruction with sparse or incomplete data through ph ysics informed neural netw ork. A cta Me chanic a Sinic a , 39 , 1–15. Xu, Y., Kohtz, S., Boakye, J., Gardoni, P ., & W ang, P . (2022). Physics-informed mac hine learning for reliabilit y and systems safet y applications: State of the art and c hallenges. R eliability Engine ering & System Safety , (p. 108900). Y an, T., F u, Y., Lu, M., Li, Z., Shen, C., & W ang, D. (2022). Integration of a nov el kno wledge-guided loss function with an architecturally explain- able net work for machine degradation mo deling. IEEE T r ansactions on Instrumentation and Me asur ement , 71 , 1–12. 88 Y ang, Y., & Perdik aris, P . (2019). Adversarial uncertaint y quantiﬁcation in ph ysics-informed neural netw orks. Journal of Computational Physics , 394 , 136–152. Y e, X., Ni, Y.-Q., Sa jjadi, M., W ang, Y.-W., & Lin, C.-S. (2022). Ph ysics- guided, data-reﬁned modeling of granular material-ﬁlled particle damp ers b y deep transfer learning. Me chanic al Systems and Signal Pr o c essing , 180 , 109437. Yin, X., Huang, Z., & Liu, Y. (2023). Bridge damage identiﬁcation under the mo ving v ehicle loads based on the metho d of ph ysics-guided deep neural net works. Me chanic al Systems and Signal Pr o c essing , 190 , 110123. Y u, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2020a). Gradien t surgery for multi-task learning. A dvanc es in Neur al Information Pr o c essing Systems , 33 , 5824–5836. Y u, Y., Jiang, X., Huang, D., & Li, Y. (2022). Pidgeun: Graph neural net w ork- enabled transient dynamics prediction of netw orked microgrids through full- ﬁeld measuremen t. arXiv pr eprint arXiv:2204.08557 , . Y u, Y., Y ao, H., & Liu, Y. (2020b). Structural dynamics simulation using a no vel ph ysics-guided machine learning metho d. Engine ering Applic ations of Artiﬁcial Intel ligenc e , 96 , 103947. Y ucesan, Y. A., & Viana, F. (2020a). A h ybrid mo del for main b earing fatigue prognosis based on ph ysics and machine learning. In AIAA Scite ch 2020 F orum (p. 1412). Y ucesan, Y. A., & Viana, F. (2020b). A h ybrid mo del for wind turbine main b earing fatigue with uncertain ty in grease observ ations. In Annual Confer- enc e of the PHM So ciety (pp. 14–14). v olume 12. Y ucesan, Y. A., & Viana, F. (2021a). A probabilistic h ybrid model for main b earing fatigue considering uncertain ty in grease quality . In AIAA Scite ch 2021 F orum (p. 1243). Y ucesan, Y. A., & Viana, F. A. (2019). Wind turbine main bearing fatigue life estimation with ph ysics informed neural netw orks. Y ucesan, Y. A., & Viana, F. A. (2021b). Hybrid ph ysics-informed neural net- w orks for main bearing fatigue prognosis with visual grease insp ection. Computers in Industry , 125 , 103386. Y ucesan, Y. A., & Viana, F. A. (2022). A hybrid physics-informed neural net work for main b earing fatigue prognosis under grease qualit y v ariation. Me chanic al Systems and Signal Pr o c essing , 171 , 108875. 89 Zgraggen, J., Guo, Y., Notaristefano, A., & Goren Hub er, L. (2022). Physics informed deep learning for track er fault detection in photo voltaic pow er plan ts. In 14th Annual Confer enc e of the Pr o gnostics and He alth Man- agement So ciety, Nashvil le, USA, 1-4 Novemb er 2022 . PHM So ciety vol- ume 14. Zhang, R., Liu, Y., & Sun, H. (2020). Physics-guided conv olutional neural net work (ph ycnn) for data-driven seismic response modeling. Engine ering Structur es , 215 , 110704. Zhang, X.-C., Gong, J.-G., & Xuan, F.-Z. (2021). A ph ysics-informed neural net work for creep-fatigue life prediction of comp onents at elev ated temper- atures. Engine ering F r actur e Me chanics , 258 , 108130. Zhou, C., & Paﬀenroth, R. C. (2017). Anomaly detection with robust deep auto encoders. In Pr o c e e dings of the 23r d ACM SIGKDD international c on- fer enc e on know le dge disc overy and data mining (pp. 665–674). Zhou, K., Diehl, E., & T ang, J. (2023a). Deep con volutional generativ e adver- sarial netw ork with semi-sup ervised learning enabled physics elucidation for extended gear fault diagnosis under data limitations. Me chanic al Systems and Signal Pr o c essing , 185 , 109772. Zhou, T., Droguett, E. L., & Mosleh, A. (2022). Physics-informed deep learn- ing: A promising tec hnique for system reliabilit y assessmen t. Applie d Soft Computing , 126 , 109217. Zhou, T., Jiang, S., Han, T., Zhu, S.-P ., & Cai, Y. (2023b). A physically consisten t framew ork for fatigue life prediction using probabilistic physics- informed neural net work. International Journal of F atigue , 166 , 107234. Zhou, T., Zhang, X., Droguett, E. L., & Mosleh, A. (2023c). A generic ph ysics-informed neural netw ork-based framew ork for reliability assessment of m ulti-state systems. R eliability Engine ering & System Safety , 229 , 108835. Zh u, Q., Zhao, Z., & Y an, J. (2022a). Physics-informed machine learning for surrogate mo deling of wind pressure and optimization of pressure sensor placemen t. Computational Me chanics , (pp. 1–11). Zh u, Y., Zhou, Y., W ei, W., & W ang, N. (2022b). Cascading failure analysis based on a ph ysics-informed graph neural netw ork. IEEE T r ansactions on Power Systems , . Zh uang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2020). A comprehensiv e surv ey on transfer learning. Pr o c e e dings of the IEEE , 109 , 43–76. 90

A Review of Physics-Informed Machine Learning Methods with Applications to Condition Monitoring and Anomaly Detection

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment