Full Optical Fiber Link Characterization with the BSS-Lasso

F ull Optical Fib er Link Characterization with the BSS-Lasso Raphael Saa v edra, P edro T o v ar, Gusta v o C. Amaral, and Bruno F anzeres Jan uary 1, 2019 Abstract In this w ork, a nov el technique for optical fiber monitoring that introduces the Lasso as a signal pro cessing technique within the Baseband Sub carrier Sweep (BSS) framework, called the BSS-Lasso, is proposed. The methodology is tested in sim ulated and real-w orld environmen ts, taking into ac- coun t b oth reflective and non-reflectiv e even ts. The results show that, for fib er links ranging from 2 to 15 km with up to 3 faults, o ver 80% of faults are detected within a 50 m range, and indicate that the prop osed metho dology significantly outp erforms current state-of-the-art BSS-based super- vision tec hniques. Finally , the BSS-Lasso allo ws for precise, lo w-cost, transmitter-em b edded full c haracterization of optical fib er links. 1 In tro duction As so ciety b ecomes more dep endent on fast distribution of information, robust op eration of telecomm u- nication links app ears as a top priorit y for netw ork managers, since even the shortest service outage can affect up to millions of users; recently , it has b een estimated that 80% of all long-distance data traffic in the world is carried by optical fib ers [1]. The mec hanical fragility of the fib ers, ho wev er, p oses a threat to the robust operation of the optical fib er links since fiber b ending, fib er breaking, imp erfect fib er splices, R. Saav edra is with the Department of Electrical Engineering and the Laboratory of Applied Mathematical Program- ing and Statistics (LAMPS), Pon tifical Catholic Univ ersit y of Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil (e-mail: rsaav edra@ele.puc-rio.br). P . T ov ar is with the Cen ter for T elecommunications Studies, PUC-Rio (e-mail: pto v ar@opto.cetuc.puc-rio.br). G. C. Amaral is with the Cen ter for T elecommunications Studies, PUC-Rio and with QuT ec h and Ka vli Institute of Nanoscience, T echnical Univ ersity of Delft, Delft, The Netherlands (e-mail: gustav o@opto.cetuc.puc-rio.br). B. F anzeres is with the Department of Industrial Engineering and LAMPS, PUC-Rio (e-mail: bruno.santos@puc-rio.br). Copyrigh t (c) 2018 IEEE. Personal use of this material is p ermitted. How ever, p ermission to use this material for any other purp oses must b e obtained from the IEEE by sending a request to pubs-p ermissions@ieee.org. 1 and corrupted connectors affect the pow er budget and, thus, transmission capacit y . With the ev olution of data transmission proto cols and netw ork architectures, the link sup ervision tec hnology must also ev olv e to meet this sough t-after robustness whilst main taining a lo w impact ov er the qualit y of data transmission [2]. The most successful and well-established supervision technique of the ph ysical lay er of optical net w orks is the Optical Time-Domain Reflectometry (OTDR) [3]. The main adv antage of suc h technology is the p ossibilit y to obtain accurate precision with high resolution in long distance monitoring [4, 5]. How ever, in order to fully acquire information from a fib er stretch using this technique, the transmission data is generally susp ended for a non-negligible time frame since the OTDR pulse carries a considerable amount of optical p ow er spread through a broad sp ectral bandwidth. As a consequence, although technically efficien t, standard OTDR monitoring is usually economically burdening (see [6] and the references therein for a wider discussion). T o tac kle this issue, several other reflectometry-based techniques ha v e b een prop osed in tec hnical literature [7–11]. Generally , they seek for a monitoring routine such that a narrow sp ectral channel can b e allocated, widening the sim ultaneous data traffic capabilit y . In other w ords, an equilibrium b etw een an efficien t monitoring capacity , b oth in distance and resolution, and the co existence with data transmission is pursuit. Th us, in practice, net w ork op erators need to balance the pros and cons of different monitoring tec hniques in order to c hoose the one that b est fits their net work, both from a technical and an economical p ersp ective. F ollowing this rationale, a nov el transparent and cost-effective reflectometry-based tec hnique has been recen tly developed [12]. Due to its c haracteristics, the technique will be henceforth dubbed as Baseband Sub carrier Sw eep (BSS). F rom a technical point of view, the main goals of the BSS monitoring method are to achiev e reasonable spatial resolution and dynamic range in optical fib er monitoring, with b oth negligible a priori knowledge ab out the fiber and in terference on data traffic. Within these four goals, three ha v e b een successfully ac hiev ed, with a dynamic range limited to 7 dB b eing the ma jor hindrance of the method. F urthermore, from an implementation point of view, this tec hnique fits the architecture of the so-called Mobile F ronthaul [13, 14], an ubiquitous concept for next-generation mobile netw orks, and can b e seamlessly incorp orated into the optical transmitter (a so-called transmitter-embedded technique) with minimum cost ov erhead [14]. Generally speaking, the nature of the BSS-based monitoring tec hnique is to measure the fib er’s transfer function by detecting the Ra yleigh backscattered p ortion of the propagating optical signal mo dulated by a sw ept tone cov ering several frequencies within an allo cated low-frequency bandwidth. Even though this had b een previously studied in [15], it was only in [12] that the manipulation of the Rayleigh bac kscattered signal b ecame the fo cus of the monitoring solution. The resulting probing signal has a p erio dic structure in the frequency domain, which can b e represented as a linear com bination of spatial-dep endent phasors that take the fault p ositions as arguments [10]. Identifying which spatial-dep endent phasors are presen t in the monitoring signal allows one to consequently identify the set of fault p ositions present in the 2 fib er. Although intuitiv e, the key dra wback of this methodology is the necessity to p erform an extensive com binatorial search in order to precisely determine the fault p ositions, an utterly non-trivial task to be conducted by naiv e optimization methods [12, 16] in reasonable computational time. In this con text, tec hniques based on high-dimensional analysis emerge as an effective to ol. In this w ork, a widely-used high-dimensional signal interpreter metho d known as Least Absolute Shrink age and Selection Operator (Lasso) [17] is employ ed in order to p erform a computationally efficien t fault detection [18]. F undamen tally , the Lasso performs a v ariable selection from an o v er-complete dictionary , electing the comp onents that b est fit the original signal. Therefore, by designing the ov er-complete dictionary with the appropriate sinusoidal-based functions asso ciated to all p ossible fault p ositions (e.g., performing a meter-by-meter discretization of the fib er length), the Lasso methodology can be used to ev aluate the optical fib er link in practical time. In fact, the Lasso has already b een adapted to fit in BSS-based monitoring tec hniques [10]. Never- theless, the key issue regarding this previous w ork w as the absence of a complete mathematical mo del describing the impact of reflectiv e and non-reflectiv e ev ents on the acquired signal; only non-reflectiv e ev ents were considered within the dictionary . Characteristics of even ts allow their classification as reflec- tiv e or non-reflectiv e, and a correspondence with the underlying physical mec hanism can b e observ ed: fib er breaking and corrupted connectors usually causing reflective even ts; and fib er b ending and imper- fect fiber splices causing non-reflectiv e ev en ts. Since the incidence of these cannot b e predetermined, the absence of either causes the model to b e incomplete. In order to comp ensate for the unaccounted con tribution of reflectiv e even ts, the incomplete mo del either pro duces unreal even ts or induces shifts on the real ev ent positions, thus reducing the accuracy and robustness of the monitoring method. Therefore, the main ob jectives and contributions of this w ork are threefold: 1. T o extend the metho dology proposed in [10] to account for a hybrid reflective and non-reflectiv e mo deling framew ork, which represents most practical cases of fiber link sup ervision. The prop osed mo del mak es use of the signal description derived in [12] to construct the ov er-complete dictionary that feeds the Lasso. 2. T o devise an ex-p ost analysis to enhance the fault p osition estimation. By making use of the prop erties of reflectiv e even ts, a tailored heuristic, hereinafter referred to as BSS-L asso , is prop osed. 3. T o design and v alidate a metho dology to prop erly reconstruct the time-domain profile of a fib er based on the BSS-Lasso metho d. The mo del that relates the frequency- to the time-domain profile allo ws precise estimation of the fault and reflection magnitudes, a feature that enables the full c haracterization of the fib er’s profile. Finally , as a minor con tribution, a large library of fault y fib er profiles in the frequency domain is made a v ailable so they can b e used to p erform extensiv e tests and comparisons with current state-of-the-art monitoring techniques. All data of this test b ench is av ailable in [19]. This step, and the p ossibilit y of 3 recreating frequency-domain profiles that mimic exp erimental acquired data is of great imp ortance for future prop osals and allows one to v alidate and compare the capacit y of differen t monitoring routines. Due to the in terdisciplinary c haracter of this work, whic h in volv es optical bac kreflection measurements and high-dimensional data processing, an introductory background is provided. Firstly , the data acqui- sition and mathematical mo del of the fault lo cation problem are briefly presen ted in Section 2, where the high-dimensional feature of the problem naturally arises from the characteristics of the sup ervision tec hnique, and its constraints are p ointed out. In Section 3, the framework of the Lasso is introduced, along with a discussion on ho w the idiosyncrasies of the stated problem allow for the tailored heuris- tic, the BSS-Lasso, to b e prop osed. With the bac kground from these tw o Sections, experimental fault lo cation results using real-world fib ers are presented in Section 4. Moreov er, in Section 5, the compar- ison of the BSS-Lasso with state-of-the-art monitoring tec hniques is p erformed in the constructed test b enc h in order to provide a statistically relev ant analysis of the tec hnique’s monitoring capability . The pap er is concluded in Section 6, where the adv ances achiev ed b y the BSS-Lasso and on-going researc h are summarized. 2 Exp erimen tal Setup Sw eeping the frequency of an optical sub-carrier tone to ev aluate fault lo cations in an optical fib er has b een extensively studied, and is referred to as Incoherent Optical F requency-Domain Reflectometry (I- OFDR) [20]. In case the bandwidth of the frequency sw eep is sufficien tly large (in the order of GHz) to yield a granular spatial resolution (in the order of meters), a F ourier transform can b e emplo yed to extract the desired information [21], but at the high cost of consuming a significan t portion of the a v ailable transmission band, except in sp ecific cases where the transmission sp ectrum can b e tailored to accommo date the monitoring signal [14]. Con versely , if the bandwidth of the frequency sw eep is reduced (e.g., to a few kilo Hertz), the achiev able spatial resolution through a F ourier transform is not sufficien t for a precise fault lo cation, but allo ws for seamless adaptation in to certain data transmission formats, with special stress to Sub-Carrier Multiplexed optical net w ork arc hitectures, where the baseband of the optical carrier is left uno ccupied [12, 14]. The BSS tec hnique falls within the latter class, making itself attractiv e from the p oint of view of adaptation to transmitters and co existence with data transmission. The data acquisition is accomplished b y mo dulating an optical carrier electric field with a sin usoidal signal, whic h has its frequency increased in a stepwise fashion that resp ects the steady-state condition of the fib er, while the amplitude and phase of the backscattered signal inside the frequency swept bandwidth are determined by a complex frequency b eat detector. 4 2.1 Data Acquisition and Mathematical Mo del The exp erimental setup for data acquisition is presented in Fig. 1. A Net w ork Analyzer (NA) generates the low-frequency step-wise sw ept tone that directly mo dulates the curren t of a laser dio de biased with a Laser Bias Source (LBS). This causes the electric field of the optical carrier to be mo dulated accordingly , th us creating a low-frequency optical sub-carrier channel with bandwidth defined b y the step-wise sweep range, whic h propagates through the fib er; this channel is henceforth dubbed the monitoring channel . The optical circulator, placed immediately b efore the Fib er Under T est (FUT), allo ws one to direct an y coun ter-propagating signal, suc h as the Ra yleigh backscattered p ortion of the incoming signal, to a photo dete ctor. Due to the elastic character of Rayleigh scattering, the detected backscattered signal will conserv e the original modulation, so that a frequency analysis inside the lo w-frequency optical sub- carrier channel can be performed with the correct apparatus. In this case, the NA itself allo ws for complex (amplitude and phase) frequency b eat detection, so the resulting electrical signal is amplified and directed to the NA for frequency response analysis. Figure 1: Experimental Setup of the baseband subcarrier sw eep monitoring. Data acquisition is p erformed b y measuring the steady-state amplitude and phase v alues for each frequency step. LD: Laser Diode; NA: Net work Analyzer; PD: Photo dio de; LBS: Laser Bias Source; BEA T: complex frequency b eat detector. A key c hallenge to adapt BSS-based technologies in fib er monitoring is precisely the spatial resolution limitation imp osed by the F ourier transform analysis induced by a frequency sw eep with limited range. As discussed in [12], analysis of the resulting signal directly in the frequency domain p ermits one to o vercome suc h limitation and extend the achiev ed spatial resolution. In order to mo del the backscattered signal inside the monitoring channel bandwidth, the fact that the OTDR profile { P ( z ) } z ∈ [0 ,L ] of a fib er with length L meters can b e suitably approximated by a linear combination of step (breaks) and p eak (reflections) functions with a single slop e (the fib er’s atten uation co efficient) is used [22]. More sp ecifically , ∀ z ∈ [0 , L ], P ( z ) = e − 2 αz X b ∈B φ b h u ( z ) − u ( z − X b ) i + X r ∈R θ r h δ ( z − X r ) i ! , (1) 5 where { X b } b ∈B are the non-reflectiv e even ts with B the set of non-reflectiv e even ts indexes, and φ b are the linear combination co efficients. Similarly , { X r } r ∈R are the reflective even ts with R ⊆ B the set of reflectiv e even ts indexes, and θ r are the linear combination co efficients. In (1), u ( z ) and δ ( z ) denote, resp ectiv ely , the Heaviside and Dirac impulse functions. Therefore, the fib er profile contains the level shifts that corresp ond to non-reflective faults and spikes that correspond to reflectiv e ev ents. Note that, although P ( z ) translates the amount of p ow er b eing bac kscattered and/or reflected from each p osition of the fib er, the co efficients φ b are not directly related to the faults magnitudes as usually presented in an OTDR; this relation is giv en by ξ b = v u u u t 1 − φ b Y j ∈B | j w     β ≥ 0 o , (6) where y is the observ ed/dep endent v ariable, M stands for a matrix of explanatory/indep enden t v ari- ables, β is a decision v ector of coefficients, and w and λ are p ositive vector- and scalar-size parameters, resp ectiv ely . The ` 1 -norm p enalty term (second term of the ob jective function) p erforms a regularization of the β vector around zero. In other words, it p enalizes “unnecessary” deviations from zero, acting, thus, as a v ariable selection. In this regularization, w stands for the p enalty weigh t at each co ordinate of β and λ defines the p enalty magnitude for the total regularization term. It is important to note that the standard Lasso do es not restrict the v ector β to b e p ositive. Nev ertheless, since a p ermanent p ow er loss or a momen tary p ow er p eak (induced by a fault or reflection, respectively) can only generate complex frequency-domain signals that, under cartesian representation, exhibit positive co efficients, the p ositiv e- ness constraint ensures the ph ysical soundness of the mo del. A k ey c hallenge to apply the Lasso in practical applications is an adequate definition of the p enalt y w eights w and λ . Usually , practitioners set w = 1 and apply an information criterion (e.g., the Extended Ba yesian Information Criterion (EBIC) [25, 26]) to select, from a pre-defined set Λ of p enalty v alues, the magnitude of λ . Therefore, one needs to solv e the optimization problem (6) for all λ ∈ Λ and pic k the one that results in the b est (i.e., minimal) EBIC. Algorithm 1 show cases the Lasso pro cedure as conducted in this w ork 1 . Algorithm 1 Lasso for λ in Λ do ˆ β ( λ ) ← arg min β ≥ 0 k y − M β k 2 2 + λ  β > w  end for return ˆ β ← arg min ˆ β ( λ ) EBIC( ˆ β ( λ )) 1 The Lasso can be efficiently executed through several op en-source pack ages, among which w e highligh t glmnet [27], that utilizes coordinate descent with F ortran subroutines and is av ailable in several languages such as Julia, Matlab and R. 9 F or the sake of simplicit y , EBIC( ˆ β ( λ )) represents the ev aluation of the Extended BIC from the solution of (6) with p enalty λ . Henceforth, Algorithm 1 will b e referred to as Lasso( y , M , w ). 3.2 Mo del design and selection stage of the BSS-Lasso In order to identify the set of fault locations on a given fib er link based on its frequency-domain profile, equation (4) is accomo dated in to the Lasso framework discussed in Subsection 3.1. The pro cedure starts by creating a spatial discretization of the fib er length L in X = { X 1 , · · · , X q } lo cations. F or a reasonable granularit y , the fault lo cations are within a negligible distance of giv en elements in X . Additionally , a byproduct of the data acquisition arc hitecture discussed in Section 2 is a set of frequencies F = { f 1 , · · · , f m } for which the probing signal has b een precisely ev aluated. Therefore, the matrix M in (6) can be defined as M = 1 L · h M B    M R i , (7) where matrices M B and M R represen t the fault and reflection signals as follows: M B =     Re { S B ( F , X ) } Im { S B ( F , X ) }     ; M R =     Re { S R ( F , X ) } Im { S R ( F , X ) }     . (8) In (7), M contains the real and imaginary parts of the signals S B and S R defined in (5), generated by ev ery possible fault and reflection lo cation within the giv en fiber discretization in X and frequency in F . F urthermore, a normalization pro cedure (division by L in (7)) is p erformed to av oid numerical issues and an intercept with asso ciated zero p enalty can be added to accoun t for p ossible measuremen t effects that offset the acquired data. According to the model description, the dependent v ariable y is an instance of S ( f ), which is also decomposed in to its real and imaginary parts: y =     Re { S ( F ) } Im { S ( F ) }     . (9) Within this design, the decision v ector β measures the coefficients Φ and Θ in equation (4), thus b eing decomp osed in to non-reflective and reflective co efficients: β = [ { β B b } q b =1 , { β R r } q r =1 ] > . Therefore, the p osition within X in which a fault is lo cated can b e iden tified b y picking { β B b } q b =1 greater than zero after executing Algorithm 2 with M and y defined in (7) and (9), resp ectiv ely . Note that, as a byproduct of the prop osed metho dology , the net work op erator can also distinguish if the fault p osition has a reflective ev ent, i.e., if the corresp onding β R is also greater than zero. The selection stage is explicitly presented in Algorithm 2. Algorithm 2 BSS-Lasso – Selection stage ˆ β (1) ← Lasso( y , M , 1 ) return ˆ β (1) 10 Remark 1 The monitoring te chnique discusse d in [10], c al le d SincL asso, is a p articular instanc e of the sele ction stage by dr opping matrix M R fr om the mo del, ther eby not taking r efle ctive events into ac c ount. 3.3 Correction stage of the BSS-Lasso In [10], it was observed that the prop osed SincLasso monitoring sc heme regularly failed to accurately iden tify fault p ositions whenever the even t had a reflective comp onent. More sp ecifically , significant shifts in the estimated fault position with resp ect to the real position w ere observ ed. F urthermore, a similar pattern persisted in the BSS-Lasso selection stage, even with the explicit inclusion of reflectiv e ev ents in the mo deling design. Nev ertheless, it w as observed that, although the fault selection remained shifted in the presence of a reflection, its reflective counterpart w as consistently accurate. Therefore, b y making use of this empirical observ ation, a correction stage was designed to accommodate an ex-p ost analysis of the selection stage in order to impro v e the supervision of fib ers with reflectiv e even ts. The correction stage has a s imilar motiv ation to the Adaptive Lasso [28], in whic h it utilizes Lasso selections to mo dify the p enalt y v ector w . More precisely , reflections only exist accompanied b y a fault at the same p osition (i.e., R ⊆ B ). Therefore, since the selection stage has shown muc h greater accuracy for the reflective selections, an adjustment on the weigh t vector w can b e made at the respective fault p osition, reducing its p enalt y so that Algorithm 1 can b e re-executed with the adjusted w . The correction stage is presen ted next. Algorithm 3 BSS-Lasso – Correction stage Let ˆ β (1) b e the selection stage output. Let Q = { q + 1 , ..., 2 q } ,  > 0, 0 < γ < 1. Initialize p enalty v ectors w ( k ) ← 1 , k = 2 , 3. for k = 2 to 3 do i. If max j ∈Q { ˆ β ( k − 1) j } = 0 = ⇒ return ˆ β ( k − 1) ii. Adjust p enalty vector w ( k ) : w ( k ) i − q ← γ , ∀ i ∈ Q | ˆ β ( k − 1) i >  · max j ∈Q { ˆ β ( k − 1) j } iii. Run the Lasso with p enalty w ( k ) : ˆ β ( k ) ← Lasso( y , M , w ( k ) ) end for return ˆ β (3) In Algorithm 3,  is a small num ber that acts as a sensitivit y threshold. Recall from the mo del design in Subsection 3.2 that co ordinates { 1 , ..., q } in β corresp ond to non-reflective even t candidates ( { β B b } q b =1 ) and { q + 1 , ..., 2 q } the reflective even t candidates ( { β R r } q r =1 ). Therefore, γ ∈ (0 , 1) is defined to reduce the p enalty weigh t on the first q elements of β according to the reflective selections of the previous iteration, such that w ( k ) has all elements equal to one, except those that represen t faults with non- negligible reflective selections, which receiv e a reduced penalty v alue γ . More precisely , if the selection 11 stage outputs any non-zero reflective selections, the BSS-Lasso enters the correction stage, where up to t wo more iterations are computed in order to address the previously men tioned shifting phenomenon. F or eac h iteration k , a reduced p enalty vector w ( k ) is created based on the selections of the previous iteration, and the Lasso is executed accordingly . The rationale b ehind the correction stage is that, if a reflection was found at a giv en p osition, it generally indicates the presence of a fault at that lo cation. Additionally , if Algorithm 3, due to the reduced penalty , selects at the correct lo cation a fault that was previously shifted, this often causes the algorithm to reject its previous incorrect position because of the ` 1 -norm penalty , th us correcting the shifting phenomenon. F or illustrative purp oses, Fig. 3 depicts the monitoring accuracy of the SincLasso, BSS-Lasso up to the selection stage and complete BSS-Lasso for an 8 km fib er link. Note that, for the reflectiv e fault at 4 km, b oth the SincLasso and the BSS-Lasso up to the selection stage incurred in an error of o v er 400 m, while the complete BSS-Lasso presented a 10 m error due to the c orrection stage. Figure 3: Comparison of the normalized selections resulting from SincLasso, BSS-Lasso up to the selection stage and complete BSS-Lasso for an 8 km fib er link with a reflective fault at 4 km and a non-reflectiv e fault at 8 km. Remark 2 In this work, the pr op ose d algorithm has up to thr e e over al l L asso iter ations. A lthough ad- ditional iter ations might b e justifiable to further impr ove the quality of the metho d, ther e is a lack of a c onsistent stopping criterion in or der to avoid cycling. In fact, two over al l iter ations ar e sufficient for the existenc e of a c orr e ction stage. However, it was empiric al ly observe d that a thir d one, which acts as a r efinement step, signific antly impr oves the pr e cision of the metho d. F urthermor e, the b enefits of addi- tional iter ations (mor e than thr e e) wer e observe d to b e mar ginal, non-existent or even ne gative. F or these r e asons, the numb er of iter ations was fixe d at thr e e in Algorithm 3, which r esults in a c omputational ly effe ctive pr o c e dur e. 12 3.4 T reatmen t stage of the BSS-Lasso Ultimately , Algorithm 3 outputs a 2 q -dimensional vector indicating the fault lo cations. How ever, in man y cases, the result is not presented as a straightforw ard handful of singular selections, but rather as a set, or cluster, of even t positions. Although the app earance of suc h clusters ma y ha v e its roots in the limited spatial resolution of the monitoring technique, the Lasso is known to produce clusters whenev er the ov er-complete dictionary v ariables present high levels of correlation [23]. In fact, dep ending on the fib er discretization, adjacent columns of the explanatory matrix M represent close enough p ositions suc h that their induced signals are almost indistinguishable, thus strongly correlated. As a consequence, the selection and correction stages often times output multiple selections around the real fault position as a cluster (see Fig. 4 for an il lustrative example con taining an output from the correction stage and the final BSS-Lasso result) and it migh t b e useful to refine the result in order to obtain more accurate selections to b etter assist the netw ork op erator. Figure 4: Time-domain profile in the top panel and BSS-Lasso selections in the b ottom panel for a 5 km fib er link with a reflective even t at 3 km and a non-reflectiv e even t at 5 km. The reflective even t results in a single selection, while the non-reflective one induces a cluster of selections. A handful of tec hniques app ears in technical literature to handle correlated explanatory v ariables within the Lasso framework. The commonly used approach p erforms an a priori clustering of the cor- related explanatory v ariables b efore solving the Lasso problem [29]. How ever, such a priori clustering is not suited for the particular application of this work, since all explanatory v ariables are sequentially correlated to its adjacent neighbors with respect to the emplo yed distance grid in matrix M . T o ov ercome this issue, an alternativ e is to narro w down the clusters of selections into single selections after the Lasso pro cessing. F rom a technical p oint-of-view, a cluster of selections represent a degenerate pro jection of y onto the column space of M , i.e, one p ossible w a y to express this pro jection in terms of the column 13 v ectors of M . As a consequence, it is likely that the real fault is among the cluster p ositions and closer to the selections with higher magnitude. Therefore, one can interpret each cluster of selections as a prob- abilit y density for the resp ective true fault p osition. In this con text, several approac hes can b e applied to directly narro w down the clusters of selections. Firstly , since the clusters can be in terpreted as probability densities, a computationally efficient ap- proac h is to estimate the true fault p osition b y applying a w eighted av erage of the positions within the clusters, where the weigh ts are given by the selection magnitudes. Another p ossibility is to consider all p ossible combinations of p ositions with a single selection p er cluster and choose the one which outputs the least square model fitting error (minimal ` 2 norm). This approach can b e implemented b y either a com binatorial set of ordinary least squares computations or via Mixed Integer Quadratic Programming (MIQP) [30, 31]. Despite showing the b est empirical results, in theory , it is affected by the curse of dimensionalit y and can b e computationally burdening, i.e., instances with wide and/or several clusters (e.g., a fib er with n umerous faults) migh t b e intractable in reasonable computational time. Nevertheless, it should be emphasized that, for most practical cases, this approach can b e computed in the order of seconds. Finally , a viable alternativ e to the combinatorial least squares for fib ers with m ultiple even ts is the least absolute error (minimal ` 1 norm), which can b e solved using Mixed In teger Linear Programming (MILP) algorithms. Although still a com binatorial procedure due to their in teger nature, MILP problems are widely recognized to b e more computationally efficient to b e solv ed than MIQP . Due to its b etter empirical performance, in this work, the treatment of the clusters of selections is p erformed using least squares. It should b e highlighted that this choice is more consistent with the prop osed BSS-Lasso metho dology , since the Lasso already uses the ` 2 norm for the mo del fitting (see the first term of the ob jective function in (6)). In fact, this choice of ex-p ost cluster treatmen t can be seen as an instance of the Lasso’s original design [17], in which the ` 1 regularization term (second term of the ob jective function in (6)) is replaced b y the semi-norm ` 0 and written as a constraint b ounded b y 1 for each cluster. More precisely , let {C i } C i =1 b e the family of C clusters resultant from the BSS-Lasso and M =  M j } j ∈C i  C i =1 , the explanatory matrix restricted to the position indexes within each cluster. Then, the cluster treatment pro cedure can b e form ulated as the following mathematical programming problem ([30, 31] are referred to for a wider discussion and efficient form ulations for this problem): min β        y − M β   2 2          β i   0 ≤ 1 , ∀ i ∈ C ; β ≥ 0 ;      , (10) where β i = { β j } j ∈C i . Therefore, in this context, the original ` 0 -Lasso is solv ed, but av oiding its high com binatorial nature since the ` 0 searc h is performed in a drastically reduced space when compared to the original problem. The treatment stage of the BSS-Lasso is th us presen ted in Algorithm 4. 14 Algorithm 4 BSS-Lasso – T reatment stage Let ˆ β (3) b e the correction stage output. i ← 1, C i ← {} , C ← i . for j = 1 to q do If ˆ β (3) j > 0 = ⇒ C i ← C i ∪ { j } Else i ← i + 1; C i ← {} ; C ← i end for Construct M =  M j } j ∈C i  C i =1 Solv e ` 0 -Lasso (10) and return its optimal decision vector F or future reference, BSS-Lasso will b e identified as the sequen tial computing of Algorithms 2 – 4. 4 Exp erimen tal V alidation V alidation of the BSS-Lasso in a real environmen t consists in comparing its estimated even ts p ositions with the reference even ts p ositions determined using a standard OTDR device for differen t fib ers. Practically , the limitation on the num b er of exp erimentally tested links is determined by the a v ailability of fibers with differen t lengths in the laboratory and the p ossibility of connecting these fib ers to form links with differen t num b er of ev en ts and differen t lengths. Six fib er links were av ailable, for which the profiles ha v e b een individually measured; combinations of these six fibers tw o-b y-t wo, to comp ose tw o-even t links, pro duced 30 more examples. Prior to measuring the fib ers, how ever, a few exp erimental parameters concerning the data acquisition and the minimum monitoring signal mo dulation pow er must b e defined. A picture of the exp erimental setup as assembled in the lab oratory for all these tests is presented in Fig. 5. 4.1 Exp erimen tal parameters c haracterization As depicted in the exp erimental setup of Fig. 1 and shown in Fig. 5, the opto electric conv ersion and signal amplification is p erformed by a photodetector, which is composed mainly by a photodio de and t wo amplifiers. F rom an optical p ersp ective, as long as the bac kscattered optical signal reaching the photo dio de carries p ow er greater than the NEP , the output electrical signal will carry information ab out the fib er. In the same wa y , but from an electrical point of view, the signal reaching the NA must b e greater than its NEP ( N na ), so that the measured frequency profile translates the fib er’s characteristics. Since an imp ortan t c haracteristic of the Baseband Subcarrier Sweep monitoring tec hnique is its co existence with data transmission in a transmitter-embedded configuration, as discussed in [12], it is paramount that a balance betw een monitoring signal p ow er and the capacity of meeting the previously mentioned p ow er requiremen ts, in b oth the photo detector and the NA, is found. In this subsection, this balance will b e analyzed and the exp erimental parameters used throughout the BSS-Lasso v alidation will b e presen ted. 15 Figure 5: Photo of the exp erimental setup. 1) Netw ork Analyzer; 2) Distributed F eedbac k-Laser Dio de; 3) Laser Bias Source; 4) T emp erature Controller Driver; 5) Optical Circulator; 6) Photo detector; 7) Fib er under test. Analysis of the optical signal-to-noise ratio (SNR) requires knowledge of a few parameters, all of whic h are summarized in the upp er part of T able 1. The laser diode’s bias curren t is set at 70 m A, yielding an output optical p ow er of 4 dBm; this allows for a 52 mA excursion inside its linear region. An insertion loss of approximately 1 dB in the optical circulator (p ort 1 → p ort 2) sets the input optical pow er to the fiber under test (FUT) to ∼ 3 dBm. Thus, considering a Rayleigh bac kscattered co efficient C of -72 dB/m [4], an attenuation co efficient α of 0.2 dB/km, a 6372 m optical fib er, and another 1 dB of insertion loss in the optical circulator (p ort 2 → p ort 3), the optical p ow er arriving at the photo dio de in the steady-state regime would b e − 33 . 20 dBm (or 477 nW). This v alue has b een exp erimentally verified to b e − 34 . 02 dBm. The emplo y ed photo detector exhibits a NEP of 200 pW in a full bandwidth condition [32], so a comfortable 33 dB SNR at the photodio de is ensured. Note, how ev er, that this SNR represen ts the total optical p ow er arriving at the photo dio de, including the DC component. As will b e discussed presen tly , a 10% p ortion of the laser’s full modulation depth is o ccupied b y the monitoring signal, so the SNR for the signal of in terest is approximately 23 dB. Electrical signal analysis b egins right after the opto-electrical conv ersion of the photo dio de: although the tw o amplification stages of the photo detector exhibit noise figures of their own and also amplify noise coming from the photo dio de, the optical SNR is sufficien tly high so that one can consider the output signal of the photo detector to maintain the same SNR. Thus, taking into account the received optical 16 p o wer of 477 nW and the photo detector parameters (resp onsivity , transimp edance gain, and the 2 nd stage voltage gain) and also the p ortion of the full modulation depth occupied by the monitoring signal, all sho wn in T able 1, it is p ossible to calculate an amplitude of 298.37 mV for the input signal en tering p ort 2 of the NA at lo w frequencies. According to the mathematical mo del and the exp erimental results, the amplitude of the electrical signal has an in v erse dep endence with the frequency and, as the latter increases, the former is exp ected to dramatically decrease. The v alue for the low-frequency measuremen t, ho wev er, has b een exp erimentally measured to b e 303.57 mV, which is far abov e N na ( − 95 dBm [33]), setting the electrical SNR at 47.32 dB and v alidating b oth the optical and electrical signal analysis. T o address the SNR reduction as the frequency increases, whic h can translate into up to 30 dB reduction at the maximum swept frequency , an av eraging pro cess is employ ed. This consists of acquiring sev eral traces inside the same in terv al and taking the arithmetic av erage, a resource already a v ailable in the emplo y ed NA. At the same time, since the frequency-domain profile is determined in a steady-state regime, the intermediate frequency bandwidth of the NA frequency b eat detector is set to a lo w v alue (150 Hz) translating in to a considerable long time-constan t of 6.64 s. This long time constan t translates in to an intrinsic av eraging of the measured amplitude and phase of the input signal, whic h, by itself, diminishes noise contributions to the measurement, esp ecially at higher frequencies, and alleviates the n umber of samples for the av erage pro cedure. Therefore, in order to reach a compromise betw een faster data acquisition and negligible noise contribution, a total of 10 samples has b een set as default to the a veraging pro cedure; the total data acquisition pro cess amoun ts to less than tw o min utes. T able 1: Experimental Parameters for Data Acquisition Exp erimen tal P arameters Optical Fib er input pow er ( P 0 ) 3 dBm Fib er length 6372 m Circulator loss 1 dB Fib er attenuation ( α ) 0.2 dB/km Opto/Electric PD resp onsivity @ 1550 nm ( R ) 1 A/W Electrical PD transimp edance gain 626 V/A PD 2 nd stage voltage gain 1x10 4 V/V Laser Bias Source (LBS) 70 mA NA mo dulation pow er 0 dBm Mo dulation depth ( m ) 10% As mentioned in the previous analysis, the p ortion of the laser’s full mo dulation depth occupied b y the monitoring signal was set to 10% as the balance b etw een minimal monitoring p ow er that allo ws for accurate fiber c haracterization. This v alue immediately impacts on the capacity of concurrent data transmission in a direct modulation scheme and exp erimentally characterizes the BSS-Lasso technique in the context of monitoring and data co existence. T o v alidate this parameter, the signal’s amplitude 17 en tering the mo dulation p ort of the laser w as v aried from 50 mVpp to 1.6Vpp, which corresp onds, appro ximately , to 2% and 62% of the laser’s full mo dulation depth, respectively . The BSS-Lasso was tested for eac h signal amplitude in the same fiber testb ed (with parameters describ ed in T able 1), with the frequency-domain fib er profile b eing determined with the BSS setup, and errors betw een the real p ositions of even ts and the ones obtained from the tec hnique b eing ev aluated based on the output of the BSS-Lasso. The exp erimental result is presen ted in Fig. 6. Figure 6: Impact of the low-frequency monitoring signal’s amplitude – written in terms of percents of the laser’s full modulation depth – on the BSS-Lasso even t p osition estimation. The 5% mark represents a clear lo w er b ound on the amplitude of the monitoring signal so that reliable results are obtained. The results v alidate the exp erimen tal parameters presen ted in T able 1 and show that, for monitoring signals with amplitude greater than 10% of the modulation depth, position estimates are within a 30 m distance error from the actual p ositions. When the monitoring signal’s amplitude falls b elow 5% of the mo dulation depth, on the other hand, the distance error rapidly increases. This indicates that the BSS-Lasso can b e implemented to contin ually monitor a fib er link while o ccupying only a tenth of the laser’s full mo dulation depth, thus making a v ailable a considerable p ortion of the laser’s electro-optical transfer function for the purpose of data transmission. F urthermore, w e note that the precision plateau attained at 10% of the full mo dulation depth is asso ciated to a limitation of the spatial resolution of the tec hnique, which is related to the maxim um monitoring frequency . All the experimental results presen ted in this paper ha v e b een acquired using the parameters describ ed in T able 1. 4.2 Exp erimen tal results In Fig. 7, the even t position estimation error of the BSS-Lasso is compared with the reference results of a standard OTDR device and presented in the form of a scatter plot. Tw o error thresholds are defined in the scatter; the first one delimits a low-error interv al, where the results are deemed to b e very close to the reference, in whic h almost 70% of the results hav e fallen; the second delimits results that are fairly close to the reference p osition, and holds ov er 93% of the total results. A few outliers can b e observ ed with errors ab ov e 100 m. The error distribution, mainly concentrated around the zero error p oin t and 18 within the 0-50 m error interv al attests the prow ess of the BSS-Lasso in characterizing the assem bled exp erimen tal fib er links. Figure 7: Distribution of position estimation errors for all the experimentally measured fib er profiles. Apart from a few outliers, the p ositions are within a 100 meters error interv al from the actual real fault p ositions. In teresting observ ations can b e made from the results of Fig. 7. First of all, it is clear that the estimation error tends to b e closer to zero for single-fib er links. As will also be discussed in Section 5, the estimation is hindered by the amount of even ts present in the fib er link. Secondly , it is noticeable that the presence of reflective ev en ts induces a better estimation of the even t’s p ositions, as all but one of them fell within a 50 m error ev en with a higher num b er of even ts (tw o non-reflective ev ents and one reflectiv e ev ent). This characteristic, which will also b e studied in Section 5, indicates that the mo deling of reflective ev ents paired with the correction stage increases the robustness of the estimation. Up to this point, accurate determination of the faults positions has b een demonstrated using the BSS- Lasso. How ev er, determination of the fault magnitudes is of utmost imp ortance for full characterization of the optical fib er link, and can b e performed following Eq. (2). Nevertheless, even though the co efficients φ b , determined as a product of the BSS-Lasso, allo w for accurate estimation of the BSS frequency-domain profile, they include the bias naturally induced by the Lasso [17]. Moreo ver, the relationship b etw een the φ b and the actual fault magnitudes ξ b (Eq. (2)) is extremely non-linear, so even tual deviations in the former induce sev erely imprecise results for the latter. T o ensure correct estimation of the fault magnitudes, a r e c onstruction procedure of the time-domain profile of the fib er, that do es not dep end on the amplitudes determined by the BSS-Lasso, is prop osed. It is as follows: (i) utilizing the ev en t p osition estimates, fiber link profiles are created in the form of Eq. (1) for whic h the resp ectiv e non-reflective magnitudes ξ b are sorted within a predetermined in terv al of [0, 5] dB in a sequence of tw o steps, a coarser 0.5 dB, and then a finer 0.1 dB; (ii) the reflective 19 magnitudes θ r are sorted within a predetermined interv al of [0, 20] dB in steps of 2 dB; (iii) from the created P ( z ), frequency-domain profiles S ( f ) are calculated using Eq. (3) and then compared to the estimated frequency-domain profile given by the BSS-Lasso; (iv) the created S ( f ) are compared with the BSS-Lasso estimated profile using the ` 2 error norm; and (v) the b est reconstructed fib er profile according to this metric is defined as the reconstructed fib er profile. Presented in T able 2, for four exp erimen tal fib er links, are: the real fault magnitudes, calculated using the OTDR profile measured by a standard OTDR device; the magnitudes calculated using Eq. (2) and the BSS-Lasso co efficien ts; and the magnitudes determined through the reconstruction pro cedure. T able 2: F ault Magnitude Comparison Real Calculated Reconstructed Magnitudes [dB] Magnitudes [dB] Magnitudes [dB] 1.9, 23.0 2.77, ∞ 1.6, 22.0 2.0, 20.7 1.79, ∞ 2.0, 22.0 2.9, 17.5 3.53, ∞ 3.0, 22.0 0.9, 22.2 1.92, ∞ 1.0, 22.0 In other words, the reconstruction pro cedure is equiv alen t to the second analysis path presen ted in Section 2.2, but using artificially created P ( z ) based on the results of the BSS-Lasso. The result, as can be p erceived from T able 2, is a muc h more accurate estimate of the fault magnitudes. Finally , an example of the reconstruction pro cedure for the first fib er link of T able 2, i.e., the artificially generated P ( z ) that translates in to S ( f ) that b est approximates the estimated frequency-domain signal given by the BSS-Lasso, is depicted in Fig. 8. 5 Sim ulation Results T able 3: Selection errors for eac h set of simulated links 1 fault 2 faults 3 faults Error [m] SincLasso BSS-1 BSS- Lasso SincLasso BSS-1 BSS- Lasso SincLasso BSS-1 BSS- Lasso [0, 50] 52.20% 52.20% 89.80% 49.60% 49.45% 81.50% 50.70% 50.77% 77.50% (50, 100] 1.60% 1.60% 1.60% 1.20% 1.45% 2.95% 2.60% 2.87% 4.03% (100, 200] 11.80% 11.90% 7.20% 4.40% 5.85% 5.65% 3.33% 4.93% 4.30% (200, ∞ ) 34.40% 34.30% 1.40% 44.80% 43.25% 9.90% 43.37% 41.43% 14.17% In order to illustrate the robustness of the proposed monitoring metho dology , in this section, extensiv e computational tests are conducted on the BSS-Lasso. T o do so, a large-scale test b ench w as created: a total of three sets of 1000 fib er links each containing, resp ectively , one, t w o, and three faults w ere randomly 20 Figure 8: Time-domain profile reconstruction based on the results of the BSS-Lasso. In the upp er panel, the estimated frequency-domain profile of the BSS-Lasso is plotted against the S ( f ) that corresp onded to the minim um ` 2 error norm during the reconstruction step. In the low er panel, the original OTDR profile acquired with a standard OTDR device is depicted along with the reconstructed profile. T able 4: Con tingency table ( ± 50 m) F ault Present F ault Absent SincLasso BSS-1 BSS-Lasso SincLasso BSS-1 BSS-Lasso F ault F ound T rue Positiv es T rue Positiv es T rue Positiv es F alse Positiv es F alse Positiv es F alse Positiv es 3035 3034 4853 2808 3251 1379 F ault Neglected F alse Negatives F alse Negatives F alse Negatives T rue Negatives T rue Negatives T rue Negatives 2945 2966 1147 2959557 2958128 2957685 Measures Sensitivity Sensitivity Sensitivity Specificity Sp ecificity Specificity 50.58% 50.57% 80.88% 99.91% 99.89% 99.95% generated based on the following steps: 1. Fib er lengths are sampled ev enly within L ∈ [2 , 15] km. 2. Given the fib er length sampled in Step 1, the fault lo cations are also sampled evenly within [2 , L ], with a mandatory fault at the end of the fib er. 3. In order to sample reflective ev en ts, a 50% chance of ha ving an asso ciated reflection is attributed to each fault sampled in Step 2. 4. The magnitudes of the even ts are then randomly c hosen, with faults ranging evenly b etw een [1 , 5] dB and reflections up to 20 dB. 21 5. The time-domain profile P ( z ) of the fiber link is constructed using Eq. (1) and the parameters sampled in Steps 1–4. 6. The methodology presen ted in Section 2 is utilized to obtain the frequency-domain profile S ( f ) from P ( z ), using Eq. (3) with a set of frequencies ranging within [100, 100000] Hz discretized in 100 Hz. F or repro ducibility purposes, the complete test b ench of fiber links used in this section is av ailable in [19]. All tests were conducted in Julia language with an In tel Core i7-490K CPU at 4.00 GHz and 32 GB of RAM memory . The sim ulation parameters used throughout the testb ench were: 10 m length discretiza- tion; reduced penalty factor γ = 0 . 5; and sensitivit y threshold  = 0 . 05. F or the sake of comparison, the p erformance of the BSS-based monitoring technology presen ted in [10], known as SincL asso , w as also ev aluated for the same test b ench. F urthermore, in order to quan tify the con tribution of the correction stage describ ed in Algorithm 3, the results of the BSS-Lasso up to the selection stage (named BSS-1 for presen tation purp oses) are also studied. The test results are presented in t wo separate tables, for more rigorous analysis. In T able 3, the errors are stratified among four distance in terv als and the resp ective percentage of position estimates within eac h in terv al is ev aluated for the three techniques. T able 4 presen ts the so-called contingency table. The idea is to p erform a binary classification of fault/no fault even ts based on the output of the tec hniques. More precisely , in the con tingency table, T rue Positiv es are accounted b y estimations within the lo w-error in terv al, i.e., when the selection error is 50 m or low er. Similarly , F alse Positiv es are estimations outside the lo w-error in terv al. It should b e highlighted that T able 4 w as constructed b y com bining the results of the three sets of fib er links, totaling 6000 ev en ts. The first conclusion based on the analysis of both tables is the clear dominance of the BSS-Lasso o ver the other tw o tec hniques; the BSS-Lasso achiev es o ver 80% of fault estimates within the low-error in terv al while the other tec hniques barely reac h 50%. A second observ ation is that the num b er of ev en ts clearly impacts the accuracy of the methodology , as it diminishes from 89.80% for the set of fib er links with a single even t to 77.50% for the set with three even ts. The increase in estimation difficulty as the n umber of ev ents gro ws larger is exp ected, as the metho d needs to distinguish more comp onents that are comp ounded in the signal. It is also striking that the selection stage of the BSS-Lasso has p erformance extremely similar to the SincLasso, indicating that the inclusion of the reflections in the dictionary of the Lasso is not sufficient to accurately detect reflectiv e even ts. F urthermore, the near 50% accuracy limit of the SincLasso shares a strong correlation with the definition of Step 3 in the test b ench creation proto col, since the probability of a reflectiv e ev en t w as set to 50%. This result translates the lac k of precision induced by a reflection and indicates that the BSS-Lasso not only deals with such reflectiv e even ts but uses their presence to optimize the estimation through the correction stage. The BSS-Lasso also excels when analyzing the contingency table, with a n umber of F alse Negatives and F alse Positiv es less than half of those of the other tec hniques. Even though a finer analysis of the 22 sp ecificit y 2 is hindered by the high num b er of p ositions that do not present a fault even t, the results are coheren t with the sparse nature of the problem itself. F urthermore, BSS-Lasso’s precision 3 of 77.87% indicates that a selection do es generally represent a true fault. This is extremely imp ortant from a practical point of view, as sc heduling an in-field unit to repair a non-existing fault can b e costly . At the same time, although the sensitivit y 4 measure is roughly 80%, analysis of the ev en ts which were not iden tified sho ws a t w o-fold b ehavior: either the fault magnitude was low and the BSS-Lasso neglects its presence; or the magnitude of an asso ciated reflective ev ent was lo w enough so that it w as not identified and the shift correction was not emplo yed, th us ensuing an error of ov er 50 m. In the first case, whic h corresp onds to the ma jority of errors, the small faults do not en tirely compromise the link’s op eration capacit y . In the second case, though represen ting a more serious defect in fault lo cation, the op erator w ould hav e knowledge of the presence of the fault but with reduced accuracy . 6 Conclusions Manipulation of the detected backscattered Rayleigh signal inside the bandwidth of a frequency sw ept optical sub-carrier propagating into an optical fib er p ermits an efficient lo calization of faults through a F ourier op erator. When the bandwidth is restricted, analysis in the frequency domain can o v ercome the spatial resolution limitation while also inducing a high-dimensional problem. In tro ducing the Lasso as a signal pro cessing tec hnique paired with the BSS framework, a metho dology to consisten tly ev aluate fib er defects, named BSS-Lasso, is designed. The BSS-Lasso presents sev eral desired prop erties in the context of optical fib er link characterization, as corrob orated by the results: (i) high specificity and sensitivit y for reflective and non-reflectiv e ev ents; (ii) successful detection of multiple faults within a low-error in terv al; and (iii) low computational burden, ha ving required less than t wo minutes for all links simulated and exp erimentally acquired in this work. Both exp erimental and sim ulation results show the efficacy of the ex-p ost analysis, or correction stage, whic h besides accurately iden tifying the reflectiv e ev ents, also makes use of their presence to increase the precision of the fault lo cations. F urthermore, the BSS-Lasso framework allows for the reconstruction of the time-domain profile of fibers and the estimation of the magnitude of faults. In conclusion, the BSS-Lasso allo ws for precise, low-cost, transmitter-em b edded full characterization of optical fib er links in practical computational time. W e highlight sev eral ideas that can b e explored in future works to impro v e the prop osed tec hnique. F rom a data-acquisition p oint of view: (i) the analysis of the impact of the optical carrier’s bandwidth and polarization on the measured frequency-domain profile; (ii) the extension of the frequency sweep range and whether this allo ws for increased estimation accuracy; and (iii) the employmen t of a lo w er NEP detector with the goal of extending the limited dynamic range of the BSS-Lasso. In the signal 2 Specificity = #T rue Negatives / (#T rue Negatives + #F alse Positiv es) 3 Precision = #T rue Positiv es / (#T rue Positiv es + #F alse Positiv es) 4 Sensitivity = #T rue Positiv es / (#T rue Positiv es + #F alse Negatives) 23 pro cessing part: (i) determining whether the dynamic range is limited by an imp ossibilit y of acquiring signals from distant portions of the fib er or by the sparse c haracteristic of the Lasso, which neglects signal contributions that are to o small; and (ii) identifying a more consisten t metho dology for the cluster analysis step. Finally , the p ossibility of em b edding the softw are into a micro-con trolled unit running Julia allied with a dedicated low-cost complex frequency beat detector w ould allow the creation of an indep enden t measurement device, suc h as a standard OTDR, and is also a viable future developmen t of this work. Ac knowledgemen ts The authors would like to thank Christiano Nascimen to and Breno P erlingeiro for technical supp ort. This study was financed in part by the Co ordena¸ c˜ ao de Ap erfei¸ coamen to de Pessoal de N ´ ıvel Sup erior - Brasil (CAPES) - Finance Co de 001. References [1] S. Kumar and M. J. Deen, Fib er optic c ommunic ations: fundamentals and applic ations . Wiley , 2014. [2] P . J. Urban, G. V all-Llosera, E. Medeiros, and S. Dahlfort, “Fib er plant manager: An OTDR-and OTM-based PON monitoring system,” IEEE Communic ations Magazine , vol. 51, no. 2, pp. S9–S15, 2013. [3] M. K. Barnoski, M. D. Rourk e, S. M. Jensen, and R. T. Melville, “Optical time domain reflectome- ter,” Applie d Optics , vol. 16, no. 9, pp. 2375–2379, Sept. 1977. [4] D. Derickson, Fib er Optic - Test and Me asur ement , 1st ed. Pren tice Hall, 1998. [5] F. Liu and C. J. Zaro wski, “Even ts in fib er optics given noisy OTDR data—Part I. GSR/MDL metho d,” IEEE T r ansactions on Instrumentation and Me asur ement , v ol. 50, no. 1, pp. 47–58, F eb. 2001. [6] P . Eraerds, M. Legr ´ e, J. Zhang, H. Zbinden, and N. Gisin, “Photon coun ting OTDR: adv antages and limitations,” Journal of Lightwave T e chnolo gy , vol. 28, no. 6, pp. 952–964, Mar. 2010. [7] S. Hangai and Y. T aki, “Detection of faults in short fib er by the phase comp ensated reflectometer,” IEEE T r ansactions on Instrumentation and Me asur ement , vol. 39, no. 1, pp. 238–241, F eb. 1990. [8] X. Dong, A. W ang, J. Zhang, H. Han, T. Zhao, X. Liu, and Y. W ang, “Combined attenuation and high-resolution fault measuremen ts using chaos-OTDR,” IEEE Photonics Journal , vol. 7, no. 6, Dec. 2015. 24 [9] G. C. Amaral, J. D. Garcia, L. E. Y. Herrera, G. P . T emp orao, P . J. Urban, and J. P . von der W eid, “Automatic fault detection in WDM-PON with tunable photon counting OTDR,” Journal of Lightwave T e chnolo gy , v ol. 33, no. 24, pp. 5025–5031, Dec. 2015. [10] G. C. Amaral, J. D. Garcia, B. F anzeres, P . J. Urban, and J. P . v on der W eid, “Multiple fib er fault lo cation with lo w-frequency sub-carrier tone sweep,” IEEE Photonics T e chnolo gy L etters , vol. 29, no. 13, pp. 1116–1119, Jul. 2017. [11] F. Calliari, L. E. Y. Herrera, J. P . v on der W eid, and G. C. Amaral, “High-dynamic and high- resolution automatic photon counting OTDR for optical fib er net w ork monitoring,” in Pr o c e e dings of the 6th International Confer enc e on Photonics, Optics and L aser T e chnolo gy - V olume 1: PHO- TOPTICS, , INSTICC. SciT ePress, 2018, pp. 82–90. [12] G. C. Amaral, A. Baldivieso, J. D. Garcia, D. C. Villafani, R. G. Leibel, L. E. Y. Herrera, P . J. Urban, and J. P . von der W eid, “A low-frequency tone sw eep metho d for in-service fault lo cation in sub-carrier multiplexed optical fib er netw orks,” Journal of Lightwave T e chnolo gy , vol. 35, no. 10, pp. 2017–2025, Ma y 2017. [13] A. Pizzinat, P . Chanclou, F. Saliou, and T. Diallo, “Things you should know about fronthaul,” Journal of Lightwave T e chnolo gy , v ol. 33, no. 5, pp. 1077–1083, Mar. 2015. [14] P . J. Urban, G. C. Amaral, and J. P . von der W eid, “Fib er monitoring using a sub-carrier band in a sub-carrier multiplexed radio-ov er-fiber transmission system for applications in analog mobile fron thaul,” Journal of Lightwave T e chnolo gy , vol. 34, no. 13, pp. 3118–3125, Jul. 2016. [15] J. Nak ay ama, K. Iizuk a, and J. Nielsen, “Optical fib er fault lo cator b y the step frequency metho d,” Applie d optics , v ol. 26, no. 3, pp. 440–443, 1987. [16] B. K. Natara jan, “Sparse approximate solutions to linear systems,” SIAM Journal on Computing , v ol. 24, no. 2, pp. 227–234, 1995. [17] R. Tibshirani, “Regression shrink age and selection via the Lasso,” Journal of the R oyal Statistic al So ciety. Series B (Metho dolo gic al) , v ol. 58, no. 1, pp. 267–288, 1996. [18] M. P arrilla, J. Ana ya, and C. F ritsc h, “Digital signal pro cessing tec hniques for high accuracy ul- trasonic range measurements,” IEEE T r ansactions on Instrumentation and Me asur ement , vol. 40, no. 4, pp. 759–763, Aug. 1991. [19] R. Saa v edra, P . T ov ar, G. C. Amaral, and B. F anzeres, “Optical fiber link test b ench.” [Online]. Av ailable: http://dx.doi.org/10.21227/H2N66K [20] S. Liehr, N. N¨ other, and K. Krebb er, “Incoheren t optical frequency domain reflectometry and dis- tributed strain detection in p olymer optical fib ers,” Me asur ement Scienc e and T e chnolo gy , vol. 21, no. 1, pp. 1–4, Nov. 2010. 25 [21] S. G. Pierce, A. MacLean, and B. Culshaw, “Optical frequency-domain reflectometry for microb end sensor demo dulation,” Applie d optics , vol. 39, no. 25, 2000. [22] J. P . von der W eid, M. H. Souto, J. D. Garcia, and G. C. Amaral, “Adaptive filter for automatic iden tification of m ultiple faults in a noisy OTDR profile,” Journal of Lightwave T e chnolo gy , vol. 34, no. 14, pp. 3418–3424, Jul. 2016. [23] T. Hastie, R. Tibshirani, and J. F riedman, The elements of statistic al le arning: data mining, infer- enc e, and pr e diction. , 2nd ed. Springer, 2009. [24] F. P eres and F. F ogliatto, “V ariable selection methods in m ultiv ariate statistical process control: A systematic literature review,” Computers & Industrial Engine ering , vol. 115, no. 1, pp. 603–619, Jan. 2018. [25] A. F. M. Smith and D. J. Spiegelhalter, “Bay es factors and c hoice criteria for linear mo dels,” Journal of the R oyal Statistic al So ciety. Series B (Metho dolo gic al) , v ol. 2, no. 42, pp. 213–220, 1980. [26] J. Chen and Z. Chen, “Extended BIC for small-n-large-p sparse GLM,” Statistic a Sinic a , vol. 2, no. 22, pp. 555–574, 2012. [27] J. F riedman, T. Hastie, and R. Tibshirani, “glmnet: Lasso and elastic-net regularized generalized linear mo dels,” R p ackage version , vol. 1, no. 4, 2009. [28] H. Zou, “The adaptiv e lasso and its oracle prop erties,” Journal of the Americ an Statistic al Asso ci- ation , vol. 101, no. 476, pp. 1418–1429, Jan. 2006. [29] P . B ¨ uhlmann, P . R ¨ utimann, S. v an de Geer, and C.-H. Zhang, “Correlated v ariables in regression: Clustering and sparse estimation,” Journal of Statistic al Planning and Infer enc e , vol. 11, no. 143, pp. 1835–1858, No v. 2013. [30] D. Bertsimas, A. King, and R. Mazumder, “Best subset selection via a mo dern optimization lens,” The A nnals of Statistics , vol. 44, no. 2, pp. 813–852, Apr. 2016. [31] D. Bertsimas and A. King, “OR Forum – An algorithmic approach to linear regression,” Op er ations R ese ar ch , vol. 64, no. 1, pp. 2–16, Jan.–F eb. 2016. [32] 10-MHz A djustable Balanc e d Photor e c eivers , New F o cus. [Online]. Av ailable: h ttps://www.newport. com/p/2117- FC [33] E5061B Network A nalyzer , Keysight T ec hnologies. [Online]. Av ailable: h ttps://literature.cdn. k eysight.com/lit web/pdf/5990- 4392EN.p df ?id=1790097 26