Variational Shape Completion for Virtual Planning of Jaw Reconstructive Surgery

V ariational Shap e Completion for Virtual Planning of Ja w Reconstructiv e Surgery Amir H. Ab di, Mehran P esteie, Eitan Prisman, Purang Ab olmaesumi, and Sidney F els Univ ersity of British Columbia, V ancouver, Canada { amirabdi,mehranp,purang,ssfels } @ece.ubc.ca, eitan.prisman@ubc.ca Abstract. The premorbid geometry of the mandible is of significan t relev ance in jaw reconstructiv e surgeries and o ccasionally unknown to the surgical team. In this paper, an optimization framework is intro- duced to train deep mo dels for completion (reconstruction) of the miss- ing segments of the b one based on the remaining healthy structure. T o lev erage the contextual information of the surroundings of the dissected region, the v o xel-weigh ted Dice loss is introduced. T o address the non- deterministic nature of the shap e completion problem, we leverage a w eighted m ulti-target probabilistic solution which is an extension to the conditional v ariational autoenco der (CV AE). This approac h considers m ultiple targets as acceptable reconstructions, each weigh ted according to their conformit y with the original shape. W e quan tify the performance gain of the proposed metho d against similar algorithms, including CV AE, where we rep ort statistically significant improv ements in b oth determin- istic and probabilistic paradigms. The probabilistic mo del is also ev al- uated on its abilit y to generate anatomically relev ant v ariations for the missing b one. As a unique asp ect of this w ork, the mo del is tested on real surgical cases where the clinical relev ancy of its reconstructions and their compliance with surgeon’s virtual plan are demonstrated as neces- sary steps tow ards clinical adoption. Keyw ords: Conditional V ariational Auto-encoder · 3D Shape Comple- tion · V-Net · Mandible Reconstruction 1 In tro duction Head and neck cancer comprises of a set of malignant tumors in the upp er respiratory tract which constitutes 3-4% of cancer cases in North America [9]. Surgery is the first line of treatmen t for the ma jority of these cases. Despite the adv ances in tools and techniques, mandibular reconstruction after segmen tal mandibulectom y is still a c hallenging pro cedure. Moreov er, due to the mandible’s vital role in mastication, sp eec h, and sw allowing, the functional and aesthetic requiremen ts of mandibular reconstructions are quite high. The v ascularized free fibula flap is the most utilized technique for mandibu- lar reconstruction [4] where the linear shap e of the fibula b one is dissected, 2 Ab di and P esteie et al. con toured, and mo delled to complete the curved geometry of the ablated b one. In the past decade, three-dimensional (3D) virtual surgical planning (VSP) has gained traction. In the VSP-enhanced free fibula flap, the mo delling and shaping of the fibula are virtually planned based on the pre-op erativ e records. One of the challenges in mandibular reconstruction is the unkno wn pre- inciden t shape of the missing (distorted) bony elements. In cases where the shape of the target b on y element is unknown, three strategies are tak en to an ticipate the original anatom y [11]. In rare cases where previous medical records of the craniofacial sk eleton is av ailable, patien t’s own b one morphology serves as the reference. F or unilateral defects, the unin v olved contralateral side is mirrored and man ually p ositioned to create an estimated reconstruction reference. Ho wev er, this approach b ecomes less reliable for defects with anteromedial extensions and is inapplicable in midline crossing lesions. Standard anatomic templates are sel- dom utilized to fill the missing b ony elements after sub jet-sp ecific adjustmen ts. Related W ork Shap e completion is an ill-p osed inv erse problem in computer vi- sion and graphics. Data-driv en models, including deep neural net works, are more in triguing as they directly learn the completion under sup ervision particularly b ecause of the non-deterministic ground-truth and multiple acceptable solutions for the reconstruction problem. In the realm of 3D generative deep mo dels, ap- plicabilit y of adversarial training (GAN) in 3D shap e and p oint cloud syn theses are in vestigated [2,13]. V aritional auto encoders (V AE) ha ve also b een able to learn seman tically meaningful latent spaces for realistic completions [6]. The closest research to our work is a recen t study where they empirically prov ed the feasibilit y of generating mandible shapes from a predefined set of landmarks [1]. Con tributions W e introduce optimization approaches to train deep con vo- lutional mo dels to reconstruct the missing b one segment from the remaining health y mandible. Our contributions are three-fold. First, w e design a frame- w ork to randomly generate samples for training of shap e completion mo dels in deterministic and probabilistic paradigms. W e in tro duce the V o xel-weigh ted Dice loss that prioritizes the target (remo ved) region o ver the rest of the geometry while ensuring consistency in reconstruction by leveraging the contextual infor- mation from the rest of the shape. Our main contribution is the T arget-w eighted v ariational ob jective function whic h addresses the one-to-many reality of shap e reconstruction. W e rep ort the quantified gain in performance and demonstrate the mo del’s abilit y to predict v ariations of the missing b one. Moreo ver, as a unique asp ect of this work, qualitativ e results on real surgical cases are provided as a notion of clinical relev ancy and compliance with surgeon’s virtual plans. 2 Metho d 2.1 Net work Arc hitecture The input ( X ) to the deep mo del is a 3D binary vo xel-grid of size c 3 con taining a mandible with a missing (dissected) contiguous segment. The output of the V ariational Shap e Completion 3 mo del is a 3D probability map ( ˆ Y ) of size c 3 where the predicted volume is the union of all v oxels with Y ij k > 0 . 5. Architecture of the learning system is summa- rized in Fig.1. W e inv estigate tw o reconstruction approaches, dete rministic and probabilistic. In heart of b oth lies a V-Net [5] consisting of four do wn-transition and four up-transitions with skip connections to forward feature maps from the encoding to the deco ding stream. Each down-transition consists of strided 3D conv olutions and batch-normalization. Similarly , the up-transitions contain transp osed strided 3D con volution (deconv olution) and batch-normalization. ELU activ ation la yers are used throughout the V-Net. Activ ations of the main stream of the last tw o down-transitions and the first tw o up-transitions are dropp ed out with a probability of 0.5. The final up-transition generates four 3D feature maps of size c 3 , same size as the input and the target o ccupancy maps. More details of the arc hitecture are av ailable in the supplementary material. In deterministic shape reconstruction, the final four feature maps are con- v olved with P g en k ernels of size 2 follow ed by a sigmoid function. In the prob- abilistic approach, the four feature maps of the V-Net are first concatenated with the tiled latent v alues (see Section 2.2) and conv olv ed with 8 kernels in tw o consecutiv e lay ers of P comb prior to P g en (Fig. 1). Fig. 1. Arc hitecture of the deterministic and weigh ted multi-target probabilistic learn- ing framework for anatom y completion with 3D conv olutional mo dels. 2.2 Loss F unctions During the mini-batc h gradien t decent optimization, each training sample is randomly dissected by a cub oid ( B ) with an arbitrary size and orien tation, whic h forms a binary occupancy map inside a 3D grid of size c 3 . The cub oid cuts the mandible to create input and target shap es as follows, X = S ◦ B 0 and Y = S ◦ B , (1) 4 Ab di and P esteie et al. where P ◦ Q denotes the Hadamard pro duct of the tensors P and Q , and B 0 is the complemen t of the binary cuboid. X and Y are the input and target (Fig. 2). V o xel-w eighted Dice This ob jective function enables the model to leverage the con textual information from the surroundings of the dissected region. Hence, v oxels of the predicted 3D probability map are weigh ted adversely by their dis- tance from the remo ved b on y segment. T o do so, a 3D normal distribution, N w ( ¯ Y , σ 2 w I ), with a diagonal co v ariance matrix is instan tiated at the cen ter of the target segmen t ( ¯ Y ). The 3D weigh t matrix W of size c 3 is then defined as W ij k = ( 1 B ij k = 1 (2 π σ w ) 3 2 N w ( i, j, k ) B ij k = 0 . (2) Eq. 2 prioritizes the v oxels inside the volume of the dissection cub oid ( B ) and ignores distan t vo xels. Here, σ w is set to c/ 3 to encourage smo other contours. Based on the w eight matrix, W , the prop osed V o xel-weigh ted Dice loss is L VWDice ( X, Y , ˆ Y , W ) = 1 − 2 ∗ P ij k  ˆ Y ◦ ( Y + X ) ◦ W  ij k P ij k  ( ˆ Y + Y + X ) ◦ W  ij k . (3) The proposed V o xel-weigh ted Dice prioritizes the dissected region for shap e com- pletion and main tains consistency b et ween the syn thesized structure ( ˆ Y ) and the remaining anatom y ( X ). T arget-w eigh ted Loss Shap e completion in the anatomical domain is in- heren tly a non-deterministic pro cess, i.e. , there is no single ground-truth that completes the dissected input anatom y . Therefore, our ob jective is to learn the v ariations of the remov ed b ony segmen t from a dataset of mandibles and generate m ultiple solutions to reconstruct a giv en dissection. In the proposed probabilistic learning paradigm, training samples are randomly dissected, on- the-fly , and the remov ed segment is considered the b est known reconstruction target, referred to as Y 0 , hereafter. In the prop osed multi-target approach, m random training samples are selected and dissected at the same lo cation using the same cuboid and registered on Y 0 to create the set of p ossible reconstructions, Y =  Y 0 , Y 1 , ..., Y m  . Among Y , only Y 0 p erfectly completes X while others do not necessarily match the input. Therefore, the metric Λ ij ( Y ) is defined as the degree of geometrical conformit y b et ween members i and j of the set Y . All target nominees ( Y ) are then indep enden tly concatenated with X and mapp ed to the laten t p osition µ i , with uncertaint y σ i , via the p osterior enco der net work ( P post ). The resultant mappings are considered as parameters of an indep enden t m ultiv ariate normal distribution ( N ( µ i , σ i )). A sample from this distribution is tiled and concatenated with the feature maps of the V-Net. The resultan t 4D tensor is pro cessed b y P comb and P g en to predict the target: ˆ Y i = P g en ( P comb ( V N et ( X ) , µ i )) , P post ( . | X, Y i ) = N ( µ i , σ i ) . (4) V ariational Shap e Completion 5 This pro cess is rep eated for all m + 1 nominee targets in the Y set of sample S . Eac h predicted shap e ( ˆ Y i ) is compared with its corresp onding target ( Y i ) using the V o xel-weigh ted Dice. Since target nominees, other than Y 0 , inv olve deviations from the input morphology ( X ), each target’s conformity ( Λ i 0 ) with the b est known solution ( Y 0 ) is taken into account in the prop osed T arget- w eighted (TW) loss function. As a result, the final ob jectiv e function is the w eighted av erage of all loss functions with resp ect to the conformities of their targets. The prop osed ob jectiv e is formulated as follows: L VWDice-TW = α m X i =0 Λ 0 i ( Y ) L i VWDice ( X, Y i , ˆ Y i , W i ) + γ K L ( P post ||N (0 , I )) , (5) where, α is a normalizing parameter set as the sum of all m conformity v alues, and γ is a w eighting constan t. As shown in Eq. 5, during optimization, the Kullbac k–Leibler (KL) divergence of the p osterior latent distribution with a fixed normal distribution, from which we sample during inference, is minimized. The L VWDice-TW loss function considers all targets as partially acceptable solutions for the probabilistic completion. In our experiments, Λ ( . ) w as set as the Dice co efficien t b et w een the shap es, where clearly Λ 00 ( Y ) = D ice ( Y 0 , Y 0 ) = 1. 3 Exp erimen ts 3.1 Data and T raining A total of 117 surface meshes of health y mandibles were collected from three publicly av ailable sources [7,12,14]. Some mandibles from the archiv es of an anon ymized center were also included through data sharing agreemen ts. The 3D meshes were rigidly registered based on their point clouds using the group-wise studen t’s-t mixture mo del algorithm with 50 mixture comp onents [8]. Using ra y testing, each surface mesh w as vo xelized into an isotropic binary o c- cupancy vo xel map with 1 mm increments to mimic mandibles segmented from CT scans with isotropic vo xels. The o ccupancy maps of all mandibles were sym- metrically zero padded to create vo xel-grid cub es of size 141 3 , i.e. size of the largest sample, whic h also matches the maximum facial width rep orted in the comprehensiv e dataset of the F aceBase pro ject [3]. The dataset was randomly partitioned in to test (15%), training (70%), and v alidation (15%) sets. During the mini-batch gradien t decent optimization, eac h sample w as ran- domly rotated, translated (shifted), and mirrored across the sagittal plane. Adam optimizer was used with default momentum parameters along with ` 2 regular- ization of 1 e − 5. The learning rate w as initialized at 1 e − 2 and exponentially deca yed with a rate of 0.98 at each ep och un til con vergence. Size of the laten t space w as set to 8. Same random seeds were used for all the experiments The data pro cessing pipeline and the models were implemen ted using the PyT orc h deep learning platform and made publicly av ailable: github.com/amir- ab di/prob-shape-completion . F or the exp erimen ts to b e repro ducable, the v ox- elized v ersion of the data accompanies the code, according to each dataset’s resp ectiv e license and data sharing agreemen ts. 6 Ab di and P esteie et al. T able 1. Quantitativ e comparison of the prop osed metho ds against other baselines. Metho d DSC% Comp Acc HD95 Determ. L Dice ( X + Y , ˆ Y ) 0 . 4 N/A N/A N/A L Dice ( Y , ˆ Y ) 84 . 9 ± 0 . 2 0 . 85 ± 0 . 13 0 . 61 ± 0 . 13 2 . 95 ± 1 . 43 L VWDice ( X, Y , ˆ Y ) (ours) 88 . 3 ± 0 . 2 0 . 65 ± 0 . 10 0 . 44 ± 0 . 10 2 . 64 ± 1 . 83 Prob. CV AE basic [10] 79 . 8 ± 0 . 6 1 . 20 ± 0 . 34 0 . 87 ± 0 . 18 3 . 98 ± 3 . 16 CV AE VWDice-TW (ours) 80 . 8 ± 0 . 5 1 . 11 ± 0 . 31 0 . 83 ± 0 . 14 3 . 74 ± 2 . 44 3.2 Ev aluation and Results Shap e completion is a non-deterministic pro cess where no single answer is the ground-truth. Ho wev er, to quan tify the p erformance of the prop osed learning metho ds, a den tist man ually remo ved b one segmen ts from the healthy mandibles of the test set and created nearly 100 test cases. The manually remo ved b one segmen ts were considered as fair targets for ev aluation and compared with the predicted reconstructions. W e assessed the mo dels based on Dice co efficien t (DSC; 2 × in tersection / sum of volumes), completeness (Comp; av erage distance from target surface to predicted surface), accuracy (Acc; av erage distance from predicted surface to target surface), and Hausdorff distance at the 95th per- cen tile (HD95). Except for the DSC, low er v alues of the metrics are preferred. F or a fair comparison of non-deterministic CV AE mo dels, and with only a sin- gle target reconstruction a v ailable, laten t v alues during the quan titative analysis w ere set to the mean of the fixed distribution N (0 , I ). The quantitativ e results are rep orted in T able 1 and a set of reconstructed samples from the test set are visualized in Fig. 3. As rep orted in the top row of T able 1, a v anilla Dice loss with the entire shap e as its target ( i.e. L Dice ( X + Y , ˆ Y )) equally treats all the vo xels and ignores the dissected region. This mo del acts like an auto enco der which only regenerates the input ( X ). On the contrary , fo cusing only on the dissected target b one ( i.e. L Dice ( Y , ˆ Y )) do es not p enalize the discrepancies in the margins of the reconstruction. Therefore, the predicted shap e shows inconsistencies and discon tinuities with resp ect to the surrounding anatom y . The p erformance gain ac hieved with the prop osed ob jective function (Eq. 3) w as assessed to b e statistically highly significant ( p < 0 . 001). T o demonstrate the effectiv eness of the prop osed con tributions, t wo CV AE mo dels were trained: one without any enhancements (CV AE basic ), and one with the ob jectiv es function describ ed in Eq. 3 and 5 (CV AE VWDice-TW ). Net work arc hitectures and h yp er-parameters were kept the same across exp erimen ts for results to b e comparable. The differences betw een the Dice metrics of the tw o mo dels was observed to b e statistically highly significant ( p < 0 . 001). Thanks to the prop osed T arget-w eighted v ariational ob jective, a strong neg- ativ e correlation was observ ed b et ween the deviation of a latent v ector from its distribution’s mode ( | z − ˆ P post | ) and the similarit y of its corresponding predicted shap e with the main target ( Y 0 ). This phenomenon is demonstrated in Fig. 2. The same w as not true for the CV AE basic mo del. V ariational Shap e Completion 7 Fig. 2. Negative correlation b et ween the deviation of the latent v alues from the mo de ( ˆ P post ) and the conformity of the predicted shap es with the main target ( Y 0 ). Fig. 3. Reconstructed samples of the test set along with their Dice metric when com- pared with the original anatom y . Surgical Cases W e ev aluated the performance of the v ariational mo del on real surgical cases who were treated with the virtually planned free fibula flap tec hnique. Here, the affected mandibular bones w ere already remov ed b y the clin- icians. The dissected mandibles were reconstructed using the trained v ariational mo del. An exp ert assessed the reconstructions against surgeon’s virtual plans of the free fibula flap surgery and found them clinically acceptable. Figure 4 visu- alizes some of these surgical cases b y sup erimposing the predicted mandibular b one (green) on the fibular segments. 4 Discussion and Conclusions In this pap er, we introduced optimization approaches for training of deep v aria- tional mo dels f or anatomical 3D shape completion. The proposed V o xel-w eighted 8 Ab di and P esteie et al. Fig. 4. Comparison of mo del predictions (green) with virtual surgical plans (VSPs). ob jective improv es the accuracy of reconstructions compared to similar ap- proac hes and guarantees smo othness b etw een the predicted b ony segmen t and the remaining contours of the mandible. The prop osed v ariational metho d takes in to accoun t the man y acceptable solutions for the shape completion problem and is able to generate realistic v ariations for the missing segment. Among the limitations of our study is the inconsistency in the presence of teeth across the samples. While there were to othless mandibles in the dataset, the ma jority of samples had all or parts of their dentition. How ever, except for their correlation with the b one-loss, teeth hav e little to no role to play in determining the missing anatom y of mandible. Therefore, the presence of dentition in the target dissections adv ersely affected the p erformance. T o mitigate this issue teeth should b e excluded from the data, either manually or automatically . The prop osed probabilistic approach is among the first works in deep anatom- ical shap e reconstruction. It can be applied to other anatomies as well as general computer graphics. Comparison with real surgical cases is demonstrated here as a step for clinical adoption. Our metho d requires no p ost pro cessing, except in con verting vo xel-grids to and from surface meshes. Therefore, our next log- ical step is to switch from v oxel-grids to graph representations to sp eedup the pro cessing pip eline for b etter clinical acceptance. References 1. Ab di, A.H., et al.: AnatomyGen: Deep Anatomy Generation F rom Dense Repre- sen tation With Applications in Mandible Synthesis. T ech. rep. (2019) 2. Ac hlioptas, P ., et al.: Representation learning and adversarial generation of 3d p oin t clouds. arXiv preprint arXiv:1707.02392 2 (3), 4 (2017) V ariational Shap e Completion 9 3. Brinkley , J.F., et al.: The F aceBase Consortium: a comprehensive resource for craniofacial researchers. Developmen t 143 (14), 2677–88 (2016), www.facebase.org 4. Hidalgo, D.A.: Fibula free flap: A new metho d of mandible reconstruction. Plastic and Reconstructive Surgery 84 (1) (1989) 5. Milletari, F., et al.: V-net: F ully con volutional neural net w orks for v olumetric med- ical image segmen tation. In: 2016 F ourth International Conference on 3D Vision (3D V). IEEE (2016) 6. Nash, C., Williams, C.K.I.: The shap e v ariational auto encoder: A deep generativ e mo del of part-segmen ted 3d ob jects. Computer Graphics F orum 36 (5), 1–12 (2017) 7. Raudasc hl, P .F., et al.: Ev aluation of segmentation methods on head and neck CT: Auto-segmen tation c hallenge 2015. Medical Physics 44 (5), 2020–2036 (2017) 8. Ra vikumar, N., et al.: A multi-resolution T-mixture model approach to robust group-wise alignment of shapes. In: Medical Image Computing and Computer- Assisted Interv ention (MICCAI 2016). vol. 9902 LNCS, pp. 142–149. Springer, Cham (2016) 9. Siegel, R.L., et al.: Cancer Statistics , 2017 67 (1), 7–30 (2017) 10. Sohn, K., et al.: Learning structured output representation using deep conditional generativ e mo dels. In: Cortes, C., et al. (eds.) Adv ances in Neural Information Pro cessing Systems 28, pp. 3483–3491. Curran Asso ciates, Inc. (2015) 11. Stranix, J.T., et al.: A virtual surgical planning algorithm for delay ed maxillo- mandibular reconstruction. Plastic and Reconstructiv e Surgery p. 1 (2019) 12. W allner, J., Egger, J.: Mandibular ct dataset collection (2018) 13. W u, J., et al.: Learning a probabilistic latent space of ob ject shapes via 3d generativ e-adversarial mo deling. In: Adv ances in neural information pro cessing systems. pp. 82–90 (2016) 14. Zuley , M.L., et al.: Radiology data from the cancer genome atlas head-neck squa- mous cell carcinoma collection (2016) C o n v 3 D B a t c h N o r m 3 D i n p u t C o n v 3 D + i n p u t o u t p u t C o n v 3 D B a t c h N o r m 3 D B a t c h N o r m 3 D P ge n D o w n T r a n s i t i o n T r a n s p o s ed C o n v 3 D + i n p u t o u t p u t C o n v 3 D B a t c h N o r m 3 D B a t c h N o r m 3 D U p T r a n s i t i o n A D r o p o u t D r o p o u t S k i p C o n n ec t i o n D r o p o u t C o n c a t e n a t e T r a n s p o s ed C o n v 3 D + i n p u t o u t p u t C o n v 3 D B a t c h N o r m 3 D B a t c h N o r m 3 D U p T r a n s i t i o n B D r o p o u t S k i p C o n n ec t i o n D r o p o u t C o n c a t en a t e C o n v 3 D B a t c h N o r m 3 D C o n v 3 D B a t c h N o r m 3 D i n p u t I n p u t T r a n s i t i o n + R ep ea t C h a n n el o u t p u t C o n v 3 D B a t c h N o r m 3 D S i g m o i d o u t p u t C o n v 3 D i n p u t P Co m b C o n v 3 D o u t p u t X P p o s t o u t p u t C o n v 3 D Y C o n c a t en a t e C o n v 3 D C o n v 3 D C o n v 3 D C o n v 3 D C o n v 3 D S u p p F i g u r e 1 . D et a i l s o f n eu r a l n et w o r k s . A ) A r c h i t ec t u r e o f n et w o r k s B ) In p u t , o u t p u t a n d In t er m ed i a t e t en s o r s o f n et w o r k s ( A ) B l o c k S h a p e o f La y e r Ou tp u t In p u t 1 × 141 × 141 × 141 In p u t Tran sitio n 2 × 141 × 141 × 141 D o wn T ran si tio n 1 4 × 70 × 70 × 70 D o wn T ran si tio n 2 8 × 35 × 35 × 35 D o wn T ran si tio n 3 16 × 17 × 17 × 17 D o wn T ran si tio n 4 32 × 8 × 8 × 8 Up T ran siti o n B1 32 × 17 × 17 × 17 Up T ran siti o n B2 16 × 35 × 35 × 35 Up T ran siti o n A1 8 × 71 × 71 × 71 Up T ran siti o n A2 4 × 141 × 141 × 141 In term e d iat e t e n s o r s o f th e V - N e t w ith its s kip co n n e cti o n s La y e r S h a p e o f La y e r Ou tp u t In p u t 12 × 141 × 141 × 141 Co n v 1 8 × 141 × 141 × 141 Co n v 2 8 × 141 × 141 × 141 I n term e d iat e t e n s o rs o f t h e P C o mb Block La y e r S h a p e o f La y e r Ou tp u t In p u t 4 | 8 × 141 × 141 × 141 Co n v 1 2 × 142 × 142 × 142 Co n v 2 1 × 141 × 141 × 141 In term e d iat e t e n s o r s o f t h e P g en in t h e p ro b ab ili s tic (8 in p u t ch an n e ls ) a n d d e term in is tic ( 4 in p u t ch an n e ls ) p at h s La y e r S h a p e o f La y e r Ou tp u t In p u t 2 × 141 × 141 × 141 Co n v 1 2 × 70 × 70 × 70 Co n v 2 4 × 34 × 34 × 34 Co n v 3 4 × 16 × 16 × 16 Co n v 4 8 × 7 × 7 × 7 Co n v 5 8 × 3 × 3 × 3 Co n v 6 8 × 1 × 1 × 1 I n term e d iat e t e n s o rs o f t h e P p o s t Block ( B ) Su p p Figu r e 1 (c on t i n u ed ) . D et ails of n e u ral n e t w or k s. A) A rch it ec t u re o f n et w or k s B ) In p u t , ou t p u t a n d I n t e rme d i at e t e n sor s o f n et w or k s