AI-Skin : Skin Disease Recognition based on Self-learning and Wide Data Collection through a Closed Loop Framework

AI-Skin : Skin Disease Recogniti on based on Self-lea rning and Wide Data C ollect ion thro ugh a Closed Lo op F ramework Min Chen a, ∗ , Ping Zhou a , Di W u b,c, ∗ , Long Hu a, ∗ , Mohamma d Mehedi Hassan d , A tif Alamr i e a Scho ol of Compu ter Scienc e and T e chnolo gy , Huazhong University of Science and T e chnolo gy , Wuhan, China b Scho ol of Data and Computer Scienc e, Sun Y at-sen University, Guangzhou, China c Guangdong Pr ovinc e Key La b or atory of Big Data Anal ysis and Pr o c essing, Guangzhou, China d Chair of Pervasive and Mobile Computing and Information Systems D ep artment, Col le ge of Computer and Information Sciences, King Saud Univ ersity, Riyadh 11543, Saudi Ar abia e Chair of Pervasive and Mobile Computing and Softwar e Engine ering De p artment, CCIS, King Saud University, Riyadh 11543, Saudi Ar abia Abstract There are a lot of hidden danger s in the change o f hu man s k in co nditions, s uch as the sunb urn caused b y long - time exp osure to ultraviolet ra diation, whic h not o nly has aesthetic impact causing psychological depress io n and lack o f self- confidence, but a lso may even b e life-threatening due to skin cancera tion. Cur- rent skin disease researches adopt the auto-cla s sification system for improving the accuracy r ate of s k in dis ease classification. How ever, the exce ssive dep en- dence on the image sample database is unable to provide individualized diagno sis service for differen t p opulation groups. T o ov e r come this problem, a medical AI framework based on data width evolution and self-le a rning is put for ward in this pap er to provide skin disease medical servic e meeting the requirement of real time, extendibility and individualiza tion. First, the wide co llection of data in the c lo se-lo op information flow of user and remo te medical data center is dis- cussed. Next, a data s et filter a lgorithm ba sed on infor mation ent ropy is given, ∗ Corresp onding authors: Min Chen, Di W u, Long Hu Email addr esses: minc hen2012@ hust.edu .cn (Min C hen), pingzhou.cs @qq.com (Ping Zhou), wudi27@sysu. edu.cn (Di W u), hulong@hus t.edu.cn (Long Hu), mmhassan @ksu.edu .sa (Mohammad Mehedi Hassan), atif@ksu.edu .sa (At if Alamr i) Pr eprint submitte d to Information F usion June 6, 2019 to lighten the load of edge no de and mea nwhile improv e the learning a bilit y of remote cloud analysis mo del. In addition, the framework provides a n externa l algorithm load mo dule, whic h can b e co mpatible with the application require- men ts accor ding to the mo del s elected. Three kinds o f deep lear ning mo del, i.e. LeNet-5, AlexNet and VGG16, are loaded and compar ed, which ha v e v erified the universalit y o f the algor ithm load module. The exp eriment platform for the pr op osed real-time, individualized and extensible skin disease reco g nition system is built. And the sy stem’s c omputation and communication delay under the interaction scena r io b etw een tester and r e mote data c ent er are analyzed. It is demonstrated that the sy stem we put forward is reliable and effective. Keywor ds: Skin Disease Recognition, Data Width Evolution, Self-learning Pro cess , Deep Learning Mo del 1. In tro duction A skin disea se is the pathologica l state a ffecting the b o dy surface. Long-time exp osure to ultraviolet radiation or the radia tion from high- frequency wire- less eq uipment may induce the skin cancer ation. According to the statistica l data r ep ort fr om American Cancer So ciety [1], it is estimated that 91 ,270 new melanoma case s ar e diagnosed in the United States in 2018, and mean while it is estimated tha t a b out 9,3 20 p eo ple will die from melanoma . Melano ma has a high c ure rate at e arly detection, with 99% of 5-year relative surviv al r a te. How- ever, since it is easier to spread to other parts of the b o dy than no n-melanoma skin cancers , the 5-year relative surviv al r ate at long -term stage dro ps to 2 0%. The symptom of skin disea ses is a long and constantly c ha nging pro cess. Generally , the hea lth care provider should pro vide as s essment where c hanges hav e o ccurred in certain ar ea of the skin for ov er a month or longer . Ho w ever, due to multiple fac to rs such as p o o r medical conditions a nd cum ber some medical pro cess, patient s a lwa ys ignore suc h changes o f their skin, or wrongly iden tify them as other skin injuries. Meanwhile, no family physician or super vision organiza tion ca rries o ut the re gular maintenance and treatmen t since it is not 2 required to r ep ort medical r ecords to the cancer r e g istry . The developmen t of b o dy senso r netw ork [2, 3], artificial intelligence, cloud computing [4] and wireless netw ork communication [5 , 6] has brought opp or tu- nities to the cognitive medical servic e [7, 8, 9]. The r emote hea lth monitoring, health guidance and feedback can b e rea lized through m ulti-sensor data fu- sion [10, 1 1] with the help of remote medical devices , such as mobile phone [12], wearable device [13], intelligen t r o b ot, autono mous vehicle and unmanned ae r ial vehicle [14, 15]. And an op en-sour ce progra mming framework to supp ort ra pid and flexible prototyping and management of human-cen tered applica tions is crit- ical [16]. In the pap er [1 7], it is p ointed out that mobile devices eq uipped with deep neur al netw ork can p o tent ially extend the ra nge of dermatologis ts o utside the outpa tient ser vice. It is estimated that by 202 3, the num ber o f smar t phone users will reach 7 .2 billion [18]. I t is p ossible to provide g eneral diagnosis servic e with low c o st [19]. The automatic r ecognition of patients’ skin conditions ma y bec o me a g o o d promoter for the c ognitive medical monito ring framework. On one side, it ca n reduce the consumption of reso ur ces deployed to the medica l industry center, and mea nwhile automatica lly feed back the pa tients’ co nditio ns a nd ser vice ex- per iences ev a lua tion. On the other side, it pr op erly takes into account the consumption of pa tient s’ time and money cost as w ell as the concerns on pri- v acy . The cloud computing technology deploy ed o n the medical center ca n solve the problem that the lo ca l devices, under the big data environment, have insuf- ficient computing and stora ge capacities to pro vide the co mputation-intensiv e services [20, 2 1, 22]. Ho w ev er, the huge amount of data trans mission and com- m unication will caus e a cons umption of netw o rk co mm unication resources , it is still unable to meet the delay-sensitive characteristic of cognitive medical ser- vices [23, 24, 2 5]. The deployment of edge computing tec hnology on the netw ork edge can s olve the pressure caused by the large scale of computation-in tensiv e and rich-media tasks [26, 27, 28]. Also, the tec hno logy is b eneficial to facil- itate sec ur e data ma nagement and conv enient da ta trading in mobile hea lth care [29]. Some existing adv anced computation offloading schemes such a s [3 0] 3 and nov el routing mechanisms suc h as [31] ca n be in tegrated into the system to achiev e highly reliable coo p erative computing and communication betw een lo cal terminals and remote clo uds [32]. The traditional skin disease detection system complete the classificatio n out- put through characteristics extraction o f image da ta set as the input. The ex- isting r e searches ado pt the deep ar chit ecture to automate the learning of char- acteristics [33, 34, 35], and the prio ri knowledge based on pathologica l s k in da ta set is obtained to improv e the accuracy of automatic classification. Estev a et al [17] put forward the adoption of Deep Con volutional Neural Netw o r ks to cla s- sify skin diseases, and demonstrate the ac hievemen t of expert-le vel diag nosis. In the literature [36], it is discussed that the pre-tr ained deep neur al net work mo del has a n effect supe r ior to the model trained from the b eginning, and the problem of insufficient lab eled image da ta of skin disea se can b e so lved by pr e- training the Conv olutional Neural Netw ork (CNN) with the images from o ther medical fields. Due to the limited intelligence o f c ur rent system, a o ne - time tes ting result be concluded by inputting the collected users’ s kin ima ge data in to the system, but the function of monitoring the changes of skin conditions c a nnot be rea liz ed. Meanwhile, curr ent system is a centralized system with a static and centralized database r e q uired an activ e update by exper t, which limits the user mo bility and c annot realize convenien t and high-efficiency self-checking. In addition, the centralized system is unable to provide sufficient resour ces to supp or t the indi- vidualized database for different po pulation groups. Due to the c entralization of the databa se, it is unable to g ive a g o o d judgment fo r paroxysmal dise ases. W e consider that the future skin dise ase monito ring system will meet follow- ing characteristics: • Real- time: The user’s individual da tabase keeps accumu lating and storing. The system ana ly zes user’s skin sta te ba s ed on pe rsonal histor ical data and current data, and monitors skin state c ha nges r egularly . The camera o n smart terminal capture user ’s skin ima ge, and skin ana lysis repo rts feed 4 back to terminals. Users reco r d their skin sta te changes based on rep or ts. • Dynamic: The physical lo cation of users o ften changes dynamically , but mobile devices are relatively static for skin disease detection. Through mobile terminals, users can easily and efficiently co llect individual skin images. With the computing p ower of termina ls, fast a na lysis res ults ar e provided to users. • Shar ing mo de: User’s skin images can b e lo cally stor e d for a nalysis. Mul- tiple user s also send their data to the cloud fo r shar ing. The cloud co llects data from different users, and conducts mor e ac c urate a nalysis with its powerful stor age and computing capacity . Different from the traditional op en-ended input/output system, w e in tend to build a user-centered close- lo op system, and co nsider connecting the mobile terminal users’ p ers onal data collection, the communication betw een mobile terminals and remote data center, and the real-time up date of training model. Based on this, a deep s kin disea s e monitoring system based on edge-to-c loud cognitive medical framew ork is put forward. Spec ifically , the contributions of this pap er are divided into three p oints as b elow: 1. A medical AI framework based on da ta width e volution and self-learning is prop osed. Under such fra mework, the pro cess of infor mation in teraction betw een users and termina l device s , and the wide co llection of da ta in the close-lo o p information flow of us er and remote medical data cen ter are considered. 2. A data set filter a lgorithm based o n information entropy is g iven, so a s to lighten the load o f no-lab e l data sets in terminals and edge cloud in the meantime of improving the data qua lity of r e mote cloud data base and the lear ning ability of analysis mo del. 3. A load mo dule specially for a nalysis algorithms is designed. Under suc h mo dule, it can be compatible with the application requirements acc ord- ing to the learning mo del selected. Meanwhile, three learning mo dels are 5 deploy e d successively in the load mo dule and the training pr o cess is com- pleted. The r emained parts of this pap er are orga nized as b elow. In Sectio n 2, the AI medical framework is presented, and the entities in a nd function o f the AI med- ical framework a re introduced in detail. In Section 3, based on the fra mework raised, the data width c ollection and the self-learning pr o cess are elab o rated. In Section 4 , the training details of the three deep lear ning mo dels deployed on cloud are provided, including the data set acquisition, mo del building and training precisio n. In Sectio n 5, the skin disease re c ognition prototype system is demonstrated, the sp ecific scene case is given, and the computing and communi- cation delay of the system are analyzed. Finally , the whole pap er is summarized and future works are discus sed in Section 6 . 2. Medical AI F ramew ork The medical AI framework based on da ta width evolution and self-lea rning is shown as Fig. 1. The framework contains user terminal, edge no des, radio access netw ork (RAN), cloud platform and remote medical site. T r y to ima gine an application scenario lik e this. A user finds abnorma l changes in facial s kin tissue and ha s plag ued. A t this time, the user can, o n his/her mobile devic es such as mobile phone, e a sily send the skin images to the edge nodes through taking photos b y camera or uplo ading ima g es from mobile pho ne gallery . The edge no des, after da ta filtering, transmit the skin ima g es to the clo ud through the RAN. The cloud provides analysis results o n the user’s skin conditions based on the deployed lea rning mo del, and mean while transmits the results to the remote medical site. Upo n receipt o f the user ’s data, the spec ia list physician feeds back the medical measures to the user, a nd meanwhile archiv es the medica l records to ev alua te the c hanges of the user’s skin conditions. The main parts inv olved in the fra mework are intro duce d in details. 6 Photo Apps Smart sk in healthcare Edge no des Cloud platf orm Compu ting resource Storage resource Networ k resource Algorithm load modu l e Resource cognitive Data cognitive Online med ical serv ices RAN Data flow Data flow Data flow Therapeu tic meas ure feedback Model par amete r update Figure 1: The Prop osed Medical AI F ramework. 2.1. User t erminal It refer s to users’ terminal device s , mainly including smart phone, sma rt bracelet, camera, humanoid robot and other intelligent devices. The terminal device itself co ntains the data storage mo dule, data sending mo dule, data pro- cessing module, a nd data receiving module. The user, after collecting his/her skin image s thr ough the device’s sho oting App o r from the device gallery , firstly makes simple pre-pro ce ssing by the da ta process ing module and then uploads the skin images to edge no de by the data sending mo dule, and the data receiv- ing mo dule will r eceive the medica l feedback data trans mitted fr om the r emote cloud or remote site. These devices are characterized by high mobility , low c o m- puting reso urces, and low storage resource s. T o deal with wir eless transmiss io ns in high mobilit y situations such as vehicular environments, we ca n employ ex- isting self-org a nized coo p e rative transmis s ion scheme like [37] or some adv anced routing pro to cols [38]. 2.2. Edge no de It refers to no de equipment deploy ed on the net work edge with relatively high computing resources and sto rage r esources , s uch as the loc al ser ver. The lo cal server has deploy ed the le arning mo del, w hich can car r y out skin condition recognition a ccording to users’ lo cal da ta . The edge no de transmits the skin 7 image to the r emote cloud thro ugh RAN, a nd meanwhile receives the up da ting parameters of the trained lear ning mo del from remote cloud, so as to provide high-efficiency lo ca l services . 2.3. Cloud platform The clo ud pla tform provides computation-intensiv e task pro ce s sing service s . The learning abilit y of model in algo rithm load module is improved through receiving the skin image data fr om the edge no des. The algorithm load mo dule realizes the adaptation with the a pplica tion requirements by loading different learning mo dels, and the updated mo del parameters are transmitted to the edge no des. The resource cognition mo dule co gnizes the netw ork r esources , and the data cognition module cogniz e s the application con text a nd ne tw ork environmen t context. The tw o mo dules act up on ea ch other to carry ing out the net work resource management and allo ca tion to meet service r equirements of applications. 2.4. R emote me dic al site It refers to the remote medical resour ces, including do c tors, nurses, medical devices and e tc. The re mo te dermatolo gists receive the user skin co nditio ns analysis re s ults from the cloud, provide online medical ser vices a nd feed back to user terminals. Users receive the suggested treatment fro m remote der matolo- gists, and meanwhile ev aluate the ser vice conten ts. 3. Data Wi dth Ev olutio n and Self-le arning Pr o cess Next, w e discuss the clo se-lo op data flow in the framework, and give the data width co llection and the self-learning pr o cess. It is shown as Fig. 2. 3.1. Data Width c ol le ction First, the termina l devices a cquire users’ skin images and tra nsmit them to the remote medica l cloud platform, a nd the c lo ud provides skin disea se diag- nosis s ervice for users by the traditional metho d based on skin databas e and 8 Skin Database Algorithm extension interface Train Model Updating Predicting Input Data Updating Model Me dical Dat a C enter Ski n Anal ysis Emot ion Anal ysis He alth Anal y sis Doctor Advice User pool1 pool2 pool4 conv1 conv2 conv3 pool3 conv4 conv5 fc1 fc2 Edge Cloud New dataset with label Selecting the data based o n the entropy Figure 2: The Illustration of Data Width Col l ection and Self-learni ng Pro cess. deep lear ning. When the e dge no des rece ive the co ntin uously accumulated im- age data from user, the edge cloud pre-pro cesses the lo ca l data ba sed on the lo cal cognition, and then send the us e r da ta to the remote medical cloud plat- form. The r e mote clo ud r eceives the da ta s et fro m multiple ter mina ls a nd deep mo del parameters up dated based on global cognition, and then fur ther feeds back the up dated parameters to the edge no de for a better local co g nition. In the whole clos e-lo op process , the skin image data o f us e rs, the local cognition data o f edge no des, and the global cognition of the cloud are transmitted and communicated mutually , so a s to contin uo usly explor e v a luable informatio n. In addition, the remote cloud cognizes users’ skin c o nditions according to the deep learning mo del, and meanwhile judges users ’ health condition and emotional state and feed back to user s. While reques ting services , user s may a lso provide the skin condition da ta, hea lth condition data, emotional state data and sur - rounding e nvironment information for remote in telligent analysis. The use v alue of the framework can b e expa nded ho rizontally through contin uous infusio n of information base d on user, environment and mo del into the system. 9 3.2. Self-le arning Pr o c ess After contin uous infusion of information ba sed on user, en vironment a nd mo del into the sys tem, the huge amo unt of data lo aded in the sys tem ar e unla - bele d. It cannot b e guaranteed that the unlab eled da ta set plays a p ositive role in the mo del tr aining and global cognition of the cloud platform. Mean while, the transmission of h uge amount of unlab eled data set will consume the net w ork communication reso urces and th us reduce the service exper ie nce of users [39]. Deploy a data filter alg orithm in the edge cloud to filter out the worthless data, and upload the v aluable data to the cloud [40]. The edge cloud, based on the information entropy , filters and provides the v alua ble data to the remote c lo ud. The remote cloud further adjusts and optimizes the mo del pa r ameters to lo ngi- tudinally explo re more v a lua ble information. W e a ssume that the lab eled data sets are x l = [ x l 1 , x l 2 , · · · , x l n , · · · , x l n ] , (1 ≤ n ≤ N ), where N is the num ber of labeled data se ts . The label classes cor - resp onding to the lab eled da ta sets are y l = [ y l 1 , y l 2 , · · · , y l m , · · · , y l M ] , (1 ≤ m ≤ M ), resp ec tively , where M is the num b er of lab e l class e s , and for bi- nary class ification problem, M = 2 . Assume that the unlabe le d data sets a re x u = [ x u 1 , x u 2 , · · · , x u k , · · · , x u K ] , (1 ≤ k ≤ K ), where K is the num ber of unla- bele d data sets. W e consider making skin color classifica tion on the unlabeled data sets which is denoted a s c u = [ c u 1 , c u 2 , · · · , c u s , · · · , c u S ] , (1 ≤ s ≤ S ), where S denotes the num ber of sk in c olor classifica tions. Then with the conditions of x u i already lab eled as class c s , the probability of b eing predicted as skin disease class j is p j i = p  y x u i = j | c s  , and it is c oncluded that the prediction pr obabil- it y of x u i is p x u i = { p 1 i , p 2 i , · · · , p M i } . On this basis , the prediction pro bability ent ropy of unla b e led data is defined as : E  p x u i  = E  y x u i = j | c s  = − M X j =1 p j i log  p j i  . (1) If the entropy v alue is les s than a cer tain threshold v alue E T , i.e. E  p x u i  < E T , the unlab eled data is selected. The thr eshold v alue E T is rela ted to the sample size of lab eled data, the accura cy rate of mo del classification, and the 10 quality of service r equired by users. When the entrop y v alue is relatively small, the newly selec ted data ha s a low er pr ediction uncertaint y . A large a mount of unlabele d data c ollected from users’ persona l terminal devices is stor e d in the edg e cloud. The filtration of the unlab eled data in the edge no des decrea s es the tr ansmission amount of image da ta w ith low v alue, and reduces the communication delay of user service. Mea nwhile, o we to the pre- liminary filtra tion o p eration in edge no des, the cloud utilizes the mo st v alua ble data to upda te the knowledge base, which gua r antees the precision of classifica - tion. In addition, the data selection a ccording to the skin color lab eled by user s can form new database classifying b y po pulation groups, so a s to supp ort the individualized databas e for different groups. 4. CNN Mo del T raining and Comparison A data set use d for the classification of human face skin disease is built. The human face images o n web pa ges a r e crawled by keyword sea r ch. The key- words ar e h uman face pictures, h uma n face skin diseas e. The first 20 pages of dynamic web pages a re selected for eac h keyword. T otally 6,14 4 imag es are obtained through crawling. The der matologists from W uhan Union Hospital are invited to classify a ll skin imag es. The lab els of ima ges contain 14 cla sses, including facial a cnes, for ehead a cnes, a lar acnes, a cne marks, c hloasma, pr eg- nant sp o ts, sunburn sp ots, ra diation sp o ts, age sp ots, dark circles , blackheads, nevus, large p ores , wrinkles. Consider ing the c haracteris tics of diseases, fac ial acnes, forehead acnes, alar acnes and acne marks are unified as skin acnes. And chloasma, pregnant spots, sunburn spots , r a diation sp o ts and age sp ots a re uni- fied as skin sp o ts . Unusable images are remov ed fro m the data set. Finally the images are classified as five types of skin diseases . In the classifica tion of skin acnes, skin spo ts, skin blac kheads, dark circles a nd clean face, the images having skin ac nes ar e taken as the p o s itive s ample, and o thers having skin sp o ts , s kin blackheads, dark circles and clea n face are taken as the negative samples. In the selec tio n o f negative samples, the num b er of images for each diseas e type is 11 R E L U P O O L R E L U P O O L R E L U P O O L conv1 pooling1 conv2 pooling2 conv5 pooling5 Ċ Ċ fc Ċ fc softmax Decision function R E L U P O O L R E L U P O O L R E L U P O O L conv1 pooling1 conv2 pooling2 conv5 pooling5 Ċ Ċ fc Ċ fc softmax R E L U P O O L R E L U P O O L R E L U P O O L conv1 pooling1 conv2 pooling2 conv5 pooling5 Ċ Ċ fc Ċ fc softmax R E L U P O O L R E L U P O O L R E L U P O O L conv1 pooling1 conv2 pooling2 conv5 pooling5 Ċ Ċ fc Ċ fc softmax R E L U P O O L R E L U P O O L R E L U P O O L conv1 pooling1 conv2 pooling2 conv5 pooling5 Ċ Ċ fc Ċ fc softmax 224*2 2 4*3 Input 224*2 2 4*3 Input 224*2 2 4*3 Input 224*2 2 4*3 Input 224*2 2 4*3 Input Skin acnes Skin spots Skin blackheads Dark circles Clean face Figure 3: The Diagram of CNN Mo del f or Skin Dis ease Classification. selected ac c o rding to the num b er of images fo r p ositive sample, so as to make the ratio of positive samples and negative samples is ab o ut 1:1, which av oids the problem o f sample imbalance. The diagr a m of CNN mo del for skin disease cla s sification is shown as Fig. 3. The system uses the Co nv o lutional Neural Net work mo del to extrac t sk in image characteristics. Three learning models , i.e. LeNet-5 [41], AlexNet [42] and V GG16 [43], are adopted to carry out the training , classification and asses s ment pro cesses . In the exper iment , 85% of the data set is us ed as the training set, and the re ma ining 15% is us ed as test set. Firstly , three dee p Co nv o lutional Neural Net works ar e pre-trained on Ima- geNet [44]. Then a fine tuning is carr ied out on all lay ers. The la st lay er is the softmax lay er, which allows to do classificatio n on tw o dia gnosis classes. LeNet- 5 is s e t as 2 conv o lutio nal lay er s, 2 ma x-p o oling layers and 3 fully connected lay ers. The sizes o f all input images are adjusted as 228* 228* 3 , a nd the input images are nor malized. AlexNet is set a s 5 convolutional lay ers, 3 max-p o oling 12 lay ers and 3 fully c o nnected layers. The max-p o o ling op era tion is carr ied a fter the 1 st, 2nd and 5th conv olution. The s izes o f all input imag e s are adjusted as 227*2 27*3, a nd the input images are normalized. The learning rate in the exp eriment is set as 0.0 01, a nd the num b er of iter ations is 15 0. The batch sizes of the training set and test set are r esp ectively 6 4 a nd 5, and the dr op out r ate is 0.6. VGG 16 is set as 13 conv olutional layers, 5 max - p o oling layers and 3 fully connected layers. The s izes o f a ll input images ar e adjusted as 227*2 27*3 , a nd the input images are norma lized. The batch size of the training set is 32 . The nu m ber of itera tions is 200. Other par ameters are s et a s the same with LeNet-5 and AlexNet. T he accur acy rates of the three learning mo dels on five classes of skin diseas e are s hown as T able. 1. It can b e seen from the statistical data that the AlexNet mo de l has the b est overall effect. T able 1: Skin Disease Classification Accuracy Rate of Three CNN M odels. Skin acnes Skin sp ots Skin blackheads Dark circles Clean face LeNet-5 0.63 0.65 0.70 0.58 0.87 AlexNet 0.79 0.80 0.91 0.78 0.95 VG G16 0.68 0.75 0.87 0.76 0.90 5. A dem onstration system for skin disease recognition 5.1. Pr ototyp e Platform AI sk in disease recognition pro totype platform is built, a s shown in Fig . 4. The har dware environment includes AIW A C (Affectiv e Interaction thro ugh Wide Learning and Cognitive C o mputing) rob o t [4 5], lo cal ser ver, and remote clo ud platform. The AIW AC rob ot is equipp ed with AI skin App, and captures the user ima ge thro ugh the camera a b ove the display scr een. The loca l ser ver is the edge no de. The remote clo ud is equipp ed with AMD FX 8-Core pro cess o r in 4 GHz with 32 GB RAM DDR3. Under current environmen t, the commu- nication gateway is the communication bridge for the AIW A C rob ot and lo cal server, the loca l ser ver and remote cloud, and the remote cloud and AIW AC rob ot. 13 Local Server (Edge node) Communication Gateway AWAIC Robot (Intelligent ter minal) Local Environment Remote Environment Big Data Center (Cloud for computing and analysis) Figure 4: The AI-Skin Pr otot ype Pl atform. The AI skin diseas e detection proces ses are as follows: Firstly , the tr a ining of skin dis e a se classification mo dels is e x ecuted on the r emote algo rithm server based on our own skin database . After the completion of training, the trained mo dels are stor ed in the cloud platform, and meanwhile migrated to lo ca l server for exe c ution. Then, the skin image s of user capture d fr o m A W AIC rob ot ar e transmitted to edge no de. When edge node receiv es those da ta, the unlabeled data are selecting based o n data set filter algo r ithm, and then lab eled together with the re c ognition mo del. The labeled data trans mitted to r emote cloud for deep tr a ining and the up dated mo del para meters are fed ba ck to the edg e no de. The algorithm extensio n interface deploy ed in cloud s erver can load different algorithm models. T o execute the recog nition alg orithm in edge device, it is required to deploy T enso rFlow environment [46]. 5.2. T est Sc ene An exp e r imental test is carr ied o ut on the AI s k in prototype platform. The tester captures her own face s k in image through the mobile ter minal camera, a nd the OpenCV [47] face detection classifier is utilized to lab el and segment fac e area a s the input of model. The skin disea se r e c ognition algorithm deploy ed in lo cal server is AlexNet model, whic h is selected by the optimal result in cloud tra ining . Based on the types of sk in diseases given by T a ble. 2, the 14 (a) Sk in Disease De tecti on (b) Anal y sis Repo rt Figure 5: The Skin Disease Recognition Demo and Analysis Report. skin co nditio ns of the tester a re analyz e d for fiv e classes of skin diseases, i.e. (skin acnes, no skin acne), (sk in sp ots, no skin s po t), (skin blackheads, no s k in blackhead), (dark circles , no dar k circle), and (clean fac e, unclean face). After the co mpletion of skin disease analy sis, the skin condition r e po rt of the tester is fed bac k to the mobile terminal. The execution re s ults of real-time a nalysis of skin co nditions are shown as Fig. 5. T able 2: Fiv e Types of Skin Diseases Cl assification. T yp e Class Class 1 Skin acnes No skin acne 2 S kin spots No skin spot 3 Skin blackheads No skin blackhead 4 Dark circ les No dark circle 5 Cle an face Unclean face The sk in disease recognition rep o rt is s hown as Fig. 5 (b). Fir s t, the s ystem gives an ov er all score bas e d on the sk in condition of the tester and the sco re is weight ed b y different skin disease s . Next, the exac t analysis result o f each disease class is given. It is p ointed out that, the pro ble m o f blackheads is 15 0 5 10 15 20 25 30 Number 200 400 600 800 1000 1200 1400 Time/ms AlexNet-Model Edge Computing Transmission Figure 6: Edge Computation D ela y and T ransmissi on Dela y with AlexNet Mo del. relatively s erious, other skin diseas e s, including the skin acnes and skin s p o ts a re only in mild degr ee. Moreover, the skin co lor of the tester is the yello wish skin which is common in Asian p eo ple. It ca n b e see n that these res ults are co nsistent with the skin state o f the tester. In addition, according to the skin ana ly sis, the r ep ort indicates that the tester b elo ngs to the damp-hea t constitution and the excessive oil secretion leads to the enlarg ed p ores or acnes . The rep or t a lso prop oses measures for improving the skin conditions , such a s increasing outdo or sp orts to pr o mote metab olism. 5.3. Delay Analysis T o ev aluate the system’s reliability and v a lidit y , tw o differen t mo dels , i.e. LeNet-5 and AlexNet, are compared for the sys tem’s computation dela y a nd transmission delay . In the exp er iment , the comm unication ba ndwidth is 2 Mbps. W e conducted 30 exp er iments under tw o mo dels, and the sequence num bers are from 1 to 30. The results of system delay under AlexNet and LeNet-5 mo dels are shown in Fig. 6 and Fig. 7 , resp ectively . F rom the fig ure, we can see 16 0 5 10 15 20 25 30 Number 200 400 600 800 1000 1200 1400 Time/ms LeNet-5-Model Edge Computing Transmission Figure 7: Edge Computation Delay and T ransmissi on Delay with LeNet-5 Mo del. that the computation dela y of each mo del c hanges smoothly with the n um ber of exper iment s. Without considering the tra nsmission dela y o f ins tr uctions in communication, we can see that the factor determining the total time delay in the system is the computation delay of edge no des. The average delay of edge computation under AlexNet mo del is 1.2s, but the end-to- end c o mmunication delay (the s um of computation and transmiss ion delay) b etw een terminal device and edge no de is still in the order of 1s. The standard deviatio ns of the total communication delay under the tw o mo dels are 7 5 ms and 63 ms, res pe ctively , which can show the effectiv eness and flexibilit y of the re a l-time skin disease recognition system. Moreov er, it is found that the edge computing time o f image s with high resolution is shorter than that with low reso lution, as shown in Fig. 8. In the exp eriment, the size of original image is 1 23363 4 bytes, and the size of the compressed ima ge is 4 830 bytes. The computing time of high- resolution ima ge is sho rter than that o f lo w-resolution image under b oth AlexNet and LeNet mo del. The image has a 1.0 pro bability of detecting skin blackhead disease 17 0 5 10 15 20 25 30 Number 200 400 600 800 1000 1200 1400 Time/ms Edge Computing HR Image with AlexNet Model LR Image with AlexNet Model HR Image with LeNet-5 Model LR Image with LeNet-5 Model Figure 8: Edge Computation D ela y of Images w i th High- and Low- Resolutions. at b oth res olutions. Upon testing the images of other skin disea ses, it is found that the high- and low-resolution images o f blackheads and skin sp ots hav e little impact on the classifica tion a ccuracy . 6. Conclusion In this pap er, a r eal-time, individualiz ed and ex tensible s kin diseas e r ecogni- tion system is presented. A medical AI framework based on data width evolution and self-lea r ning is prop osed. The c lose-lo o p infor mation flow b etw een user and remote medical da ta center is disc ussed based o n the up dating of the data sets such as user’s skin images, user’s health conditions, environment informa tion, and mo del para meters in AI skin detectio n pro ces s. In a ddition, a data set filter algorithm base d o n information entropy is given. Thr ough the filtra tio n of v alu- able da ta sets in the edge no de, the data qua lity o f the r emote cloud da tabase and the lear ning ability of mo dels can be further impro ved. The universality of algorithm extension in terface is verified based on the three learning mo dels trained on the cloud, i.e. LeNet-5, AlexNet a nd V GG16. A skin disea se recogni- 18 tion proto type system is built. And skin disease a nalysis res ult with the tester’s face skin image s hot on the mobile ter minal camer a is co nducted. Meanwhile, the edge computation delay and trans mission delay o f the system is tested, so as to verify the reliability and v alidity of the system. In our exp eriment, the end-to-end comm unication dela y b etw een terminal device and edg e node is in the order o f 1s. W e found that the high- and low-resolution images of some skin dis eases hav e little impact on the cla ssification accura cy . There is a trade- off b etw een transmissio n delay a nd classifica tion accura cy . In the future, lower transmission dela y can be realized through the deploymen t of imag e compr es- sion a lgorithms o n terminals [48]. Mo r eov er, the clas sification acc ur acy of skin disease can b e further improved by the improv emen t of learning mo del [49]. Ac knowledgmen t The author s ar e grateful to the Deanship o f Scien tific Resear ch at King Saud Univ er sity for funding this w ork thro ugh the Vice Deanship of Scientific Research Chair s : Chair of Perv asive and Mo bile Computing. References References [1] A. C. So ciety , “Cancer F acts & Figures 2018”, Atlanta: Ameri c an Canc er So ciety , 2 018. [2] G. F or tino, S. Galza rano, R. Gr avina, et al., “ A framework for colla b orative computing and m ulti-sensor data fusion in b o dy sensor netw o rks”, Informa- tion F usion , vol. 2 2 , pp. 50 -70, 2015. [3] H. Car ol, M. Ab dalla h, D. Rony , and C. Rapha?l, “Health r isk assess- men t and decision-ma king for patient monitoring and dec is ion-supp ort using Wireless Bo dy Sensor Netw or ks”, Information F usion , v ol. 47, pp. 10 -22, May 20 19. 19 [4] C. Xu, X. Ma, R. Shea, H. W ang and J. Liu, “Enhancing Perfor- mance and Energy Efficiency for Hybrid W or kloads in Virtualized Cloud Environmen t”, IEEE T r ansactions on C loud Computing , 2018, DO I: 10.110 9/TCC.20 18.28 37040. [5] H. Jia ng, C. Cai, X. Ma, Y. Y ang, a nd J. Liu, “ Smart Home Bas e d o n WiFi Sensing: A Survey”, IEEE A c c ess , vol. 6, pp. 133 17-13 325, 20 18. [6] M. Hu, Z. Chen, K. Peng, X. Ma, P . Zho u, a nd J. Liu, “Perio dic Charging for Wireless Sensor Netw orks with Multiple Portable Charger s”, IEEE A c c ess , vol. 7, pp. 261 2 -262 3, 20 19. [7] N. H. F riday , T. Y. W a h, M. Ghulam, and A. G. M. Ali, “Data fu sion and m ultiple classifier sy stems fo r human a ctivity detection and hea lth monitor- ing: Review a nd op en rese a rch directions” , Information F u sion , vol. 46 , pp. 147-1 70, March 2019. [8] L. A. T aw a lb eh, R. Mehmo o d, E. Benkhlifa and H. Song, “Mobile Clo ud Computing Mo de l and Big Da ta Analysis for Hea lthcare Applications” , IEEE A c c ess , vol. 4, pp. 6171 -6180 , 201 6. [9] R. Gravina, C. Ma, P . Pace, et al., “Cloud-ba sed Activity-aaService cyb er - Cph ysical framew ork for h uma n activity mo nitoring in mobilit y”, F utu r e Gener ation Computer Systems , vol. 75 , pp. 158 -171, 2017. [10] R. Gravina, P . Alinia, H. Ghasemzadeh, G. F ortino, “Multi-senso r fusion in b o dy sensor netw or ks: State-of-the-ar t a nd rese arch challenges” , Infor- mation F usion , vol. 35, pp. 68- 80, 2017 . [11] E. Ka njo, E. M.G. Y ounis, a nd N. Sherk at, “T ow a rds unr av elling the re - lationship betw een on-bo dy , environmental and emotion data using sensor information fusion a pproach”, Information F usion , vol. 40 , pp. 18 -31, 2018. [12] C. Ca i, M. Hu, D. Ca o, X. Ma, Q. Li, a nd J. Liu, “Self-Deploy able Indo or Loca lization with Acoustic-enabled Io T Devices Exploiting Par- 20 ticipatory Sensing ” , IEEE In ternet of Things Journal , 2 019, DO I: 10.110 9/JIO T.2019.2900524. [13] J. Cao, W. Li, C. Ma, and Z. T ao, “Optimizing multi-sensor deployment via ensemble pruning for w earable activity rec o gnition”, Information F u sion , vol. 41 , pp. 68-7 9, May 201 8. [14] M. Hu, W. Liu, K. Peng, X. Ma , W. Cheng, J. Liu, and B. Li, “J oint Rout- ing a nd Scheduling for V ehicle- Assisted Multi-Drone Surveillance”, IEEE Internet of Things Journal , 2018, DOI: 10 .1 109/ JIOT.20 18.2878602. [15] M. Hu, W. Liu, J . Lu, R. F u, K. Peng, X. Ma, and J. Liu, “On the Joint De- sign of Routing and Scheduling for V ehicle-Assisted Multi-UA V Insp ection” , F u tur e Gener ation Computer Systems , vol. 9 4 , pp. 214 -223, 2019. [16] G. F ortino, R. Giannantonio, R. Gravina, et al., “E nabling effectiv e pro- gramming and flexible management of efficien t b o dy s ensor netw or k applica - tions”, IEEE T r ansactions on Human-Machi ne Systems , vol. 43, no. 1 , pp. 115-1 33, 201 3 . [17] A. Estev a , B. Kuprel, R. A. Nov o a, J. K o , S. M. Swetter, H. M. B la u, a nd S. Thr un, “Der matologist- le vel classificatio n of skin ca ncer with deep neural net works”, Natur e , vol. 542 , no . 763 9, pp. 115 -118, 20 17. [18] P . Cerw all, Ericsso ns mobility rep ort 2018 , ht tps://www.erics s on.com/as sets/lo ca l/ mobility- rep ort/do c uments/2018 /Erics son-mobility-report-june-2018.p df. [19] J. Chen, C. W ang , Z . Zha o, K. Chen, R. Du, G. Ahn, “ Uncov ering the face of android ra nsomw a re: Char acterizatio n and r eal-time detection”, IEEE T r ansactions on Information F or ensics and Se curity , v ol. 13, no. 5, pp. 1286 - 1300, 201 8. [20] G. F o rtino, D. Parisi, V. Pirr one, et al., “ Bo dyCloud: A SaaS a ppr oach for communit y b o dy sensor netw orks”, F utur e Gener ation Computer Syst ems , vol. 35 , pp. 62-7 9, 2014. 21 [21] C. W ang, et a l., “A L ow Pow e r Cardiov ascular Healthcare Sy stem With Cross-L ayer Optimization F r om Sensing Patc h to Cloud P latform”, IEEE T r ansactions on Biome dic al Cir cu its and Systems , vol.13, no. 2, pp. 31 4-329 , 2019. [22] B. H. Ma lik , S. N. Cheema, I. Iqbal, Y. Mahmo o d, M. Ali, A. Mudasser, and N. Asna fi, “F rom Cloud Co mputing to F og Computing (C2F): The key technology provides services in healthcare big data”, MA TEC Web of Confer enc es , vol. 1 89, pp. 03010 , 201 8. [23] P . Pace, G. Alo i, R. Gravina, G. Caliciuri, G. F o r tino, A. Liotta, “An Edge-bas e d Ar chitecture to Suppor t Efficient Applications for Healthcar e Industry 4.0”, IEEE T r ansactions on Indust rial Informatics , v ol. 15, no. 1, pp. 481 -489 , 20 19. [24] Y. Zhang , et al., “Health-CPS: Healthcare Cyb er-P hysical System Assisted by Cloud and Big Data”, IEEE Systems Journ al , vol. 11, no . 1, pp. 88-9 5, 2017. [25] T. Muhammed, R. Mehmo o d, A. Albes hri, and I. Katib, “Ub eHealth: A Personalized Ubiquitous Cloud and Edge-E nabled Netw orked Healthcare System for Smart Cities”, IEEE A c c ess , vol.6, pp. 32 258- 3 2285 , 201 8. [26] M. Chen, Y. Hao, L. Hu, M. Hossain, A. Ghoneim, “Edge-CoCaCo : T o - wards Joint Optimization of Computation, Caching and Co mmunication on Edge Cloud” , IEEE Wir eless C ommunic ations , vol. 2 5, no. 3, pp. 21 -27, 2018. [27] D. W ang, Y. Peng, X. Ma, W. Ding, H. Jiang , F. Chen, a nd J. Liu, “Adap- tive Wireless Video Streaming based on Edge Computing: Opp or tunities and Approa ches”, IEEE T r ansactions on Servic es Computing , 2 018, DOI: 10.110 9/TSC.20 18.28 28426. [28] Z. Deng, Y. Zhou, D. W u, G. Y e, M. Chen, and L. Xiao, “ Utilit y Maxi- mization of Cloud-ba sed In-Car Video Recording over V ehicular Access Net- 22 works”, IEEE In ternet of Things Journ al , vol. 5, no . 6, pp. 5213- 5 226, Dec. 2018. [29] X. Li, X. Huang, C. Li, R. Y u and L. Shu, “ EdgeCar e: Leveraging Edge Computing for Collabo rative Data Ma nagement in Mo bile Healthcar e Sys- tems”, IEEE Ac c ess , vol. 7, pp. 2201 1-22 0 25, 2 019. [30] J. Zhou, D. Tian, Y. W ang, Z. Sheng, X. Duan, and V. C. M. Leung, “Reliability-oriented o ptimization of computation o ffloading for co op erative vehicle-infrastructure sys tems ” , IEEE Signal Pr o c essing L etters , vol. 26, no. 1, pp. 10 4-10 8, 20 1 9. [31] J. Chen, K . He, R. Du, M. Zheng, Y. Xiang, Q. Y uan, “Dominating set and ne tw ork co ding-ba sed ro uting in wireless mesh netw o rks”, IEEE T r ans- actions on Par al lel and Distribute d Systems , vol. 2 6, no. 2, pp. 423-43 3 , 2015. [32] W. Huang, Y. Z ho u, X. Xie, D. W u, M. Chen, and E. Nga i, “Buffer State is Enough: Simplifying the Design of QoE- Aware HTTP Adaptiv e Video Streaming”, IEEE T r ansactions on Br o adc asting , vol. 64, no. 2, pp. 590- 601, June 2018. [33] U. O . Dorj, K. K. Lee, J. Y. Choi, and M. Lee, “ The skin cancer clas- sification using deep conv olutional neural net w ork”, Multime dia T o ols and Applic ations , pp: 1-16, 2018 . [34] A. Rezv antalab, H. Safigholi, and S. Ka rimijeshni, “Derma tologist Lev el Dermoscopy Skin Cancer Classific a tion Using Different Deep Lear ning Con- volutional Neura l Netw orks Algorithms”, arXiv preprint arXiv:1 8 10.10 348, 2018. [35] J. W u, et al., “What evidence do es deep lea r ning mo del use to clas sify Skin Lesions?”, arXiv pr eprint arXiv:1 811.0 1051, 201 8 . [36] N. T a jba k hsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Got w ay , and J. Liang, “ C o nv o lutional neural netw orks for medical 23 image analys is : F ull training or fine tuning?”, IEEE tr ansactions on me dic al imaging , vol. 35, no. 5, pp. 129 9-131 2, 2016 . [37] D. Tian, J. Zhou, Z. Sheng, M. Chen, Q. Ni, and V. C. M. Leung, “Self- organize d r elay selection for co op erative transmission in vehicular ad-hoc net works”, IEEE T r ansactions on V ehicular T e chnolo gy , vol. 66 , no. 1 0, pp. 9534- 9549 , 2017 . [38] D. Tian, K. Zheng, J. Zho u, X. Du an, Y. W ang , Z. Sheng, and Q. Ni, “A micr obial inspir ed r o uting pr oto col for V ANETs”, IEEE In- ternet of Th ings J ournal , v ol. 5, no. 4 , pp. 2293-23 03, 201 8. DOI: 10.110 9/JIO T.2017.2737466. [39] M. Chen, Y. Qian, Y. Hao, Y. Li, J. So ng, “ Da ta-Driven Computing and Caching in 5G Netw orks: Architecture and Delay Ana ly sis”, IEEE Wir eless Communic ations , vol. 25, no . 1, pp. 70-7 5, 20 18. [40] M. Chen, Y. Hao, K. Lin, Z. Y uan, and L. Hu, “ L a b el-less learning for traffic co ntrol in an edg e netw ork”, IEEE Network , vol. 32 , no. 6, pp. 8-1 4, 2018. [41] Y. LeCun, L. Bottou, Y. Bengio, et al., “Gradient-based lear ning applied to do cument recognition”, Pr o c e e dings of the IEEE , vol. 86, no. 11, pp. 2278- 2324 , 1998 . [42] A. Kriz hev sky , I. Sutskev er, G. E. Hin ton, “Imag enet classification with deep conv olutional neural net w orks”, A dvanc es in neur al information pr o- c essing syst ems , pp: 1 0 97-1 105, 2012. [43] K. Simony a n, A. Zisserma n, “V ery Deep Conv olutional Netw o r ks for Large- Scale Image Rec o gnition”, Computer Scienc e , 2014. [44] O. Russakovsky et al., “Imagenet large scale visual recognition challenge”, International Jou rnal of Computer Vision , vol. 115, no. 3 , pp. 21 1-252 , 2015. 24 [45] M. Chen, J. Zhou, G. T ao, et al., “W ear able Affective Rob ot”, IEEE A c c ess , vol. 6, pp. 647 6 6-64 776, 20 18. [46] http://www.tersorflow.org/. [47] https://openc v .org/ . [48] X. Li, X. W ang, P . J. W an, Z. Han, V. C. M. Leung, “Hierarchical edg e caching in devic e -to-device aided mobile netw o rks: Mo de ling , optimization, and design”, IEEE J ournal on Sele cte d A r e as in Communic ations , v ol. 36, no. 8, pp. 1768- 1785, 20 18. [49] X. W ang, Y. Han, C. W ang, Q. Z hao, X. Chen, M. Chen, “In-edge AI: Intel- ligentizing mobile edg e c o mputing, c a ching and communication by federated learning”, to app e ar in IEEE Network Magazine , 201 9. 25