Integration of Machine Learning Techniques to Evaluate Dynamic Customer Segmentation Analysis for Mobile Customers

International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 DOI: 10.5 121/ijdkp.2 017.7102 13 I NTEGRATIO N O F M ACHINE L EARNING T ECHNIQUE S T O E VALUATE D YNAMIC C USTOME R S EGMEN TATION A NALYSIS F OR M OBILE C USTOME RS Cormac D ullaghan and Eleni Rozaki School of Com puting, National College of Ireland, Dublin, Ireland A BSTRACT The telecomm unicatio ns industry is highly c ompetitive, which mean s that the m obile providers ne ed a business intelligenc e mo del t hat can be used to achiev e an optimal level of churners, as well a s a mini mal level of cost in marketing activities. Machine l earning appl ications can be used t o provide guidan ce on marketing st rategies. Furthermore, data mining te chniques c an be u sed in t he process of customer segmentation. T he purpose of this paper is t o provide a deta iled analysis of the C.5 algorithm, within naive Bayesian modelli ng for the task of se gmenting t elecom munic ation cu stomers behaviour al profiling according t o their billi ng and soci o-demogra phic aspects. Results have bee n ex perimentall y implemented . K EYWORDS Bayesian Mo delling, Dat a mining, Customer Rel ationship Managem ent, Chu rn Manageme nt 1. I NTRODUC TION More mobile customers are using large a mounts of da ta as they watch vide os, live strea ming programs, an d view la rge numbers of picture s via faster 4G networks. As customers consume larger amounts of data because of the activities in which t hey engage on t heir mob ile devices, mobile data service r evenues for mobile providers also increase. In addition, as more apps for iOS and And roid devices a re developed, mor e c ustomers download more a pps and use them, which further increases th eir data usage.[1] Because of i ncreasing mobil e data pr ices, l arge numbers o f subscribers ch urn from one provider to a nother in search o f better rates. The y al so churn providers in order to receive benefits f or si gning up with a new carri er, such as receiving a free or deeply discounted phone. In addition, t he lower si gnup fees associated wit h prepaid mobile serv i ces also enc ourages customers to churn. The ability of mobile customers to keep their existing mobile numbers thr ough the Wireless local number portability (WLNP) reduces b arriers to churning within the industry, which is a major pr oblem f or companies in the telecomm unications industry.[2] Because of the li kelihood of customers to change providers, the deals that tele communication companies offer m ay diff er based on th e nee ds of i ndividual customers and their wiliness to pay for particular services . iD Mobile Ireland are the company that is the basis f or the work performe d in this pape r. They are a start-up telecommunications provider in t he Republic of Ireland. The company differentiates itself in the competitive Irish market by separating th e mobile t ariff from t he handset. This allows customers the flex ibilit y to enter or leave a 12, 18 o r 24 mont h c ontract without pe nalty, and purchase a n ew h andset every t hree months, should they wish to do so, once the previous handset International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 14 cost is fu lly paid of f. Additionally as the customer i s not tie d down int o an ex tended contract where the cost of the phone i s subsid ised by t he tarif f price, cu stomers may change their tariff call, text and data allowances every month, to s uit their i ndividual nee ds, allowing them more control o ver their acc ount charges. They could, for example, increase t heir call mi nutes bundle amount f or the month of D ecember should they env isage ma king more calls du ring this peak holiday period. The company has access to a wide range of data, wit h the prospect to capture even more d ata, growing a t a rapid rate. The data that the company can access are currently not being used to their full potential as a means of understanding th e customers that are served, their sale patterns, the potential f raud risks, and churn patterns. In thi s paper, the goal i s to coll ect, clean, categorise, and gain i nsight f rom a large d ataset spanning 16 months of Bill Pay customer account data that contains 26 717 r ows and 86 columns of attributes . The d ata also contain an additional 11 columns comprised of formula derived values or classes used to categorise the data. The primary aim of this effort is to better meet customer needs, improve customer satisfac tion, developing cus tomer loyalty to the brand as a means of improv ing customer retention. The initial step in carrying ou t thi s effort was to acquire the r elevant data to generate the various reports, using the attributes available, and to cross check the results in the production customer care system as a means of confirming the a ccuracy of the data. Th is verif ication step proved to be very important because as several tabl es o f data where combined, erroneous results oc curred. This meant that separate r eports had to be created because all of t he data could not be in one report due to the databa se tables not containing t he required logic to be joined together or the report data outputted ex ceeding t he current maximum capable by the system, which wa s 70,000 cells of data. The primary contribution of this paper is churn p rediction improvement through the pr ocess of applying some well-known machine learning algorithms to perform customer data segmentation. After experimenting with two different algorithms, decision tree rules models were intr oduced that showe d the segmentation rules within a Bay esian modelling sc heme that displaye d the significance of the probability of predicting customers bu ying patterns. [3] The methodology that is suggested in t his study i ndicates how to improve services for VIP customers. However, t his methodology includes a tuni ng parameter that can be manipulated to p redict different customer preferences on deals and t he probability of a customer switching to a different telecommunication provider. As a r esult, th e machine learning algorithms can be us ed to pr edict which customers are likely to switch carriers.[3] It should also b e noted that anoth er important purpose of thi s project was to provide a way for telecomm unication companies to be able t o target deal s and sp ecial programmes or incentives i n order t o prevent cu stomers fr om churni ng. Targeted proactive pr ograms h ave t he p otential advantages for telecommunications companies of having l ower incentive costs due to the fact t hat customers are not trained to ne gotiate for better deals unde r the threat of churning.[2] 2. C USTOMER R ELATIONSHIP M ANAGEMENT I N M OBILE C OMMUNICATIO NS E NVIRONMENT Large numbers of companies are adopting CRM technology as a way of managing customer relationships. CRM is us ed to create, maintain and enhance strong relationships with customers and stakeholders.[1] By t aking advantage of machine learning techniques with the CRM database, i t is poss ible to f ind the be st cu stomers. This pro cess is known as cu stomer classification.[1] International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 15 Machine l earning te chniques can b e used to extract important custome r information from a much larger set of data that may be irrelevant fo r a particular purpose, such as preventi ng customer churn. Man y CRM st udies are based on the us e of decision trees. Data mini ng can b e u sed to discover what might oth erw ise be hidden customer behaviours from l arge am ounts of data. In t his regard, the real value of data mining is the ability to transform l arge amounts of raw data into usable data to address bus iness problems.[1] The idea is t o group and profile customers according to di fferent socio-demog raphic aspects ( age, gender, l ocation, e tc.) It is i mportant to notice that s tudies including these variabl es all ow identifying specific demographic se gments of the population to be the target of certain policies and strategies . However, based on T.Garín-Muñoza, T.Pérez-Amar alb,N.C.Gijónb_R.L ópez 2015 the main complaints of telecom munication customers are based on the foll owing ty pes: Reputation of the co mpany: The likelih ood t hat the comp any will ac tually i nvestigate and address a customer c omplaint.[ 4] Level of expenditure : The higher the cos t on mobile p hone services to the custome r, the greater the likelihood to complain.[4 ] Overall level of satisfaction: The o verall level of satisfaction of the cust omer, i ncluding an assessment of a problem situation in light of overall satisfaction, and the evolution of a part icular problem over time.[5] Type of contract: It is expec ted that consumers who have post-paid contracts will have a higher likelihood to complain than prepaid customers. The r eason for this is because prepaid customers will not incur problems r elated to billing or breach of offers as they do not receive a monthly bill.[4] 2.1. Mining c hurning behaviours The variables t hat are generally used for market segmentation are Demographic, socioeconomic, and geographic characteristics of the customers. A v ery useful technique for behavioural-based data min ing method in the RFM analysis, which involves the extraction of c ustomer profiles by using a few criteria, which r educes the complexity of analysis.[5] In RFM analysis, customer data are classified b y R ecency (R), Freq uency (F) and Mo netary (M) variables. It has b een noted that RFM enables the practitioners to ob serve customer behaviour, as well a s to segment customers in order to determine imme diate customer value.[5] It should also be noted t hat using decision ru les al gorithms f or t he p urpose of customer segmentation may result in an efficient evaluation of a segmentation plan. Dec ision trees ca n be identified into sets of if-then rules, which means t hat they can be used to solve a v ariety of problems, such as custome r segmentation and customer churn p rediction. In fact, m any researchers have used this method to study c ustomer seg mentation. A customer s atisfaction survey can be used to construct a customer segmentation system based o n demographic variables and even customer reviews.[1] Researchers have pr ovided idea s about modelling customer satisfaction using unstructured data with a Bayesian approach. They explain that the transformation of uns tructured data taken from customer’s reviews i nto a semi-structured form ass ociated with e ach aspect reflecting the frequency counts for po sitive, negative, and n eutral sentiments. One assumption of this mod el is International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 16 that the r ating of each aspect is b ased on a particular combination of t he positive, neutral and negative s entiments o f th at particular aspect. T he result is th at th e overall aspect rating depends upon how many times an aspect has been associated with positive, neutral and negative sentiments in a single customer r eview. Furthermore, th ere i s also an overall rat ing that is assigned to each review by the contributor.[6] 2.2. Applications of Data Mining Tec hniques in Telecom Churn Prediction The data regardi ng user co mmunica tion characteristics consists of Call Data Reco rd information. The telephone u sage habits of customers are indicated by t he start time, the number of users to make the call, the sum, duration, call type , call variation information, and other sta tistical data. As fraudulent behaviour may have a fixed behaviour pattern, most fraudulent behaviour can be fou nd by ex amining th e CDR data. It is typically the c ase th at normal users sometimes delay pa yment because of special reasons or habi tual delay s. Howe ver, these t ypes o f delays ar e different from fraudulent behaviour as they generally have a fixed behaviour patterns. In order to id entify late paying users, the late payment behaviour must be compa red with customer payment records.[7] Fraudulent behaviour can r esult in telecom providers suffering he avy short-term financial losses. Late payments that are due to non-fraudulent behavi our may not immed iately cause significant financial losses , but they m ay result in o ther type s of losses such as cash flow r eductions, increased labo ur costs related to debt collected custo mer churn.[7] Based on the info rmation that has be en reviewed, po ssible variables for modelling the decision tree were selected.[8] The creation of a t est to determine proper costs, as well as t o p revent cross subsidisation is diffi cult i n the te lecommunications in dustry. The dif ficulty a rises from the pervasive common and jo int costs that arise within the industry. T elecommunications companies typically group multiple services rather th an p roviding in dividual services th at are unrelated to each other. This means t hat investment costs are common to multiple se rvices.[8] Among them, the most s ignificant c ost variables for c ustomers eva luation that h ave higher c ontribution to predict the churn are se lected. 3. C USTOMER B EHAVIOUR P ATTERN A NALYSIS A ND D ERIVED A TTRIBUTES One of the issues that arose in this study involved tr ansforming the raw d ata. An effort was mad e to collect, clean, categorise, and gain insi ght from a l arge dataset that contained 26717 instances and 86 attributes with a further 11 columns comprised of formula derived va lues or classes to categ orise the d ata. In order to deal wi th this issue, th e derived attributes were p erformed based on the needs o f the learning machine scheme.[6] It is important to understand t hat derived attr ibutes are n ew variables that are b ased on original variables. The most effective derived variables are tho se that represent something in the real world, such as customer behaviour. There are g eneral classes of deri ve d variabl es, such as total values, average values, and r atios. In this study, t he derived variable of the average value over the last six months is u sed. In addi tion, the r atio betwee n t he average value over th e last thr ee months and the average value over all months is used.[9] 3.1. Customer Demographic Profiles Customer demographic pro files are a grouping of a demographic or market segment t hat contains likely consumer behaviour . This information t ypically includes a ge, location and gender, among other demographic variables. Th e information of ge nder and counties are available and selected as two new features in this study.[10] International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 17 3.1.1 Customer Age Group The extracted attr ibute of customer age groups categorises t he customer’s age int o five group s, 0- 14, 15-24, 25-44, 45-64 and 65+. These groups were chosen based on the Ir ish Census groupings to allow f or future compa ri sons to be made. First it performs an error check t o ensure the age column is a nu mber, then it checks wh ich group that n umber falls within and returns that group value. The decision t ree classifier facilitates th e pr ocess of classification i n regards to the customers age. Figure 1. Decis ion tree of Customer a ge 3.1.2 Customer Location County This variable is the appro ximate county location of the customers obtain ed from the county name in the Customer Bill Address column. Because t he data are manually entered at point of sa le, they are not consisten t and may no t always contain t he county name. However, Eircodes, which is Ireland’s Post Code, h as become mandatory as part of t he sales proc ess, w hich means that precise location ca tegorisation can occur in the future. Figure 2. Conf usi on Matrix fr om Location C ount y Model The diagonal of t he confusion matrix shows the d ifferent costs for t he locations classification, all of which are p ositive values. The use of thi s type of con fusion matrix i s to crea te classifiers t hat minimise the pr ediction cost as oppos ed to the prediction error that occurs in customers’ segmentation based on the county. Moreover, the locations with t he higher prediction errors in cost benefits shown in Dublin and Co rk as well as in Wa terford and Meath. International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 18 3.2. Sales information about Day & Time The variable for sales information about day & time contains the types of service packages, credit controller indicators, and the first date of using t he services. The variable also contains the creation d ate and time, th e bill frequency, the account ba lance, e quipment rents, payment t ypes, and contract duration.[10] 3.2.1. Customer Length of Service This variable contains the t otal nu mber of days a customer account has b een in service. The variable was created by checking the C ustomer Net work Status, and if inactive, minuses the Subscriber inactive Date from the Customer Activati on Date. If no inactive date was present, the variable w as created b y subt racting the Customer Activation Date f rom today’s date to find the current length of service in days . 3.2.2. Service Sale Date The service sales date attri bute was cr eated from the value of t he sale d ate column with the format of D D/MM /YYY and converte d it to a day in the week. For example, 28/ 07/2015 was converted to Tuesday. 3.2.3. Sale Time of Day This v ariable contained the sales ti me in the 24 hour format HH:MM:SS and categorised into 4 types. If the hou r was less than 6, it was defined as Night. If the hour was less than 12, it was defined as Morning. If the hour was less than 17, it was defined as Afternoon. Othe rwise, the time of day was defined as Evening. For example, 10:17:55 was categ orised as Morning. 3.3. Customer account information about bills and payments This variable contains t he bi lling information f or each customer and service f or a certain number of yea rs. The l ast 16-months of Bill Pay customer account data are available and only used i n o ur prediction sy stem. However, dif ferent customers may h ave had different bi ll occurrences depending on when they join ed th e mobile network and their l ength of service. In t his regard, th e durations of customer bills mig ht have been different.[10] 3.3.1. Total Invoice Am ount Excluding Brought Forward This v ariable contains the Total Invoice Amount Excluding Br ought Forward in order to add in every month’s invoice amount mi nus any amount unpaid and brought f orward from the pre vious month. By performing this calculation, it was possible to f ind the total revenue generated by each customer’s acc ount. 3.3.2. Total Num ber of Invoices The attr ibute of Total Number of Invoices is to count the tot al nu mber of invoices a customer h as received. It requir ed an additional row be created above t he attribute names, with the string “Invoice” bein g pl aced above each column of data where t he months’ invoice amo unt is. T his is due t o the data layo ut hav ing t he b alance brought forward a ttributes be tween e ach month’s invoice amount and the Count function only being able to se lect a single range of columns. International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 19 3.3.3. Average Invoice Amount Excluding Brought Forward The attribute of Average Invoice Amount Excluding Brought Forward checks to see if t he value for Total Invoice Amount Excluding Brought Forward is zero, and if so returns zero, otherwise divide it by the va lue in Total Number of Invoices to find an average invoice amount. 3.3.4. Total Paid Amount o f Invoices The Total Paid Amount of Invoices attribute takes a count of every paid invoice. As the attribute data is located in columns beside each other, a single range c an be d efined as paid, unpaid and decline status of the i nvoices. N is a set of Total Paid Amount of Invoice s attributes: N = {P ji, DCl ji, U ji ,} 4. C USTOMER S EGMENTATION M ODEL A set of rules can be generated f or checking customer willingness to pay based on a verage invoice amount. The ru le that was created wa s that if the average invoice amount i s less than €15, then define as “Low Spender”. If t he average invoice amount is between €15 and €29, then define as “Average Spender” If the ave rage invoice a mount is betwee n €29 and €50, then define as “Above Average Spender”. If the average in voice amount i s between €50 and €70, t hen “High Spender”. If the average in voice amount is above €70 then “Very High Spender”. If none o f these apply, then “Investiga te” is returned. The ClassA ji presents the lo w spe nder st atus, Class B ji presents the A verage Spender status, ClassC ji presents the A bove Av erage Spender, ClassD ji presents the High Spe nder, ClassE ji presents the Very High Spender status. 4.1. Learning al gorithm: Seeking the customer profile rules se t This algorithm categ orises each customer acc ount into four different classes based on their invoices being paid and spender status. If th e Total P aid Amount of Invoices i s greater than or equal t o one less of the Total Number of Invoice s, and if t heir Spender St atus is either a Low Spender or an Average Spender, then they are defined as Standard. If the spender status is either a High Spender or an Above Average Spender, then t hey are defined as Premium. Al ternatively, i f the spender status is a Very High Spender, then they are defin ed as VIP. If the Total Paid Amount of I nvoices is less than one less o f the Total Number of Inv oices, then i t i s d efined as Unpaid Invoice to highlight that a cus tomer account is not paid up to date. International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 20 Figure 3. Cus tomer Seg mentati on algorithm 5. E XPERIME NTAL R ESULTS A ND D ISCUSSION In order to validate th e various test option efficiencies , a coll ection of r esults were gathered from repeatedly r unning the same classifier test. The test dat a must be di fferent to the t raining d ata t o ensure accurate test r esults. As all of t he data used in this study was f rom a single d ataset, a separate test set f ile was not used. However, cross-validation and percentage split test op tions were used. Firstly, in order to selec t t he p aramete rs of t he model, the data were divided into training and t esting s ets. T he training data was used for parameter e stimation, while the test set was used for evaluation of t he methodology.[11] To e nsure an accurately di fferent result each of the ten times , a new r andom seed was provided in t he opt ions for the percentage split each time. The Sample Mean was calculated by averaging all ten r esults, and the Sample Standard Deviation was used to m easure the r ange of the numb ers. After all percentage spl it tests were completed, cross validation was used and set to 10 folds. [1 2] This m eant that the enti re dataset w as divided into 10 pieces wit h each used in tur n for training. Then, an eleventh test was performed u sing th e entire dataset as the test data. Once complete d, the final result was displayed. 5.1. Experiment I -Customers segmentation us ing decisi on tree classification rules In th is study, the d ata classification was used to determine the lik elihood of a result from creating a decision tree. Th e attri butes of an i nstance wer e used against the decision tree to determine th e likelihood of the result by p rogressing through each step u ntil t he f inal decision. It should be noted that this is one of the mos t popular and widely used c lassification techniques. First, classifying a customer’s VIP status is explained. Because of the l arge n umber of attri butes in the d ataset, the C.5 algorithm was or iginally not available. After removing several derived attributes that were deemed not to b e relevant for this classification, th e C.5 alg orithm with 86 attributes was available for use. International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 21 Figure 4. C ustomer Seg mentati on decision rules The decision tree algorit hm extracted the rules f or customer pr ofiling and evaluation. The segmentation decision r ules were based on spender s tatus and the payment behaviours, such as the total amount of paid and unpaid invoices and the customer preferences in regards to the deals offered by t he p rovider. The size of the decision tree was 9124 and ha d 9062 le aves. The algorithm resulted in classification of the custome r profiles int o four categ ories, such as standard, unpaid invoice, premium a nd VI P sta tus. The rules e xtracted from the tr ee sh ows t he most reliable customer, their spending status, f requency of purchases during date and time, how to track unpaid invoices and block accounts and, whe ther t hey were VIP customers. The VIP Status could be deemed to be a successful classifier because of the high accuracy of correctly classified i nstances . The deci sion tree algorit hm showed additi onal rules about t he VIP churners and their length o f time, the services that have received, wh ether th ey were more likely to p urchase a d eal in regards o f time and d ay, and the most popular addresses at which VIP customers were loca ted. After completing all eleven tests , the Sample Mean of Correctly Classified Instances was 97.70315% for Percentage Split, while Cross-validation 10 Folds showed Correctly Classified Instances of 97. 6669%. Given the very slight 0.03625% difference i n correctly classified instances, along with t he reduced time to run a single cross-validation 10 f olds’ test, the most efficient test method appe ared to be cross validation. [12] 5.2. Experiment 2- Bayesian mod elli ng of customer profiles In th is study, t he data set was an alysed using t he Naïve Bayesian mod elling. A ma chine learning technique based on Naïve Bayes model assumes the presence of a part icular feature in a customer profile class that is unrelated to the presence of any othe r feature. As has been shown, the Bayesian network provided estimation s about the utility of eve ry possible attribute value in th e spender status domain. In ord er to use these e stimations to eli cit customer preferences, a notion was needed regarding the relative importance of t he attribute s relative to each other. In t he approach used in this study, the i mportance of an attribute d epended on three factors : C ustomer demography, Customer L ength of Service Sale Da y & Time of Day and financial considera tions. [13] International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 22 Figure 5. Naïve Ba y esian profile c lassification r esults Through the use of semi-structured data, it is proposed that a Bayesian approach be used to model the overall custome r preferences in terms of the aspects identified fr om the consumption and the billing behav iour associated with a cus tomer’s willingness to pay. This Bayes ian model considered t he overall rating of each deal a s a weight and sum of the probability to select the individual offers. This model allowed for t he determination of an estimation of t he tendencies for each deal aspect from each customer’s perspective.[6] The use of t he Naïve Bayes model provided a good, but still l ess accurate result than the C.5 model, of 87.7189% Correctly Classified Instances. The VIP type in the VIP Status class is where the precision is weakes t and th us cause d the reduced overall accuracy. One might consider a point in the customer pr eferences where the model shows a high precision and ac curacy related to the “ premium offer” as evidence withi n the naïve Bayesian n etwork leading to the posteriori probability-distributions shown i n Fig.5. Evalua ting the Influences as have been explained, the learning algorithm used in this study showed that the cust omer tends to choose a premium off er due to the additional need for cell services and mobile internet connectivity.[13] The Naïve Bayes results showed that the probability of se lecting the premium offer was higher for t he customers. However, th ere was also a ve ry low independence among t he preferences related to the “ standard deal” offered by t he mobile provider. Based on these re sults, churn management models should not only identify the customers that are most l ikely to leave the current service provider but also i dentity the customers that are most likely to respond positively to the right retention deal.[1 3] Furthermore, based o n the results that were obtained, the “standard deal” n eeds to be reconsidered and reviewed in relations t o customer’s need and customer will ingness t o pay. In addition, a significant p robability of f raud detection was found to be classified in the “unpaid International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 23 Status” customer pr ofile shown on the Bayesia n results t hat need to be fu rther investigated by the network providers. 6. P ERFORMANCE E VALUATION R ESULTS Three types of variables were used to develop t he two machine le arning models th at can b e tailored according to t he custo mer needs and cu stomer’s wi liness t o p ay. In order to actually implement the plan, more retention efforts should be given to the pot ential churners who are most likely to react positively. [13] In this study, the prediction accuracy of each model for each data set in regards to t he ROC curve, precision, recall and RPC area were e xamined. In addition, the predictions of models built and evaluated on more transactions in the training data, and models built and evaluated on customer’s deals were also examined. Furthermore, i n addition to t he True Churn and False Churn illustrated in t he ROC C urve for decision tr ee models, t he results of the Naïve Bayesian classif ication. PRC, and the overall accuracy were examined, and the res ults are shown in figures 6&7. Figure 6. Deci sion Tree Figure 7. Naïve Bayes Figures 6 and 7 show the p erformance of th e model ac hieved by each machine le arnin g technique and the max imum accuracy t hat was arch ived by the decision tree model in all the deals .[15] It should be not ed that the main purpose of th e Bayesian network wa s to derive utility estimations for attribute Values from which customers’ preferences were d erived. In this regard, the churn probability in different deals shown in the N aïve Bayes results was based on the assumption t hat values that were more likely to satisfy the customer’s needs were also more useful.[14] 7. C ONCLUSIONS The Use of machine le arning t echniques wit h a datas et built upon custome r data generated from the Telecommunications Industry made i t possible to test variou s classifiers fo r categorising a Customer’s Age Group, VIP Status, Spend Status and Custome r Length of Service . All of t he features used fo r t he churn prediction in this study w ere either demographic billing, or usage features. T he goal wa s to gain an understanding about the i mportance of these thr ee types of deals in churn prediction. The con clusion that was reached wa s that t he bi lling and usage features had a ve ry high importance for a custome r segmentation scheme.[2] International Jour nal of Data M ining & Knowled ge Management P rocess (IJDKP) Vol.7, No.1, J anuary 2017 24 Finally, t he two ways i n which to evaluate customers’ seg mentation were equally important for predicting churn. The number assig n ed t o a ca tegory of a spende r showed the impo rtance of a deal in churn pr ediction. The re sult of t he decision t ree algorith m and preference given to th e features ar e a bi t d ifferent from th e B ayes ian model. Howeve r, usage and billing features were still of primary impo rtance, while demographics may also affect the churn pr ediction.[2] In the future, res earch is p lanned for mining further churning behav iours and dev eloping re tention strateg i es. R EFERENCES [1] L. Luan a nd H . Sh u, “Inte grat ion of da ta mining techni ques to evaluate promotion f or mobi le customers ' data traf fic in data plan,” 2 016 13t h Inter national C onference on Se rvice Systems and Service Manage ment (ICSS SM), 20 16. [2] A. A. Khan, S. J amwal, a nd M. Sepehri, “A pplying D ata Mini ng t o Customer Churn P rediction i n an Internet Service Provider,” Internati onal Journal of Compute r Applica tions, vol. 9, n o. 7, pp. 8–14. [3] A. Ker amati, R. Jafari-Mara ndi, M. A lianne jadi, I. Ahmadian, M. M ozaffari, an d U. A bbasi, “Improved churn predicti on i n te lecommun ication industr y usi ng d ata mining techniques ,” Applied Soft Computi ng, vol. 24, pp. 994–101 2. [4] T. Garín-Muñoz, T. Pérez- Amaral, C. Gijón, and R. López, “Consumer c omplaint behaviour in telecommunications : The ca se of mobile phone users in Spai n,” Telec ommunicat ions P olic y , vol. 4 0, no. 8, pp. 804– 820. [5] A. C. B. Durs un a nd M. Ca ber, “Using data minin g te chniques f or profil ing pr ofitable hotel customers: A n application of RFM anal ysis,” Touri sm Ma nagement P erspectives , vol. 1 8, pp. 153 – 160, 2016. [6] M. Farhadloo, R. A . Patterson, and E. Rolla nd, “ Modelin g c ustomer satisfacti on fr om u nstructure d data using a Ba yesian appr oach,” Dec ision Supp ort S ystems, vol . 90, pp. 1–11. [7] C.-H. Chen, R.-D. Chia ng, T .-F. W u, and H.-C. C hu, “A combine d m ining-based fra mework for predicting telec ommunicati ons customer pa yment be haviors, ” Expert S ystem s w ith Appli cations, vol. 40, no. 16, pp. 6561–656 9. [8] E. Rozaki, “ Financial Pre dictions Using Cost Sensitive Neural Netw orks for Multi-Class Learni ng,” Advanced En gineering Forum, vol. 16, pp. 104–11 6. [9] V. Umayaparvathi and K . I ya kutti, “Applicati ons o f Data Mi ning Techniques i n Tele com C hurn Prediction,” Internati onal Journal of Com puter A pplications, vol. 42, no. 20, pp. 5–9. [10] B. Huang, M. T. Kec hadi, a nd B. Buckle y, “Cust omer c hurn pre diction in tele communicati ons,” Expert System s with A pplications, vol. 39, no. 1, pp. 1414– 1425. [11] Brett Lantz, “Mac hine Lear ning with R” Pac kt Publishi ng Ltd. Sec ond edition Jul y 2015. [12] I.H. Witten & E. Frank, “Dat a Mining , practical m achine learnin g t ools and techniq ues,” Morga n Kaufmann publ ishers, 200 5. [13] S. Radde and B. F reitag, “U sing b a yesian netw orks to i nfer product rankings fr om u ser needs,” UMAP 2010 Workshop on Intelli gent Techni ques for W eb Pers onalization and R ecommende r Systems. [14] H. Lee, Y . Lee, H. Cho, K. Im, a nd Y. S. Kim, “Mi ning churnin g beha viors and de veloping ret ention strategies based on a p artial le ast squares ( PLS) mode l,” Decision S upport S y stems, vol. 5 2, no. 1, pp. 207–216. [15] Y. Huang a nd T. Kecha di, “An effecti ve hybrid learn ing s y ste m for tel ecommunica tion churn prediction,” E xpert S ystems with A pplications, vol. 40, no. 1 4, pp. 5635–56 47.