GEFCOM 2014 - Probabilistic Electricity Price Forecasting

GEF COM 2014 - Probabilistic Electricit y Price F orecasting Gergo Barta 1 , Gyula Borb ely, Gab or Nagy 1 , Sandor Kazi 1 , and T amas Henk PhD. 1 Departmen t of T elecommunications and Media Informatics, Budap est Universit y of T echnology and Economics, Magy ar tudosok krt. 2. H-1117 Budap est, Hungary {barta, nagyg, kazi, henk}@tmit.bme.hu http://www.tmit.bme.hu Abstract. Energy price forecasting is a relev an t yet hard task in the field of m ulti-step time series forecasting. In this pap er we compare a w ell- kno wn and established metho d, ARMA with exogenous v ariables with a relativ ely new technique Gradien t Bo osting Regression. The method was tested on data from Global Energy F orecasting Comp etition 2014 with a y ear long rolling window forecast. The results from the exp eriment reveal that a multi-model approach is significantly b etter p erforming in terms of error metrics. Gradient Bo osting can deal with seasonalit y and auto- correlation out-of-the b o x and achiev e low er rate of normalized mean absolute error on real-world data. Keyw ords: time series, forecasting, gradien t b o osting regression trees, ensem ble mo dels, ARMA, comp etition, GEFCOM 1 In tro duction F orecasting electricity prices is a difficult task as they reflect the actions of v ar- ious participants b oth inside and outside the mark et. Both producers and con- sumers use da y-ahead price forecasts to derive their unique strategies and make informed decisions in their respective business es and on the electricit y mark et. High precision short-term price forecasting mo dels are b eneficial in maximizing their profits and conducting cost-efficient business. Day-ahead market forecasts also help system operators to match the bids of b oth generating companies and consumers and to allo cate significan t energy amounts ahead of time. The metho dology of the curren t researc h pap er originates from the GEF- COM 2014 forecasting con test. In last year’s con test our team achiev ed a high ranking p osition by ensem bling multiple regressors using the Gradient Bo osted Regression T rees paradigm. Promising results encouraged us to further explore p oten tial of the initial approach and establish a framework to compare results with one of the most p opular forecasting metho ds; ARMAX. Global Energy F orecasting Comp etition is a well-established comp etition first announced in 2012 [1] with worldwide success. The 2014 edition [2] put focus on renew al energy sources and probabilistic forecasting. The GEFCOM 2014 2 GEF COM 2014 - Probabilistic Electricity Price F orecasting Probabilistic Electricit y Price F orecasting T rac k offered a unique approach to forecasting energy price outputs, since comp etition participan ts needed to fore- cast not a single v alue but a probability distribution of the forecasted v ariables. This metho dological difference offers more information to stakeholders in the industry to incorp orate into their daily w ork. As a side effec t new metho ds had to b e used to pro duce probabilistic forecasts. The rep ort con tains five sections: 1. Metho ds sho w the underlying mo dels in detail with references. 2. Data description pro vides some statistics and description ab out the target v ariables and the features used in research. 3. Exp erimen t Metho dology summarizes the training and testing environmen t and ev aluation scheme the research w as conducted on. 4. Results are presen ted in a the corresp onding section. 5. Conclusions are dra wn at the end. 2 Metho ds Previous exp erience show ed us that oftentimes multiple regressors are better than one[4]. Therefore w e used an ensemble metho d that was successful in v ari- ous other comp etitions: Gradien t Bo osted Regression T rees[5,6,7]. Exp erimen tal results w ere benchmark ed using ARMAX; a mo del widely used for time series re- gression. GBR implementation was provided by Python’s Scikit-learn[8] library and ARMAX b y Statsmo dels[9]. 2.1 ARMAX W e used ARMAX to b enchmark our metho ds b ecause it is a widely applied metho dology for time series regression [10,11,12,13,14]. This metho d expands the ARMA mo del with (a linear combination of ) exogenic inputs (X). ARMA is an abbreviation of auto-regression (AR) and mo ving-av erage (MA). ARMA mo dels were originally designed to describe stationary sto chastic processes in terms of AR and MA to supp ort hypothesis testing in time series analysis [15]. As the forecasting task in question has exogenic inputs b y specification therefore ARMAX is a reasonable candidate to b e used as a mo deler. Using the ARMAX model (considering a linear mo del wrt. the exogenous input) the following relation is assumed and modeled in terms of X t whic h is the v ariable in question at the time denoted by t . According to this the v alue of X t is a com bination of AR( p ) (auto-regression of order p ), MA( q ) (moving av erage of order q ) and a linear combination of the exogenic input. X t = ε t + p X i =1 ϕ i X t − i + q X i =1 θ i ε t − i + b X i =0 η i d t ( i ) (1) The symbol ε t in the formula ab ov e represents an error term (generally re- garded as Gaussian noise around zero). P p i =1 ϕ i X t − i represen ts the autoregres- sion submo del with the order of p : ϕ i is the i -th parameter to weigh t a previous GEF COM 2014 - Probabilistic Electricity Price F orecasting 3 v alue. The elements of the sum P q i =1 θ i ε t − i are the w eighted error terms of the mo ving a verage submo del with the order of q . The last part of the formula is the linear com bination of exogenic input d t . Usually p and q are chosen to b e as small as they can with an acceptable error. After choosing the v alues of p and q the ARMAX mo del can b e trained using least squares regression to find a suitable parameter setting which minimizes the error. 2.2 Gradien t Bo osting Decision T rees Gradien t b o osting is another ensem ble metho d resp onsible for com bining weak learners for higher mo del accuracy , as suggested by F riedman in 2000 [16]. The predictor generated in gradient b o osting is a linear combination of w eak learners, again w e use tree mo dels for this purpose. W e iteratively build a sequence of mo dels, and our final predictor will b e the weigh ted av erage of these predictors. Bo osting generally results in an additiv e prediction function: f ∗ ( X ) = β 0 + f 1 ( X 1 ) + . . . + f p ( X p ) (2) In eac h turn of the iteration the ensemble calculates tw o set of weigh ts: 1. one for the curren t tree in the ensemble 2. one for eac h observ ation in the training dataset The ro ws in the training set are iteratively reweigh ted by upw eigh ting previ- ously misclassified observ ations. The general idea is to compute a sequence of simple trees, where each suc- cessiv e tree is built for the prediction residuals of the preceding tree. Each new base-learner is chosen to b e maximally correlated with the negative gradient of the loss function, asso ciated with the whole ensem ble. This wa y the subsequent stages will work harder on fitting these examples and the resulting predictor is a linear com bination of weak learners. Utilizing b o osting has man y b eneficial prop erties; v arious risk functions are applicable, intrinsic v ariable selection is carried out, also resolves m ulticollinear- it y issues, and works well with large num b er of features without ov erfitting. 3 Data description The original comp etition goal was to predict hourly electricity prices for every hour on a given da y . The provided dataset contained information about the prices on hourly resolution for a roughly 3 year long p erio d b et ween 2011 and 2013 for an unknown zone. Beside the prices tw o additional v ariables were in the dataset. One was the F orecasted Zonal Load ( 0 z 0 ) and the other w as the F orecasted T otal Loa d ( 0 t 0 ). The first attribute is a forecasted electricit y load v alue for the same zone where the price data came from. The second attribute con tains the forecasted total electricity load in the pro vider net work. The unit 4 GEF COM 2014 - Probabilistic Electricity Price F orecasting of measuremen t for these v ariables remain unknown, as is the precision of the forecasted v alues. Also, no additional data sources were allo wed to b e used for this comp etition. T able 1. Descriptive statistics for the input v ariables and the target Price F orecasted T otal Load F orecasted Zonal Load coun t 25944 25944 25944 mean 48.146034 18164.103299 6105.566181 std 26.142308 3454.036495 1309.785562 min 12.520000 11544 3395 25% 33.467500 15618 5131 50% 42.860000 18067 6075 75% 54.24 19853 6713.25 max 363.8 33449 11441 In T able 1 w e can see the descriptive statistic v alues for the original v ariables and the target. The histogram of the target v ariable (Figure 1) is a bit sk ew ed to the left with a long tail on the right and some unusual high v alues. Due to this c haracteristic we decided to take the natural log v alue of the target and build mo dels on that v alue. The mo del performance was better indeed when they w ere trained on this transformed target. Fig. 1. Price histogram The distribution of the other tw o descriptive v ariables are far from normal as we can see on Figure 2. As we can see the shap es are very similar for these v ariables with the p eak, the left plateau and the tail on the right. They are also highly correlated with a correlation v alue of ~ 0.97, but not so muc h with the target itself ( ~ 0.5-0.58). GEF COM 2014 - Probabilistic Electricity Price F orecasting 5 Fig. 2. F orecasted T otal Load and F orecasted Zonal Load histograms T able 2. Correlation matrix of input v ariables Price F orecasted Zonal Load F orecasted T otal Load Price 1.0 0.501915 0.582029 F orecasted Zonal Load 0.501915 1.0 0.972629 F orecasted T otal Load 0.582029 0.972629 1.0 Beside the v ariables of T able 1 we also calculated additional attributes based on them: several v ariables derived from the tw o exogenous v ariable 0 z 0 and 0 t 0 , also date and time related attributes were extracted from the timestamps (see T able 3 for details). During the analysis w e observed from the auto correlation plots that some v ariables v alue hav e stronger correlation with its +/- 1 hour v alue, so we also calculated these v alues for every row. Figure 3 shows 3 selected v ariables to b e shifted as the auto correlation v alues are extremely high when a lagging window of less than 2 hours is used. Fig. 3. Auto correlation of tzdif, zdif and y M24 v ariables 6 GEF COM 2014 - Probabilistic Electricity Price F orecasting T able 3. Descriptive features used throughout the comp etition V ariable name Description do w Da y of the week, in teger, b etw een 0 and 6 do y Da y of the year, in teger, b etw een 0 and 365 da y Da y of the month, in teger, b etw een 1 and 31 w oy W eek of the year, integer, betw een 1 and 52 hour Hour of the day , integer, 0-23 mon th Mon th of the year, in teger, 1-12 t M24 t v alue from 24 hours earlier t M48 t v alue from 48 hours earlier z M24 z v alue from 24 hours earlier z M48 z v alue from 48 hours earlier tzdif The difference b etw een t and z tdif The difference b etw een t and t M24 zdif The difference b etw een z and z M24 In Figure 4 figure we can see an auto correlation plot of price v alues in sp e- cific hours and they are shifted in da ys (24 hours). It is clearly seen that the auto correlation v alues for the early and late hours are m uch higher than for the afterno on hours. That means it is w orth to include shifted v ariables in the mo dels as we did. Not surprisingly the errors at the early and late hours were m uch low er than midday and afterno on. Fig. 4. Auto correlation of price v alues at sp ecific hours, shifted in days Gradien t Boosting Regression T rees also pro vided in trinsic v ariable imp or- tance measures. T able 4 shows that (apart from the original input v ariables) the GEF COM 2014 - Probabilistic Electricity Price F orecasting 7 calculated differences w ere found to be imp ortant. The relatively high imp or- tance of the hour of da y suggests strong within-day p erio dicity . T able 4. Attribute imp ortances provided b y GBR A ttribute GBR v ariable imp ortance tzdif 0.118451 tdif 0.092485 zdif 0.090757 z 0.090276 hour 0.085597 t 0.078957 z M48 0.078718 t M48 0.076352 t M24 0.069791 z M24 0.069072 do y 0.067103 da y 0.056018 do w 0.024973 mon th 0.001449 4 Exp erimen t Metho dology In our research framework we abandoned the idea of probabilistic forecasting as this is a fairly new approach and our goal was to gain comparable results with w ell-established conv entional forecasting metho ds; ARMAX in this case. W e used all data from 2013 as a v alidation set in our research metho dology (unlik e in the comp etition where sp ecific dates were marked for ev aluation in eac h task). T o b e on a par with ARMAX we decided to use a rolling windo w of 30 days to train GBR. This means muc h less training data (a substantial dra wback for the GBR mo del), but yields comparable results b etw een the tw o metho ds. The target v ariable is kno wn until 2013-12-17, lea ving us with 350 days for testing. F or eac h day the training set consisted of the previous 1 month p erio d, and the subsequent da y was used for testing the 24 hourly forecasts. On some da ys the ARMAX mo del did not conv erge leaving us with 347 days in total to b e used to assess mo del p erformance. The forecasts are compared to the known target v ariable, we provide 2 metrics to compare the tw o metho ds: Mean Abso- lute Error (MAE) and Ro ot Mean Squared Error (RMSE). Gradien t Bo osting and ARMAX optimizes Mean Squared Error directly meaning that one should fo cus more on RMSE than MAE. 8 GEF COM 2014 - Probabilistic Electricity Price F orecasting 5 Results Figure 5 compares the mo del outputs with actual prices for a s ingle da y . While T able 5 shows the descriptive statistics of the error metrics: mae p ar max , r mse p ar max , mae p g br and r mse p g br are the Mean Absolute Errors and Ro ot Mean Squared Errors of ARMAX and GBR mo dels resp ectiv ely . The a v- erage of the 24 forecasted observ ations are used for each day , and the av erage of daily means are depicted for all the 347 days. In terms of b oth RMSE and MAE the av erage and median error is significan tly lo wer for the GBR model; surpassing ARMAX b y approx. 20% on av erage. During the ev aluation we came across several days that had very big error measures, filtering out these outliers represented by the top and b ottom 5% of the observ ed errors we ha ve tak en a t-test to confirm that the difference b etw een the t wo mo dels is indeed significant ( t = 2 . 3187, p = 0 . 0208 for RMSE). T able 5. Descriptive statistics for the error metrics mae p armax rmse p armax mae p gbr rmse p gbr coun t 347 347 347 347 mean 8.640447 10.395176 7.126920 8.496357 std 11.809438 13.822071 10.396122 11.627084 min 1.223160 1.781158 1.020160 1.302484 5.0% 2.083880 2.673257 1.439134 1.785432 50% 5.152181 6.088650 3.520733 4.144649 95% 27.049138 31.339932 27.171626 31.122828 max 101.081747 106.317998 77.819519 83.958518 6 Conclusions and future w ork The GEF COM comp etition offered a nov el w ay of forecasting; probabilistic fore- casts offer more information to stakeholders and is an approach w orth inv esti- gating in energy price forecasting. Our efforts in the contest were fo cused on dev eloping accurate forecasts with the help of w ell-established estimators in the literature used in a fairly different context. This approach was capable of ac hieving roughly 10 th place in the GEFCOM 2014 competition Price T rack and p erforms surprisingly well when compared to the con ven tional and widespread b enc hmarking metho d ARMAX ov erp erforming it by roughly 20%. The methodology used in this pap er can be easily applied in other domains of forecasting as w ell. Applying the framew ork and observing mo del p erformance on a wider range of datasets yields more robust results and shall b e cov ered in future w ork. During the comp etition we filtered the GBR training set to b etter represent the c haracteristics of the day to b e forecasted, which greatly improv ed model GEF COM 2014 - Probabilistic Electricity Price F orecasting 9 Fig. 5. Within-day price forecasts for 2013-02-19 p erformance. Automating this pro cess is also a promising and c hief goal of on- going researc h. References 1. T ao Hong, Pierre Pinson and Shu F an, ”Global Energy F orecasting Comp etition 2012”, International Journal of F orecasting, vol.30, no.2., pp 357-363, April - June, 2014 2. T ao Hong, ”Energy F orecasting: Past, Present and F uture”, F oresigh t: The Inter- national Journal of Applied F orecasting, issue 32, pp. 43-48, Winter 2014. 3. R. W eron, ”Electricity price forecasting: A review of the state-of-the-art,” Interna- tional Journal of F orecasting, v ol. 30, pp. 1030-1081, 2014. 4. L. Rok ach, ”Ensemble-based classifiers,” Artificial Intelligence Review, vol. 33, pp. 1-39, 2010. 5. S. a. S. L. Aggarwal, ”Solar energy prediction using linear and non-linear regular- ization mo dels: A study on AMS (American Meteorological So ciety) 2013–14 Solar Energy Prediction Contest,” Energy , 2014. 6. H. B. e. a. McMahan, ”Ad click prediction: a view from the trenches.,” Proceedings of the 19th ACM SIGKDD international conference on Knowledge discov ery and data mining, pp. 1222-1230, 2013. 7. T. e. a. Graep el, ”W eb-scale bay esian click-through rate prediction for sp onsored searc h advertising in microsoft’s bing search engine.,” Pro ceedings of the 27th In- ternational Conference on Machine Learning, pp. 13-20, 2010. 8. P edregosa, F abian, et al. ”Scikit-learn: Machine learning in Python.” The Journal of Machine Learning Research 12 (2011): 2825-2830. 9. Seab old, Skipp er, and Josef Perktold. ”Statsmodels: Econometric and statistical mo deling with p ython.” Pro ceedings of the 9th Python in Science Conference. 2010. 10. Ian K. T. T an, Poo Kuan Ho ong, and Chee Yik Keong. 2010. T ow ards F orecasting Lo w Netw ork T raffic for Soft ware P atch Do wnloads: An ARMA 10 GEF COM 2014 - Probabilistic Electricity Price F orecasting Mo del F orecast Using CR ONOS. In Pro ceedings of the 2010 Second Interna- tional Conference on Computer and Net work T ec hnology (ICCNT ’10). IEEE Computer So ciety , W ashington, DC, USA, 88-92. DOI=10.1109/ICCNT.2010.35 h ttp://dx.doi.org/10.1109/ICCNT.2010.35 11. Gao F eng. 2010. Liaoning Province Economic Increasing F orecast and Anal- ysis Based on ARMA Mo del. In Pro ceedings of the 2010 Third International Conference on In telligent Netw orks and Intelligen t Systems (ICINIS ’10). IEEE Computer So ciety , W ashington, DC, USA, 346-348. DOI=10.1109/ICINIS.2010.107 h ttp://dx.doi.org/10.1109/ICINIS.2010.107 12. Y a jun Hou. 2010. F orecast on Consumption Gap Bet ween Cities and Countries in China Based on ARMA Mo del. In Pro ceedings of the 2010 Third International Conference on In telligent Netw orks and Intelligen t Systems (ICINIS ’10). IEEE Computer So ciety , W ashington, DC, USA, 342-345. DOI=10.1109/ICINIS.2010.137 h ttp://dx.doi.org/10.1109/ICINIS.2010.137 13. Sh uXia Y ang. 2009. The F orecast of Po wer Demand Cycle T urning P oints Based on ARMA. In Pro ceedings of the 2009 Second International W ork- shop on Kno wledge Discov ery and Data Mining (WKDD ’09). IEEE Com- puter So ciety , W ashington, DC, USA, 308-311. DOI=10.1109/WKDD.2009.140 h ttp://dx.doi.org/10.1109/WKDD.2009.140 14. Hong-Tzer, Y. (1996). Identification of ARMAX mo del for short term load fore- casting: an evolutionary programming approach 15. Whittle, P . (1951). Hyp othesis testing in time series analysis. Uppsala: Almqvist & Wiksells b oktr. 16. J. H. F riedman, ”Greedy function approximation: a gradien t b o osting mac hine,” Annals of Statistics, pp. 1189-1232, 2001.