Training Gamble leads to Corporate Grumble?

- 1 - Training Gamble lead s to Corpo rate Grumble? David Chadwick, EuSpRIG Chair D.R.Chadwick@ gre.ac.uk Fifteen y ears of research studies have concluded unanimously that spreadsheet errors are both common and non-trivial. Now we must seek ways to reduce spreadsheet errors. Several approaches have been suggested, some of which are promising and others, while appealing because they a re easy to do, are not likely to be effec tive. To date, only on e technique, cell-by-cell code inspection, has been demonstrated to be effective. W e ne ed to conduct further research to determine the degree to which other techniques can reduce spreadsheet errors. As Ray Panko [1] correctly points out, the problem of spreadsheet errors has y et to be solved and represents a g reat, often unrecog ni sed, risk to corporate decision making and financ ial integrity . Corporations are indeed gambling with the hidden risks. Abstracts of papers prese nted at past EuSpRIG sy mposia are available on the website www.eusprig.org ; a quick review of the titles suggests that the spreadsheet risk problem may be dealt with in several way s by: • expanding awareness of the problem among st corporations and academia Rajaling ham et al [2], Ayalew et al [4], Butler R [6] ,Cleary P M [8] • setting standards for end-user spre adsheet development Butler R [11] , Hawker A [3], Knight D [17] • creating better a udit approaches with accompany ing software tools, Nixon D [18], Hock et al [7], Ettema H et al [1 6] • Creating methodologies and software based tools for spreadsheet building Rajalingham et al [5] [ 14], Chadwick et al [10] , O’Beirne [12], Paine [13], Raffensperger [15]. Many of the above are already being well researche d and, no doubt, useful results will be obtained in due course but there is a view that the very least that can be done immediately is to concentrate eff ort in the first two areas mentioned. Expanding awareness and setting standards at an early enough stage in the training of would-be spreadsheet developer s must surely have an effect in tackling the problem wher e it really begins - in human error. I t i s a belief held by the author that, often overlooked, is the need for better training and education of y oung professionals and students learning spreadshee t skills for the first time and surely de stined to become the spreadsheet builders of tomorrow. Many novice spreadsheet builders formally learn their skills by st udy ing either for a professional examination of the auditing, IT and a ccountancy bodies or from courses provided at universities, colleges and private training a genc ies. I f there is to be a true chang e in the quality of business spreadsheets then surely it is through such training and education providers that improvement and innovation must be channelled. The question, therefore, that needs to be addressed by EuSpRIG members is: 'How can EuSpRIG be actively involved in the teaching and assessment of spreadsheet skills where these occur?’ To help in answering this question it is useful to look at two areas where the influence of an interest group composed of active ac ademics and professionals (such as EuSpRI G) may have an effect: - 2 - • the sy ll abi of professional examinations and coverage of spr eadsheet risk, • the curricula of higher education courses and cover age of spre adsheet risk. 1.0 Professional Examinations And T heir Coverage Of Spreadsheet Risk 1.1 Professional Examinations I n the UK, there are seve ral bodies either directly involved in the teaching and assessment of spreadsheet skills through their own professional studies or indirectly involved in the validating and influencing of sy llabi of other educational providers. For instance, the I CAEW (Institute of Chartered Ac countants England and Wales) is the main body influencing standards of assessment for accountants, the BCS (B ritish Computer Society) is involved in setting its own examinations or in validati ng courses run a t universities for IT pr ofessionals. So, too, the Institute of I nternal Auditors (II A) operates its own examinations for computer auditors. In ter ms of gaining a prof essional qualification in computer auditing and risk management there a re probably two main avenues: the Certificate in Inf ormation Sy stems Auditing (CI SA) and the Qualification in Computer Auditing (QiCA). Their approach to the development of skills differs. CISA appea rs suited primarily t o those who have already gained some working knowledge by vi rtue of having done the pra ctitioner job for some time. It results in a multiple-choice test which basically ex amines the applicant's in-depth knowledge of practical scenarios. I t is therefore not suited to a beginner who has no or relatively limited personal experience. The QiCA, however, fr om the Institute of I nternal Auditors, is comprised of two papers at different levels; the lower leve l paper is based upon learning f rom written texts and is more suited to the true novice, the practitioner with little experience or the student at university who both need to grasp fundamentals. The paper at the higher le vel deals with practical scenarios and is more appropr iate for the experienced prac titioner. 1.2 Training Curricula An d Coverage Of Spreadsheet Risk Neither the sy llabus for CISA nor that for QiCA adequately cover the treatment of spreadsheet risks. A standard text used for teaching Computer auditing in the UK has one parag raph of four lines devoted to spreadsheet errors: 'Users of spreadsheets and database packages are necessarily immune from errors. 9%ile it is true that such packages provide a safe processing environment within w hich it is difficult if not impossible to make undetected or obscure input and output errors, it is s till possible to make errors in logic. In fact, such errors maybe more difficult to detect than t hey would have been if a procedural programming language had been employed... This may be the case if a very large spread sheet is generated, so that only small parts of it can be viewed on screen at any one time... Chambers A.D and Court L M [9] The third line admits that the problem for spreadsheets may be even worse than for normal software particularly with logical errors. Despite this, there is no literature on how to prevent errors nor on how to conduct a spreadshee t audit. Perhaps this is where EuSpRIG has a role to play . Now established as the world’s foremost action group on spreadshee t risks isn't it about time that EuSpR I G set itself up as a standards making body , a pressure group to encourag e the professional bodies to i ncorporate spreadshee t audit in their examinations 2.0 Higher Education Cou rses And Their Coverage Of Spreadsh eet Risk - 3 - 2.1 Universities Providing Courses Recognised By Professional Bod ies The professional bodies themselves provide distance learning c ourses or other arrang ements for study but there are sever al universities around the UK who offer cour ses preparing students either to sit for the professional audit examinations directly or to g ain ex emption from the same by passin g other rec ognised courses. City Business School, Guild Hall Uni versity , University of Central England and Sheffield Hallam University are but a few of those that provide courses preparing ca ndidates to sit the two papers of the I I A's Qualification in Computer Auditing. Other UK universities, such as Southampton Institute and University of Greenwich, take a different approach. The y have applied to the II A for recognition of their existing undergraduate computing courses covering auditing material at the lower level of the QiCA sy llabus and requesting that these courses be r ecognised as equiva lent to the II A's own professional exam. At the University of Greenwich the HA has permitted students passing two particular undergradua te courses to apply for exempti on from HA studies at the lower level, an arrang ement has been extremely beneficial for the teaching of computer auditing at the university . 2.2 Spreadsheet Teaching At Universities There is no doubt that spreadsheet teaching a t universities suffers from an image pr oblem: • spreadsheets are see n as trivial encompassing simple accounting models, • the package s are easy to learn therefore the intellectual content is limited, • students learn basic skills prior to university and ha ve developed bad habits. But perhaps the major problem is that of convincing ac ademics that spreadsheet teaching requires a complete re-think involving not only change s in teaching content but also chang es in teaching method perhaps accompanied by the development of a researc h ethos to find appropriate and innovative solutions. 2.3 Encouraging A Research Ethos Fig. 1 Rationale For Deve loping A Research Ethos - 4 - The rationale for developing a spreadsheet resea rch ethos is that it encourages a Research-Teac hing Feedback L oop (Fig.2). I m mediate improvements occur in feedback f rom research to classroom and feedback from classroom to rese arch with the overall effe ct being an improvement in both research a nd teaching leading to an improvement in student learning. Fig 2: Researc h-Teaching Feedback Loop The above approach ne cessitates a close liaison between teaching and researc h which in many universities tend to operate independently of each other. What is required is an approach whereby the teachers and the rese archers work tog ether to improve both the taught content (i.e. sy llabus, the actual substance of imparted knowledge) and also the teaching method (i.e. ty pes of materials, teaching sty le, organization of the student cohort). 2.4 Integrating Research And Teaching Recent research has highlighted the high incidence of errors in spreadsheet models used in industry. In an attempt to reduce the incidence of such errors, a teaching approach has been devised which aids students to reduce their likelihood of making common errors during development. The approach comprises of spreadsheet checking methods based on the commonly acce pted educational paradigms of peer assessment and self - assessment. However, these paradigms are here based upon practical techniques commonly used by the internal audit junction such as peer audit and control and ri sk self - assessment. The result of this symbiosi s between educational assessment and professional audit is a method that educates students in a set of structur ed, transferable skills for spreadsheet error checking which are useful for increasing error-awareness in the classr oom and for reducing business risk in the workplace Chadwick D, Sue R [10] Fig 3. Two Research Strands - 5 - The University of Greenwich has developed its own model (Fig.3) on integra ting teaching c ontent and teaching method and also on integra ting educational resea rch with formal software eng i neering research with reg ard to spreadsheet risks (Fig. 4). I n the latt er there is classroom observation of students and the errors they produce whilst buildi ng spreadshee ts. These findings are fe d into the formal software eng ineering re search process which is curr ently working on a software engineer ing methodology for developing spreadsheets as pieces of softwar e. As the formal resear ch progresses so it spins off new classroom techniques which are tried and tested a nd feedback g iven to the formal research. The c lassroom experience itself produces valuable findings for formal research within the arena of educa tion. Fig 4: The University of Greenwich model showing i ntegration of educational an d softw are engineering research - 6 - 3.0 The Importance of Standards In Training Regimes So far we have addre ssed the methods of training offered by professional bodies and educational institutions. It is necessary now to look at what needs to be taught and the question of providing standards. Standards concerning spreadsheet development and audit are ve ry important because without them, nobody w ould be clear just exactly what needs to be taught and how it is to be assessed. This topic generally needs more thought and debate than was possible in the preparation of this paper but suffice it to say that th e matter has been broug ht to the attention of EuSpRIG deleg ates by Ray B utler at the Amsterdam sy mposium . One of the problems reported by researchers and auditors in the field of spreadsheet risks is that of getti ng and keeping management's attention to the problem. Since 1996, the Information Systems Audit & Control Foundation and the IT Governance Institute have published CobiT ® w hich brings mainstream IT control iss ues into the corporate governance arena. This paper illustr ates how spreadsheet risk and control issues can be mapped onto the CobiT framework and thus b rought to managers’ attention in a f amiliar format. Butler R [11] There is a need for the widespre ad promotion and eventual adoption of sound spreadsheet building and spreadshee t auditing standards. From a look at the training a nd teaching curr icula of academia and hig her education, there a ppears to be no attempt to present students with working standards for spreadsheet deve lopments nor even an acknowledge ment that such may be required. It seems pertinent there fore that EuSpRIG should approach these bodies and suggest that firstly , more emphasis be placed upon spreadsheet error s and that secondly , good practice techniques be promoted wherever possible. In due c ourse perhaps EuSpRIG itself could sugg est t ried and tested guidelines of its own. The CobiT framework seems a useful starting point for EuSpRI G endeavours in this area. 4.0 How can Eu SpRIG Be Involved In Education and Training? Finally , let us consi der where EuSpRI G can go from here. As an interest g roup composed of professional and academic contac ts and with worldwide affiliations it may be able to play a decisive role in encourag ing educational ag encies to chang e their approach to spreadshe et skills training, education and assessment. At a minimum and with little effort initially EuSpRI G could be become more involved with: • promoting a grea ter emphasis to be placed upon spreadsheet tea ching, • promoting research throug h encourag ement of participation in EuSpRIG , • using its influence to seek funds for resea rch-active EuSpRI G members. 5.0 Conclusions The crux of this paper has been to look at perhaps the most straightforward a nd immediate way s in which EuSpRIG ca n grow and contribute direc tly to the problem of spreadsheet risks. EuSpRIG has a r ole to play : • in the development of sy l labi in professional studies, • in curriculum development and research in highe r education, • in the creation and promotion of spreadsheet building and auditing standar ds. - 7 - 6.0 References [1] Spreadsheet Errors: What W e Know. What W e Think We Can Do. Panko R Proceedings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2000, Greenwich, UK, July 2000. http://arxiv. org/abs/0802.3457 [2] Cl assification of Spreadsheet Errors Rajalingham K, Chadwick D, Knight B: Procee dings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2000, Greenwich, UK , July 2000. http://arxiv .org/abs/0805.4224 [3] Build ing Financial Accuracy into Spreadsheets Hawker A: Proceeding s of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2000, Greenwich, UK, July 2000. http://arxiv .org/abs/0805.4219 [4] Detecting Errors in Spreadsheets Ay alew Y, Clermont M, Mi ttermeir R: Proceedings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2000, Greenwich, UK , July 2000. http://arxiv .org/abs/0805.1740 [5] A St ructured Methodology for Spreadsheet Modelling. Rajalingham K, Chadwick D, Knight B; Procee dings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2000, Greenwich, UK , July 2000. http://arxiv .org/abs/0805.4218 [6] Ri sk Assessment for Spreadsheet Developments Butler R (HM Customs and Excise) Proceedings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2000, Greenwich, UK, July 2000. http://arxiv .org/abs/0805.4236 [7] Visual Checking Of Spreadsheets Chan H Q Chen Y: Proceedings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2000, Gree nwich, UK, July 2000. http://arxiv .org/abs/0805.2189 [8] How Important Are Spreadsheets To Organisations? Cleary P M: Proceedings of 1 st I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2000, Greenwich, UK, July 2000 [9] Comput er Auditing Chambers A.D and Court LM ( 1994) Computer Auditing 3rd Edition. P itman. [10] Teaching Spr eadsheet Development Using Peer Audit and Self-Audit Methods for Reducing Errors Chadwick D., Sue R.: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2001, Amsterdam, Holland, July 2001. http://arxi v.org/abs/0801.1514 - 8 - [11] Applying t he CobiT ® Control Framework to Spreadsheet Developme nts Butler R J : Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2001, Amsterdam, Holland, July 2001. http://arxiv .org/abs/0801.0609 [12] Euro C onversion in Spreadsheets O'Be irne P: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2001, Amsterdam, Holland, July 2001 [13] Safer Spreadsheets wi th Model Master Paine J: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2001, Amsterdam, Holland, July 2001. http://arxiv .org/abs/0801.3690 [14] An Evaluati on of the Quality of a Structured Spreadsheet Development Methodology Rajalingham K, Chadwick D, Knight B: Procee dings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2001, Amsterdam, Holland, July 2001. http://arxiv .org/abs/0801.1516 [15] New Gui delines for writing spr eadsheets Raffensperg er J: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2001, Amsterdam, Holland, July 2001 [16] Assuran ce by Control around Is a Visible Alternative to the Tra ditional Approach Ettema H, Janssen P, de Swart J : Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2001, Amsterdam, Holland, July 2001. http://arxiv .org/abs/0801.4775 [17] A Real alt ernative to spreadsheets Knight D,: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPR IG 2001, Amsterdam, Holland, July 2001 [18] Spreadsheet Auditi ng Nixon D, O’Hara M: Proceedings of 2 nd I nternational Sy m posium on Spreadsheet Risks, EUSPRIG2001, Amsterdam, Holland, July 2001