Spreadsheet End-User Behaviour Analysis

Brian Bishop, Kevin McDaid Dundalk Institute of Technology, Dundalk, Ireland brian.bishop@dkit.ie, kevin.mcdaid@dkit.ie

ABSTRACT

To aid the development of spreadsheet debugging tools, a knowledge of end-users natural behaviour within the Excel environment would be advantageous. This paper details the design and application of a novel data acquisition tool, which can be used for the unobtrusive recording of end-users mouse, keyboard and Excel specific actions during the debugging of Excel spreadsheets. A debugging experiment was conducted using this data acquisition tool, and based on analysis of end-users performance and behaviour data, the authors developed a ‘spreadsheet cell coverage feedback’ debugging tool. Results from the debugging experiment are presented in terms of end- user debugging performance and behaviour, and the outcomes of an evaluation experiment with the debugging tool are detailed.

1. INTRODUCTION

The reported usage of spreadsheet programs spans a wide variety of job functions, purposes and industries. In a survey of nearly 1600 respondents, Baker et al [2006] found that spreadsheets were used by end-users in various job functions including finance, engineering, manufacturing, marketing, sales and administration, and for many different purposes, such as maintaining lists, analysing and tracking data and determining trends. Iyengar & Svirbely [2005] reported on the usage of a website that made available medical algorithms in the form of MS Excel files, and the users of the website included physicians, nurses, healthcare professionals, computer scientists etc. Maybe more than any other industry, spreadsheets are of critical importance to the finance sector [Croll, 2005]. In a study on the use of spreadsheets in organisations in the City of London, Croll [2005] found that with regard to the financial markets:

“Excel is utterly pervasive. Nothing large (good or bad) happens without it passing at some time though Excel.”

The most recent study that investigated spreadsheet error rates was conducted by Powell, Baker & Lawson [2007], who reported that of the 50 real-world operational spreadsheets they audited, 94% contained errors. To help alleviate the problem of spreadsheet errors, a number of tools are available to aid in the debugging of spreadsheet programs. These tools are both academic [Burnett et al, 2002], [Abraham & Erwig, 2007], [Clermont & Mittermeir, 2003] and commercial products such as Spreadsheet Professional (www.spreadsheetinnovations.com ) and RedRover Audit (www.redroversoftware.com ). In order to develop tools that complement end-users natural behaviour, some knowledge of that behaviour would be required. The concept of human-centered development in software engineering could be applied to the development of spreadsheet debugging tools. Norman [1999] stated that “At its core, human-centered product development requires developers who understand people and the tasks they wish to achieve. It means starting by observing and working with users”. At present, there is very little empirical Spreadsheet End-User Behaviour Analysis
Brian Bishop, Kevin McDaid Proceedings of EuSpRIG 2008 Conference " In Pursuit of Spreadsheet Excellence " ISBN : 978-905617-69-2 Copyright © 2008 European Spreadsheet Risks Interest Group (www.eusprig.org ) and Authors 142 data available to researchers on the processes and actions of end-users while debugging Excel spreadsheets.

This paper details the design and application of a novel custom built data acquisition tool. An experiment was conducted with 47 subjects (professionals and students), the aim of which was to record and analyse the performance and behaviour of expert and novice end-users while debugging an experimental spreadsheet model. Using the data gathered by the data acquisition tool, analysis was conducted which led the authors to develop a simple ‘spreadsheet cell coverage feedback’ debugging tool. Results from the experiment are presented in terms of end-user debugging performance and behaviour, and the outcomes of an evaluation experiment with the debugging tool are detailed.

The layout of the paper is as follows. Section 2 details data acquisition methods currently available and commonly used, and the custom built data acquisition tool developed by the authors. In Section 3, an experiment in which the data acquisition tool was utilised is described, and results are presented in terms of expert and novice performance and behaviour. The debugging tool developed by the authors is detailed in Section 4, along with results of an evaluation of t

…(Full text truncated)…