Reproduced by kind permission of Paul Cragg
This paper is based on a working paper dated September 1992. The working paper reviewed 57 papers relating to spreadsheet based DSS. The working paper was not formally published and has not been updated to include research papers since 1992.
A REVIEW AND RESEARCH AGENDA
FOR SPREADSHEET BASED SYSTEMS
IN END-USER COMPUTING
P.B. Cragg, Canterbury, NZ
and
M. King, Loughborough, UK
ABSTRACT
Spreadsheet based systems are an important part of end-user computing (EUC), yet they have received relatively little attention in the EUC research literature. A review of the literature is offered in order to stimulate research. The review is based on the Ives et al (1980) research framework for MIS. A total of 57 papers are reviewed, covering a broad range of topics, including the framework's major areas of environment, process and system. Most papers are shown to concentrate on issues associated with one area, rather than consider relationships involving variables in two or more areas. This shows that there are many research topics which are relatively unexplored and specific research questions are proposed. In general, the review identifies a need for more research on spreadsheets in natural settings, particularly concentrating on cause and effect relationships.
KEY WORDS: Spreadsheet systems, Review, Research agenda
INTRODUCTION
Spreadsheet modelling is a very significant part of the rise in end-user computing (Lee, 1986). Spreadsheet packages are increasingly pervasive and used by many managers in various settings as well as by many management professionals. The use of spreadsheet packages has moved well beyond simple applications and they have become a wide ranging decision support tool used in many management situations (Clarke 1992, Eom and Lee 1990 ).
Managers are attracted to spreadsheet packages because they are "apparently" cheap, "apparently" easy to learn and quick to provide results in a form readily appreciated by managers. However, they are also valued because they enable managers to be independent of Finance Directors and MIS departments and adopt a Do-it-Yourself approach to systems development (King et al 1991). It is clear that spreadsheet modelling has the potential to allow users to develop their own reliable, robust and flexible decision support systems but the independence they facilitate can create problems (Batson 1986, Brown & Gould 1987, Creeth 1985, Mason & Keane 1989, Schofield 1987, Ronen et al. 1989). The models may contain a variety of types of errors which can easily go undetected. As a result incorrect information may be used in decision making. Though erroneous spreadsheet models used in practice are not commonly reported some typical examples are available (Hayen & Peters 1989).
In an exploratory study of spreadsheet modelling practice, involving ten organisations, Cragg and King (1993) examined 20 spreadsheet based systems in detail. 25% of the systems contained errors detected after about one hours checking. All the faulty systems contained an incorrect specification of the range. The testing processes were inadequate and less than half systems had adopted features to avoid corruption while in use or incorrect data entry. There was little documentation and few attempts to identify versions or modification dates. The majority where entered into a computer without any design taking place. Six of the developers admitted there had been problems with their models.
The potential problems of the explosion in end-user computing fuelled by spreadsheet packages have been a motivating factor for some of the research in model management and end-user computing. For instance, the statement that "the rampant proliferation of spreadsheets has caused major headaches for management" is an important starting point for a series of articles (Dolk & Konsynski, 1985, Dolk 1986 and Dolk 1988) concerned with model management systems, model databases, model administration and information resource dictionaries. Similarly the review of model management research by Baldwin et al 1991 gives prominence to the problem of spreadsheet packages. Further the growth in the use of spreadsheets is the starting point for the review of organisational issues of end-user computing by Amoroso 1988. However, there appears to be little reference in these reviews to research which examines the take up of model management theories by organisations or, more significantly, on the practical impact the proposed systems have on the quality of spreadsheet based systems in organisations.
The popularity of spreadsheets and their potential as a major decision support tool suggests that it is important to review what research has been conducted in this area and, if possible, identify areas where further research should be conducted. The aim of this paper is to provide such a review and attempt to identify important further topics for research. The framework used for analysing the research areas is based on the Ives et al (1980) framework for analysing research in computer-based management information systems. The limitations of the review are explained before each area of the framework is taken in turn to report the existing research results and suggest areas worthy of further research.
A FRAMEWORK FOR REVIEWING SPREADSHEET RELATED RESEARCH
Ives et al (1980) presented a comprehensive framework for research in Management Information Systems. This framework has become relatively popular among IS researchers (Teng and Galletta 1991). The core of the framework was the identification (p 919) of three major aspects of Information Systems research. One aspect concerned the information subsystem itself, the second the environment of the subsystem and the third was the processes associated with the subsystem. In the present application of this framework the first aspect represents characteristics of spreadsheet based systems, both in particular cases and generally in an organisation. Within each aspect of the framework Ives et al delineated several classes of variables which are shown in figure 1 together with the subgroupings suggested by Ives et al.. Figure 1 contains some additional variables not in the original framework and these are described in more detail as each aspect is discussed.
INSERT FIGURE 1 ABOUT HERE
To help classify the research areas, Ives et al represented the three aspects by three overlapping circles. This can be viewed as 7 different areas of research, which is a slightly different numbering system from that used by Ives et al. Areas numbered 1, 2 and 3 represent research which concentrates on factors within one aspect only of Figure 1. An important point of the framework is to highlight the overlapping areas numbered 4 through to 7. These areas represent research which considers the relationships between one or more variables in one aspect with one or more variables in a different aspect.
The research review proceeds by taking each of the numbered areas in turn and explaining the structure of the variables considered in that area. Those articles which fall in the area are reviewed and suggestions for further research in that area are made.
There are three limitations to the current review. First, this review has only considered literature which specifically discusses spreadsheet software. The two large bodies of literature concerned with the general issues surrounding decision support systems (DSS) and end-user computing (EUC) have not been reviewed, although a few references from these two areas are included because they were already known to have direct relevance to specific spreadsheet related issues. Nevertheless, these two areas of general issues in DSS and EUC are likely to contain a considerable amount of further research which would help provide a more complete picture of the current state of spreadsheet research. Further studies should take this body of knowledge into account.
A second limitation concerns the completeness of the review, rather than the scope. The review attempted to identify all major pieces of work using ABInform and citations. However, some major work may have been omitted, and further studies may bring others to light.
A third limitation concerns the analysis that has been conducted on the papers. Spreadsheets can be used for many tasks, for example decision support, data analysis, and graphical presentation. This review made no attempt to classify the type of application.
AREA 1 RESEARCH : SPREADSHEET MODEL CHARACTERISTICS
The variables that Ives et al (1980) suggest for the information subsystem need clarifying since the original framework was developed with organisational computing in mind, rather than end-user computing. They provided three classes of variables under the headings of Content, Time and Form. However, it is important to include in this aspect a class of variables which measure the quality of the system in terms such as the number and type of errors it contains and the potential problems it may lead to as well as assessments of whether the system incorporates good spreadsheet practice. The variables considered under time and form are still appropriate but the content class needs some additions. Clearly variables measuring the size and sophistication of the system would fall in this area and it may be important to assess whether systems are simple or complex. Content variables which describe the features used by the system and what the system does are also relevant and it seems appropriate to supplement these with assessments of the purpose of the systems. This would include consideration of why the system is built, the nature of the application and the type of decision making it is supporting and leads onto research into some aspects of how spreadsheet activity grows and spreads in organisations.
The study by Lee (1986) shows that spreadsheets are popular; 74% of his sample of 311 PC users were spreadsheet users. Some of this use may be for trivial purposes, but other studies show that spreadsheets are used for serious applications. Eom & Lee (1990) analysed DSS applications published between 1971 and 1988, and identified 24 of 203 specific DSS as being based on spreadsheet modelling. Also, an extensive review of the strategic planning literature from 1980-1990 revealed 31 users of spreadsheets as a tool in 766 strategic planning papers. This made spreadsheets the 6th most popular management science tool in strategic planning (Clarke, 1992). It is possible that these reviews under-estimate the popularity of spreadsheets, as for example, a forecasting system could be spreadsheet based, but classified in the studies under forecasting rather than spreadsheeting. Some examples of the wide use of spreadsheets are: manpower planning (Anthony & Wilson 1990); optimisation (Roy, Lasdon & Plane 1989 and Trout et al 1991); market segmentation (Winter, 1989); decision tree analysis (Jones 1986); stochastic simulation (Seila & Banks 1990); regression analysis (Nugus 1991); forecasting (Mumford et al 1991); and costing (Wellman 1992).
A common theme in much of the professional literature (eg Ditlea 1987, Freeman 1986 and Schofield 1987) is the concern about the quality of spreadsheet based systems. In at least one case of an error, legal action was threatened (Gilman & Bulkeley, 1986). A study by Cragg & King (1993) and another by Davies & Ikin (1987) looked for errors in spreadsheet models used by organisations. Cragg & King examined a sample of 20 systems and found about 25% contained logical or data errors but suggest the true error rate could be higher. They also indicate weaknesses which could lead to errors in use. For example, poor separation of data and logic and little use of cell protection. Stang (1987) reported twenty-five types of errors that he had come across during his training and consulting.
Although there has been significant research in this area, further work is needed including the following issues:
AREA 2 RESEARCH : ENVIRONMENT
The environment covers an enormous possible area for research and five classes of environmental variables are delineated by Ives et al 1980. The first such class is the external environment which included legal, social, political, cultural, economic, educational, research and industry/trade considerations. However, it is important to note that for spreadsheet based systems this set of variables includes the external technological environment which covers all aspects of commercially available software including such considerations as the potential of spreadsheets, the limitations of typical software, the merits of additional software and alternative software.
The second class is the organisational environment which covers all aspects of the organisation as it is, as well as what might be termed company policy which represents what the organisation has decided should be occurring, or what organisations have been recommended to do. This distinction is important in the other sets of variables. The remaining three classes of environmental variables remain as the user environment, the development environment and the operations environment. Policy issues are important in the user environment and the development environment and so these include policy matters in connection with training and support, as well as information systems development.
Some of the spreadsheet literature focuses on the external technology environment and discusses both the potential and the limitations of spreadsheet software. Bodily (1986) discusses the potential for spreadsheets to be used for decision tree analysis, expert systems, optimisation, risk analysis, and statistical analysis. McLaughlin & Sullivan (1988) examine the limitations of typical spreadsheet software, listing no audit trail, only two dimensions, limited statistical and optimisation capability, and weak integration with word processing and data communications. They review add-in programs, enhanced spreadsheets and database software that could be used to overcome these limitations. Keane & Mason (1989) and Mason & Keane (1989) also examined weaknesses, particularly regarding data management and posed ten questions to help a user decide whether a spreadsheet should be used. Creeth (1985) goes even further and argues for the use of modelling rather than spreadsheet software, unless the builder was to be the sole user, and there would be no need for updating. Additional software can help overcome some of the limitations. Raskin (1988) discusses three add-ons that help a developer add notes, and hence aid documentation. Spreadsheet audit software has been discussed by McMickle (1987), and Simkin (1987).
Lee (1986) showed that microcomputers are used by all levels within an organization, but particularly by professionals and middle managers. Studies of accountants show that they are heavy users of spreadsheets (Berry & McLintock 1991, Coy & O'Grady 1992), suggesting that about two-thirds of accountants use spreadsheets, with many now as long time users. Many accountants build models for external clients as well as for use within their own organization.
Influences on use and non-use of spreadsheets by individuals have also been studied. Lee (1986) identified "lead users" as playing an important role in assisting other users. Lead users tended to use PC's more and to have stronger backgrounds in computers. In Carlsson's (1989) study, eleven of fourteen employees who had shown a real desire to spreadsheet by attending a course, did not use a spreadsheet after the course for a number of rational reasons. Some decided that a spreadsheet was not appropriate. Others wanted to spreadsheet, but had trouble doing so, due to lack of time, lack of support within the organization (not just IS support, but also from superiors and colleagues), and from an inability to modify their job. King & McAulay (1989) and King, McAulay & Cragg (1991) show there are unexpected reasons for accountants and managers wanting to use spreadsheet based systems for decision support, whereas King et al (1991) point out some accountants attempt to limit usage of spreadsheet systems by other managers. They report cases where managers were deliberately deterred from going on training courses by finance directors in an effort to retain control of spreadsheet developments within the finance department. They also report other cases where the opposite view is taken and finance staff encourage other managers to develop spreadsheet modelling skills in the hope of building more integrated systems.
Turning to some of the policy issues for the organisational aspects of spreadsheet developments the little research evidence that is available suggests that many organizations have relatively few specific controls in place for spreadsheet modelling. Of the ten firms studied by Cragg & King (1993), only one had formal spreadsheet standards. For this reason, Mason & Keane (1989) argued for the appointment of data model administrators, to encourage and control spreadsheet practices within an organization. Most other relevant research on organizational policies and support applies to EUC in general, rather than spreadsheet modelling specifically. In particular the extensive model management literature (Baldwin et al 1991) is relevant and in some case specifically refers to spreadsheet problems (eg Dolk 1986 & 1988 and Dolk & Konsynski 1985).
Williams (1987) offers a comprehensive set of standards on how spreadsheeting should be supported and controlled within an organization. These cover approved software, minimum spreadsheet skills, specification, documentation, review, security, and on-going management.
There is a large amount of literature, particularly in accounting journals, which offers advice on spreadsheet development. Ronen, Palley & Lucas (1989) propose a Spreadsheet Development Life Cycle (SDLC), particularly for models which were to be used again, or by another user. Their SDLC included problem identification, model definition, model construction, testing, documentation and auditing. Others to provide comprehensive advice on the whole process are Batson (1986), Batson & Brown (1991) and Williams (1987). With the elimination of errors in mind, various validity checks have been proposed (Berry & McLintock 1991, Chussil 1988, Pearson 1988, and Simkin 1987). Many authors advise on the use of spreadsheet audit packages to help identify errors. Simkin (1987) provided a brief review of ten packages useful in auditing spreadsheets. Many authors offer advice on good design practices in order to overcome the concerns about errors an poor documentation and security. For systems that are to be used on a regular basis, Litecky (1991) offers rigourous advice on the design and programming of input functions, using a modular approach.
Despite the large amount of literature considering various aspects of the environment there are other issues of importance in this area which do not seem to have been studied, such as:
AREA 3 RESEARCH : PROCESS
Ives et al (1980) identified three major classes of process variables related to development, use and operations. The development process measures different aspects of the development of the system. The use process focuses on the impact of systems on the user. In the case of spreadsheet based systems the developer and user are often the same individual so the concept of secondary and primary user is less relevant. Hence the operations process merges with either the development process or use process, which themselves overlap. Process variables are predominantly performance measures such as resources used and different assessments of satisfaction as well as the impact on decision making and productivity.
There have been a number of studies of the development process. Some have been experimental, while others have been in natural settings. Brown & Gould (1987) discuss their experiment of how nine experienced users each built three models. Typically little time was spent on planning and testing. Many types of errors were identified, and occurred at different stages throughout the building process. Although all were experienced users, each had at least one major error, but were confident their models were error free. In another experimental situation, Floyd & Pyun (1987) requested fourteen experienced users to develop two models, and to modify two others. Over 450 errors were recorded (using video), and many of these errors were serious. They also identified excessive cursor movement during the building process, which indicated the need for improved visual feedback within the software.
There have been two studies reported of the spreadsheet development process in an organizational setting. Carlsson (1988) monitored three users over a period of thirteen months, giving particular attention to the different information sources that were used. Carlsson found that model building required more time than model use, so a lack of spare time hindered model building. Napier et al (1992) also recorded keystrokes for forty experienced users in their normal work. They report on the frequency of use of various commands and functions. They use this data to recommend improvements to the user-interface of spreadsheet package, so that modellers can more easily carry our common activities. They also found that experienced users use relatively few commands, and speculated that this was due to a lack of training at intermediate and advanced levels.
A longitudinal study of the development and use process in a natural setting involved follow-up interviews for 24 models, eighteen months after an initial interview (Klepper & Sumner, 1988). During this period two-thirds of the systems underwent "evolutionary change", to incorporate more data or variables. The study identified the turnover of skilled personnel as a major threat to end-user systems, rather than poor spreadsheet practices.
The study by Cragg & King (1993) reported many problems in the development of 20 systems. Typically users designed their models at the computer, which led to a high modification rate, including significant restructuring in the later stages. Six of the developers admitted there had been problems with their models. All the models were tested, but rarely by someone other than the builder, and 25% still contained errors which cast doubt on the efficacy of the testing.
Studies of the development process utilised various tools for data collection. Olson & Nilsen (1987) and Floyd & Pyun (1987) used video, while Napier et al. (1992) and Brown & Gould (1987) used keystroke recording software. Carlsson & Stabell (1986) discuss ways of capturing key strokes.
Although there have been a number of studies reported which relate to the process aspects of spreadsheet systems there are still areas in need of research such as:
AREA 4 RESEARCH : OVERLAP OF SPREADSHEET AND PROCESS
All the overlap areas represent research which examines the influence of one or more variables from one main area on one or more variables in another main area. Area 4 examines relationships and hypotheses linking the spreadsheet system characteristics with process variables.
No studies were found in this area, although it is clearly an important area. Some of the more obvious topics of concern are:
AREA 5 RESEARCH : OVERLAP OF SPREADSHEET CHARACTERISTICS AND ENVIRONMENT
This area represents research that considers relationships between environmental characteristics and spreadsheet system characteristics. As this review is considering the spread of the range of applications of spreadsheet systems in an organisation as part of the spreadsheet characteristic area, Area 5 includes research on what influences the use of spreadsheets and the growth of spreadsheet systems in an organisation.
Brancheau & Wetherbe (1990) studied the adoption of spreadsheet software, using innovation diffusion theory. They found that inter-personal channels of communication dominated all phases of the decision to adopt a spreadsheet, with IS staff playing a minor role. Early adopters tended to be younger, more highly educated, more attuned to mass media, more involved in interpersonal communication, and more likely to be opinion leaders.
There appears to be a rather complex set of factors influencing the growth in spreadsheet based systems. King et al (1991) show that there is often conflict between end-user managers and professionals such as accountants and IS staff. One response to this conflict is a desire for independence by end-user managers and this has led managers to the independent (and sometimes clandestine) purchase of PCs, the independent (and sometimes clandestine) acquisition of spreadsheet modelling skills and the independent development of their own spreadsheet systems.
Some further aspects of the interplay between personal power, organisational politics, IS skills and the spread of spreadsheet based systems are explained by King and McAulay (1989) through an in depth case study of a large organisation. It is shown that individuals promoted their own spreadsheet based systems for a variety of personal factors including distrust, dislike or failure to communicate with specialist staff. In other situations individuals became excited and enthralled by the fun of working with spreadsheets and so were "seduced by the technology", and others tried to use their skill in developing their own spreadsheet systems to impress others and gain power and influence.
No further studies were identified as falling in this area but there are a series of important issues which should be researched such as:
AREA 6 RESEARCH : OVERLAP OF ENVIRONMENT AND PROCESS
The relationships between environmental factors and process measures are a very common theme in much IS research. This area includes studies of how any of the environmental factors affect the various performance measures especially variables associated with the development and use processes. There appears to be relatively little research in this area which is specifically concerned with spreadsheet based systems, although there are several studies relating skills and training to the development process.
An experiment to examine the impact of two different possible control mechanisms on spreadsheet development was conducted by Alavi, Phillips & Freedman (1990). It was found that developers were happier with less control but that they produced less documentation, carried out less testing and had fewer back-up provisions. The evidence supported the need for a "refinement" phase towards the end of the development of a spreadsheet, where an independent review was undertaken by an end-user support group.
Mackay & Lamb (1991) studied novices and experts. The novices and experts were classified according to their knowledge of both the task-domain and of the particular spreadsheet software in use. It was found that both areas of expertise were important in influencing the development process. Different problem solving strategies were adopted by the different types of users, showing that spreadsheet training was essential. Bostrom et al (1988) argue that this training needs to be tailored to the type of user, as they found development performance differences among 70 novices, classified using Kolb's learning style inventory. The study by Hicks et al (1991) shows that computer-based instruction can serve an assisting role to human instruction, by helping with syntax. Subjects who received human instruction rather than computer-based instruction, had a greater ability to comprehend and apply spreadsheet concepts in the development process.
There are more general studies on the education and training needs of end-users which are relevant to the development of spreadsheet based systems. Igbaria & Zviran (1991) look at the impact of cross-cultural differences on end-user effectiveness and conclude that the type of training and support that is most effective in organisations depends on the cultural setting. Nelson (1991) conducted a survey of the perceived educational needs of end-user personnel and recommended that organisations should tailor their IS training to individual users needs as this was the most effective way to reduce the risks associated with end-user computing.
An experiment on error detection in the development process by Galletta et al (1990) found that error detection is not easy. The accountants in the study were best at detecting accounting type errors. Spreadsheet experts were faster than novices at detecting errors.
There are papers about the technology, but few studies of how features affect use. An exception is Baxter & Oatley (1991) who conducted a novice-expert experiment with two different spreadsheet packages. They found little difference between the two systems in terms of ease of learning, although good documentation did help learning. This suggests that documentation can influence the resources required for the development process. In a similar study Olson & Nilsen (1987) did find differences in the time to accomplish a task when they had experimental subjects using two different packages.
Although there are published results covering parts of area 6, there are other aspects of the relationship between environmental factors and the development and use process which clearly deserve attention. Some interesting issues are:
AREA 7 RESEARCH : ALL GROUPS OF VARIABLES
This area is concerned with the interaction of variables from all the three major aspects in the framework. It does not include research work that considers separately questions from two of the previous overlap areas. It only includes studies of more complex relationships that simultaneously bring together variables from the environment, the process and spreadsheet features, linking them together in some interactive hypothesis. None of the research covered in this review has been allocated to this area, but there are some important research topics which fall in this area, such as:
CONCLUSIONS
Before summarising the findings of the literature review, it is important to recall the review's limitations. In particular, the review focused on literature on spreadsheeting in general. This general perspective was adopted in order to provide a broad picture of spreadsheeting.
The review showed that spreadsheets are a popular software environment, particularly among accountants. However, spreadsheet applications are of questionable quality. Their use in organisations has been influenced by close colleagues rather than MIS departments. The literature offered advice on how applications should be developed, and on how spreadsheet activities should be managed.
The review identified many opportunities for research on spreadsheet based systems. Using the framework for IS research by Ives et al (1980), most of the past research has been limited to four of the framework's seven areas. Even in these areas, further research topics were identified. Furthermore, in the remaining areas of 4, 5 and 7, few studies were identified. These provide considerable research opportunities, where cause and effect relationships must be studied.
While experiments have proved to be a useful research method, there have been few studies of spreadsheets in natural settings. As a result, we have little insight into some of the most important issues concerning the development, use and management of spreadsheets in organisations. For example, there is literature that expresses considerable concern over the quality of spreadsheet applications. Yet there is little evidence of the significance of this problem. Another example concerns applications development, where there is a large literature indicating how spreadsheets should be developed. However, there is scant evidence on actual practice, and on whether such methods improve practice. A third example concerns spreadsheet use. We know that spreadsheets are popular in organisations, yet we know little about the factors that encourage use and non-use. Furthermore, we know very little about the role and impact of spreadsheets in the decision making process. A final example of our poor understanding of spreadsheets in organisations, concerns information system policies. Some advice is available on policies to support spreadsheeting in organisations. However, there is little evidence from organisations of the practicality and impacts of such policies.
The above examples show that there are some very important gaps in our understanding of spreadsheeting in organisations. Further research will require a greater emphasis on studying spreadsheets in natural settings. A greater use of longitudinal studies will be required, as many important topics involve development and use processes, which occur over time. It is hoped that this review and research agenda will stimulate research in this area, and provide it with the recognition it deserves as a major aspect of end-user computing.
REFERENCES
1. Alavi, M., Phillips, J.S. & Freedman, S.M. (1990), An Empirical Investigation of Two Alternative Approaches to Control of the End-User Application Development Process, DATA BASE, 20:4, Winter, pp. 11-19.
2. Amoroso, D.L. (1988), Organizational Issues of End-User Computing, DATA BASE, Vol. 19, No. 3/4, pp. 49-57.
3. Amoroso, D.L. & Cheney, P.L. (1992), Quality End User Developed Applications: Some Essential Ingredients, DATA BASE, 23:1, Winter, pp. 1-11.
4. Anthony, S.A.M. & Wilson, J.M. (1990), Manpower Modelling using a Spreadsheet, Omega, 18 pp. 505-510.
5. Baldwin, A.A., Baldwin, D. & Sen, T.K. (1991), The Evolution and Problems of Model Management Research, Omega, 19:6, pp. 511-528.
6. Batson, J. (1986), Spreadsheet Good Practice, Accountants Digest, 200, winter.
7. Batson, J. & Brown, A. (1991), Spreadsheet Modelling Best Practice, Accountants Digest, 2730, Autumn.
8. Baxter, I. & Oatley, K. (1991), Measuring the learnability of spreadsheets in inexperienced users and those with previous spreadsheet experience, Behaviour & Information Technology, Vol. 10, No. 6, pp. 475-490.
9. Berry, B. & McLintock, A. (1991), Accountants and financial modelling, OR Insight, 4:4, pp. 11-14.
10. Bodily, S.E. (1986), Spreadsheet Modeling as a Stepping Stone, Interfaces, 16:5, pp. 34-52.
11. Bostrom, R.P., Olfman, L. & Sein, M.K. (1988), The Importance of Individual Differences in End-User Training: The Case for Learning Style, in Awad E.M. (Ed), Proceedings of the 1988 ACM SIGCPR Conference, Maryland, April, pp. 133-141.
12. Brancheau, J.C. & Wetherbe, J.C. (1990), The Adoption of Spreadsheet Software: Testing Innovation Diffusion Theory in the Context of End-User Computing, Information Systems Research, 1:2, June, pp. 115-143.
13. Brown, P.S. & Gould, J.D. (1987), An Experimental Study of People Creating Spreadsheets, ACM Transactions on Office Information Systems, 5:3, July, pp. 258-272.
14. Carlsson, S.V. (1988), A Longitudinal Study of Spreadsheet Program Use, Journal of Management Information Systems, 5:1, Summer, pp. 82-100.
15. Carlsson, S.V. (1989), Why Johnny Can't or Won't Spreadsheet, Scandinavian Journal of Information Systems, Vol. 1, pp. 118-142.
16. Carlsson, S.V. & Stabell, C.B. (1986), Spreadsheet Programs and Decision Support: A Keystroke-Level Model of System Use, in McLean, E.R. & Sol, H.G. (Eds), Decision Support Systems: A Decade in Perspective, North-Holland, pp. 113-128.
17. Chussil, M.J. (1988), Computer-Based Planning: A PC Survival Guide, The Journal of Business Strategy, Jan/Feb, pp. 38-42.
18. Clark, D.N. (1992), A Literature Analysis of the Use of Management Science Tools in Strategic Planning, J. Opl. Res. Soc., forthcoming.
19. Coy, D. & O'Grady, W. (1992), The Changing Use of Spreadsheets by Accountants in New Zealand: A Comparison of 1991 with 1986, Working Paper, Department of Accounting and Finance, University of Waikato, New Zealand, March.
20. Cragg, P.B. & King, M. (1993), Spreadsheet Modelling Abuse: An Opportunity for OR ?, J. Opl. Res. Soc., forthcoming.
21. Creeth, R. (1985), Microcomputer Spreadsheets: their uses and abuses, Journal of Accountancy, June, pp. 90-93.
22. Davies, N. & Ikin, C. (1987), Auditing Spreadsheets, Australian Accountant, December, pp. 54-56.
23. Ditlea, S. (1987), Spreadsheets Can Be Hazardous To Your Health, Personal Computing, January 1987, pp. 60-69.
24. Dolk, R.D. (1986), Data as Models: An Approach to Implementing Model Management, Decision Support Systems, 2, pp. 73-80.
25. Dolk, R.D. (1988), Model Management and Structured Modeling: The Role of an Information Resource Dictionary System, Communications of the ACM, 31:6, pp. 704-718.
26. Dolk, R.D. & Konsynski, B.R. (1985), Model Management in Organizations, Information & Management, 9, pp. 35-47.
27. Eom, H.B. & Lee, S.M. (1990), A Survey of Decision Support System Applications (1971-April 1988), Interfaces, 20:3, May-June, pp. 65-79.
28. Floyd, B.D. & Pyun, J. (1987), Errors in Spreadsheet Use, WP 167, Center for Research on Information Systems, Leonard N. Stern School of Business, New York University, October.
29. Freeman, R.M. (1986), A Slip of the Chip on Computer Spreadsheets Can Cost Millions, The Wall Street Journal, August 4 1986.
30. Galletta, D.F., Abraham, D., Louadi, M.E., Lekse, W., Pollalis, Y.A. & Sampler, J.L. (1990), An Emprical Study of Spreadsheet Error-Finding Performance, Behavior, and Strategies, WP 707, Joseph M. Katz G.S.B., University of Pittsburgh, August.
31. Gilman, H. & Bulkeley, W.M. (1986), Can Software Firms be held Responsible when a Program Makes a Costly Error? Wall Street Journal, August 4.
32. Hayen, R.L. & Peters, R.M. (1989), How to Ensure Spreadsheet Integrity, Management Accountant, Vol. LXX, No. 10, April, pp.30-33.
33. Hicks, J.O., Hicks, S.A. & Sen, T.K. (1991), Learning spreadsheets: human instruction vs. computer-based instruction, Behaviour & Information Technology, 10:6, pp. 491-500.
34. Igbaria, M. & Zviran, M. (1991), End-user Effectiveness: A Cross-Cultural Examination, Omega, 19:5, pp. 369-379.
35. Ives, B., Hamilton, S. & Davis, G.B. (1980), A Framework for Research in Computer-Based Management Information Systems, Management Science, 26:9, September, pp. 910-934.
36. Jones, J.M. (1986), Decision Analysis using Spreadsheets, European J. of Operational Research, 26, pp. 385-400.
37. Keane, D. & Mason, D. (1989), Beyond Spreadsheets: Dollars and Sense, Interfaces, November 1989, pp. 33-35.
38. King, M., Lee, R.A., Piper, J.A. & Whittaker, J. (1991), Information Technology and the Management Accountant, CIMA, London.
39. King, M. & McAulay, L. (1989), Information technology and the accountant: a case study, Behaviour and Information Technology, 8:2, pp. 109-123.
40. King, M., McAulay, L. & Cragg, P.B. (1991), Experiences in Applying a General Modelling Methodology, Omega, 19, pp. 567-575.
41. Klepper, R. & Sumner, M. (1988), Continuity and Change in User Developed Systems, Working Paper, MIS Department, Southern Illinois University at Edwardsville.
42. Lee, D.M.S. (1986), Usage Patterns and Sources of Assistance for Personal Computer Users, MIS Quarterly, 10:4, Dec., pp. 313-325.
43. Litecky, C. (1991), Spreadsheet-Based Accounting Applications, with Emphasis on Design of Input Modules, The EDP Auditor Journal, Vol. II, pp. 69-78.
44. Mackay, J.M. & Lamb, C.W. (1991), Training needs of novices and experts with referent experience and task domain knowledge, Information & Management, 20:3, March, pp. 183-189.
45. Mason, D. & Keane, D. (1989), Spreadsheets: Solution or Problem, New Zealand Computer Interface, October, pp. 82-84.
46. McLaughlin, H.S. & Sullivan, T.G. (1988), Recent Spreadsheet Innovations: Expansion Paths from Lotus 1-2-3, Journal of Accounting and EDP, Spring, pp. 8-17.
47. McMickle, P.L. (1987), Troubleshooting Lotus Spreadsheets, Journal of Accounting and EDP, 3:2, pp. 69-71.
48. Mumford, I.G., Schultz, D.E. and Troutt, M.D. (1991), Using spreadsheets to generate a distribution of forecasts for electric power demand, Jnl. Operational Research Soc., 42, pp. 931-940.
49. Napier, H.A., Batsell, R.R., Lane, D.M. & Guadagno, N.S. (1992), Knowledge of Command Usage in a Spreadsheet Program: Impact on User Interface Design and Training, DATA BASE, 23:1, pp. 13-21.
50. Nelson, R.R. (1991), Educational Needs as Perceived by IS and End-User Personnel: A Survey of Knowledge and Skill Requirements, MIS Quarterly, 15, December, pp. 503-525.
51. Nugus, S. (1991), Spreadsheets by Design, Certified Accountant, January, pp. 38-45.
52. Olson, J.R. & Nilsen, E. (1987), Analysis of the Cognition Involved in Spreadsheet Software Interaction, Human-Computer Interaction, Vol. 3, pp.309-349.
53. Pearson, R. (1988), Lies, Damned Lies, and Spreadsheets, Byte, Vol. 12/13, December, pp. 299-304.
54. Raskin, R. (1988), Software: Note-It Plus, Noteworthy, and SmartNotes, LOTUS, July, pp. 98-101.
55. Rivard, S., Lebrun, P. & Talbot, J. (1991), Measuring the Quality of User-Developed Applications, Proceedings HICSS, pp.471-478.
56. Ronen, B., Palley, M.A. & Lucas, H.C. (1989), Spreadsheet Analysis and Design, Communications of the ACM, 32, pp. 84-92.
57. Roy, A., Lasdon, L. & Plane, D. (1989), End-user optimization with spreadsheet models, European J. of Operational Research, 39:2, March, pp. 131-137.
58. Schofield, J. (1987), Beware of Spreadsheets, Management Today, February 1987, pp. 39-40.
59. Seila, A.F. & Banks, J. (1990), Use an Electronic Spreadsheet as a Decision Support System, Industrial Engineering, 22:5, May, pp. 40-46.
60. Simkin, M.G. (1987), How to Validate Spreadsheets, Journal of Accountancy, November, pp. 130-138.
61. Stang, D. (1987), Spreadsheet Disasters I have known, Information Center, 3:11, November, pp. 26-30.
62. Teng, J.T.C. & Galletta, D.F. (1991), MIS Research Directions: A Survey of Researchers' Views, DATA BASE, Vol. 22, No. 1/2, pp. 53-62.
63. Trout, M.D., Tadisina, S.K. & Clinton, R.J. (1991), Interactive Optimization Aspects of Electonic Spreadsheet Models of Design and Planning, Jnl. Operational Research Soc., 42, pp. 349-355.
64. Wellman, F. (1992), Software cost estimating - Models and Methods, OR Insight, Vol. 5, No. 2, March-June, pp. 21-24.
65. Williams, T. (1987), Spreadsheet Standards, Touche Ross & Co., February. Winter, F.W. (1989), Market Segmentation Using Modeling and Lotus 1-2-3, Interfaces, 19:6, pp. 83-94.
Figure 1 : CLASSES AND GROUPS OF RESEARCH VARIABLES (based on Ives et al, 1980)
1. Spreadsheet System Characteristics
Classes Subgroupings
Presentation Form -
Time -
Content Size and sophistication
Features used
Purpose
Nature of application and decisions supported
Growth and spreadQuality Actual errors and potential problems
Use of good practice
2. Environmental Characteristics
Classes Subgroupings
External Technological
Non-technologicalOrganisational Goals, tasks, structure, style
Training policy, IS policyUsers Personal characteristics
Attitudes and perceptions
Tasks and functions
Education, training and skillsDevelopment EUC support and training available
Methods, techniques and structures
Planning and controlOperations Resources
Technical support
3. Process Variables
Classes Subgroupings
Development Resources used
Impact on organisation
Participation and satisfactionUse System effectiveness
Satisfaction
Producing and performance impact
Decision making impactOperations Resources used