Problems and Pitfalls

of Educational IMM Development


Rob Phillips
Curtin University of Technology


Introduction

This essay and associated discussion will be more practical than many of the topics discussed to date on ITForum, which have been rather theoretical in nature. I proposed a practical discussion because I am a practical person, and most of my involvement in Instructional Technology has been practical; in the development and management of educational interactive multimedia (IMM) software packages.

I also believe that it is appropriate that this forum seeks a balance between theory and practice. After all, the theoretical ideas discussed in many previous essays need to be tested and evaluated in the form of real pieces of software before they can be accepted as valid or invalid and new advances made.

So, what I am trying to do with this essay is to initiate a discussion about the problems and pitfalls of educational IMM Development. My audience is not necessarily the Instructional Design professional, but it may be. Instead, my audience is the set of people from around the world who have become excited about the possibilities of the new instructional technology, and who have developed their own IMM packages. There are literally hundreds of people throughout the world who have attempted this, with varying degrees of success. For instance, in Australia, $A4-5 million is available each year for teaching development grants, many of which use instructional technology. What can be learnt from the efforts of these early-adopters?

Many of these people/groups would have started from scratch, with little appreciation of the existing body of knowledge in the area. I certainly did. A typical model is that an academic has a good idea for a teaching innovation, attracts funding, and then simply hires a programmer to do the "work." This approach pays scant attention to the range of skills required to develop educationally effective IMM on time and on budget. In addition to programming, a range of other skills are necessary. What about graphic design? What about project management? Dare I suggest Instructional Design skills are necessary?

Format

I would like to use this opportunity to experiment with a different type of scholarly discourse. This "essay" will be relatively brief, to set the scene, so to speak. Then, I would like you, the reader, to contribute your reflections on the problems you have experienced in your own projects. I will then moderate the responses and suggest ways in which people could improve the processes by which they develop IMM. My responses may lead to further discussion.

I intend this to be a form of ITForum paper, which can lead to active participation. However, if YOU don't respond, the attempt will fail.

The alternative approach would be for me to simply write about a range of common problems and how to avoid them. You might agree with all I say, say "ho-hum," and go on to the next message. More importantly, some of you may have had different problems than I have identified, and this might not come to light. Everyone will get more out of the process if you reflect on your problems, and then describe them to ITForum.

Background

Curtin University is a new and dynamic university of 25,000 students, with a main campus, four satellite campuses and numerous students taking courses overseas, either in adjunct institutions, or by distance education. Curtin was the second university in Australia to have a strategic plan for Information Technology, and this strategic plan led to the modern network infrastructure which is in place, and to a range of developments in the educational use of technology. The Information Technology Strategic Plan Services section, which has four academics and a range of technical staff. Academic Services has a general role to promote the use of all aspects of Information Technology in academic life. Part of this role has been to develop educational IMM software in collaboration with teaching academics from across campus. Over 20 projects have been developed since 1992, and I have had a role in most of them, either as project coordinator, project manager, and/or instructional designer.

It is unusual, but historically fortuitous, that a support department such as a Computing Centre should have academic staff. However, the availability and flexibility of a pool of technical experts , and the presence of computing academics to bridge the gap between the techo's and the teachers, has contributed greatly to the success of our projects.

Lloyd Rieber was prompted to invite me to contribute to ITForum because of a book written by a range of Curtin staff, and edited by myself, which sought to dissect the processes of IMM development and highlight problems experienced in such development. The Developers Guide to Interactive Multimedia was initially intended for internal use only, but it became clear to me that it could have a wider appeal in the industry. The points that I will make later in this week's discussion will largely arise from the Developers Guide, which has been accepted for publication by Kogan Page (London). (If only I could get the legal issues sorted out!)

Questions

If you have been involved in the development of courseware of any type, then I invite your contribution. I would like you to reflect on the development process, and ask yourself some questions (outlined below). Even if you decide not to contribute, it will still be worthwhile to you personally to think about the process. Think about the successes of the project. More importantly, think about what went wrong. What would you improve next time?

I would like you to be forthright about your contributions. What were the real problems? You need to be honest about this and share your experiences so that others can learn from your mistakes. I want you to depart from the paradigm of a traditional published paper. If that medium is to be believed, every project ever started was successful, with a few minor glitches!

Culturally, it is probably easier for Australians to be so open about lessons learned from failures than for people from other cultural backgrounds. If you find it difficult to contribute openly, please send your comments directly to me, and I guarantee to repost them anonymously as part of one of my responses to the list.

Some questions you might consider in your reflection on your project are:

Planning

Were your plans too ambitious? Did you bite off too much to chew? If you did it again, what would you change? Would you ever do it again? Did you have enough funding? Did it take longer than you thought? Did the project finish? Is it in use?

Design

Are you satisfied with the result overall? Are you satisfied with the instructional design? Was there sufficient student interaction with the software? Are you happy with the way it looks?

Process

Did parts of it turn out differently than you envisaged? Were there arguments about who said what, when? Did design flaws surface late in the project? Are there bugs in the program? How many? How much effort did it take to get rid of them?

In framing your responses, give a brief overview of the project so that readers get a context in which to read your analysis. What is its place in the curriculum? What are the average hours of use by students? Then describe your views about the project, with the assistance of 20-20 hindsight.

The case study below, adapted from Chapter 9 of the Developers Guide to Interactive Multimedia, is an example of how you might present your thoughts, although it might be longer than necessary.

Case Study

In the Bachelor of Science (Nursing) program, the subject "Dosage Calculations" represents only a small component of the curriculum, yet it is vitally important. Important not only in terms of the patient's well being and safety but also in respect to the students' welfare and academic progress.

Prior to this project, the teaching of dosage calculations used a lecture format with worksheets for practice, both of which were devoid of clinical contextual reality. Students had three opportunities to demonstrate competency, with a performance level set at 100%. This approach was problematic, mainly in terms of its administration and educational validity. Organizing worksheets and assessments for student groups of 80 to 120 presented a logistical nightmare. The teaching and assessment matrix was inflexible to the needs of large student groups with diverse mathematical backgrounds. The Dosage Calculations program was developed to circumvent these problems.

It was one of the first developed at Curtin and it was mostly developed before we had adopted our current design methodology. Some aspects of the design were not comprehensive enough to define clearly how the program would behave, particularly in respect of the testing modules, which should have been designed at the same time as the rest of the program, instead of being added at a later date.

One of the first problems we encountered was involved with transferring the program from the stand alone machines on which it was developed to the network server where it was to run. In order for the program to work successfully from the server it had to be "locked" to prevent multiple users trying to change the document at the same time. Since the file was locked, the program reverted to its original state every time the user changed screens. When users of the test section clicked to go on to the next question, the program would flip to the next card, revert to its saved form and present them with the original question again!

This problem was initially circumvented by a quick patch up job, but a comprehensive rewrite and redesign was necessary to completely solve the problem. The solution was quite simple, but, because the issues of use on a server had never cropped up before, it was never considered during the initial design. Consequently, more than 50% of the total development time was spent on unfunded maintenance and debugging of the program.

Communications between the server administrator, the programmer responsible for bug fixes and the tutors running the program was poor at some times. Many communication problems were exacerbated by the project leader's maternity leave. This caused the chain of communication to become stretched. Quite often a tutor would attempt to report a bug to the programmer, who wasn't available. The tutor would then contact the project leader at home with a verbal bug report, which she would subsequently pass on to the programmer, who then had similar problems in communicating that the bug was fixed, and requesting that the server administrator update the production version on the server.

Despite all this, the program is now working well and is in regular use as part of the curriculum.

Closing Remarks

At the conclusion of the week, I will consolidate all the threads of discussion onto a series of linked web pages, so that new members of the list can follow it. It may also be easier to read on the web.

So now, all of you who got this far through the essay, it is up to you. What are your experiences in educational software development?

Rob Phillips
Computing Centre
Curtin University of Technology
GPO Box U1987
Perth 6001, Australia

E-mail: r.phillips@curtin.edu.au



The buttons that appear below will be found at the bottom of each page of the discussion. The first button will take you back to the previous page (in this case, to the beginning of paper #15). The middle button will take you to the ITForum home page. The last button takes you forward into the discussion as it progressed on-line.