The value of students as part of the design team for educational software


Dolf Steyn
University of Pretoria
This article originated with a project where students were involved in the design and development process of a multimedia simulation game on the use of analogue multimeters. The project included the development of the progam as well as the evaluation of the learner contribution and learning value of such an approach
Introduction
 
Using students to help in designing and developing followed an approach not unlike that of Rieber (1996). This was done to address the needs created in the current South African educational environment. According to Morris (1996) the South African National Qualification Framework (NQF) expects learners to think analytically and critically. Therefore students were enabled to take control of their learning situation as active participants and to observe their own learning experience along the way.
 
“The multimedia concept involves more than using multiple media for a given instructional purpose. It involves integrating each medium format into a structured, systematic presentation. Each medium is designed to complement the others so that the whole multimedia system becomes greater than the sum of its parts” (Heininch, Molenda and Russel, 1986: 172).
 
In line with this inclusive approach, the potential of the development process as a learning vehicle itself, needed closer attention.
 
Researchers and developers are struggling to find innovative ways to exploit the interactive potential of the learning environments afforded by computers while remaining consistent with psychological and philosophical beliefs about how people learn (Hannafin, 1992). This hesitation to move away from ‘trusted’ ways could be as a result of the invisible nature of the learning functionality within process driven approaches to learning. So much easier to believe that students understand and remember all that is said in class!
 
Another possible reason for the struggle could be found in the highly linear and behavioural structured fashion of traditional computer based training. In contrast, cognitive learning assumes that knowledge is a constructed, rather than a learned response, acquired by active mental processing and collaborative interpretation (Perkins, 1986).
 
Educators often seem hesitant to move away from ‘trusted’ ways.
 
Overview
The project revolved around a single program. Figure 1 shows the accretion of the whole project.

Figure 1: The accretion of the whole project


 

These can be regrouped into three main components to form the scope of this account.
  1. Design and development phase.
  2. Primary testing and evaluation.
  3. Reporting.
The scope of the program
The use of testing equipment is very important in all electrically orientated careers and applications. This is because the presence or flow of electricity and electrical charges are not visible to the naked eye. Testing equipment however opens the possibility of quantification of these concepts. Unfortunately many people never take time to study the practical use of multimeters formally, but rather rely on trial and error to be the tutor. Many aspects in the use of the meters are very simple at face value, but are often neglected in practice.

The software was named Multimaxand is a multimeter simulation game designed to give 12 to 19 year old students the opportunity to apply their knowledge of multimeters in the quest to meet the challenges posed by the program.

Focus

This article addresses two questions:

  1. What is the value of using students as part of the design team for educational software?
  2. What value can students add to the project?
  3. What value can students derive from the experience?
Previous research
 
Pappert (1981) and Rieber (1996) have used students in the creation of microworld program developments. They do not generalise, but show valuable indicators of the possible strengths of students within the development process.
Researchers like Clark (1992), Lanza and Roselli (1991) and Fouche (1995) have done studies to show the effectiveness of computer based training programs. Many of these studies show a tentative preference for this medium of instruction, but without any quantifiable proof.
 
Approach
 
The following process was followed:
  • Development
  • Obtain buy-in from a group of students to participate over a fairly long period of time.
  • Train the team members.
  • Design the program.
  • Author the program.
  • The program
     
    The high cost of multimedia development and the rather small number of technical students, make the South African electronics market an unlikely target population for commercial development. This program can act as example of the value of such projects and could be used to persuade authorities to invest in similar development. The main aim was never the end product, but rather the learning value of the process.
     
    The Development Team

    The students involved in the development will have gained experience within the working environment of software development. Similar career guidance opportunities within the field are limited and could lead to a more informed career choice or a closer defined study field.

    Aim

    In addition to the learning approach experimentation already mentioned, the value of simulation games can be better advocated with the aid of proven examples. It is hoped that this research may therefore contribute to the use of both learner participation and simulation utilisation.
     

    Literature study
     
    Recent shifts in higher education
     
    Factors around education, of which financial constraints are not the least, have forced stakeholders to take note of the changing situation and to adopt new ways. The learners themselves are much more outspoken about their demands on educational structures and the combined result leaves a drastically altered reality for which educators should be prepared.
     
    For another forum it may have been necessary to list the trends and changes in education, but since this is an almost permanent thread on ITForum, I will suffice to say that the traditional educational approach can no longer meet the needs of the learner and that alternative ways should be investigated.
     
    Motivation.
     
    Most humans do not study as a result of natural drive, but rather as the result of some intrinsic and extrinsic factors. There are two main motivational theories. The one from Malone and the other from Keller. Both theories are important but, since they are often referred to, they too will not by repeated here.
    It is the view of the author that motivation be viewed inclusively rather than exclusively.
     
    Flow theory
     
    This is yet another theory sometimes mentioned in ITF postings, but this may however be less known and could bear some explanation.
     
    For some twenty years, Dr. Mihaly Csikszentmihalyi has been involved in research on topics related to flow (Csikszentmihalyi, 1990).
     
    What is flow?

    In spite of various motivational theories, people often report doing tasks because they were caught up in the "flow" of it. Csikszentmihalyi’s respondents describes flow as “It was like floating” or “I was carried on by the flow” (Csikszentmihalyi, 1990).

    Rieber (1996, p.6) defines "Flow" as follows:

    “Flow theory gets its name from the way so many adults have described a peculiar state of extreme happiness and satisfaction. They are so engaged and absorbed by certain activities that they seem to "flow" along with it in a spontaneous and almost automatic manner - being "carried by the flow" of the activity.

    Csikszentmihalyi (1990, p. 4) defines flow as"...the state in which people are so involved in an activity that nothing else seems to matter; the experience is so enjoyable that people will do it even at great cost, for the sheer sake of doing it."

    Enjoyment

    The quality of life does not depend directly on what others think of us or on what we own. The bottom line is, rather, how we feel about ourselves” (Csikszentmihalyi, 1990, p44).

    Implications on educational design

    The fact that flow seems to be a universal experience, means that it should be kept in mind when designing educational interventions. This should be done for the following reasons:

    Disorder in consciousness

    Csikszentmihalyi speaks of disorder in consciousness, or psychic entropy (Csikszentmihalyi, 1993). He states that this indicates information that conflicts with existing intentions, or factors distracting attention from these intentions. Pain, fear, anxiety or jealousies all fall in this category. Prolonged experiences of this inner disorder can weaken the self to the point that it is no longer able to invest attention and pursue goals.

    This negative state in itself is not desirable, but it does create the environment from which growth can be accomplished. The lesson approach of starting with a problem statement comes amongst others from Piaget’s theory that learning can not take place unless an individual is in a state of disequilibrium (Phillips, 1981; Piaget 1952). Learning is described as the construction of new knowledge resulting from the resolution of this conflict.

    Psychological growth

    Piaget’s disequilibrium arises from the differentiation and integration needs of the individual. On the one hand there is the need to remain unique and differentiate between different factors, but, on the other hand there is also the need to find common denominators and integrate new information into the existing frame of reference (Van der Stoep and Louw, 1976). After each cycle of differentiation and integration the self is enriched by the new information that has been added. This new order in consciousness adds complexity to the self, producing psychological growth (Csikszentmihalyi, 1990).

    From the above it can be deduced that Piaget’s disequilibrium is the first step in the cycle towards growth and that as a result of this it should be seen as a positive and desired condition. Within limits that would be true, but when the disequilibrium becomes too big it can cause frustration and anxiety. Too little challenge can change into boredom. “Enjoyment appears to be at the boundary between boredom and anxiety” (Csikszentmihalyi, 1990, p.52).

    Figure 2 The balance between anxiety and boredom

    In all Csikszentmihalyi’ s studies, enjoyment came at a specific moment when the individual's skills matched the challenges of the activities. In any given situation one person may be bored while another less skilled person may experience anxiety.Therefore the point where enjoyment is experienced is a dynamic one, which is dependent both on the skills level of the individual and on the challenge level of the activity. The diagram should be revised to incorporate these two factors:

    Figure 3 The effects of skill and challenge on enjoyment

    This enjoyment must not be seen as mere pleasure, for during these enjoyment periods, flow is reached and a cycle of discrimination and assimilation takes place. Positive feedback from each flow experience strengthens the self and more attention is freed to deal with the outer and inner environment.“In flow we are in control of our psychic energy, and everything we do adds order to consciousness” (Csikszentmihalyi, 1990, p.40).

    Figure 4 The learning cycle (Csikszentmihalyi,1990, p 74)

    In the diagram above, it can be seen that as A learns during the flow experience, the new skills acquired have the potential to take the individual back into the state of boredom ( A1 to A2). This state of boredom would be as undesirable as ever, but by moving the challenge to a higher level (A3) the boredom can be counteracted to leave room for a new learning experience (A3 to A4).In order to keep the learner within this flow channel the changes to the difficulty level should be within context of the skills level of the individual.

    Life long learning

    The continuous spiral of movement between boredom and anxiety, clearly sets the stage for an approach where the individual can be assessed on an on-going basis to constantly readdress needs as new competencies are acquired.

    The South African education system recognises the need for lifelong learning (South Africa, National Department of Education, 1997). This approach to education makes room for a more flexible time-frame which, without stating the fact, clearly makes it possible for learners with different abilities to steer their path of intellectual growth in such a way that they stay within the borders of the flow channel.

    Prerequisites

    From his studies Csikszentmihalyi (1990) draws the following eight factors that make flow possible if one or more of them are met:

    Challenge is optimised.

    Attention is completely absorbed in the activity.

    The activity has clear goals.

    The activity provides clear and consistent feedback as to whether one is reaching the goals.

    The activity is so absorbing that it frees the individual, at least for the moment, from other worries and frustrations.

    The individual feels completely in control of the activity.

    All feelings of self-consciousness disappear.

    Time is transformed during the activity (e.g. hours pass without noticing).

    Since flow is conducive to quality learning, the educator has to try to incorporate as much of these factors into any learning program as possible. It should not be the sole consideration. Rieber (1996) warns of the difficulties, but because the flow theory provides intriguing psychological insights into the effects of certain experiences, its value is again underscored.

    Simulation design within simulated roles

    The use of students within the roles of game architects, instructional designers and programmers are in itself a form of simulation where all the reasons for the game itself apply to the design team. This warrants a closer look at simulations and games.

    Simulations and games

    Students do not share the self-discipline that adults (should) show in completing tasks that are not enjoyable, but excel in most activities they choose. Children’s play is an engaging and deliberate activity to which they devote great effort and commitment. It would not be true to say that play is irrelevant or inconsequential to either formal or informal learning (Rieber, 1990).

    When discussing advantage, it must be said what holds advantage over what. In this regard the simulation is such a powerful medium that authors like Alessi and Trollip (1991) not only state advantages of simulations over other methods of instruction, but also the advantages that simulations hold over reality.

    Papert (1973) writes about “learning by doing”. Alessi and Trollip (1991) echo this and maintain that it holds superior motivational value to simulate (e.g. Flight), rather than merely reading about it. The likeness between simulation and reality makes the student more confident when confronted with reality. This improvement in transfer of learning often justifies the cost involved in the creation of a simulation. The efficiency of transfer in any given time period also compares favourably to any other learning platform. Add to this the intrinsic motivational strengths of the simulation as a result of the high fantasy level involved in the interactions and it becomes clear that the simulation holds great advantages.

    Simulations vs. reality

    Simulations as instructional tools hold advantages over the real world as a classroom. Alessi and Trollip (1991) back this statement by listing improved safety concerns as a prime reason. The ability to manipulate time is another example of how the simulation makes things possible that would not have been possible in the real world.

    The fact that most pilots and other personnel in charge of dangerous environments such as nuclear power facilities are trained extensively in simulators, could well be because a simulation creates scenarios so unlikely and potentially hazardous that real exposure could not have been endeavoured.

    In the Multimax game, a simulation structure was chosen as combined approach, because of the ability to control the environment to focus on the mistake of the moment. For example ;In real life it is possible for another person to switch on the current while a linesman is working. Therefore a notice to say that the power is down for a specific reason, should be posted. Because the worst does not happen each time a risk is taken in real life, workers grow lax and students sometimes think they can also get away without the necessary precautions. Multimax is scripted to make the worst happen every time, to make it clear to the user what the danger can be when performing a particular action.
     

    The value of using students as part of the design team for educational software.
     
    Design phase
     
    At the onset of the project, the students were assured of anonymity, so pseudonyms are used. They were also given a clear indication of what was expected. Some did not feel that we could manage. They saw software development as too complex. Others were very exited. They felt sure it could be done. This compares well with Malone’s (1981) view that the challenge should be adjusted for the student. By varying the difficulty of the individual tasks, student capabilities were matched to the challenge throughout the project.
     
    Once the drawings for the various views were done on paper, scanned and coloured, and more serious work commenced, Slower students could not keep up with the learning curve to acquire programming skills by themselves. They very often had to ask for guidance from their peers and from the researcher. It seemed as if enjoyment made room for anxiety. Csikszentmihalyi (1990) suggested that this might happen once the balance between challenge and skill was lost.
     
    Never the less, the energy shown by all the students who took part, arrested any reservations regarding student commitment.
     
    The following contributions were very valuable during the design phase:
    Students contributed many creative ideas.

    They have extensive experience in the latest multimedia games and contribute with relevant suggestions.

    Suggestions for appropriate language for the particular age groups came easily from the students who still express themselves in a similar fashion.

    As representatives of the target group, with the same level of subject knowledge, they could suggest realistic mistakes to include in simulation distracters.

    It should be noted that all of the above mentioned contributions relate to Malone’s (1987) rules regarding control.

    The creativity of especially Tony, Juan and Gerald were of constant value. These three students applied themselves to the generation of new ideas with dedicated effort. They took initial remarks by the author regarding cognitive scaffolding to heart and were often bouncing ideas off one another in an effort to get an even better result. Although they deny this, the author is convinced that there were elements of pure competition. This does not change the quality of their contributions, but ties up with motivation derived from confidence and satisfaction (Keller and Suzuki, 1988).

    Development phase

    Students are very honest sounding boards. They do not only help with the workload, but also add affective value by, for instance, the wrong joke at the right time.

    They also brought specific abilities. Juan could do some Delphi programming and Wayne and Ryan were good with Paint Shop Pro. This lead to a culture of cross teaching and learning. They seemed to be motivated by curiosity to learn the other programs (Malone, 1981) and found sufficient confidence from their own contributions, not to feel threatened by the possible disorder in consciousness as a result of confrontation with new knowledge (Csikszentmihalyi 1990).
     

    What value can students derive from the experience?
    The students indicated that they have benefited in many ways. Wayne said that he could not see the need for all the different software packages: “Can’t they make one proper program that can do everything?” In raising this question, he appears to admit to be in the state of disequilibrium that Piaget (1952) states as a prerequisite for learning. Gerald came a step further. He mentioned that he would no longer be afraid when confronted with new software, since he has learned so many programs in a fairly short time and came to see that they are not completely dissimilar. From this it is clear that Gerald’s anxiety threshold has been pushed up by the acquisition of a larger skills’ base (see figure 5). Tony, who expressed a broadening of his existing computer user abilities, echoes this psychological growth.
     
    Papert (1973) writes about “learning by doing”. Tony’s remark that he thought he knew the subject matter, but was proven wrong, shows that knowledge needs to be backed up by practice to become skill. Juan and Tony mentioned that they feel more confident about the subject matter and less frightened by new knowledge. In the time following the project, they have approached the author with snippets of new information they stumbled onto. The total experience and the enjoyment derived from the success of the project, appear to have awakened a zest for new knowledge.
    Multimax development as a learning platform.
     
    Officially, the completion of the program was the end of the project for the students of the development team. However, the effect of the increased contact with the author also showed. Gerald surprised with a routine project that now came as a self-executable file with text, diagrams and hypertext links instead of the traditional paper based hand-in project. The computer knowledge necessary to complete such a project was gained from the Multimax exposure. The Author marked a definite increase in all the participating students’ work quality during the semester after the project. They show improvement by having no assignments late or incomplete. Ryan shows an increase in self-confidence, asking questions in class, where he just kept quiet in the past.
     
    Showing that students learn from the program.
     
    The pilot testing group was asked beforehand and taped on video to later compare facial expression, activity and situational relation within the simulation.

    By applying the flow theory rules (Csikszentmihalyi, 1990) stated above, it follows that there could be a correlation between losing track of time and learning. All the members of the group claim that they have learned from the program and claim to have lost track of time. Observations made, seem to support this:

    All participants worked for an hour while being videographed. None of them even stopped for a moment to stretch an arm or to look around the room.

    They were completely absorbed by the activity. Brian actually started humming and Sunél-Marie talked to herself.

    All of them started off rather slowly, but wanted to continue even after the time was up. Three of them wanted a personal copy of the program. This supports a move away from anxiety, towards enjoyment, again correlating with Csikszentmihalyi’s (1990) theories on learning.

    In what way can the learning experience be structured according to open-ended constuctivist principles and still be useful for carrying a specific narrowly focused message?

    The development team

    The team took a far greater value from the program than mere development skills. They were not bound by any formal educational structure. No prior objectives were set and no tests were written. At no stage was any one forced to acquire any skill and all participation was strictly voluntary. Yet, the students never once objected to address any problem. The common goal of the Multimax program was enough to focus the energy. When a task was complete or if something needed doing more urgently, the students were all too eager to find a way to solve the next problem.

    Once, Tony almost gave up and said that he did not think he was getting anywhere with the scripting of the question-generating engine. Immediately Juan and Gerald stopped their own tasks and came over to help him find a solution. Juan, writing a data library file (.DLL), in Borland Delphi eventually solved this particular problem. Gerald then linked it to a Macromedia Authorware function call. During this time they telephoned twice to postpone other activities and ended up leaving with beaming faces and red eyes at 23:30 that evening. This commitment coincides with Csikszentmihalyi’s (1990, p4) definition of flow: “…the state in which people are so involved in an activity that nothing else seems to matter”.

    The value of teamwork and the joy of a job well done, could hardly have been taught in any better way.

    The time investment needed, does not make this method suitable for carrying a specific message in a short time frame. However, the longevity of the learning experience and the scope of competencies picked up by the participants, indicate ample application possibility.

    The quality of the end product

    The design team was most successful in finding ways to carry the subject message, while still making it fun for the end user. Most of the users confirmed that it felt as if the program were ready for any eventuality. The game character of the program made it possible for the users to experience control while they were still moving within the boundaries of the game. All users were quite happy with the structure of the program and did not feel threatened.

    The simulation content is strict and can only be done in the correct way, but depending on options taken, the users can learn skills like the value of good manners or the danger of vandalism. The students fared well with the simulations. The mistakes made were the same ones encountered in the electronics workshop. This is encouraging as it indicates applicability while still enjoying the benefits of safety and convenience as suggested by Alessi and Trollip (1991).

    What others think of us is less important. What is important is how we feel about ourselves (Csikszentmihalyi, 1990). It is interesting to note that no one of the pilot testing group, asked for help immediately. They were confident they could master the game without accessing the help file. Hence, they did not know the background information or what was expected of them. Once they came round to doing so, they worked more structured. By making the information optional, students could ask for the information when they needed it and could use the information well once they obtained it. The selective nature of assistance ensures that students do not feel pressured into a particular way of working. It echoes Malone’s (1991) call for control to provide choice and empowerment and follows the suggestions on confidence from Keller and Suzuki (1988).

    Problems experienced

    Some aspects of the program deserve mention as they proved to be undesirable.

    The question bank consists of 50 questions that rotate according to the variable interval performance (VIP) structure (Alessi and Trollip, 1991). However, the students needed so many credits, that some questions eventually repeated.

    Since many labs have hardware restrictions, high specifications cannot be taken as the minimum requirement.When running Multimax in 32 bit environments, some animations proved to be unstable. This problem was not consistent, but should it occur; reapproaching the scene usually helps.

    None of the students complained, but they did ask about the voice artists, indicating that the quality of the speaking drew their attention.
     

    Conclusion
     
    What is the value of using students as part of the design team for educational software?
     
    Self-regulated learning requires more than a smooth flowing simulation. The learner must find the activity interesting (Kinzie, 1990; Kinzie and Sullivan, 1989). Since we have shown that Multimax was received favourably by the whole pilot group, it follows that the contributions made by the design team was valuable. As mentioned, scenarios that the author thought of as useful, were turned down as boring by the student members of the design team. The reverse was also true; students suggested the use of a variety of very obviously wrong possibilities as options during the dialogue design. Some of these were included and the pilot group actually chose some of them! Using students can prove useful in helping the design team to adapt to the world of the target population in order to address things interesting to them.
     
    Value students add to the project.
     
    Martin was unable to keep up with the challenge and eventually lost interest. Others managed and were excited up to the end. This echoed the theories of Malone (1981). Plot difficulty holds a lot of the challenge and motivation in games. All four the pilot testers expressed satisfaction with the degree of plot difficulty of Multimax. The extensive gaming experience that the design team students could muster among themselves, has provided the team access to up-to-date knowledge of current levels of acceptable challenge.
     
    Value students derive from the experience.
     
    Learning is an active process in which meaning is developed based on experience. Wayne’s remarks have shown that initial exposure to new knowledge can lead to a state of disequilibrium and that sufficient interaction with new information can lead to the construction of new perceptions. Gerald improved his skills-base and in the process pushed up his anxiety threshold to such an extent that he had the self-confidence to approach an interactive project on his own.

    From the above it follows that students do not only gain in knowledge when participating in projects like these, but also improve their self confidence.

    The joy that the students experienced with the VIP question structure was apparent. Projects like Multimax can provide these liberating experiences of successful completion of difficult tasks.

    Recommendations for further research

    The Multimax project gave rise to the following hypotheses. Further research is required to investigate their validity.


    References and Further reading
    Alessi, S.M. and Trollip, S.R. (1991). Computer based instruction - methods and development. Second Edition. Eaglewood Cliffs, NJ: Prentice Hall.

    Cortinovis in Nielsen, J. (1993). Usability engineering. California: AP Professional.

     
    Cronje, J.C. (1996) “How to make interactive television interactive”, paper presented at the forum on Telecomunications for tertiary education, CSIR, Pretoria
    Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper and Row.
     
    Department of education, (1997).Curriculum 2005, Lifelong learning for the 21st century, Pretoria: Goverment Press.\
     
    Duffy, T. M. and Jonassen, D. H. (Eds.). (1992). Constructivism and the technology of instruction: a conversation. Hillsdale NJ: Lawrence Erlbaum Associates.

    Fouché, J. (1995). An interactive Multimedia Program for the Teaching of Electrical Principles to Secondary Pupils. Universty of Pretoria.

    Gates, W. H. (1995). The road ahead. Harmonsworth: Viking.

    Gayeski, D. M. Getting started in Multimedia - avoiding common pitfalls. Unrefferenced article.

    Hannafin, M. J. (1992). Emerging technologies, ISD, and learning environments: Critical perspectives. Educational Technology Research and Development, 40(1), 49-63.

    Heinich, R., Molenda, M. and Russel, J.D. (1986). Instructional Media and the technology of instruction, MacMillan: New York.

    Jaspers, F. (1991). Interactiviy or instruction? A reaction to Merrill. Educational Technology Research and Development, 31(3).

    Keller, J.M. and Suzuki, K. (1988). Use of the ARCS motivation model in courseware design. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Kinzie, M. B. (1990). Requirements and benefits of effective interactive instruction: Learner control, self-regulation, and continuing motivation. Educational Technology Research and Development, 38(1), 5-21.

    Kinzie, M. B. and Sullivan, H. J. (1989). Continuing motivation, learner control, and CAI. Educational Technology Research and Development, 37(2), 5-14.

    Koehl, D. (1997). Expand the use of instructional Design. ITFORUM (on-line electronic listserver). The University of Georgia.

    Kommers, P. (1994). Conceptual Metaphors for Visualisation in Interactive Learning. In Schoeman, J. and Stanchev (Eds.), Visualisation: Principles and tools for Instructional Visualisation. Enschede: Anderson Consulting.

    Kozma, R.B. (1987). The implications of cognitive Psychology for Computer-Based Learning Tools. Educational Technology Research and Development, 27(11), 20-25.

    Lanza, A and Roselli, T. (1991). Effects of the hypertexual approach versus the structured approach on students’ achievements. Journal of computer Based education, (18), 48.

    Lippert, R. (1992). Face to interface. Centre for Computer-based Instruction, University of Pretoria.

    Lippert, R. (1993). Creating meaningful Learning Environments in Hypermedia. Centre for Computer-based Instruction, University of Pretoria.

    Malone, T. (1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 5(4), 333-369.

    Malone, T. W., and Lepper, M. R. (1987). Making learning fun: taxonomy of intrinsic motivations for learning. In R. E. Snow and M. J. Farr (Eds.), Aptitude, learning, and instruction: Conative and affective process analysis, (pp. 223-253). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Marais, D, Calitz, L.P. and van Wyk, C.F.M. (1983). Die integrering van Onderwysmedia in lesaanbieding. McGraw-Hill: Johannesburg.

    Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59, 43-64.

    Morrison, G. R. (1994). The Media Effects Question: “Unresolvable” or Asking the Right Question. Educational Technology Research and Development, 42(2), 41-44.

    Mouton, J. and Marais, H.C. (1990), Basiese begrippe: Metodologie van die geesteswetenskappe, Pretoria: RGN Uitgewers.

    Nielsen, J. (1993). Usability engineering, California: AP Professional.

    Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas, New York: BasicBooks.

    Papert, S. (1981). Computer-based microworlds as incubators for powerful ideas. In R. Taylor (Ed.), The computer in the school: Tutor, tool, tutee, (pp. 203-210). New York: Teacher's College Press.

    Perkins, D. N. (1986). Knowledge as design. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Phillips, J. L. (1981). Piaget's theory: A primer. San Francisco: W.H. Freeman.

    Piaget, J. (1952). The origins of intelligence in children. New York: BasicBooks.

    Ralston, G. (1991). Hypermedia, not multimedia. The expanded desktop, 1(4), 58.

    Rieber, L. P. (1992). Computer-based microworlds: A bridge between constructivism and direct instruction. Educational Technology Research and Development, 40(1), 93-106.

    Rieber, L. P. (1993). A pragmatic view of instructional technology. In K. Tobin (Ed.), The practice of constructivism in science education, (pp. 193-212). Washington, DC: AAS Press.

    Rieber, L. P. (1996). Animation as feedback in a computer-based simulation: Representation matters. Educational Technology Research and Development, 44(1), 5-22.

    Rieber, L. P. (1996). Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research andDevelopment, 44(2), 43-58

    Schuman, H. and Presser, S. (1981). Questions and answers in attitude surveys. New York: Academic Press.

    Steyn, D. (1999) The value of using students as participants in educational software development: differentiated outcomes within an open environment. Campus-wide Information systems, Volume 16 November 5 1999, MCB University press

    Tapscott, D. (1995). Learning in the age of networked intelligence. EDUCOM: Keynote address. Portland,Oregon

    Van der Stoep, F. and Louw, W.J. (1981). Inleiding tot die didaktiese pedagogiek, Pretoria: Academia.


    ITFORUM PAPER #53 - The value of students as part of the design team for educational softwareby Dolf Steyn of the University of Pretoria. Posted on ITFORUM April 28, 2001. The author retains all copyrights of this work. Used on ITFORUM by permission of the author.Visit the ITFORUM WWW Home Page at http://itforum.coe.uga.edu/home.html