From Art to Alchemy:
Achieving Success With Online Learning

Roderick C. Sims, PhD
Director, Learning Environments
Deakin University, Victoria (Australia)

Pompeii to PLATO: The magic revealed

As a young child in the early 1960s I emigrated from Great Britain to Australia. As we sailed from Southampton on a cold night in late November, I began a journey that was to expose me to new and different places. I can still recall the rough voyage through the Bay of Biscay and our first port of call at Gibraltar, where I watched monkeys scrambling up the famous rock, not cowering behind bars. Then the smooth blue of the Mediterranean and our journey from Naples to the silent homes and temples of Pompeii - where small figures were captured in their final attempt for escape, encased in stone forever - with the smoking Vesuvius towering above. But it was in Port Said where I first encountered a glimpse of the magic our world can offer. On a balmy evening, strolling through the sandy grey-brown streets, our family was confronted by two men in long flowing robes, one of whom proceeded to pull an egg out of my ear! The magic fascinated me, but as he tried the same trick with my more conservative father, the man was given a few pieces of change and we moved on.

Over the ensuing years, my wonder at the magic open to us has not diminished. Towards the end of high school, I can vividly recall the amazing sights of inland Australia and the emu-dance performed by a local Aborigine. Here was a man in old baggy pants, a jacket festooned in badges, a tilted army hat and a weathered, bristly face who transformed almost instantly into a desert bird hunting and scavenging through the scrub. Not long after, as an undergraduate in the early 1970s, I encountered my first computer. While writing (well punching cards is more accurate) a Computerised Crook Catching program, a simulated exercise to compare witness descriptions with the characteristics of known criminals to identify likely suspects, I learned much about both the power and simplicity of this technology. In the same way that a person could magically transform into a bird so too the computer could transform meaningless data into valuable information. This was best demonstrated when I was fortunate to view a presentation by the designer of PLATO (Programmed Learning for Automated Teaching Operations) to the 1976 Australian Computer Society conference in Perth. Of particular fascination was the moment when Dr Bitzer was, by touching the display, moving bees from one screen location to another. During this demonstration he paused to make observations to the audience but was interrupted by the computer saying "Dr Bitzer - you still have a bee on your finger"! This was a definitive moment for me, as I perceived a potential for communication and interaction between computer and human that could engage, humour and educate.

In the same way that these images of entombed figures, street magician and desert impersonator have engaged my senses, so has that initial magic of PLATO provided me with a context to understand computers as a learning tool. This is the magic of surprise, like an egg appearing from nowhere or the magic of awareness when links between people and the land emerge. Perhaps more simply, it is the magic of our dynamic, living planet. However it is this magic which we have the capability to harness and expose through computer-based learning experiences. Over the past twenty years I have endeavoured to apply this analogy of magic through my work as a computer-based learning analyst, courseware developer and teacher of educational technology. However, as the challenge to make educational technology work better remains, I wonder whether the potential of computer-based learning technology has in some way been lost - its magic diffused in a maze of marketing and technological hype.

myth, art and alchemy

It is an honour to be part of the extensive, dynamic and insightful ITFORUM community. I remember when it was first conceptualised, and its value as a resource for our community remains. In 1995 I was invited to present a paper to ITFORUM which I titled “Interactivity: A Forgotten Art?” (see Sims, 1997) an attempt to contextualise the notion of interactivity. Since then I have continued to explore the various elements of interactivity through research and practice and had initially thought to present an updated reflection of this critical element of educational technology. However, I believe there are more critical issues to reflect upon.

First, the purveyors of software and hardware often argue and advertise that their products will provide solutions to educational problems. Recent history reveals this as a myth. Second, it appears that the art of using computers in education has been lost. Yes, this is a generalisation, but look at a current reality. How many “online learning” sites are simply screens of wasted content? This is not how computer-based learning should be manifested, and it won’t work! Third, I offer the concept of alchemy, because we have the challenge to ensure that the gold, in terms of our exemplary face-to-face teaching and learning practice, is not turned into lead through inappropriate use of computer-based technologies.

I recently presented these ideas at the 2001 EdMedia Conference in Finland and offer the following expanded version for discussion and comment. With so many new teachers and learners exploring the online world, the analysis provides a context in which these environments might better be understood. The basic concepts articulated were generated in collaboration with my colleagues Graeme Dobbs and Tim Hand from OTEN in Sydney, Australia. Since then they have been expanded and I anticipate the feedback from ITFORUM will help develop this as one means to recover the magic that educational technology often lacks.

From a different perspective, this paper might be seen as a form of interactive archaeology or palaeontology – seeking the structures and fossils left by those who preceded us – attempting to put the pieces of the jigsaw together to give meaning to the many-layered complexity of educational technology. I look forward to your comments and inputs, especially as online learning is so much a part of contemporary learning environments.

introduction

The imperative for tertiary education providers to embrace online learning as the primary mode for access to teaching and learning resources has never been greater. While many practitioners are familiar with the issues and processes associated with the production of these materials, many institutions are demanding their creation without necessarily having staff competent in all aspects of the online paradigm. At the same time, research studies have demonstrated online environments to have both positive and negative impacts in terms of effectiveness and achievement of outcomes (Franklin, Peat, Lewis & Sims, 2001). Given this environment, it is critical that online development projects implement levels of quality control to ensure the learners receive the most effective resources.

The most frequent form of quality assessment within the educational environment is that of evaluation, which is typically placed at the end of the development process and as the last chapter in books focusing on educational and learning design (compare Smith & Ragan, 1999; Morrison, Ross & Kemp, 2001). This form of evaluation is typically reactive, examining the functionality of the learning resources and the achievement of the associated learning outcomes once the materials have been developed and implemented. The evaluation data thus collected is then to inform developments and maintenance with those resources. As an alternative to this environment, and to provide specific support for the creation of on-line materials, this paper argues for a repurposed development environment in which evaluation is integrated into all considerations of the design process. In concept, the production team are exhorted, through a proactive evaluation framework, to focus on the criteria by which their products and resources would be evaluated and thereby ensure that all factors associated with a successful evaluation are addressed during the design and development process. Theoretically, a product created using proactive evaluation will be more likely to achieve the educational and learning goals.

While there is no intention to suggest the replacement of formative and summative evaluation protocols, which are critical to the iterative nature of educational development, the strategies proposed appear to provide an integrated framework for an online pedagogy. In fact the proactive aspect of the concept is one that places evaluation within all facets of the design, development and implementation process.

Proactive Evaluation

The following classification of factors and associated influences provide the basis for a concept referred to as proactive evaluation. The essence of this concept is that by first considering the complex interactions between educational design and online environments, designers and developers with new or limited skills in online learning will reduce the risk of producing poor-quality or ineffective materials as well as the likelihood of critical, negative evaluation. The ideas presented can be viewed as an extension to current practice in instructional or learning design, as many of the factors identified stem from that theoretical base. However, the underlying principles assume that the environment for which the development is being undertaken is not a traditional instructional setting, but an online world in which the familiar relationships between teacher, learner and content no longer always hold true. By applying the proactive evaluation concept, development teams will not only ensure all aspects of creating online learning resources and activities are addressed, but that subsequent formative and summative evaluation will be more rigorous and meaningful. In addition, the concept focuses the decision making process on the complex interaction between disciplinary content, learning outcomes and computer-based learning environments. Better understanding and addressing these relationships will consistently produce more effective teaching and learning resources.

Strategic Intent

One of the first questions we always ask our clients is “Why are you attempting to place these resources or activities into the online context”? If they are unable to provide an explicit answer, then it is likely that the strategic intent or rationale for the product has not been defined succinctly. Without a clear understanding of the purpose of the product and the stakeholders who have an investment and/or interest in its outcomes, the chances of success are reduced. Within the context of the tertiary education environment, these stakeholders include the administration, the faculty, the development group and most importantly the students. Without full commitment to the concept from all stakeholders, even with the best intentions, effectiveness in terms of learning outcomes being achieved may not be realised.

The critical issue is the extent to which the online component(s) being considered will add value to the overall teaching and learning process. For example, if an institution decides to “go online” without allocating sufficient funds to recreate materials so that they will be consistent with learner-computer communication, then the decision could prove extremely costly with few, if any, positive educational outcomes. Indeed, anecdotal feedback is suggesting a rebellion against online materials; for example, where online is seen to be an economic solution used by institutions in response to government funding cutbacks and academic workload increases. In some cases the intent has been to introduce online studies to reduce class contact time, resulting in student anger and resentment for being short-changed, not to mention issues of quality.

Content

Many online projects have focused on the conversion of existing paper-based resources into their digital equivalent, with a proliferation of unit outlines and study guides in either HTML or .PDF formats made available for student access. But this is not online learning, and if portrayed as such then is a misrepresentation of the capabilities and benefits of the technology. If content materials and learning activities are to be placed online, then a significant level of thought must be placed on the very nature of the online medium and the underlying implications for current practices of teaching and learning.

As shown by the influences identified in Table 1(a), the online environment caters for a range of content formats: from predetermined static media to a dynamic state where content is sourced, repurposed, constructed and enabled by and for all participants in the learning process. Between these two extremes are the more typical options for delivery of course content in terms of resource material being contributed to by both the teacher and the learner. It is important that this range of options is understood and that the implication for the discipline base is considered. To fully exploit the online environment means having to reassess the overall approach to the content, how it should be presented or accessed and the relationship between teacher and learner in that process. The components associated with the content must also be addressed by considering their interaction with the influences and their availability to the learning community. As detailed in Table 1(b), the issues to consider will have a significant impact on the role of the content and subject matter in the broader learning process. Underlying this notion is the understanding that content can no longer be seen as being “owned” by the teacher or discipline but rather as an information base that can be perceived and worked with in many different ways.

 

STATIC <------------------------------------------------------------------------------------------> DYNAMIC

Predetermined and Presented

Teacher
Contributed

Learner
Contributed

Captured Dialog
(Interactivity)

Constructed

Content defined and prescribed by the teacher, and does not change for one iteration of delivery.

Content defined and prescribed, but additions or modifications made by teacher if and when required.

Content defined and prescribed, but learner additions and contributions enhance the resource base.

Through collaborative endeavours, content material is added to the overall resource base for the program.

Content defined through research by participants and subsequent interpretation and construction.

 

Table 1(a): Influences Affecting Online Content

 

COMPONENT

Issues

Structure Organisation & Information

If adopting strategies that enable the dynamic construction of knowledge, traditional forms of information presentation may have to be modified.

Matches
Goals & Outcomes

The extent to which program goals and objectives are predefined may be affected by strategies that enable the learner to use knowledge construction techniques.

Contextual and/or Situated

With a dispersed cohort of learners, content must be considered in terms of the context in which the learner is situated rather than that of the teacher’s particular experience.

Accuracy, Integrity, and Totality of the Information Base

Recognition of the learners’ ability to contribute to the knowledge base presents questions as to accuracy and integrity – from whose perspective are these characteristics of the content to be measured and assessed?

Extensibility of Content

Is the discipline base so rigid that no options for new content are considered possible, or can new alternatives be considered for collaboratively constructing and extending the knowledge base?

Quality of Expression
(language, grammar, image resolution)

To what extent can traditional norms for quality of presentation be maintained if a more dynamic approach to content is considered appropriate, and what impact might this have on roles in the development process?

 

Table 1(b): Online Content – Major Components

Learning Design

The term learning design is used instead of instructional design to emphasise the learner-centred environments online resources can provide. Taking this stance is particularly important because it forces designers to conceptualise the development process from the learner’s perspective rather than that of the content. However this does not preclude developers from adopting an instructivist (I) or presentational strategy compared to a constructivist (C) or generative approach, but does require careful thinking about the learner and the options provided for interacting with the content and their learning partners.

As detailed in Table 2(a), the design of resources will be influenced by the pedagogy, outcome and resources considered appropriate for the task. When considered in terms of the specific components that are critical to the overall design task shown in Table 2(b), the complexity of addressing educational and technological elements of the process are emphasised. Embarking on the design and development of resources for online environments requires new layers of thinking to be added to the well-established principles of course development.

 

I < -------- Pedagogy --------> C

Learning Outcome

Resources (Media, Modes)

Individual teachers and learners have different philosophies on the most appropriate ways that knowledge is gained and learning acquired. As online environments can be perceived as supporting the constructivist paradigm, adopting rigid instructivist strategies may degrade the overall effectiveness of the encounters experienced by the learners.

These options link learning strategy to outcome and affect each component of learning design:

problem solving; declarative knowledge; concept learning; principle learning; procedural learning; cognitive strategies; attitude & motivation and psychomotor (Smith & Regan, 1999)

The ways in which media elements are used and extent to which they are accessible will influence the individual components of learning design.

 

Table 2(a): Influences Affecting Online Learning Design

 

COMPONENT

Issues

Prior Experience

Online learning is a new environment, and learners must have the requisite skills to effectively work within this paradigm.

Learning Styles

Does the provision of a range of media elements address the different motivational and perceptual styles of learners, and what impact will this set of options have on the overall development effort.

Learning Environment

The extent to which the environment is designed as a digital page or a virtual world will impact on the overall development effort (see Interface Design)

Pathways/Sequencing

The strategies for online learning can establish predefined pathways or enable students to explore and discover different facets of the content. Managing these options to minimise information overload becomes critical.

Outcomes

In providing an online environment, are the stakeholders providing for a range of outcomes or are the consistent with predefined objectives.

Assessment

Closely linked to outcomes, are new forms of assessment being considered for the online environment, such as collaborative understanding and concept formation.

Level of Learning

What impact might governmental standards have on the design – and do those standards influence or constrain the preferred modes of delivery within the online contect?

 

Table 2(b): Online Learning Design – Major Components

Interface Design

The interface between learner and computer is one of the two most neglected aspects of online learning, the other being interactivity, as both are integral to successful and ongoing communication. As detailed in Table 3(a), the options available for on-line productions can range from the non-contextual through to the theatrical, where the learner is conceptualised as an active player in the overall learning process rather than a passive observer (Laurel, 1991; Sims, 2000).

Overall, the conceptualisation of the interface must consider the strategies employed to position the learner within the illusion of the virtual learning environment; the way in which representations, metaphors, icons are employed to support communication; how design decisions affect connectedness and interference within the learner-computer interface and the extent to which animations and sound effects impact on cognitive load and degradation of learning (Sweller, 1988). The associated components for Interface Design are identified in Table 3(b) and are linked explicitly to the elements of Learning Design, one area that can be neglected by developers new to the creation of online resources. To maximise the online learning experience it is not sufficient to apply rigorous educational design to content materials as the means by which that content (resource, activity, conference, reading) is presented to the learner will impact on its overall effectiveness.

 

Non-Contextual <----------------------> Contextual <----------------------> Narrative <----------------------> Theatrical

Information Design

Interaction Design

Input/Output

Navigation Design

Aesthetics

What procedures have been employed to ensure maximum communication of information?

Have the various interactivity options been catered for and communicated to the learner?

How clear are the options for entering and accessing content and responses?

Does moving between resources affect continuity of delivery or context?

How does the “look and feel” contribute to or detract from the communication experience?

 

Table 3(a): Influences Affecting Interface Design

 

COMPONENT

Issues

 

 

User comfort - connectedness

Has appropriate usability testing determined the extent to which users are able to work with the resources and make the necessary connections between content elements?

 

User control
User centred

In what ways are users able to control the learning process and link the activities to their own learning requirements?

 

Supports content structure

Has the interface been conceptualised to be consistent with the content structure while maintaining acceptable standards?

 

Supports learning design approach

Has the interface been designed to be consistent with the particular paradigm employed for the course?

 

Alignment of Mental Models

What strategies have been employed to ensure the mental model of the design group has been effectively communicated to the learner?

 

Customisation vs Individualisation

In what ways can the learner structure the environment to meet their own individual learner needs or preferences?

 

Table 3(b): Online Interface Design – Major Components

Interactivity

Interactivity is about successful communication and, in the context of online learning environments, one of the most crucial factors to address. As a component of the human-computer relationship or encounter (Sims, 2000; Sims, 2001), interactivity can include passive presentation, navigation, undirected exploration, directed involvement and specific manipulation. The extent to which these constructs of interactivity impact on the continuity of communication between learner and interface, content, other learners or other teachers is critical to the overall effectiveness of the experience and is inextricably linked to the factors and influences associated with content, learner design and interface design. As shown in Table 4, elements of interactivity can encompass both human-computer activity and human-human communication. The ability of the learner to “inhabit” the interactive world presented to them is naturally critical to its success as a learning environment. The way in which the motor, cognitive and collaborative elements of an interactivity coalesce with the task being undertaken will contribute to the successful engagement of the learner with the activity.

 

INTERACTIVITY

Motor

Cognitive

Collaborative

Learner : Learner

 

 

 

Learner : Teacher

What aspects of the design and interface might enhance or impede the success of the different, but often simultaneous, interactions?

Learner : Content

Learner : Interface

 

 

 

 

Table 4: Elements of Interactivity

Assessment

Much discussion takes place in educational institutions about how best to deploy multiple-choice or short-answer questions in online environments, and what form of authentication should be installed to verify the electronic submission of assignments or completion of remote examinations. However, these strategies seem to contradict contemporary approaches to learning, which advocate active participation by learner and teacher and enable self-assessment and reflection. One challenge therefore is to determine how you can take advantage of the online environment, not to see how you can replicate traditional testing strategies in another medium. How is the assessment designed to take place? As shown in Table 5 following, assessment can be either teacher, peer or student directed and within that context, the way in which assessments items are presented becomes critical.

 

ASSESSMENT

Teacher-Directed

Peer-Directed

Student-Directed

Assignments

Is the assessment being proposed just old wine in new bottles, or does it represent a new model for assessment and learning that is consistent with the medium and environment being provided for the learner?

Examinations

Group/Project Work

Work Placement

Authentication

 

Table 5: Elements of Interactivity

Student Support

Providing appropriate support for the learner cohort is even more critical in the online environment because in many instances they will be working independently in their preferred environment. Even though this environment may include collaborative work, the learner’s only medium of communication is the computer, and therefore support becomes critical to ensure their mental model is consistent with that of the other stakeholders in the process. In addition to the typical help systems, announcements and guides, recent research (Sims, 2000) has suggested that more explicit support is required to bring the learner into the online environment, especially by eliminating assumptions that learners will know what to do and why they are doing it.

 

STUDENT SUPPORT

On-Campus

Mixed-Mode

Off-Campus

Auxiliary Information

What expectations do you have for your students? How ado you plan to nurture them into the collaborative world of online learning? What support personnel and resources have been identified to ensure they will feel integral to the learning environment?

Institutional Support

Features

Personalisation

Security

 

Table 6: Elements of Student Support

Utility of Content

Within Australia, new digital copyright legislation and the proliferation of digital resources have provided new incentives to focus on the international standards for online learning environments. A crucial component of any development exercise therefore is to examine the extent to which content can be used in multiple environments (within and outside the product being developed), the means stakeholders might have to customise the materials and the interoperability between other learning objects in the wider curriculum. Complicating these factors is the increase in legislative and compliancy conditions; at the time of writing, ensuring online resources do not breach copyright and are accessible for learners with disabilities are part of the quality control process. Underpinning this is the need for a robust technological infrastructure.

 

COMPLIANCE

Copyright

Accessibility

Infrastructure

Multiple Use

What benefits would accrue from a Digital Object Management System (DOMS)? Are the learning objects compliant with international standards?

Customisability

Interoperability

 

Table7: Elements of Content Utility

Outcomes

The final factor to consider relates to the ways in which outcomes have been successfully achieved. For example, measures of learning associated with both intra-curricula and extra-curricula activities; the level of learner satisfaction with the overall experience; the completion rates and the extent to which pass rates and grades are consistent with alternative delivery options. Overall the design effort needs to include items to enable a comparative analysis of student outcomes in relation to the overall development parameters.

 

OUTCOMES

Program Maintenance

Quality Audit

Teacher Performance

Learning

Very simply, did we get it right? What needs to be done to make it better? Knowing the parameters that will be used to validate both quantitative and qualitative outcomes of the learning experience will be critical to its ongoing success.

Satisfaction

Results

Outcomes v Objectives

 

Table 8: Outcomes

 

Conclusion

The capacity of computer-based technology to display combinations of media elements and respond meaningfully to user actions and manipulations has been established for many years. However, the power and capability of the computer to support the learning process is often lost in a maze of marketing publicity and technical gadgetry. Unfortunately, without the requisite skills, it has become all too easy to create web-based materials without understanding the underlying principles of online, interactive, engaging learning. Rather than creating effective learning environments, many materials have proven ineffective, with learning activities a confused labyrinth of information, links, colleagues and navigation. The factors and influences presented here, which are critical to effective online learning, have been presented as a framework for designers and developers to proactively evaluate their product. By enabling development teams to address the critical issues associated with the creation of learning resources for delivery in an online environment we will have a greater chance of ensuring the achievement of educational outcomes, with learners gaining significant value from their online experiences. More importantly, perhaps we will see the magic return.

References

Franklin, S., Peat, M., Lewis, A. & Sims, R. (2001). Technology at the Cutting Edge: A Large Scale Evaluation of the Effectiveness of Educational Resources. Short paper presented at the 2001 EdMedia Conference. Tampere, Finland; June.

Laurel, B. (1991). Computers as Theatre. Reading, MA: Addison-Wesley.

Morrison, G.R., Ross, S.M., & Kemp, J.E. (2001). Designing Effective Instruction. New York: John Wiley & Sons

Sims, R. (1997). Interactivity: A Forgotten Art? Computers in Human Behavior, 13(2), 157-180.

Sims, R. (2000). Learners as Actors: Strategies for Computer-Enhanced Learning Encounters. Unpublished Doctoral Thesis. University of Wollongong, NSW, Australia.

Sims, R. (2001). Usability and Learning in Online Environments: A Case of Interactive Encounters. Full paper presented at the 2001 EdMedia Conference. Tampere, Finland; June.

Smith, P.L. & Ragan, T.J. (1999). Instructional Design. New York: John Wiley & Sons.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285.