[This paper is an abbreviated version of the 1997 Peter Dean Lecture presented for the Division of Learning and Performance Environments (DLPE) at the 1997 National Convention of the Association for Educational Communications and Technology (AECT), February 14, 1997, Albuquerque, New Mexico.]
My intention in this piece is to provoke discussion on the direction of the technology-based training/education/performance environment field (TBT) as a profession. I intend to include in this discussion all forms of computer-based or computer-aided media, in all venues: K-12 and post-secondary education, adult education in all forms, military and business/industry training, and home educational and "edutainment" software. I intend to address both professional practitioners and academic programs for research and professional preparation, as well as the supporting industries of authoring tools and systems, as well as the various forms of workshops and other non-academic training for the field. In doing this, I will offer only observations and opinions from my 26 years of practice. I hope that others will disagree and offer theirs, and that out of this will come a useful dialog on how to advance the field.
The field of TBT is now about 30 years old. I believe the field is moving from "adolescence" to "adulthood." My evidence for emerging "adulthood" of the field is that across industry, education, homes, and the military in the U.S. and some other industrialized countries, many senior managers in positions with programmatic responsibility (and without any kind of instructional design or other TBT training) are growing increasingly willing to invest considerable resources in TBT as a way of achieving mission-critical objectives. In TRO's own markets for its PLATO(R) system, there has been in the past few years a dramatic increase in size of contract, so that now six and seven-figure contracts are not uncommon from educational, military, and industry/adult sectors. This signifies that the investment in TBT is no longer an "experiment" for our employers, clients, and customers; it is a strategic investment. The best estimate I was able to assemble from investment banking, market research, and management sources is that the total expenditure for TBT software in the U.S. in 1996 was in the vicinity of USA$3 billion. That's not R&D any more.
But with "adulthood" and that kind of commitment of resources comes a serious expectation to deliver on the promise of TBT--with an implied "or else." With the possible exception of the home market, accountability (in the form of standards, external assessments, and even formal evaluation or ROI studies) are a clear and growing trend. But it's absolutely not clear to me that the field will successfully make the transition to "adulthood" from its current "adolescence," or that good instructional design (ID) will become the dominant design model. There are a number of reasons for my concern:
(1) I believe the average instructional quality of TBT has gone down over the last decade. On average, the attractiveness and costs of multimedia have simultaneously contributed to the popular acceptance of TBT, but in a typical project now, the greatest resources are consumed by the multimedia and the software technologies, not the instructional design. In the home market, for example, I recently attended an investment presentation by one of the largest producers for the home/school "edutainment" sector. According to the CEO, their production budget per product now is in the range of USA$800,000 to $1 million. Yet I think ID practitioners would agree that the instructional design of these products--particularly the best sellers--is so simple as to be trivial. Whatever the company is spending their money on, it couldn't possibly include much ID. I believe there are analogous situations in industrial and military training, where millions are spent on realistic simulations, but necessary instructional components, such as coaching, student modeling, strategy-level dialog, etc., are trivial or completely absent. And, the same can certainly be said of most of the hypertext developed since the introduction of HyperCard and HTML.
(2) Most academic ID/TBT programs are experiencing a demand for their graduates which far exceeds supply. For the lucky few graduates, this is good news. But for the field, it's bad news, because it means that most of the people who are designing TBT enter the field with no formal training in the ID at all, or with nothing more than a few days of workshop training from an authoring system or seminar vendor. Instead, the training they receive is in content areas and design fields such as graphics (magazines), textbooks, instructor-led training, technical writing, entertainment games, film, simulation software, and so on (note that my argument here is not that these other fields have nothing to contribute to the design of TBT; only that ID contributes so little). Many of the major producers of TBT don't even have degreed ID people on staff, or don't have any in positions of influence. Nor do they necessarily acknowledge that they are missing an important skill set.
Furthermore, regardless of how they entered the field, there is virtually no continuing professional education available in TBT, unless a practitioner happens to be compulsive enough to read journals and books consistently. Since academic TBT programs have been functioning for about 30 years, that means that we now have an entire generation of degreed practitioners in the field who are working with design skills which may not have fundamentally advanced beyond what they learned in their formal entry-level training.
In neither entry-level, nor continuing professional education in TB, is there any possible way that the present combination of residential academic programs, workshops, and seminars could be scaled up to meet the need fully.
(3) There is no particular reason to believe that ID is the leading or preferred paradigm for development of TBT (if good design could be counted on to win in the market place, Microsoft wouldn't be able to sell versions 1 or 2 of anything, or the .0 version of any release). If you agree that the average quality of instructional design has declined, then there is a good chance that your clients, employers, or customers have never encountered TBT which represents good ID by any definition. That means (if you believe, as I do, that good ID contributes to effectiveness) that their experience with the effectiveness of TBT is probably mediocre, at best. This, in turn, may cause them to lower their expectations for what good TBT can be, thereby creating a descending spiral of expectations.
Meanwhile, there is an ascending spiral of expectations for multimedia production values. I believe good ID in a TBT product is subtle, delayed in its effect, and so, virtually transparent to the consumer. By contrast, multimedia is immediately obvious. So, our clients, consumers, and customers conclude that each advance in multimedia production values and technology is the main factor defining quality in TBT. Furthermore, the production value expectations are set by what the purchasers have seen, and what they have most probably seen is a retail interactive game. The result for any TBT producer which listens to its customers is millions for multimedia (literally), but pennies for ID.
All these factors lead me to believe that there is a substantial chance of TBT becoming "last year's thing," instead of realizing its potential as a major contributor to the transition to the knowledge-based economy. At the very time that "prime time" is ready for TBT, I'm not convinced that the TBT field is really prepared for "prime time."
To make sure TBT delivers on its promise, I believe we must recognize a number of trends and barriers to performance both within the field and externally (in its environment). Space precludes a thorough discussion, but I will briefly comment on a few of each.
Each new advance in hardware/software technology seems to lead to a flurry of journal articles explaining the technology and describing how it might be used for TBT. In many journals, much of the time, such articles dominate. In the 1980's, the advent of personal computer technology seemed to me to dominate the field so thoroughly that very little real theoretical advancement of the field took place: we "lost" a full decade of progress. Meanwhile, in many journals there is proportionally little space devoted to advancement of ID theory and research, independent of the hardware/software technologies. Of course, we need to know what any new hardware/software technology is, and how it might help us. But we also need to know what its cost/benefit tradeoffs are, and what it can't do (what a physician would call "indications" and "contraindications"). Somehow, that kind of information never makes it into the journals--or does so only much later. The result, in my perception, is that with each new technology we suffer through years of "gee-whiz" articles that help the practitioner little, if at all, while distracting precious research and thinking time from more important matters.
For example, in the 1970's, the field (correctly, in my view) examined, and then largely abandoned design practices involving individualization of TBT by learning style. There were good theoretical and methodological reasons for this skepticism, as articulated by Cronbach and Snow (1977), and others. More recently, I wish the same could be said about such trendy topics as constructivism. I would have hoped to see reasoned and balanced examination of research evidence, discussion of the strengths and weaknesses in professional practice, and a serious attempt to incorporate into practice what seems to be of value. Instead, I think I've seen a quasi-ideological contest of true believers on each side (though a recent article by Jonassen (in press) seems to be a more balanced view). The positions seem to be held as fervently as those in the Macintosh versus Microsoft religious wars.
My first faculty appointment was in a medical school, where I learned quickly the distinction between "medical research" and "clinical practice."
In medicine, there are generally acknowledged standards for proof of safety and (cost-) effectiveness which must be met before it is considered professionally responsible to incorporate a research treatment into standard clinical practice. And, there are a number of things practitioners do just because they have been shown to work, even though the theory has no adequate explanation, yet. Of course, we all are aware of the complexities and criticisms medicine has encountered over the issues surrounding this distinction, and my intent is not to represent the medical system's model as flawless. My only intent is to make the point that the TBT field seems to have no comparable self-discipline. (Parenthetical note: that was the original thinking behind the creation of what is now Educational Technology Research & Development (ETR&D)/Development section as an alternative to (ETR&D/ Research section. I will let the ITForum debate whether this distinction has held in the editorial content of the two sections.)
There's another way of looking at this point. Practitioners, by definition, must deal with the full, ugly complexity of real-world problems. They do not have the option afforded researchers of choosing what portion of reality to investigate and theorize about. Since theories (especially in our field) tend to work only because they deal with a circumscribed subset of reality, there is therefore a hard upper limit on how far a given theory can take a practitioner in solving a problem. The ID practitioner thus is left with the task of solving his/her problem by assembling some combination of multiple theories and personal judgment to reach a design solution. This should come to no one as a shock: it's simply another way of saying that the connection between theory and practice is never direct and simple. From research and theory the practitioner can get expectations of how cause and effect are probably related, all other things being equal. This hardly is a new discovery about our field (for an analogous argument about Human Performance Technology, see Foshay and Moller, 1992). And yet, this basic principle seems to have escaped most of those whose writings present new theory in the field.
(4) We're a decade behind advancements in the theory of related fields.
By its nature as an applied field, ID and TBT draw from more basic research done in many other fields, including educational psychology, organizational psychology, measurement and evaluation, systems theory, etc. But I believe that while the field was distracted by the task of absorbing the microcomputer revolution in the 1980's, we lost the initiative in areas such as instructional principles derived from cognitive learning theory, systems design methodology, and human performance technology. Only now are we finally getting around to "catching up" on the decade or more of research in these fields. While I believe this is leading to a "golden age" of ID theory development, we also need to recognize that we are seriously behind our cognate fields in the incorporation of cognitively-based instructional models, the change from a "waterfall" design methodology to a rapid application development (RAD)-based, short cycle time, iterative design methodology, and quality management (Tennyson and Foshay, in press). This lag has to be one of the reasons why ID has had only a marginal influence on the practice of TBT.
I believe a productive response would include a dialog to survey the cognate fields of ID and TBT for developments we should be investigating and synthesizing into our own theories and models. The result would be a widely-recognized agenda for research and theory in the field. With the notable exception of the ID2 project, I have seen no such disciplined effort.
The dominance of Windows has led to consolidation of the authoring tool market down to about three major players, none of which meet the requirements of knowledgeable TBT practitioners. Full discussion of this point would be the topic of another paper, but it is enlightening to discover the kinds of tools now expected in any mature RAD-based professional software environment, and to contrast that tool set with the feeble capabilities of our current tools.
Unfortunately, I'm pessimistic about the long-term future of our authoring tools. The lack of substantial ID training by most TBT developers means that the market for authoring tools is hopelessly fragmented. At the low end, the tools are being crowded by the general-purpose multimedia presentation tools, such as HTML tools, Powerpoint and Director. At the high end, the tools are being crowded by general-purpose object-oriented development environments, such as Visual Basic or Visual C++ and the complete suite of design and development tools for each (sorry, Java doesn't qualify yet). That has left the authoring tools scrambling to keep up with progress in popular capabilities such as HTML, Java support, and multimedia support--all of which are available (better) elsewhere. Meanwhile, the truly unique requirements of TBT aren't being addressed, such as answer analysis (none of the present tools have as much power as the 30-year-old, ugly approach of Tutor), or instructional simulation. Why? Because the TBT market is so fragmented that these "unique" TBT requirements have very small demand. I would not be surprised if, within a few release levels, the low-end users will have migrated to the general-purpose presentation tools, and the high-end users will have no alternative but to build their own class libraries of objects and authoring environments, just to remain competitive on cost and cycle time.
The net result of this development, in combination with the rising expense of multimedia production, will be to raise the cost of entry for production of TBT. One-person shops or small consulting groups will no longer have the resources to capitalize their development projects or to develop a competitive authoring environment. They will be able to do no more than serve as hired hands for the larger companies, if they survive at all. We are already seeing that kind of consolidation in the retail/home sector, and in certain specialized high-end TBT markets (such as commercial aviation, where TRO is virtually the only viable independent producer remaining). I predict consolidation will become the dominant scenario in all TBT domains over the next few years.
It's at best very complex to describe objectively what makes well-designed and effective TBT. That, plus the strong likelihood (discussed above) that our clients, customers and employers have never seen well-designed TBT, means that our clients don't know how to recognize or ask for high-quality design.
The usual market mechanism in such situations is for consumers to recognize certain brands as having high quality, even if they don't know what that means. For example, you don't have to know anything about automobile design to know the difference between the brands "Mercedes" and "Yugo."
I believe that in the TBT context, ID has almost no "brand integrity" among most of our clients, customers and employers. It's up to us to establish it, through a combination of quality standards (which could be based on any combination of accreditation, certification, or even simple recognition awards, addressed to practitioners, processes, or products). For example, Craig Johnson at Florida State has been leading a long-term effort to establish an ISO 9000 guideline for training and education. The U.S. version (ASQC, 1996) has now been published and adopted by the American National Standards Institute (ANSI). This makes it possible for a U.S. training or education organization to be ISO 9000 certified in an ISD-like process. Once there is a definition of what the "ID brand" means, and a means for assuring the brand's quality, then the principles for establishing the brand's integrity in the minds of consumers are well understood by marketing professionals. We would do well to take their advice and act accordingly.
TBT is a classic knowledge-based business. As such, it experiences the problems common to any knowledge-based business when dealing with the financial community (banks, investors, etc.). The most fundamental problem, in my view, is the failure of standard accounting principles to adequately value knowledge assets. For example, when a manufacturer goes to the bank to get a loan, the bank will accept as assets the factory building, the inventory, and the order backlog. For a knowledge-based business such as TBT, none of these typically has much value. Instead, the real asset of the business is its knowledge. But that's not tangible, so it doesn't count. So, it's hard to raise capital.
Some of the more sophisticated knowledge-based businesses in the world are beginning to work on the problem (one of the arguments in favor of EPSS is that it renders a company's knowledge tangible). And at least one major accounting firm (Arthur Andersen) has a project under way to address this issue (can anyone provide a reference?). But for now, at least, I see little help.
The need for capital is particularly acute because of the rapid growth of the market. The sectors I described in the introduction all are said to be growing in the range of 10% to over 40% annually. If the capital to fuel such growth isn't available through the open market, then it will serve as probably the major factor driving consolidation of the field under the umbrella of large and cash-rich companies.
However, I'm profoundly skeptical of the ability of high-quality TBT to thrive in the face of such consolidation, in the long term. For example, textbook publishers have (correctly) recognized the strategic threat posed to them by TBT, and have been purchasing TBT companies for at least 15 years. I believe the fate of most such acquisitions has been stagnation or death within a few years. In my view, the current trend of acquisition by media conglomerates (such as Viacom, Time-Warner, or Disney) may not fare much better in the long run. In companies engaged in other businesses who build (or buy) TBT capability for internal use, TBT probably is not seen as a strategic or "core" technology of the enterprise, and willingness to grow it will depend entirely on the ability of TBT to deliver on its promises and continue to convince senior management of its value.
When was the last time you heard of a venture capitalist sinking significant funds into R&D in ID? It's quite common for software start-ups, even though risk is very high. Why the difference? Because there's only a weak market for high-quality ID, even though there's a $3 billion market for TBT. If we succeed in proving ID's merit in the TBT market, then perhaps support for R&D in sophisticated ID will result.
Meanwhile, a number of other disciplines (most notably cognitive science) has been able to compete successfully for R&D funding from the traditional sources (such as ARPA and NSF). I would argue that the reasons include excellent salesmanship by proponents, and a loose agreement on a research agenda for the field, so funding sources get a consistent message about what they are "buying" with their investment of research funds. I see little such coherent representation of TBT and ID.
The heavy demand for ID services has created a de facto "brain drain." There are significant inducements for academic ID people to spend on consulting what would otherwise be time spent on research. Of course, the consulting is valuable "real world" experience, but it can also be a distraction. I also believe that the net flow of high-level talent between the private sector and academic ID departments is unbalanced in the direction of the private sector. Simultaneously, it seems that nearly every other department related to education or training has discovered TBT, and yet they often seem unaware of what they could learn from the ID field or their own university's department.
The major reason for this trend is probably the overall contraction of university budgets, particularly in colleges of education, where most ID departments are housed. We have a few examples of departments whose leaders have overcome the academic politics and used the demand for TBT as a point of leverage to build new strategic alliances within the university and with corporate and other external partners, and so to emerge as a major player on their campuses. But accomplishing this appears to me to require an unusual combination of leadership and circumstances, so it does not qualify as a trend. As a result, I am pessimistic about the likelihood of most academic ID departments to overcome their constraints.
In a recent presentation at AECT, Allison Rossett characterized this as the best of times, and the worst of times for TBT. I heartily agree. The opportunities for technology to systemically transform the worlds of education and training are real, growing rapidly, and within our reach, if not our grasp. At the same time, success of this magnitude is by no means a foregone conclusion. There are real and substantial factors which limit the growth of TBT. Taken together, I believe they have the potential of causing the growth of TBT to self-limit.
In spite of this, I am cautiously optimistic about our field. With adequate leadership and cooperation among researchers, those who train people entering the field, and practitioners, the limiting factors can be overcome, mitigated, and circumvented. If we can overcome our isolation and empower ourselves, then together the field can reach its potential. I am part of a private-sector TBT organization, and my company is vigorously competitive, yet I believe such cooperation is possible and in the strategic interest of my company: a rising tide floats all boats. But let us move quickly: windows of opportunity such as TBT has now do not stay open long.
American Society for Quality Control. (1996). American national standard: Quality assurance guidelines for the application of ANSI/ASQC Q9001 or Q9002 to education and training institutions (Document ANSI/ASQC Z1.11-1996). Milwaukee, WI: ASQC.
Cronbach, L.J., & Snow, R.E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington Publishers.
Foshay, W.R., & Moller, L. (1992). Advancing the field through research. In H.H. Stolovich & E.J. Keeps, (Eds.). Handbook of human performance technology. San Francisco: Jossey-Bass.
Jonassen, D. (in press). Instructional design models for well-structured and ill-structured problem-solving outcomes. Educational Technology Research and Development.
Tennyson, R. and Foshay, W. (in press). Instructional Systems Development. In S. Tobias & D. Fletcher, (Eds.), Handbook of instructional design (working title).
The buttons that appear below will be found at the top and bottom of each page of the discussion. The first button will take you back to the previous page (in this case, to the beginning of paper #19). The middle button will take you to the ITForum home page. The last button takes you forward into the discussion as it progressed on-line.