As we continue to ride out the information explosion, we also continue to see increasing demand for performance support systems (PSS). This demand has been somewhat ameliorated by the current economic boom in the United States but has nonetheless been increasing. The "better, cheaper, faster" mantra adopted by NASA is being embraced by training organizations everywhere. Despite a paucity of research on the efficacy of performance support systems, these organizations believe that PSS's can indeed improve their training quality, reduce training costs, and reduce training time.
One reason companies might want to build a performance support system is downsizing. When companies scale back the number of employees they do not often scale back the number of tasks which must be accomplished in the organization. Instead the remaining employees are forced to become proficient at a wider variety of tasks. This need for increased proficiency and skills could be met by training, but companies may be reluctant to invest in training if they foresee continued downsizing in the future. At the same time the nature of the tasks employees must complete is changing as well. Tasks are becoming more and more complex as we shift toward an information-based economy. For example, the job category "typist" has all but disappeared over the last several years. Instead it been replaced by the category " "word processor." Word processing is already significantly more complex than mere typing and becomes even more complex with each new release of the popular word processing programs. Microsoft Word 97 has at least 1200 commands that the user can employ. Training would need to be almost continual for employees to stay current on all the intricacies of the tasks they must now complete.
One way to meet the demands of a shrinking workforce and an increasing task complexity without resorting to massive and perhaps somewhat risky training operations, is to employ a performance support system. A performance support system is a seamless integration of information, tools, training, policies and procedures designed to situate learning in real performance contexts and enhance that performance at the moment of need. They are a natural evolution of the just-in-time training concept. However unlike the just-in-time training concept, a PSS developer recognizes that training alone is not always the best solution to performance problems, therefore training makes up only one component of PSS.
Performance support systems were originally and are still frequently referred to as electronic performance support systems (EPSS). However, the addition of the word "electronic" limits performance support systems and tends to negate the importance of the last word of the phrase. "Electronic" implies that performance support is based solely on hard technologies such as computers and the Internet. While computer-based technologies are necessary in modern performance support, they are not sufficient to comprise an effective system. In this paper we present lessons learned from the development of a large-scale Internet based performance support system. We present an overview of PSS as embodied in that project, and problems encountered and lessons learned. We also address the importance of taking a systemic approach to performance support. Finally, we argue that it is perhaps time to rethink the entire concept of performance support systems in light of developments in general systems theory.
Early in 1996 a large telecommunications corporation announced plans to establish several new call handling centers with approximately 500 employees per center. Employees at these centers utilize a software program designed to allow the input of pertinent data through a graphical interface into a database of customer and supplier data. Each center is designed to handle thousands of calls per day. New employees attended an approximately three week long instructor-led training program to learn the job processes and to learn to utilize the computer-based data system. Putting thousands of new employees through the traditional training program could cost tens of millions of dollars. Clearly a new approach was needed.
After a thorough analysis it was determined that the new approach had four main objectives.
To meet these objectives, a comprehensive Intranet-based Performance Support System (IPSS) was designed to support the initial core training for new employees and provide both new and experienced employees specific task and procedural support while performing real "on-the-job" work. The IPSS also provides employees with immediate just-in-time knowledge of any changes and modification relevant to their particular duties.
We completed an extensive task analysis to determine where it would be desirable to support employees work. We collected reams of data detailing every step of the job processes. We initially divided work on the support system into three main categories: information, tools and training.
We determined that there were three types of information needed in the system. One was reference information. This was the kind of information that employees might need to access frequently in the course of their day, but was impractical to learn. For example this might include telephone numbers, or product codes. Part of this information might be categorized as declarative knowledge. However, another part of this type of information referred to the procedural steps employees might follow in performing their jobs. In this particular situation, employees had three major job categories with several subcategories of jobs in each. On a given day, depending upon demand, an employee might be expected to work in anyone of the several categories. In addition, as a call center in one region of the country experienced excessive demand, calls might be automatically forwarded to a different call center in a different region. Each region had a somewhat different set of methods and procedures. Thus an employee in New York might handle a New York call followed by a California call followed by a Florida call, requiring her to use three different sets of procedures in succession. This combination of multiple job types with diverse procedural information meant that employees had to deal with an extremely large and complex set of performances.
Another type of information that the system required was time sensitive information. For example there could be daily bulletins about changes in suppliers or procedures, even announcements about upcoming events or programs. Employees had to be made aware of these changes and announcements, and importantly, the organization needed to know that the employees were aware.
A third type of information the system required was expert advice. It was possible that the employee might encounter complex problems that were not included in the regular job methods and procedures. Employees needed a means of dealing with these problems. Temporal and financial constraints prevented our constructing a true expert system so we created an online help center employees could access when needed.
There were three types of tools included in the system. The first type was templates workers could use in completing tasks. These typically took the form of an already completed example template coupled with blank templates that the workers could fill out. The second type of tool was Decision Aids that could assist the worker in determining which procedures to follow. These typically took the form of a series of branching proposition statements that eventually presented the user with a narrow range of options that might apply in a particular situation. The third type of tool included in the system was an information management tool. In the course of completing their work, call center employees generated quite a bit of data about their tasks. They also required data from numerous subsystems to completed tasks. The information management tools allowed them to collect data in one place and then automatically disburse it to its proper subsystem or conversely collect data for multiple subsystems and have it displayed in one place in a manner relevant to the task at hand.
In addition to the tools and information components of the system, we also developed a training component. There were two broad issues of training associated with the system. The first issue was content training. This included both training on the job content itself and also training on the use of the performance support system. Employees required training on job content that was mission critical or time dependent. Certain aspects of work were so important to the overall success of the system that we were reluctant to depend solely upon employee application of the PSS functions. We wanted to ensure that employees had a more fundamental understanding of certain content and its relation to the overall task for these content areas. We included online training in the form of traditional CBT. Other aspects of the work required employee interaction with customers in real-time. In these situations the employee needed automatized skills that could be applied without the delay of accessing another part of the PSS.
The second type of content training was training on the PSS itself. While we tried hard to avoid needing a performance support system for our performance support system, the PSS ultimately became complex enough that not all aspects of it were intuitive. Further, employees were unaccustomed to working with a performance support system. We found it necessary to train employees on the PSS and on how to work with the PSS.
The second issue regarding training with PSS's was the timing of the training. We determined that there were three aspects of timing of training. The first was new hire preparation. Performance Support Systems to do not allow anyone to come in off of the street and immediately do a job. At least in this instance there was a base level of knowledge and skills required before the employee could begin to make productive use of the PSS. It may be of interest to know however that with the performance support system we were able to reduce the length of new hire training from three weeks to six days. This six-day training was instructor led. All the materials for this training program including presentations, exercises, handouts and so forth were incorporated into the PSS. Instructors used the PSS to deliver their training and students had access to the same training once they were out on the job.
The second kind of timing involved in the PSS was on the job as needed training. There are two aspects of this category. The first is when employees needed training during the performance of a task. When employees were completing non-mission or time critical tasks, they had the option of doing additional training on those tasks as they worked. They also had the option of doing additional training during their downtime. For example employees could choose a job category in which they were not yet efficient and study it while waiting to come on shift or after they went off shift. Employees were motivated to undertake this self-initiated training since they were rewarded with higher pay as they became qualified to complete more tasks. The second aspect of on-the-job training was required training. Frequently the company would implement new systems or procedures and would require all employees to be trained in these new systems or procedures. In addition, the company had mandatory safety training sessions as well. They wanted to make sure that every employee received training updates, but did not want to pull employees off the line and run a training class every time something changed. Therefore the PSS had the ability to lock employees out of their work systems until they had completed the required training.
When we began working on the project we envisioned that our development team would work closely with subject matter experts and technical infrastructure developers to create the performance support system we would then implement with the workers. This turned out not be the case. We soon realized it was necessary for our development team to work not only with the subject matter experts and technical infrastructure developers but also with methods and procedure writers, trainers, customers, subsystem developers, unions, and management. Our initial development plan was based on an artificial model of a simple, relatively closed system. The reality was that we were working in a complex opened system. It soon became apparent that if our PSS were to have any hope of success it was necessary for us to identify the components of this larger system and determine their interrelationships.
For example it was quite difficult to determine who actually "owned" the content for which we were developing the PSS. In this case the subject matter expert we were working with was a line manager for the call center employees. This manager had a fundamental understanding of the tasks that her employees needed to be proficient in. However, the pre-existing training for these employees differed, in some cases significantly, from the information we were getting from the subject-matter expert. To further complicate matters, the organization had a methods and procedures group who quite literally "wrote the book" for the tasks the employees were performing. And the systems designers, that is the people who wrote the software that the employees used in their jobs, had highly developed ideas about how their software should be used. Even the union to which most of the employees belonged occasionally had something to say about how the job should be performed. As if that weren't enough, our own instructional designers were, through the result of their task analyses, developing very specific ideas about how the work ought to go. Who owned this content? All of the groups mentioned as a valid claim to at least part.
Another problem we encountered was that when a new hire first went online at a call center with the performance support system, he would not be as initially productive as someone who had had more extensive training about the job, at least theoretically. His unfamiliarity with both the work and system would reduce productivity because of the extra time needed to determine how to do a task that that an employee who had been through the traditional three-week training might already have mastered. As result of this extra time the PSS trainee might not be able to handle as many calls in the course of a shift as a traditional trainee might. However, the PSS trainee would be on the job 15 days sooner than the traditional trainee and should eventually reach a higher level of performance in a broader variety of tasks. The reaction of quota driven line management was that the initial low productivity levels were unacceptable. They were reluctant to allow the PSS employees' time to complete online training.
Unless you work for NASA, work does not occur in a vacuum. It occurs in a rich, dynamic environment of subtle and sometimes not so subtle interactions. Relationships form, change and dissolve continuously. Continual changes in the competitive environment require continual changes in the performance environment. Changing personal and political situations require changing operational strategies. It other words, it's a jungle out there.
We had initially developed a performance support application, not a performance support system. It worked fine in the artificial context of the development environment, but it failed to take into account the complicated interactions that were occurring in the actual performance environment. The part of our definition that addressed "enhancing performance at the moment of need" was wrong. By the time the "moment of need" rolled around it was far too late. We realized that we had neglected the second "S." Our performance support application had to be seen as only one component of the overall system. We needed to make changes throughout the system if we were to be successful in performance support. This was not an easy proposition. We had been asked to change the way the organization did training. Instead we needed to change the way the organization did business.
One lesson from General Systems Theory is that a change in any one component in a system may affect the operation of all of the other components of the system. PSS's require a major change in the way employees do their jobs, if not permanently then at least initially. It would be shortsighted to suppose that we could make that change and not affect the rest of the organization. Yet, training departments may not always have the political wherewithal to make the changes needed in the rest of the organization. Management might not be amenable to providing the reward and support structures needed in the new work environment. Other units in the organization might not understand the importance of collaborative interaction in job design. In short, it may not be possible for a training department to implement the kind of systemic changes needed for effective performance support, if indeed they can even be identified. Perhaps what is needed is a rethinking of our approach to performance support systems.
One problem with traditional PSS development is that it assumes a regular symmetrical system. That is to say that it assumes a work environment in which it is possible to predict how operations in one part of the environment will affect operations in other parts. Such predictions are reasonable given a broad enough perspective of a system. For example from a distance it is easy to discern a forest. Move closer and you can recognize individual trees and branches. You might also observe some interactions of these components (i.e. how a branch shaded by a larger branch is stunted.) But if you continue to shrink the unit of analysis you eventually reach a point where such recognition of parts and relationships cannot be modeled. At the molecular, atomic and particular levels we can theorize about how the molecules of the tree interact. However we cannot know the interactions of specific individual molecules.
On close analysis most social systems turn out not to be so easy to predict. A shrinking unit of analysis quickly brings us to a point where actions or interactions are difficult if not impossible to pre-determine. These systems, or in this case performance environments are complex.
Complexity can then be characterized by lack of symmetry or"symmetry breaking", by the fact that no part or aspect of a complex entity can provide sufficient information to actually or statistically predict the properties of the others parts. (http://pespmc1.vub.ac.be/COMPLEXI.html)
We might be able to predict with some accuracy how a corporation might react to a given input, or how a department or unit might react. But when it comes to modeling individual workers in a system symmetry breaks down. There are too many variables that might influence interactions among workers or the workers' individual performance at a given task on a given day. Mary's attention is wandering because she's excited about a weekend trip to Puerto Rico. Bob is short-tempered because he ate a bad egg at breakfast and is suffering the effects. But even these situational variances aren't fully illustrative of our individual unpredictability.
We all work in somewhat different ways. Jim's modus operandi may be more intuitive, where his decisions result from an inductive approach. Elizabeth may be more methodical, carefully following a deductive path to arrive at a solution. Smith and Ragan (1998) do an excellent job describing both the relatively stable and the varying similarities and differences in learners. The same distinctions can be applied to performers. Each of these can influence how we approach a task. My physical characteristics, training, temperament and experiences all influence how I do something. I venture that my driving style and your driving style are fairly different. Yet we both most always can get wherever we're going safely. In the words of American folk singer John Hartford:
I tried real hard not to make this song sound like any other song that I might have written before, But if I did that's Ścause it's my style and style is based on limitation. And I tried real hard not make this song sound like any other song that any other singer or songwriter might have written before, But if I did that's 'cause it's music, and music's based on repetition. --John Hartford, Trying to Do Something to Get Your Attention.
We support the music of task performance, but we make no allowance for style. Traditional PSS's are built on the assumption that all workers will complete a task in the same way. No options are built in for situational or stable differences in performance style.
Perhaps what is needed is a different approach to the design of PSS's. Rather than require workers to perform tasks in a set series of steps, perhaps we should borrow a page from hypermedia design and purposefully give performers the ability to perform steps in the order that seems most natural to them. True hypermedia systems allow users to go from any one piece of information directly to any other piece of information, thus actualizing the node-link structures of their individual semantic nets. Might it be possible to design and build hyper-performance support systems? Workers would be able to complete tasks in the way most suited to their individual abilities and styles. Workers would like these systems because they feel natural. Perhaps they would facilitate "flow." (Management would like these systems because they introduce the term "hyper-performance." Perhaps they would facilitate profits.)
These systems would also provide the flexibility needed to account for complex performance environments. Different performers might use them in different ways depending on with whom else in their environment they were interacting. They would be dynamic in that performers could use them in a different way each time they attempted a task or indeed, change them if they so desired. In much the same way that we can create a link to a meaningful site on the World Wide Web, workers could create performance links to tools or information that suit their styles. The Hyper-PSS would be organic, growing and changing to meet the complexities of its changing environment.
We're still developing the concept of "hyper-performance support." We're not sure yet just how such a system would look. At this stage we recommend focusing on the second "S." When building Performance Support Systems be sure and consider the larger organizational system. Design a system, not an application. Don't be taken in by the electronic "E." If you lack the power to effect systemic organizational change then a performance support application may be the best you can hope for, at least until hyper-performance arrives.
The buttons that appear below will be found at the top and bottom of each page of the discussion. The first button will take you back to the previous page (in this case, to the beginning of paper #36). The middle button will take you to the ITForum home page. The last button takes you forward into the discussion as it progressed on-line.