We intend to study how human attention interacts with computer software. The long term vision is to make all software interact with people through a flexible mechanism that customizes the timing and content of interaction to conform with the needs of the user and helps improve the focus of attention, without causing distraction This research will focus on the extreme case where the normal human attention mechanism has been disrupted. As a result, this research will expose difficult software interface questions and highlight issues that might otherwise remain obscure. We will determine what kind of model of attention is required for reasoning about user-interaction in an educational system. We expect that dynamic models are required, and this research will explore their development. Our proposed research will generate fundamental insights critical to the successful deployment of attention models.
We intend to improve the interaction between computers and people by developing software models of human attention. Systems that have models of attention and use them to reduce distraction will help people complete tasks faster and more accurately. We propose to develop our computational theory of attention as a component in educational software designed to teach mathematics to children with Attention Deficit/Hyperactivity Disorder (AD/HD).
Extremely poor attention is a fundamental characteristic in children with AD/HD which "affects 3 to 5 percent of all children, perhaps as many as 2 million American children." (National Institute of Health). Because issues of attention are so fundamental to these children, our model of attention will be vigorously tested. Methods that help these children should be more than adequate for the needs of typical adults.
Our group has extensive experience in the development of educational software, clinical treatment of AD/HD and psychology of education. Our advisory panel includes one of the leading researchers and best read authors on the subject of AD/HD; a Co-Founder of Carnegie Learning, Inc. which distributes software for mathematical education; and computational expertise in cognitive modeling, diagrammatic reasoning and medical reasoning.
We will develop a tutoring system that reasons about the knowledge and capabilities of each student, in order to make high quality decisions about instruction. We will incorporate a model of the student's attention and demonstrate how this model can be used to improve instruction. We will conduct laboratory studies with eye-tracking devices for initial validation of our models. We will conduct field studies involving hundreds of students in local schools to further evaluate the system. We will analyze large databases of student behavior collected during these studies and use machine learning methods to incorporate the results into the software models.
Knowledge of mathematics is vital in our technological society, but the subject is challenging and uninteresting to many students; as a result many people have trouble concentrating on it. Students can't learn if they aren't paying attention. Therefore, the problem of teaching mathematics to children with AD/HD is ideal for research on computational models of attention. The resulting empirical results will provide scientific guidance to the selection and deployment of cost-effective techniques for incorporating knowledge of attention into user interface design. This research will address critical problems in the development of advanced software interfaces that model attention.
Figure 1. A Simple Fraction Recognition Problem.
2. RESEARCH PLAN
We have developed a prototype intelligent system for teaching early mathematics skills to elementary school students. The system is called AnimalWatch, because students perform simulated research on the ecology of endangered animals, which gives mathematical problem solving a meaningful context, Figure 1.
The system reasons about student performance and learning style to determine the appropriate problem solving support for each student. Problems are individualized to challenge each student without being too difficult. Graduated hints and interactive environments provide support for student needs at different stages of concept mastery. Numerous procedural and declarative hints are available in AnimalWatch. Manipulatives, such as "Cuisenaire rods" provide students with concrete demonstrations of specific mathematical skills, Figure 2. Procedural support allows the system to break a problem into manageable steps, Figure 3.
Although AnimalWatch adapts its instruction in a variety of ways, there is considerable room for improvement in the realm of maintaining a student's attention. We performed a case study with a 6 year-old AD/HD boy who worked with the system for an hour while we observed his reactions and level of attention.
Our observations suggest that the system has the potential to work effectively for children with AD/HD but should be modified as follows:
We expect that students using the enhanced software will make fewer mistakes and suffer less fatigue. We also expect an increased perception, among students and teachers, that the software is useful and valuable. Hundreds of students from elementary and middle schools will use the system during evaluation.
All of these enhancements require a model for reasoning about attention,
both at the design level and for dynamically adapting instruction. Successful
deployment of the enhanced system will validate the underlying theory of
attention and provide scientific guidance to the selection and deployment
of cost-effective techniques for incorporating knowledge of attention into
user interface design.
The existing prototype was designed to increase students' interest in mathematics and confidence in their skills as they complete elementary school, which is a point when many girls begin to lose interest in math (Beal, 2000).
Evaluation with over 400 10 and 11 year-old students demonstrated the system's educational value and found different styles of interaction among different groups of students. Abstract feedback was only effective for students with higher cognitive abilities; girl's performance was enhanced by interactivity and structure but those styles of interaction were not optimal for boys (Arroyo, 2000). AnimalWatch has a positive influence on students' self confidence and their belief in the value of learning mathematics (Beck et al., 1999; Beal et al., 1998).
We concluded that it is important to model these student characteristics
and modify instruction appropriately.
Children with AD/HD have poor attention control when compared to other children, making this group the best subjects for our study of attention. Psychology research often studies extreme cases for the building of behavioral models, and this has proven to be an efficient methodology. We will work with educators and psychologists who are familiar with AD/HD to develop principles for adaptive tutoring designed to support these children.
We will examine the many educational modifications used in classrooms, and empirically determine which can be implemented in software. Our system will reason about when a student is most distractible. This explicit model of attention will allow the system to optimize the use of a limited set of attention getting methods.
We will use eye-tracking equipment to record the location of a student's current focus of attention during laboratory studies of approximately 10 to 20 students using the AnimalWatch tutor. By monitoring eye movements, we will determine when students are not paying attention. Using eye-tracking to determine attention lapses has two benefits: it allows us to develop a predictor for when students will lose focus and it will reveal specific aspects of the AnimalWatch system that need improvement.
It is not practical to equip classrooms with eye-tracking apparatus. We will use machine learning methods to build a model that predicts loss of attention, based upon tutoring data that is available without using eye-tracking equipment. We will use existing data on a large group of students to leverage the results from our eye-tracking experiments. This will allow the software to predict loss of attention, either due to distraction or impulsiveness, based on data that will be available during normal classroom use. We will then experimentally validate our mechanisms by comparing the behavior of AD/HD children and non-AD/HD children using the tutor, as described in the following section. We have had success at predicting future student responses for accuracy and correctness (Beck et al., 2000) based on observable data. However, the ability to detect loss of engagement will greatly increase the educational value of the software.
2.3 Evaluating Adaptations
Clearly, an educational system should be made to interest all students as much as possible. However, methods for enhancing a student's focus of attention may conflict with other goals, such as moving quickly through the curriculum. A highly interactive system may be able to maintain students' attention, but may unnecessarily delay students with high attention skills.
Implementation of simple and coarse-grained adaptivity to different attention levels would be sufficient if a single attention model was appropriate for all students, or for easily distinguished groups students. However, attention is a complex dependent process and we expect the system to require integrated reasoning about context and attention abilities.
Experiments will demonstrate that our adaptive mechanisms reduce the frequency of loss of engagement and decrease the periods when students are distracted. We will measure student performance and attitudes along multiple dimensions, such as confidence, problem preference, subjective interest and objective score.
We will study students with different degrees of attention control in a 2 by 2 experimental design, varying both AnimalWatch and the students' ability to concentrate. We will test both an enhanced system and the baseline system on children with and without AD/HD.
The baseline AnimalWatch system will contain all of the media and content, but only the enhanced system will reason dynamically about the student's current level of attention to help focus attention. We expect that students with a greater ability to focus will be less affected by the changes we make, while students who have more difficulty concentrating should show larger increases across a variety of measures (problems solved, how much they like the tutor, etc.). Table 1 provides a summary of this design.
There is a trade-off between activities designed to increase motivation and time spent on the real curriculum. Motivating one student might require extended multimedia that does not benefit other students. If so, how big a penalty is involved, and will possible gains in motivation offset the time required? It may or may not be profitable to try to adapt the amount of support students receive to help them stay engaged with the tutor.
We can directly observe the impact such features have by comparing students
in the experimental and control groups (from the previous section). However,
this only determines the effects of our modifications at a gross level.
Perhaps certain interventions are harmful and others beneficial? We can
estimate the effects of individual interventions by observing how student
performance changes after each intervention is applied. Deviance from predicted
student reactions can be attributed to the new interventions (Beck et al.,
If the adaptations designed for AD/HD students are helpful to everyone, there is no need to increase AnimalWatch's level of adaptivity; we can simply leave these features turned on for all of the users. In this case, AnimalWatch (or any other computer software for that matter) does not need enhanced adaptation capabilities. This would be a very useful result for the general design of user interfaces, because it would justify simpler designs, although it is not the result we expect.
A second possibility is a dichotomy between the experimental and control groups. In this case, students with AD/HD perform much better with the adaptations, and other students perform better without the adaptations. Either a user preference or a short questionnaire could be used to evaluate the student's attention problem. If it appears likely that the student has problems concentrating AnimalWatch will enable support for attention lapses. This would be have less favorable implications for determining general rules for making software support attention disorders. However, there may be some subset of the adaptations that are always beneficial. These could be added to all versions of AnimalWatch, as well as serve as guidelines for other user interface development.
A third possibility is that there is no systematic pattern: some students do worse in situations where others do better; some students prefer or work better when given certain types of adaptation, etc. In this case, it is necessary for AnimalWatch to maintain an estimate of each student's current level of attention and to dynamically reason about the alternative intervention that will best focus attention and improve learning. While this implies that software must incorporate complex mechanisms, we believe that this research will indicate how to obtain excellent performance in the development of such advanced technology products. We have experience incorporating other cognitive factors such as Piagetian level of development (Arroyo, et al. 2000) to dynamically modify our tutor's teaching (Beck et al., 2000), with successful results.
Clearly, an educational system should be designed, as much as possible, to interest all students, but, methods that are effective for some students, in some situations, may be less effective for other students, in other situations. There is no theory that software developers can use to determine the best way to support student motivation in different contexts. The results of this research will help select among alternative solutions to this problem.
If we can show that differences in attention can be ignored or easily categorized, then future developers of intelligent tutoring systems can confidently use simple models of attention. If we demonstrate how a system can adapt to people's attention needs, our methods can be incorporated into future systems. In either case, completion of this research will provide useful guidance to developers of all kinds of human computer interfaces.
Considering attention is vital in the development of advanced interactive software, especially educational software.
Our group of educators, psychologists, multimedia experts and computer scientists will integrate diverse knowledge, techniques and theories in the development of models of attention. Behavioral psychologists at UMass have spent over 1,000 hours providing one-on-one remedial instruction to children with AD/HD. Research at the University of Massachusetts has produced educational software in many areas during the past 15 years, including cardiac resuscitation (Eliot & Woolf, 1995; Eliot, 1996), combat trauma, chemistry, engineering, history and language. Results from this research will contribute to the development of computational attention models in this project.
3.1 Intelligent Tutoring
Intelligent Tutoring Systems (ITS) evaluate the current knowledge, goals and attitudes of individual students. Intelligent tutors are among the most sophisticated implementations of educational software, but, until recently few have been deployed outside of the laboratory, despite 30 years of ITS research. These systems use explicit models of teaching theory, domain content and educational psychology to optimize the learning interaction (Anderson, 1993). Tutoring systems dynamically reason about the knowledge and capability of each student, generate new problems and select hints appropriate to the individual's needs (Beck, 1997). Intelligent problem selection and tutoring support are designed to enhance learning. Many of these systems have proven to be engaging, but the underlying theory of attention has not been properly formalized.
During the last five years, we have developed an intelligent system for teaching early mathematics skills to elementary school students. All instruction was designed in collaboration with classroom teachers to ensure that curriculum standards are met. Many problems involve graphics (charts, maps, diagrams) to encourage students to develop data interpretation skills. The tutor motivates students to use mathematics for practical problem solving as they observe, monitor and manage the endangered animals in a simulated research environment. The system incorporates cognitive factors such as Piagetian level of development (Arroyo et al., 2000) to dynamically modify software tutoring (Beck et al., 2000).
3.2 Attention and AD/HD
Research on attention includes substantial work in the areas of vision (Allport, 1989), education (Brickner, 1970) and cognitive issues (Norman, 1969), but little relates directly to computer interfaces, except see (Gluck et al., 2000). Therefore, our research on dynamically supporting user interaction based on attention models will be an extension to research that has been done in the fields of human-computer interaction, psychology, instructional technology and education.
Members of our research team have been involved with assessment and educational intervention on behalf of children with AD/HD for five years. Over 60 children have been given extensive human tutoring using methods based upon psychological theory and research. This familiarity with AD/HD will contribute to the development of principles of attention control for adaptive tutoring.
Children who have been diagnosed with AD/HD often have trouble learning because of difficulty with task organization, sustaining attention, following instructions, focusing on details and completing activities before attention is diverted (Beers & Berkow, 1999). It is known that these children are more quickly distracted and require greater frequency of reinforcement than most children. Many educational methods are used in classroom teaching to support children with attention problems, including reducing off-task interruptions, making on-task activities more interesting, eliminating distractions and providing organizational support to help children complete activities. "Recent research suggests that providing more stimulation and variety can improve the performance and behavior of students with ADD. You can alter the type of assignment, the activities involved, or even the color of the paper used." (Office of Special Education and Rehabilitative Services, United States Department of Education.)
AD/HD is a serious concern. "ADHD often continues into adolescence and adulthood, and can cause a lifetime of frustrated dreams and emotional pain." (National Institute of Health). Research provides ample documentation of the problem "AD/HD is one of the best-researched disorders in medicine. The overall data on its validity is far more compelling than that for most mental disorders and even for many medical conditions." (JAMA, April 8, 1998-Vol, No. 14, pg. 1105). Treatment for children with AD/HD typically combines use of medication, particularly stimulants such as methylphenidate, and behavioral techniques to improve self-control. Attention problems are often combined with hyperactivity and impulsiveness.
We will use cognitive, educational and computational theories to develop advanced theories for reasoning about attention. The AnimalWatch mathematics tutor will be designed using these theories to reason about instruction, based upon empirical data and the insight of educators and psychologists. The resulting software will integrate sophisticated models of student attention in a powerful multimedia tutoring system.
The model of student attention will be designed to meet the learning needs of the most distractible students and then evaluated to determine how well other students learn from the same system. Machine learning algorithms will determine when and how to select actions to maintain the student's attention. The explicit model of attention will allow the system to greatly improve the user interaction.
We will study mechanisms that reason about attention to improve the interaction between people and software. Our proposed research will generate fundamental insight critical to the successful deployment of attention models. In addition, the mathematics tutor makes clear social contributions to education in our technological society, and millions of students with AD/HD can benefit from specialized support.
Allport, A. Visual Attention. In M.I. Posner (Ed.) Foundations of cognitive science. Cambridge, MA: MIT Press, pp.625-648 (1989).
Anderson, J. Rules of the Mind. Hillsdale, NJ: Lawrence Erlbaum Associates (1993).
Arroyo, I., Beck, J.E., Woolf, B.P., Beal, C.R. and Schultz K., Macroadapting Animalwatch to Gender and Cognitive Differences with Respect to Hint Interactivity and Symbolism, In Proceedings of the Fifth International Conference on Intelligent Tutoring Systems (2000).
Arroyo, I., Animalwatch: an arithmetic ITS for elementary and middle school students, Workshop at the Fifth International Conference on Intelligent Tutoring Systems (2000).
Barkley, R. A., ADHD and the nature of self-control, Guilford, New York, (1997).
Barkley, R. A., Taking charge of ADHD: the complete, authoritative guide for parents, Guilford, New York, (1995).
Beal, C. R, Woolf, B. P., Beck, J. E., Arroyo, I., Schultz, K., Hart, D. M.: Gaining Confidence in Mathematics: Instructional Technology for Girls. Proceedings of International Conference on Mathematics/Science Education and Technology. pp. 57-64 (2000).
Beck, J., Woolf, B. and Beal, C. R., ADVISOR: A machine learning architecture for intelligent tutor construction Proceedings of the Seventeenth National Conference On Artificial Intelligence (2000).
Beck, J.; Stern, M.; Woolf, B. Using the student model to control problem difficulty. Proceedings of the Sixth International Conference on User Modeling (1997).
Brickner, C. Ann. (1970) The Analysis of Eye-Movement Recordings from a Sample of Underachieving Secondary Students and a Sample of Underachieving Primary Students. Philadelphia: Hanley & Belfus, 1996, pp. 7-11.
Eliot, C. R., Cohen, P., Amant, R. S., The Dynamics of Pointer Manipulation, NSF Final Report (1998).
Eliot, C. R., and Woolf, B. P., Multiple Agents Acting in Parallel within an Intelligent Real-time Tutor, Proceedings of the 13th National Conference on Artificial Intelligence, AAAI-96. Cambridge, MA: AAAI/MIT Press, 1996, pp. 409-415.
Eliot, C. R., An Intelligent Tutoring System Based Upon Adaptive Simulation, Ph.D. thesis (1996).
Eliot, C. R., and Woolf, B. P., An Adaptive Student Centered Curriculum for an Intelligent Training System, Journal of User Modeling and User-Adapted Interaction (1995).
Eliot, C. R., Williams, K, and Woolf, B. P., An Intelligent Learning Environment for Advanced Cardiac Life Support, Proceedings of the 1996 AMIA Annual Fall Symposium, Washington, DC, J.J. Cimino (Ed.),
Gluck, K.A., Anderson, J.R., Douglass, S.A. : Broader Bandwidth in Student Modeling: What if ITS were Eye TS? In Proceedings of the Fifth International Conference of Intelligent Tutoring Systems
Hallowell, E., and Ratey, J., Driven to Distraction, Pantheon, New York, (1994).
JAMA, April 8, 1998-Vol, No. 14, pg. 1105
Kafai, Y. Electronic Play Worlds. In Constructionism in Practice: Designing, thinking and learning in a digital world. Lawrence Erlbaum Inc. pp 97-123
Nadeau, K. G., Adventures in fast forward: life, love and work for the ADD adult, Bunner/Mazel, New York, (1996).
NIH Publication No. 96-3572 Printed 1994, Reprinted 1996
Norman, D.A. (1969) Memory and Attention. New York: Wiley, Chapter 2: Attention. Pages 7-36
Office of Special Education and Rehabilitative Services, United States Department of Education.