ITForum Paper #6

What Leads the Research

in Computer-Based Learning:

Research Questions, Methodologies, or...?

Dr. Ron Oliver
Edith Cowan University


This paper is written in a less formal style than previous papers and its intention is to stimulate debate and discussion more of a practical than theoretical nature. It is my hope that ITForum subscribers who have not had much to say in the past may find something to add to the discussion about this paper. I have included some hints for shy subscribers at the end of the paper to help those who are new to the service to overcome the pain of their first contact.

When I submitted a topic to Lloyd several months ago, I suggested a topic that might naturally follow on from the discussion of Tom Reeves' paper. I was disappointed that more people didn't argue with Tom but he seemed to make his point so well that anyone who disagreed seemed to end up with egg on their face. I know.

In the last ITForum, Tom Reeves [ITForum #5] wrote a paper that took a long and hard look at research in the field of instructional technologies. He concluded that much of the research was extremely lax from a methodological point of view and as a consequence lacked application and real value. Tom suggested that a bit of soul-searching and perhaps some new directions would be of considerable benefit if we want truly to make some contributions to the quality of the educational process. He introduced the term "social relevance" as one possible measure of research quality. Judging by the lack of argument that made its way onto the board, one could assume that there was a high degree of agreement among readers with the points that were raised. The difficulty is knowing where to go from here. What can be done to raise the social relevance of research in this field? The purpose of this forum is to invoke some discussion that might suggest some possible strategies.

The Motivation and Purpose of Educational Research

When one considers the reasons for the large amount of educational research that goes on these days, it is helpful to consider both the motivation and the purpose of the research as distinct and separate entities. Educational research is done for many reasons apart from a desire to improve the system. In fact I would posit that the majority of educational research that is undertaken is initiated and motivated for reasons other than a genuine desire to advance knowledge. Some of the reasons are esoteric, others are opportunistic, and some entirely pragmatic. In the world of academia, educational research is primarily a means to an end rather than an end in itself. In the main, it is done to gain qualifications, to demonstrate scholarship, and to advance career prospects. Under these circumstances it seems quite natural that some or much of the work will lack social relevance. Research is based on the identification and solution of problems. Research is seen to be done when solutions are found even when problems do not necessarily exist. It is the reasons why we do research rather than the way in which we do research that I think leads to the problems of social relevance which Tom spoke about in his paper.

A Case Study

This issue first came to my attention after I had just completed my own Ph.D. in which I investigated the ways teachers in local schools taught computer programming. I questioned the capability of conventional teaching methods to develop students' understanding of programming concepts and the programming process. My own experience had been that most students when learning to program, gained very little understanding of what they were doing, but instead gained a broad knowledge of programming syntax. An initial study in local schools demonstrated that this was the case. From there I went on to develop and trial a model of teaching that could be used to improve students' understanding of programming semantics. Applications of the model proved to be very successful and I was able to conclude that if teachers applied this instructional strategy, their students would understand more about what they were learning than they might if the conventional model was used. The study served my purposes, it resulted in a Ph.D., provided some material for conference presentations, was useful in my teaching activities, and yielded several publications, but it all appeared to end at this point.

No one in local schools, not even those involved in the study showed anything but a passing interest in my findings or the alternative instructional strategy. I had created the proverbial solution where a problem was not perceived to exist. The local teachers weren't at all worried that their students might not be developing an understanding of programming concepts. They indicated that the more able students did understand what they were doing and there was little that could be done about the others. They noted that the external exam by which students' achievement and knowledge was assessed was heavily directed towards a knowledge of programming syntax and that few questions demanded the understanding that my method was able to develop. In short, there was little to motivate teachers to change from what they were doing.

Factors Motivating Research Topics

Why did I choose the topic? In the first instance, I needed a topic for study, so I chose one in an area of interest to myself that could serve my purpose, that is provide the scope and opportunity that would satisfy three external examiners. From my perspective, the topic was important and something needed to be done. I judged it to be very socially relevant. From the perspective of the other stakeholders, it wasn't important and although I had loads of support in conducting the study, few teachers were really interested in what was being studied and learned. My judgment of its social relevance was not matched by the judgments of others. The problem had nothing to do with how well the research was conducted, whether it was pseudoscience or not. It was all about the motivation and purpose behind the research. I wonder how many other readers have found themselves in this position. Is this a common outcome among other research projects? Should we expect our research to have application in the contexts in which it was conducted, or is it adequate for a researcher to satisfy him or herself with the fact that the body of knowledge has been extended?

Currently, I still see this problem emerging in the topics which many graduate students come to me with as possible Ph.D. and Masters theses. How many times have you had to deal with research students who know what they want to do in a study and how they are going to go about it but have extreme difficulty describing what they are researching and why it is important? Many students and academics tend to choose questions that emerge from the fields of work in which they are currently involved. This would seem to be a sensible place to start. In the next instance, questions are framed with a distinct methodology in mind. Again this is being very practical. It recognizes the need for an environment in which a study can be successfully completed and is very much influenced by external factors. But the consequences of this practicality can be quite restricting. Questions tend to emerge that seek to find solutions to what are quite contrived problem sets.

Considering the Target Audience

Given the many different reasons that motivate research, the question that needs to be answered is what can be done to guide research so that it will be meaningful and relevant? One possible solution is to consider more carefully before commencing who or what is the target audience of the research. The target audience is the beneficiary of the research outcomes. When one considers and includes the target audience in the planning process, issues of relevance and meaning become clearer.

When I read educational research journals, I see a good deal of research whose target audience appears to be the academic community. Journals like Journal of Computer-Based Instruction and Educational Technology Research & Development reviewed in the last paper tend to be good examples of this. I would guess that these journals see academics and researchers as their main audience. It is only natural, then, that they will be filled with papers and articles that may be judged as socially irrelevant from the perspective of influencing teaching practice. On the other hand, these articles may be very relevant for researchers working in the field. Likewise journals created for teachers and practitioners tend to use different criteria in selecting the sorts of papers and articles they publish. These papers would suit one audience well and another less well.

When the target audience of the research is considered in the planning process, the questions that are posed, the methods that are employed, and the manner in which the results are reported all become important issues. The likelihood of creating research that has contextual value and application will increase.

Where to from here?

I am wary that some might find this whole discussion socially irrelevant. After all, the prime motivation behind this paper was to provide input to the ITForum. But I was thinking about the audience when I wrote it. I'm sure many will have an opinion of one form or another on this topic.

In summary, the points I have made in this paper are:

  1. The motivation for, and the purpose behind, educational research are two entirely different issues.

  2. These issues have significant impact on what research is done and how it is done.

  3. The prime motivation for conducting educational research is not necessarily a wish to improve current practice nor to extend existing knowledge.

  4. Measures of research relevance and value must be made from the viewpoint of the target audience.

  5. Consideration of the target audience in the planning of research can aid considerably in ensuring research is relevant and of value to the community at large.

The question that I would like to pursue now is what else can be done to guide research planning and conduct so that the outcomes will be meaningful and relevant? I think that a useful way to commence would be through a discussion of research projects that participants have undertaken that they consider are examples of relevant, meaningful, and worthwhile inquiry. In describing the project, you should explain the means by which you judged the research to have been relevant and worthwhile. Alternatively, you might like to describe a project as I did, that you thought would be worthwhile but you now feel was misjudged or had less of an impact than you originally thought it might. With the assistance of participants, I hope we can draw some common threads from examples and non-examples of meaningful and relevant research topics and end the forum with some clear guidelines on how to choose such topics.

Hints for Shy Subscribers

I appreciate that some readers may be a bit shy about interacting for the first time but may still have some very valid points to make. There are a number of useful strategies that can be used for entering the discussion without bringing the full weight of the Internet down on your shoulders. One way to overcome this is to introduce your discussion with the line, "a friend of mine once did a study that ..." or you could write and sign the letter in an anonymous fashion, for example, "Members of the Computer-Human Interface Research Group," so that you don't have to wear ownership of what you are saying, in case someone disagrees with you. The best method is to use a friend's computer. Then you don't even have to say who you are. I will now sit back and read the responses and come back on-line when Lloyd tells me I have to.

Ron Oliver, Senior Lecturer
Department of Library and Information Science
Edith Cowan University
2 Bradford St, Mt Lawley, 6050
Western Australia

Phone: 09 370 6372
Fax: 09 370 2910

The buttons that appear below will be found at the bottom of each page of the discussion. The first button will take you back to the previous page (in this case, to the beginning of paper #6). The middle button will take you to the ITForum home page. The last button takes you forward into the discussion as it progressed on-line.