Media comparison studies: Problems and possibilities

Background/Context: Media comparison studies aim to compare the relative effectiveness of different media at promoting educational outcomes. While these types of studies remain popular, they have been under attack for more than two decades. Critics of media comparison studies claim that continued studies are unhelpful because a great number of research projects have already shown that media produce "no significant difference" in learning outcomes. They also claim that the studies that do find a difference among media are flawed because of a conflation of "media" with "method." These claims suggest the need for conceptual clarification. Purpose of Study: The purposes of this study are: (1) to uncover and evaluate the conceptual assumptions that are made in debates about media comparison studies, (2) to make some suggestions for how media comparison studies could be made more useful and interesting. Research Design: The methodology used in this study is a conceptual and philosophical analysis. Findings: The analysis produces three major findings. First, the debate so far has not examined the underlying assumptions involved in the process of making a comparison. Second, there has been a failure to appreciate how general comparative statements across media technologies must stipulate contexts of use, as well as a failure to examine the implications for the debate that flow from this requirement. Third, the concept of a "medium" has not been explored with sufficient depth, either with regard to how this concept is different from "method," or to how this concept is itself a contestable metaphor for classroom technologies. Recommendations: In the end, the following suggestions are offered for media comparison researchers: (1) Media researchers should realize that claims about a "significant difference" and about whether something "matters" to learning, will always be limited to particular educational goals and contexts of use - no global statements can be made about such things. (2) New media technologies can and should be compared with regard to the different educational ends they make possible rather than as mere means to an end that is already assumed. (3) Media researchers need to explore the educational importance of media that extend beyond intended aspects. Such a research program would benefit from adopting the "space" metaphor, from looking at educationally practices more holistically, and from a robust qualitative element in the comparative research.