Reading as the Interpretation of Signs

This study examines the challenges of analyzing texts that are multimodal - that is, combining more than one mode of communication, such as written text, images, video, and/or audio - and considers the theoretical and practical implications of two very different approaches. The more common current approach, which adapts the work of Halliday and Saussure, is criticized for failing to account for critical differences in the ways that language, both written and spoken, and other modes of signification convey meaning and consequently rely on linguistic structures in the analysis of images. A second approach, based in Peirce's triadic approach to semiotic analysis, which allows Saussurean modes of interpretation in its final stages, is proposed and demonstrated through the analysis of two news videos about the Occupy Wall Street movement produced by CNN and Al Jazeera. In conclusion, the implications of the two approaches for how literacy and multimodality are defined and analyzed are considered.