Remaking IO: Semiotic Remediation in the Design Process

Our experiences with, and understandings of, literacy are rapidly shifting as new technologies reorganize our inscriptional tools and practices and alter our semiotic landscape. A growing body of research (for example, Baldry and Thibault, 2006; Bolter and Grusin, 1999; Kress and Van Leeuwen, 2001; Lemke, 2002; Manovich, 2001) has begun exploring ways to understand multimodal artifacts. However, the situated and mediated practices of producing new media have received limited attention (see, however, Johnson-Eilola, 2005). This chapter examines multimedia production in an Art and Design program at the University of Illinois, focusing on two Art and Design professors and two of their graduate student research assistants in the process of remediating (in Bolter and Grusin’s, 1999, sense) a web-based art object called IO, an interactive website mixing words, sounds, and images. Extending earlier analyses of the design process (Suchman and Trigg, 1993; Tang, 1988) and metapragmatic work on gestures and talk (for example, Goodwin, 2000; Hanks, 1990; Haviland, 2000, 2007; LeBaron and Streeck, 2000; Ochs, Jacoby, and Gonzales, 1994), the analysis in this chapter focuses on the semiotic remediation involved in the design production work of this group as they engaged in talk, drawing, listing, data entry, programming, and gesturing (especially over screens and paper).