Subject: dialogs -- material Date: Tue, 01 Oct 2002 13:59:35 +0200 From: "A. Eliens" To: claire@cs.vu.nl, eliens@cs.vu.nl http://www.cs.vu.nl/~eliens/demo/levels.html http://www.cs.vu.nl/~eliens/demo/@online/media/course/research-2-3.html directions -- narrative structure Where do we go from here? What is the multimedia computer, if not a new medium? To close this section on multimedia authoring, let us reconsider in what way the networked multimedia computer differs from other media, by taking up the theme of convergence again. The networked multimedia computer seems to remediate all other media. Or, in the words of [Hamlet]: convergence (p. 27) ... merging previously disparate technologies of communication and representation into a single medium. The networked computer acts like a telephone in offering one-to-one real-time communication, like a television in broadcasting moving pictures, like an auditorium in bringing groups together for lectures and discussion, like a library in offering vast amounts of textual information for reference, like a museum in its ordered presentation of visual information, like a billboard, a radio, a gameboard and even like a manuscript in its revival of scrolling text. In [Hamlet], an analysis is given of a great variety of computer entertainment applications, varying from shoot-em-up games to collaborative interactive role playing. [Hamlet] identifies four essential properties that make these applications stand out against the entertainment offered by other media, which include books and TV. Two key properties determine the interactive nature of computer entertainment applications: interactive procedural -- 'programmed media' ... participatory -- offering agency All applications examined in [Hamlet] may be regarded as 'programmed media', for which interactivity is determined by 'procedural rules'. With agency is meant that the user can make active choices and thus influence the course of affairs, or at least determine the sequence in which the material is experienced. Another common characteristic of the applications examined is what [Hamlet] calls immersiveness. Immersiveness is determined by two other key properties: immersive spatial -- explorable in (state) space encyclopedic -- with (partial) information closure All applications are based on some spatial metaphor. Actually, many games operate in 'levels' that can be accessedonly after demonstrating a certain degree of mastery. Networked computer applications allow for incorporating an almost unlimited amount of information. Some of the information might be open-ended, with storylines that remain unfinished. Closure, then, is achieved simply by exhaustive exploration or diminishing attention. multimedia authoring Coming back to the question what the 'new medium', that is the networked multimedia computer, has to offer from the perspective of multimedia authoring, two aspects come to the foreground: multimedia authoring narrative format procedural authorship The narrative format is incredibly rich, offering all possibilities of the multimedia computer, including 3Dgraphics, real-time sound, text. In short, everything up to virtual reality. But perhaps the most distinguishing feature of the new medium is that true authorship requires both artistic capabilities as well as an awareness of the computational power of the medium. That is to say, authorship also means to formulate generic computational rules for telling a story while allowing for interactive interventions by the user. Or, as phrased in [Hamlet], the new cyberbard must create prototypical stories and formulaic characters that, in some way, lead their own life and tell their stories following their innate (read: programmed) rules. In section 7-3, we will present a framework that may be used as a testbed for developing programmed narrative structures with embodied agents as the main characters. http://www.cs.vu.nl/~eliens/demo/@online/media/course/research-3-3.html navigating information spaces Having agents everywhere might change our perspective on computing. But, it may also become quite annoying to be bothered by an agent each time that you try to interact with with your computer (you know what I mean!). However, as reported by Kristina Höök, even annoyance can be instrumental in keeping your attention to a particular task. In one of her projects, the PERSONAS project, which stands for PERsonal and SOcial NAvigation through information spaceS the use of agents commenting on people navigating information space(s) is explored. As a note, the plural form of spaces is mine, to do justice to the plurality of information spaces. As explained on the PERSONAS web site, which is listed with the acronyms, the PERSONAS project aims at: PERSONAS investigating a new approach to navigation through information spaces, based on a personalised and social navigational paradigm. The novel idea pursued in this project is to have agents (Agneta and Frieda) that are not helpful, but instead just give comments, sometimes with humor, but sometimes ironic or even sarcastic comments on the user's activities, in particular navigating an information space or (plain) web browsing. As can be read on the PERSONAS web site: Agneta & Frieda The AGNETA & FRIDA system seeks to integrate web-browsing and narrative into a joint mode. Below the browser window (on the desktop) are placed two female characters, sitting in their livingroom chairs, watching the browser during the session (more or less like watching television). Agneta and Frida (mother and daughter) physically react, comment, make ironic remarks about and develop stories around the information presented in the browser (primarily to each other), but are also sensitive to what the navigator is doing and possible malfunctions of the browser or server. In one of her talks, Kristina Höök observed that some users get really fed up with the comments delivered by Agneta and Frieda. So, as a compromise, the level of interference can be adjusted by the user, dependent on the task at hand. Agneta & Frieda In this way they seek to attach emotional, comical or anecdotal connotations to the information and happenings in the browsing session. Through an activity slider, the navigator can decide on how active she wants the characters to be, depending on the purpose of the browsing session (serious information seeking, wayfinding, exploration or entertainment browsing). As you may gather, looking at the presentations accompanying this introduction to multimedia and [Dialogs], I found the PERSONAS approach rather intriguing. Actually, the PERSONAS approach is related to the area of affective computing, see [Affective], which is an altogether different story. The Agneta and Frieda software is available for download at the PERSONAS web site. http://www.cs.vu.nl/~eliens/demo/@online/media/course/quote-history.html some phenomena in the media are older than is generally recognized Again, some of the conventions of the 20th century comic books draw directly or indirectly on an even longer visual tradition: speech balloons can be found in the eighteenth century prints, which are in turn an adaptation of the 'text scrolls' coming from the mouths of the Virgin and other figures in medieval religious art. Anon, The Vision of St. Bernard, Book of Hours c. 1470 p 40 (1500) Another development of this period was the narrative strip or picture story. The ancestor of the 20th century comic strip. The visual narrative in which the viewer 'reads' the episodes, visually from left to right and from top to bottom, was already known in the middle ages but its importance increased with the rise of woodcuts in the Renaissance. Finally: http://www.cs.vu.nl/~eliens/demo/paper-platform.html Dialogs in Virtual Environments Desktop VR is an excellent medium for presenting information, for example in class, in particular when rich media or 3D content is involved. At VU, we have been using presentational VR for quite some time, and recently we included dialogs using balloons (and possibly avatars) to display the text commenting on a particular presentation, [Eliëns et al. (2002)]. See figure (b) for an example displaying a virtual environment of the VU, a propaganda movie for attracting students, and two avatars commenting on the scene. The avatars and their text are programmed as annotations to a particular scene as described below. Each presentation is organized as a sequence of slides, and dependent on the slides (or level within the slide) a dialog may be selected and displayed. See the observer fragment in section 3. Our annotation for dialog text in slides looks as follows: dialog ... In figure (b), you see the left avatar (named cutie) step forward and deliver her phrase. This dialog continues until cutie remarks that she always wanted to be an agent. The dialog is a somewhat ironic comment on the contents of the movie displayed, which is meant to introduce the VU to potential students. Clearly, our approach is reminiscent to the notorious Agneta & Frida characters developed in the Persona project. See http://www.sics.se/humle/projects/persona/web Furthermore, there are a number of style parameters to be dealt with to decide for example whether the avatars or persona are visible, where to place the dialogs balloons on the display, as well as the color and transparancy of the balloons. To this end, we have included a style attribute in the phrase tag, to allow for setting any of the style parameters. style parameters Apart from phrases, we also allow for gestures, taken from the built-in repertoire of the avatars. In section 4.3 we discuss how to extend the repertoire of gestures, using a gesture specification language. Both phrases and gestures are compiled into DLP code and loaded when the annotated version of the presentation VR is started. design issues levels of complexity level 0: basic material -- dialog text, media object level 1: combined -- media object + dialog level 2: with avatars -- media object + dialog + avatars level 3: with attributes -- media object + dialog + avatars + objects level 4: with context -- media object + dialog + avatars + context