introduction multimedia
[] readme preface 1 2 3 4 5 6 7 appendix checklist powerpoint resources director

talk show tell print

multimedia authoring

It is tempting to identify a presentation with the information space it presents. This is what users often do, and perhaps should do. When that happens, the presentation is effective. But you must remember that the actual presentation is just one of the many possible ways to engage a user in exploring an information space. Making the choice of what to present to the user is what we understand by (multimedia) authoring.

Authoring is what we will discuss in this section. Not by giving detailed guidelines on how to produce a presentation (although you may look at the online assignment for some hints in this respect), but rather by collecting wisdom from a variety of sources.

visualization

Let's start with our explorations by looking at the problem of visualisation with a quote from David Gelernter, taken from  [User]:

visualization


Grasping the whole is a gigantic theme, intellectual history's most important.

Ant vision is humanity's usual fate; but seeing the whole is every thinking person's aspiration.

David Gelernter, Mirror Worlds 1992


Now, consider, there are many ways in which the underlying information space may be structured, or speaking as a computer scientist, what data types may be used to represent the (abstract) information.

data types


The visualisation problem then is to find a suitable way to present these structures to the user. Basicall, following  [User], there are two paradigms to present this information:

Storytelling may be very compelling, and does not force the user to interact. On the other hand, storytelling may lead to information consumerism alike to television enslavement.

An interaction paradigm that combines 'storytelling' with opportunities for interaction, as for example in the blendo approach discussed in section 3.2, would seem to be most favorable. Interaction then may result in either changing the direction of the story, or in the display of additional information or even transactions with a third party (for example to buy some goodies).

multimediocrity

Multimedia is a promising technology, and (nowadays) affordable. So we see that multimedia (which includes 3D-graphics, video and sound) is increasingly being used, also in information visualisation. But what is it good for? To quote  [Magic]:

multimedia's promise is terribly generalized,

it simply lets you do anything.

As with any new technology, the early multimedia productions (in particular CDROM and CD-I) were not optimal with respect to (aesthetic) quality. To quote  [Magic], again:

shovelware -- multimediocrity


... far from making a killing, it looked as if the big boys ... had killed the industry by glutting the market with inferior products.

Perhaps the industry in the late eighties did not have the right business model. But, then again, what are the chances of multimedia in our time. One more quote from  [Magic]:

if multimedia is comparable to print then yes, we'd be crazy to expect it to mature in a mere ten years.

eliminating complexity

So now, in the new millenium, we are (sadder and wiser) in a position to approach the effective deployment of mutimedia afresh. What we look for is aesthetic quality. How do we find it? Easy enough, just be authentic.

"Learning how to not fool ourselves is, I'm sorry to say, something that we haven't specifically included in any particular course that I know of. We just hope you've caught it by osmosis."

Richard Feynman


Authentic in creating mutimedia means, apart from not fooling yourselves, that you must become aware of the message orinformation you want to convey and learn to master the technology to a sufficient degree. But beware, an effective multimedia presentation is not the same as scientific argumentation:

the media equation


We regularly exploit the media equation for enjoyment by the willing suspension of our critical faculties. Theatre is the projection of a story through the window of a stage, and typically the audience gets immersed in the story as if it was real.

These quotes, as well as the following one have been taken from an online essay on eliminating complexity which provides an argument against inessential gadgets and spurious complexity and bells and whistles in whatever you can think of, including user interfaces and scientific theories. Back to the subject, what does master the technology to a sufficient degree mean? Just remember that what you do is a form of engineering.

engineering


"engineering is the art of moulding materials we do not wholly understand ... in such a way that the community at large has no reason to suspect the extent of our ignorance."

A. R. Dykes.


In other words, learn the tool(s) that you are using to a degree that you master the basics and easily cut through its apparent magic.

theories of creativity

 Producing multimedia, in whatever form, has an element of craftsmanship. But, given the need for aesthetic quality, whatever way you approach it, there is an element of creativity. That means, you're in for a challenge. And, to quote  [Magic],

The best thing is to empower yourself. But before you can do that, you need to understand what you are doing -- which is a surprisingly novel thing to do.

Now it is tempting to look for a set of guidelines and rules that give you a key to creativity. So let me be straight with you:

there is no theory of creativity

On the other hand, there are techniques for producing ideas. And some recommend a sequence of steps, such as:

steps

browse, explore; chew it over; incubation, let it rest; illumination (YES); verification,does it work?

And in addition, still following  [Magic], there are some general rules:

general rules

  • if you aim to please everybody, you will please nobody
  • constraints come with the territory, you must learn to love them
  • emotional charge is the key to success
Now, if you'd ask me, I would say, just make your virtual hands dirty. But read on, there is more

persuasive technology

Whatever your target audience, whatever your medium, whatever your message, you have to be convincing if not compelling.

In the tradition of rethorics, which is the ancient craft of convincing others, a new line of research has arisen under the name of persuasive technology. In the words of my collegue, Claire Dormann, persuasion is:

persuasion

  • a communication process in which the communicator seeks to elicit a desired response from his receiver
  • a conscious attempt by one individual to change the attitudes, beliefs or behaviours of another individual or group individual through the transmission of some messages.
In other words,

The purpose of persuasion is to accomplish one of the following goals: to induce the audience to take some action, to educate the audience (persuade them to accept to accept information or data), or to provide the audience with an experience.

In the area of multimedia, one may think of many applications. Quoting Claire again

In interactive media, the field of application of persuasive technology ranges from E-commerce, social marketing (like an anti-AIDS campaign) to museum exhibits. Also E-commerce provides an obvious example. To convince people to buy more, more persuasive messages and technologies are developed through the use of humorous and emotional communication, agents (such as price finders) or 3D representations of products and shops. For health campaigns (or any campain of your choice) one can imagine 3D information spaces with agents presenting different point of views and where users are given different roles to play. In a museum you might want to highlight key points through innovative and fun interactive exhibits.

Although the subject of persuasive technology is far less technology-oriented than the name suggests, multimedia (in a broad sense) form an excellent platform to explore persuasion. You may want to look at the site given below

www.captology.org -- Computers As Persuasive Technology

As concerns multimedia authoring, set yourself a goal, do the assignment, explore your capabilities, convey that message, and make the best of it.

(re)mediation

What can you hope to achieve when working with the new media? Think about it. Are the new media really new? Does anyone want to produce something that nobody has ever seen or heard before? Probably not. But it takes some philosophy to get that sufficiently clear.

In  [Remediation], the new media are analyzed from the perspective of remediation, that is the mutual influence of media on eachother in a historical perspective. In any medium, according to  [Remediation], there are two forces at work:

(re)mediation -- forces at work

  • immediacy -- a tendency towards transparent immersion, and
  • hypermediacy -- the presence of referential context
Put in other words, immediacy occurs when the medium itself is forgotten, so to speak, as is (ideally) the case in realistic painting, dramatic movies, and (perhaps in its most extreme form) in virtual reality. Hypermediacy may be observed when either the medium itself becomes the subject of our attention as in some genres of modern painting, experimental literature and film making, or when there is an explicit reference to other related sources of information or areas of experience, as in conceptual art, many web sites, and also in CNN news, where apart from live reports of ongoing action, running banners with a variety of information keep the viewers up to date of other news facts.

Now, the notion of remediation comes into play when we observe that every medium draws on the history of other media, or even its own history, to achieve a proper level of immediacy, or 'natural immersion'. For example, Hollywoord movies are only realistic to the extent that we understand the dramatic intent of cuts, close-ups and storylines, as they have been developed by the industry during the development of the medium. As another example, the realism of virtual reality can only be understood when we appreciate linear perspective (which arose out of realistic Renaissance painting) and dynamic scenes from a first person perspective (for which we have been prepared by action movies and TV).

Even if you may argue about the examples, let it be clear that each (new) medium refers, at least implicitly, to another medium, or to itself in a previous historic phase. So, what does this mean for new media, like TV or virtual reality?

Let's start with virtual reality.

This is not like TV, only better -- says Lenny Nero in Strange Days

 [Remediation] comment on a statement of Arthur C. Clarke

Virtual Reality won't merely replace TV. It will eat it alive.

by saying that

... he is right in the sense that virtual reality remediates television (and film) by the strategy of incorporation. This strategy does not mean that virtual reality can obliterate the earlier visual point-of-view technologies, rather it ensures that these technologies remain as least as reference points by which the immediacy of virtual reality is measured.

So, they observe "paradoxically, then, remediation is as important for the logic of transparency as it is for hypermediacy". Following  [Remediation], we can characterize the notions of immediacy and hypermediacy somewhat more precisely.

immediacy

  • epistemological: transparency, the absence of mediation
  • psychological: the medium has disappeared, presence, immersion

hypermediacy

  • epistemological: opacity, presence of the medium and mediation
  • psychological: experience of the medium is an experience of the real
Now, sharpen your philosophical teeth at the following statement:

p. 224: Convergence is the mutual remediation of at least three important technologies -- telephone, televison and computer -- each of which is a hybrid of technical, social and economic practice, and each of which offers its own path to immediacy.

The telephone offers the immediacy of voice or the interchange of voices in real-time.

Television is a point-of-view technology that promises immediacy through its insistent real-time monitoring of the world.

The computer's promise of immediacy comes through the combination of three-dimensional graphics, automatic (programmed) action, and an interactivity that television can not match.

As they come together, each of these is trying to absorb the others and promote its own version of immediacy.

Let us once more come back to virtual reality and its possible relevance in our information age:

p. 225: In the claim that new media should not be merely archival but immersive, the rhetoric of virtual reality finally enters in, with its promise of the immediacy of experience through transparency.

So, with respect to the new media, we may indeed conclude:

new media


... what is in fact new is the particular way in which each innovation rearranges and reconstitutes the meaning of earlier elements.

What is new about media is therefore also old and familiar: that they promise the new by remediating what has gone before.

The true novelty would be a new medium that did not refer to the other media at all.

For our culture, such mediation without remediation seems to be impossible.

research directions -- narrative structure

 Where do we go from here? What is the multimedia computer, if not a new medium? To close this section on multimedia authoring, let us reconsider in what way the networked multimedia computer differs from other media, by taking up the theme of convergence again. The networked multimedia computer seems to remediate all other media. Or, in the words of  [Hamlet]:

convergence

(p. 27) ... merging previously disparate technologies of communication and representation into a single medium.

The networked computer acts like a telephone in offering one-to-one real-time communication, like a television in broadcasting moving pictures, like an auditorium in bringing groups together for lectures and discussion, like a library in offering vast amounts of textual information for reference, like a museum in its ordered presentation of visual information, like a billboard, a radio, a gameboard and even like a manuscript in its revival of scrolling text.

In  [Hamlet], an analysis is given of a great variety of computer entertainment applications, varying from shoot-em-up games to collaborative interactive role playing.  [Hamlet] identifies four essential properties that make these applications stand out against the entertainment offered by other media, which include books and TV. Two key properties determine the interactive nature of computer entertainment applications:

interactive


  • procedural -- 'programmed media' ...
  • participatory -- offering agency
All applications examined in  [Hamlet] may be regarded as 'programmed media', for which interactivity is determined by 'procedural rules'. With agency is meant that the user can make active choices and thus influence the course of affairs, or at least determine the sequence in which the material is experienced.

Another common characteristic of the applications examined is what  [Hamlet] calls immersiveness. Immersiveness is determined by two other key properties:

immersive


  • spatial -- explorable in (state) space
  • encyclopedic -- with (partial) information closure
All applications are based on some spatial metaphor. Actually, many games operate in 'levels' that can be accessedonly after demonstrating a certain degree of mastery. Networked computer applications allow for incorporating an almost unlimited amount of information. Some of the information might be open-ended, with storylines that remain unfinished. Closure, then, is achieved simply by exhaustive exploration or diminishing attention.

multimedia authoring

Coming back to the question what the 'new medium', that is the networked multimedia computer, has to offer from the perspective of multimedia authoring, two aspects come to the foreground:

multimedia authoring


  • narrative format
  • procedural authorship
The narrative format is incredibly rich, offering all possibilities of the multimedia computer, including 3D graphics, real-time sound, text. In short, everything up to virtual reality. But perhaps the most distinguishing feature of the new medium is that true authorship requires both artistic capabilities as well as an awareness of the computational power of the medium. That is to say, authorship also means to formulate generic computational rules for telling a story while allowing for interactive interventions by the user. Or, as phrased in  [Hamlet], the new cyberbard must create prototypical stories and formulaic characters that, in some way, lead their own life and tell their stories following their innate (read: programmed) rules. In section 7-3 and appendix platform, we will present a framework that may be used as a testbed for developing programmed narrative structures with embodied agents as the main characters.
[] readme preface 1 2 3 4 5 6 7 appendix checklist powerpoint resources director
eliens@cs.vu.nl

draft version 1 (16/5/2003)