We live in the digital era, [Negroponte (1995)].
We are surrounding ourselves with gadgets and we are consuming immense amounts of information, that is increasingly being delivered to us via the Internet. We play games, and we still watch (too much) television.
Some of us watch televion on our PCs, and may be even looking forward to watch television on their mobile phone.
This is multimedia.
For others, the PC is still a programmable machine. Being able to program it might earn you a living. Understanding multimedia, however, might even provide you with a better living.
In this chapter, we study what trends may currently be observed in the
creation and delivery of multimedia information, and we explore what impact the digital revolution may have from a commercial perspective.
convergence
- content -- 2D/3D graphics, data, video, audio
- distribution -- broadcast, wireless, DVD, internet, satelite, cable
- platform -- PC, television, game machine, wireless data pad, mobile phone
the players
Alltel, AT&T Wireless, AtomShockwave, Cingular Wireless, Clear Channel, HitHive, Ifilm, Infinity, KDDI, Liquid Audio, LMIV, Mannesmann, MP3.com, MTV, NetCom, Myplay, Nortel Networks, NTT DoCoMo, Omnitel, Sprint, Telefonica, Telstra, Vitaminic, Verizon Wireless, Virgin Megastores, Vodafone, Voicestream.
information society
the new term 'information society' gave form to a cluster of hitherto more loosely related aspects of
communication -- knowledge, news, literature, entertainment, all exchanged through different media and different
media materials -- paper, ink, canvas, paint, celluloid, cinema, radio, television and computers.
From the 1960s onwards, all messages, public and private, verbal and visual, began to be considered as 'data',
information that could be transmitted, collected, recorded, whatever their point of origin, most effective through
electronic technology.
projects & further reading
As a project, consider
the development of a Java-based mobile game
using J2ME, see [Morrison (2005)],
or a web-based game using Visual Basic .NET,
see [Santos Lobao and Hatton (2003)].
You may further explore
multiplatform game development,
and find arguments to choose for either Java-based or managed code
based implementations.
For further reading, I advice to have a look
at the special issues of the Scientific American, [American],
and the CACM on the next 1000 years of computing, [CACM (2001)],
and, for getting an idea where this all leads to,
Schneidermann's Leonardo's laptop, [Shneiderman (2003)].
For Second Life, see [Rymaszweski et al. (2007)].
hypermedia information spaces
learning objectives
After reading this chapter you should be able to
define information spaces in a precise manner,
position the hypertextual capabilities of the
web in a historical perspective,
explain the difference between multimedia and hypermedia,
and argue why computational support for
narrative structure in multimedia applications is desirable.
However entertaining it might be presented to you, underlying every multimedia presentation there is an information space. That is to say, irrespective of the medium, there is a message. And being confronted with a message, we might want to inquire for more information.
In this chapter, we will define the notion of information space more precisely.
We will extend this definition to include information hyperspaces, by looking at the history of hypertext and hypermedia.
Finally, we will discuss visualisation as a means to present (abstract) information in a more intuitive way, and we will reflect on what is involved in creating compelling multimedia.
Current day multimedia information systems
distinguish themselves from older day
information systems not only by what information
they contain, that includes multimedia objects such
as images and sounds, but also by a much more extensive repertoire
of query mechanisms,
visual interfaces and rich presentation facilities.
See [Chang and Costabile (1997)].
S.K. Chang and M.F. Costabile -- Visual Interfaces to Multimedia Databases
The Handbook of Multimedia Information Management
multimedia information systems
- storage technology -- multimedia databases
- wideband communication -- distribution accross networks
- parallel computing -- voice, image and video processing
- graphic co-processors -- visual information with high image quality
multimedia applications
geographical information systems,
office automation,
distance learning,
health care,
computer aided design,
scientific visualization, and
information visualization.
multimedia databases
- the size of data,
- synchronization issues,
- query mechanisms, and
- real time processing.
an information space is a representation of the information
stored in a system or database that is used to present
that information to a user.
we must distinguish between a visual information space
(for presentation),
a logical information space (in which we can reason
about abstract information objects) and
a physical information space
(where our concrete multimedia objects are stored).
- physical information space -- images, animations, video, voice, ...
- logical information space -- abstract database objects
- presentational information space -- to present information to the user
a logical information space is a multidimensional
space where each point represents an object from the
physical information space (read database).
- information object -- a point in the (logical) information space
- query -- an arbitrary region in this information space
- clue -- a region with directional information, to facilitate browsing
XML in 10 points
- XML is for structuring data
- XML looks a bit like HTML
- XML is text, but isn't meant to be read
- XML is verbose by design
- XML is a family of technologies
- XML is new, but not that new
- XML leads HTML to XHTML
- XML is the basis for RDF and the Semantic Web
- XML is license-free, platform-independent and well-supported
related technologies
- Xlink -- hyperlinks
- XPointer -- anchors and fragments
- XSL -- advanced stylesheets
- XSLT -- transformation language
- DOM -- object model for application programmer interface
- schemas -- to specify the structure of XML documents
XML
- separate data from presentation
- transmit data between applications
information hyperspace
the logical information space may further be structured in a logical information hyperspace, where the clues become hyperlinks that provide directional information, and the information space can be navigated by the user following directional clues.
history
- 1945 -- Vannevar Bush (Memex) -- as we may think, [ Bush (1995)]
- 1963 -- Douglas Engelbart (Augment) -- boosting the human intellect [Engelbart (1963)]
- 1980 -- Ted Nelson (Xanadu) -- everything is intertwinkled, [Nelson (1980)]
- flash 1: we are in trouble (human mankind)
- flash 2: we need to boost mankind's ability to deal with complex urgent problems
- flash 3: aha, graphic vision surges forth of me ...
- flash 4: hypermedia -- to augment the human intellect
- flash 5: augment (multimedia) workstation -- portal into an information space
hypermedia systems
- components -- text, graphics, audio, video
- links -- relations between components
- presentation -- structured display
A curriculum promotes a false simplification of any subject,
cutting the subject's many interconnections and leaving
a skeleton of sequence which is only a charicature
of its richness and intrinsic fascination.
classification of hypermedia systems
- macro-literary systems -- publishing, reading, criticism
- problem exploration tools -- authoring, outlining, programming
- browsing systems -- teaching, references, information
- general hypermedia technology -- authoring, browsing, collaboration
- embedded hypermedia -- CASE, decision support, catalogs
component
- content -- text, graphics, video, program
- attributes -- semantic description
- anchors -- (bi-directional) links to other documents
- presentation -- display characteristics
(CMIF) multimedia model
- data block -- atomic component
- channel -- abstract output device
- synchronization arc -- specifying timing constraints
- event -- actual presentation
Amsterdam Hypermedia Model
- contents -- data block
- attributes -- semantic information
- anchors -- (id, value)
- presentation -- channel, duration, ...
research issues
- search and query -- for better access
- composition -- for imposing structure
- virtual structures -- on top of existing structures
- computation -- for flexibility and interaction
- versioning -- to store modification histories
- collaborative work -- sharing objects with multiple users
- extensibility and tailorability -- to adapt to individual preferences
visualization
Grasping the whole is a gigantic theme,
intellectual history's most important.
Ant vision is humanity's usual fate;
but seeing the whole is every thinking person's aspiration.
David Gelernter, Mirror Worlds 1992
data types
- 1-D linear data -- text, source code, word index
- 2-D map data -- floor plan, office layout
- 3-D world -- molecules, schematics, ...
- temporal data -- 1 D (start, finish)
- multi-dimensional data -- n-dimensional (information) space
- tree data -- hierarchical
- network data -- graph structure
- interactive -- overview first, zoom and filter, then details on demand
- storytelling -- as a paradigm for information presentation
Whatever your target audience, whatever your medium, whatever your message, you have to be convincing if not compelling.
persuasion
- a communication process in which the communicator seeks to elicit a desired response from his receiver
- a conscious attempt by one individual to change the attitudes, beliefs or behaviours of another individual or group individual through the transmission of some messages.
(re)mediation
- immediacy -- a tendency towards transparent immersion, and
- hypermediacy -- the presence of referential context
Virtual Reality won't merely replace TV. It will eat it alive.
immediacy
- epistemological: transparency, the absence of mediation
- psychological: the medium has disappeared, presence, immersion
hypermediacy
- epistemological: opacity, presence of the medium and mediation
- psychological: experience of the medium is an experience of the real
Convergence is the mutual remediation of at least
three important technologies -- telephone, televison
and computer -- each of which is a hybrid of technical,
social and economic practice,
and each of which offers its own path to immediacy.
The telephone offers the immediacy of voice or the interchange of voices in real-time.
Television is a point-of-view technology that promises immediacy through its insistent real-time monitoring of the world.
The computer's promise of immediacy comes through the combination of three-dimensional graphics, automatic (programmed) action, and an interactivity that television can not match.
As they come together, each of these is trying to absorb the others and promote its own version of immediacy.
convergence
(p. 27) ... merging previously disparate technologies of communication and representation into a single medium.
The networked computer acts like a telephone in offering one-to-one real-time communication, like a television in broadcasting moving pictures, like an auditorium in bringing groups together for lectures and discussion, like a library in offering vast amounts of textual information for reference, like a museum in its ordered presentation of visual information, like a billboard, a radio, a gameboard and even like a manuscript in its revival of scrolling text.
interactive
- procedural -- 'programmed media' ...
- participatory -- offering agency
immersive
- spatial -- explorable in (state) space
- encyclopedic -- with (partial) information closure
multimedia authoring
- narrative format
- procedural authorship
web 2.0
video sharing /
online gaming /
social networking
daft punk -- technologic (cn / jp)
Buy it, use it, break it, fix it.
Trash it, change it, melt -- upgrade it.
Change it, point it, zoom it, press it.
Snap it, work it, quick -- erase it.
Write it, out it, paste it, save it.
Load it, check it, quick -- rewrite it.
Plug it, play it, burn it, rip it.
Drag and drop it, zip -- unzip it.
Look it, fill it, curl it, find it.
View it, coat it, jam -- unlock it.
Surf it, scroll it, pose it, click it.
Cross it, crack it, twitch -- update it.
Name it, rate it, tune it, print it.
Scan it, send it, fax -- rename it.
Touch it, bring it. Pay it, watch it.
Turn it, leave it, stop -- format it.
mashup(s)
- substituting a single pragmatism for ideal design
- using light weight programming models
web 2.0 design pattern(s)
- web 1.0 -- the web as platform
- web 2.0 -- architecture of participation
- web 3.0 -- data is the (intel) inside
Learnlog:
XML Is The Fabric Of Web 2.0 Applications
- the client side is AJAX (Asynchronous Javascript and XML)
- the server application typically exposes data through XML
- the interaction model is web services
- mashups combine multiple webservices to create new types of applications
service(s)
REST
- representation -- encoding in a particular format
- state -- data encapsulated in an object
- transfer -- using HTTP methods
the most powerful mashups don't just mix code and data, they mix cultures.
which provides a challenge that trancends all issues of mere technological correctness.
built-in(s)
resource(s)
- /seen?user=SomeAvatar -- records the presence of SomeAvatar
- /touched?user=SomeAvatar -- invokes flickr API with users tag
- /set_tag?user=SomeAvatar&tag=FavoriteTag -- records SomeAvatar's favourite tag
flash/quicktime in SL
Quicktime supports Flash, but only up to Flash version 5. We're up to version 9 on that now!
Luckily, I have been dabbling with Flash since the early days,
so already knew how to do this 'the old way'... So,
Flash is doing all the work. No LSL at all... I heart feeds.
Did I say 'I heart feeds?
Referring to section 7.4 for a more detailed discussion, we may observe
that there is no meaning in merely putting things together.
Without mechanisms of personalization and recommendation we would simply be flooded
by data and information, in a way that even search would not be able to cope with.
Context, narratives and personalized presentation(s), notions from the past,
reappear as keywords for the future of the web 2.0 and beyond.
concepts
technology
projects & further reading
As a project, I suggest the development of a virtual tour
in a city, museum or other interesting locatoion.
You may further explore
the implementation of traversal within a context,
taking into account the history of navigation
when backtracking to a particular point,
issues in
hyperlinking and interaction in multimedia applications, and
computational support for narratives.
For further reading I advice you to take a look
at the history of hypermedia and the web,
using online material from the
W3C, or the history of
media as accounted for in [Briggs and Burke (2001)] and [Bolter and Grusin (2000)].
- book covers -- [Weishar (1998)], [Eco (1994)], [Burger (1981)], [Kunst], [Betsky (2004)]
- Federico Campanale -- Oxygen, fragments from video installation, 2004
- Vasarely -- [Diehl 1973].
- Vasarely -- [Diehl 1973].
- Vasarely -- [Diehl 1973].
- Federico Campanale -- Oxygen, more fragments.
- student work -- from introduction multimedia 2000.
- Rutger van Dijk -- mobius, interactive story, opening screen,
see section 2.3.
- edgecodes -- screenshots, see section 2.3
- signs -- people, [ van Rooijen (2003)], p. 244, 245.
