digital convergence
learning objectives
After reading this chapter you should be able to
discuss the multimedia development process,
to indicate the need for information system support
in the cultural heritage domain,
to characterize the notion of digital dossier,
to provide solutions for navigating
complex information spaces,
and to discuss the data representation issues involved.
As you gather from reading this book, the field
of multimedia is widely divergent.
However, when you develop a multimedia application,
you will find that all topics treated so far will become
relevant.
There will be a need to mix multiple media formats.
You will have to find suitable codecs for your video.
You will be asked whether search is possible.
And, not the least important, you will have to balance
navigation and presentation.
This chapter is based on the work we, that is my students,
have been doing in the domain of cultural heritage.
In the first section, we will introduce the notion of
digital dossier
and outline our general approach.
We will then in section 2 look at some examples,
and describe how we deploy concept graphs as a universal
navigation tool for complex information spaces.
Finally, in section 3, we will explore the options
for presenting multimedia material
and discuss the design issues as well as the technical issues
that have arisen in the course of our work.
multimedia casus
The assignment in the multimedia casus is to develop a virtual environment for some cultural or governemental institute or company.
The practicum takes the form a stage, in which external supervision plays an important role.
In the multimedia casus, techniques learned in previous courses (see the afterthoughts) will be applied to create the application.
At the start of the course the actual assignment will be determined.
Examples of possible assignments are: the development of a virtual exposition hall for the Dutch Royal Museum of the Arts, a virtual city square, which gives information about both the present and the past, a virtual shop, with online buying facilities, or an online broker, which offers facilities for inspecting houses.
Given an information space, create a VR that resolves
the duality between information and presentation,
using
intelligent multimedia
technology.
The VR must offer access to all relevant
information entities, organized in a suitable spatial layout,
and must allow for presentations from a variety
of perspectives, making full use of graphical
and rich media facilities.
|
|
| front page of the INCCA website |
INCCA
In 1999, a group of eleven international modern art museums and related institutions applied to the European Commission (Raphael Programme) under the umbrella International Network for the Conservation of Contemporary Art (INCCA). The INCCA project was accepted and work started in January 2000 led by the organiser, the ICN (Netherlands Institute for Cultural Heritage) and the co-organiser, Tate, London.
objectives
INCCA's most important set of objectives, which are closely interlinked, focuses on the building of a website with underlying databases that will facilitate the exchange of professional knowledge and information. Furthermore, INCCA partners are involved in a collective effort to gather information directly from artists.
mission
INCCA's guiding mission is to collect, share and preserve knowledge needed for the conservation of modern and contemporary art.
checklist
- roles -- create a team
- project goal -- develop a vision
- production -- construct the assets
- quality assesment -- test and control
- delivery -- present and archive
- manage -- all along
- document -- track project's history
judgement
- group -- (2) effort, 5 (product), 3 (documentation)
- individual -- (4) responsibility, (3) productivity, (3) quality
deliverables
- group -- project plan, design, project report, product
- individual -- detailed weekly account of activities
schedule
- project organisation
- project definition
- planning and design
- construction and development
- integration and delivery
- presentation and archiving
|
|
|
| physical control cube | head-mounted display |
| application area | metaphor | familiar knowledge |
|---|
| operating environment | desktop | office tasks |
| spreadsheets | ledger sheet | columnar table |
| object-oriented environment | physical world | real world |
| hypertext | notecards | organization of text |
| learning environment | travel | tours, guides, movement |
| file storage | piles | categorizing |
| multimedia environments | rooms | spatial structures |
| cooperative work | multi-agents | travel agents, servants |
interaction styles
- command entry
- menus and navigation
- forms fills and spreadsheets
- natural language dialog
- direct manipulation
digital dossier
Create a VR that realizes a digital dossier
for a work of a particular artist.
A digital dossier represents the information
that is available for a particular work of art, or a collection
of works, of a particular artist.
The digital dossier should be multimedia-enhanced,
that is include photographs, audio and other multimedia material
in a compelling manner.
Webster New World Dictionary
- dossier (dos-si-er) [ Fr < dos (back); so named because labeled on the back ] a collection of documents concerning a particular person or matter
- archive -- 1) a place where public records are kept ... 2) the records, material itself ...
everything must be highly intertwinkled
|
|
| presentation of video clips from Marina Abramovic |
Reconstruction of Terra della Dea Madre in VRML.
style issues
- what icons should be used to identify the elements of the concept graph?
- what categories and relationships are most appropriate?
- how should the information be displayed, simultaneously or more focussed?
- how do we allow the user to choose between multiple information items?
- how do we avoid visually disturbing elements?
|
|
|
|
| no light | half light | full light |
user-centered design methods
field studies,
user requirement analysis,
iterative design,
usability evaluation,
task analysis,
focus groups,
formal/heuristic analysis,
user interviews,
prototype (without user testing),
surveys,
informal expert review,
card sorting,
participatory design
usability evaluation
- learnability -- time and effort to reach level of performance
- throughput -- the amount of work done
- flexibility -- accomodating changes in the task
- attitude -- of users to the system
task world ontology
- task -- activity performed by an agent to reach a certain goal
- goal -- a desired state in the task world or system
- role -- a meaningful collection of tasks
- object -- refers to a physical or non-physical entity
- agent -- an entity that is considered active
- event -- a change in the state of the task world
next generation dossier(s)
- adaptation of representation to Dublin Core (+ annotation needed for presentation)
- XML-based content management, with php forms (extending what we have now)
- there should also be a possibility to present the information and material in a 'plain' web format
- as well as in (a new version of) 3D dossiers
- we should think about the proper presentation parameters.
structures
- Video -- to display video fragment, including interviews
- Picture -- to present pictures of the artwork
- Artwork -- contains all information connected to a work of art
- TextItem -- to present text, from the interview or any other source
- MaterialItem -- to present information about material(s) used
- GroupNode -- to combine nodes in the concpet graph
- Information -- acts as the outer container for all nodes
concept graph
Information {
informationNodes [
GroupNode {
ID "MAIN"
shortName "Main"
longName "Main"
urlModel "models/conceptGraph/main/modelMain.wrl"
description [ "Central information node" ]
connectedNodesIDs [ "ARTWORKS", "KEYWORDS",
"INTERVIEWS", "REPORT" ]
}
GroupNode {
shortName "Artworks"
longName "Artworks"
description [ "Node that connects to all the artworks" ]
ID "ARTWORKS"
connectedNodesIDs [ "MAIN", "TRANSITORY",
"ULAY", "VIDEOINSTALLATION", "ABRAMOVIC" ]
urlModel "models/conceptGraph/artworks/artworksGroup.wrl"
}
## ...
]
}
Artwork {
shortName "Terra degla Dea Madre"
longName "Terra degla Dea Madre"
description ["15:40 min, colour, sound."]
ID "AV24"
connectedNodesIDs ["VIDEOINSTALLATION", "DTV24",
"TTV24", "PV24", "CV24", "VV24", "G0"]
urlPreviewImage "images/previewImages/AV24.jpg"
widthPreviewImage 479
heightPreviewImage 349
}
Video {
ID "CV24"
shortName "Interview clip Terra degla Dea Madre"
longName "Interview clip showing Terra degla Dea Madre"
url "interviewclips/interview_terra_degla.avi"
width 320
height 360
urlPreviewImage "images/previewImages/interview_terra_degla.jpg"
widthPreviewImage 320
heightPreviewImage 240
description [""]
connectedNodesIDs ["CLIP", "AV24"]
}
TextItem {
shortName "Instruction"
longName "Green Dragon Lying instructions for the public."
description ["Text explaining the way the public has to interact with the artwork."]
ID "ITO05"
connectedNodesIDs ["AO05", "INTERACTION"]
url "text/AO05_instruction.txt"
}
Dublin Core
- title -- name given to the resource
- creator -- entity primarily responsible for making the content of the resource
- subject -- topic of the content of the resource
- description -- an account of the content of the resource
- publisher -- entity responsible for making the resource available
- contributor -- entity responsible for making contributions to the content of the resource
- date -- date of an event in the lifecycle of the resource
- type -- nature or genre of the content of the resource
- format -- physical or digital manifestation of the resource
- identifier -- unambiguous reference to the resource within a given context
- source -- reference to a resource from which the present resource is derived
- language -- language of the intellectual content of the resource
- relation -- reference to a related resource
- coverage -- extent or scope of the content of the resource
- rights -- information about rights held in and over the resource
|
|
| location of Tower of Babel project |
I-GUARD
Contemporary art is an intrinsic part of our cultural heritage.
Installations, performances, video and other forms of media art,
as for example web art, have the interest of a small
group of adherents, but are in comparison with more traditional
art forms, far more difficult to present to a general audience.
Another problem presents itself, due to the type of materials
used and the context-specific aspects of these art forms,
in the conservation of the works.
In our research we address the issue of providing access
to these contemporary art forms from a wide variety
of perspectives, ranging from the interested layman
to the expert that has to deal with archiving,
conserving and the possible re-installation of the art works.
intelligent guidance
- filtering the information space according to the user's
perspective, and
- intelligent agents, that (pro) actively aid the user in searching
the information space.
digital dossier(s)
- representation of information of one or multiple works
of art,
- presentation of that information in a
rich media presentation environment,
- intelligent navigation and interaction, and
- support for interaction with loosely-structured
natural language.
|
|
| outside view of Tower of Babel project |
Inside view of Tower of Babel project.
(new) media art
- audio art -- no definition available
- computer art -- any art in which computers played a role in production or display of the artwork.
- digital art -- art created on a computer in digital (that is, binary) form.
- electronic art -- entry to game producer,
should be Leonardo.
- generative art -- art or design generated, composed, or constructed through computer software algorithms,
or similar mathematical or mechanical autonomous processes
- hacktivism -- the writing of code, or otherwise manipulating bits, to promote political ideology
- interactive art -- a piece of art that involves the spectator in some way.
- internet art -- art or, more precisely, cultural production which uses the Internet
as its primary medium and, more importantly, its subject.
- performance art -- art where the actions of an individual or a group at
a particular place and in a particular time, constitute the work.
- robotic art -- page does not exist
- software art -- is an intersection of two almost non-overlapping realms: software and art.
- video art -- is a subset of artistic works which relies on "moving pictures" and is
comprised of video and/or audio data.
- video game art -- involves the use of a computer game for the creation of a digital artwork.
frame(s) of reference
It is one of the most important formal qualities of film
that every object that is reproduced appears simultaneously in two
entirely different frames of reference, namely the two-dimensional
and the three-dimensional, and that as one identical object it fulfills
two different functions in the two contexts.
rules vs fiction
Game fiction is ambiguous, optional and imagined by
the player in uncontrollable and unpredictable ways, but the emphasis
on fictional worlds may be the strongest innovation of the video game.
concepts
technology
projects & further reading
As a project, develop a data format for text, images
and video in XML,
and implement stylesheets in XSLT
to convert the format for display,
for example in HTML frames or using SMIL.
You may further explore the formulation
of criteria for selecting software and tool support
for developing multimedia applications.
For further reading I suggest, apart from the manuals
and learning materials that come with your tools,
to study example projects and in particular the
workflow, that is the dependencies between stages in
the production, as for example explained in [Effects].
- website of Montevideo Collection Catalogue.
To avoid being parochial here, I should also mention similar
institutes abroad, such as Electronic Arts Intermix
from New York, USA,
and LUX, from London, UK.
- website of INCCA.
- tangible virtual museum -- from [Tangible], see section 10.1.
- digital dossier -- concept graph for abramovic dossier,
see section 10.2.
- digital dossier -- presentation gadget in abramovic dossier, with video of
Relation in Time, with Ulay.
- digital dossier -- installation Terra dea degli madre,
as 3D model.
- conservator's studio --
Self-Portrait with Braid,
see section 10.2
- diagram -- task world ontology, [Euterpe].
- tower of babel -- location where the event took place, see below.
- tower of babel -- projection of tower of babel project,
see section 10.3, submitted by Katelijne Arts.
The project is a concept of
Katelijne Arts, Tineke Goemans, Franka van de Goor, Leidi Haaijer en Bert Vogels.
- tower of babel -- a view from the inside of the building.
- PANORAMA architecture -- from [PanoramaWeb].
- signs -- sports, [Signs], p. 278, 279.
