I-GUARD -- Intelligent Guidance in Artist's Digital Dossiers
www.cs.vu.nl/~eliens/research/i-guard.html
PDF
1) I-GUARD
1a) Intelligent Guidance in Artist's Digital Dossiers
1c) Dr. A. Eliëns
Address: Vriej Universiteit, Faculty of Sciences/Informatica,
De Boelelaan 1081, 1081 HV Amsterdam, email: eliens@cs.vu.nl
2) Summary:
2a) Summary
Contemporary art is an intrinsic part of our cultural heritage.
Installations, performances, video and other forms of media art,
have the interest of a small
group of adherents, but are in comparison with more traditional
art forms, far more difficult to present to a general audience.
Another problem presents itself, due to the type of materials
used and the context-specific aspects of these art forms,
in the conservation of the works.
We focus
on intelligent navigation and interaction
with the the digital dossier. that contains all (multimedia) information
necessary for an adequate understanding of one or more works of art.
An essential part of our research is to support guided tours,
that may take the form of a walk-through in virtual space, an automated story-line
in a concept map, or allow the (professional) to experiment with the (de-)construction of
for example an art installation, or to experiment with exhibition parameters, such as light and space.
An important research aspect here is to provide for the possibility to adapt the guided tour to the
personal information need of the user.
The results of our research will on the one hand contribute to making
contemporary art forms accessible to a larger audience
and on the other hand are explicitly meant
to support the complex task of the conservation
and re-installation of works of art in an effective manner.
These results are also of interest for presenting more traditional forms
of cultural heritage,
in particular with respect to the development of personalised guided tours.
2b) Abstract for layman (in Dutch)
Installaties, performances, video en ander vormen van media kunst,
hebben de interesse van
een kleine groep liefhebbers, maar zijn in vergelijk met meer
traditionele kunstvormen, zoals de schilderkunst, moeilijker
toegankelijk te maken voor een breed publiek.
Ook doen zich, door ondermeer materiaal gebruik en de
situatie-gebonden aspecten van deze kunstvormen, problemen
voor bij de conservering ervan.
In ons onderzoek richten we ons op intelligente
navigatie en interactie met
het digitale dossier, dat de (multimedia) informatie bevat
om een of meerdere kunstwerken adequaat te begrijpen.
Essentieel is de ondersteuning van guided tours,
die de vorm kunnen aannemen van een 'walk through' in een virtuele ruimte,
een 'story-line' in een concept map, danwel de (professionele) gebruiker
in staat stellen tot interactie met het kunstwerk, zoals bijvoorbeeld
de (de-)constructie of het experimenteren met exhibitie-parameters zoals licht en ruimte.
Een belangrijk onderzoeks-aspect hier is het aanpassen van de
guided tour aan de persoonlijke informatie behoefte van de gebruiker.
De resultaten van ons onderzoek zullen enerzijds een bijdrage
leveren aan het toegankelijk maken van hedendaagse kunstvormen
voor een breder publiek en zijn anderzijds expliciet
bedoeld de ingewikkelde taak van de conservering
en re-installatie van de kunstwerken effectief te ondersteunen.
Ook voor meer traditionele vormen van cultureel erfgoed zijn deze resultaten van groot belang, met het oog op de ondersteuning van ge-personaliseerde
guided tours in een media-rijke omgeving.
3) Classification:
computer science / digitale beleving
4) Composition of the Research Team:
| name | expertise | affiliation | hours/week |
|---|
| Dr. A. Eliens | multimedia/game development | VU/IMSE | 8 (supervision) |
| Drs. C. Visser | agent technolgy/DLP | VU | 12 (programmer) |
| C. van Riel *) | student MMC | VU | 32-36 (OIO) |
| | | 2 (promotor) |
The research will be executed within the VU, under the daily supervision of
Dr. A. Eliens
*) C. van Riel will obtain his master degree Information Science, within the
specialisation Multimedia and Culture, 11 october 2006.
As student/contributer to the digital dossier(s), he is the proposed candidate for
the OiO position within the project.
5) Research School: SIKS
All project-members fall under SIKS
(the Dutch Research School for Information and Knowledge Systems,
www.siks.nl).
6) Description of the Proposed Research
Contemporary art is an intrinsic part of our cultural heritage.
Installations, performances, video and other forms of media art,
have the interest of a small
group of adherents, but are in comparison with more traditional
art forms, far more difficult to present to a general audience.
Another problem presents itself, due to the type of materials
used and the context-specific aspects of these art forms,
in the conservation and re-installation of the works.
In our research we address the issue of providing access
to these contemporary art forms from a wide variety
of perspectives, ranging from the interested layman
to the expert that has to deal with archiving,
conserving and the possible re-installation of the art works.
Our research is centered around the notion of digital dossier,
that is the collection of information needed to (properly)
understand one or more works of art in the context of the oeuvre
of the artist.
The information consists of descriptive text, images depicting
the work(s), and possibly video registrations as well as
interviews with the artist.
From the perspective of the expert, in charge of conserving
the work(s), the dossier must contain all the information needed
for (proper) re-installation.
For the presentation of the digital dossier
we deploy a rich media presentation environment
with which we gained ample experience in our
research in intelligent multimedia,
among others in previous pilots with ICN,
in which we realized such environments for
the Dutch
artist Marinus Boezem,
the Serbian-Dutch
artist Marina Abramovic
and the Australian media artist
Jeffrey Shaw.
In our present research we will focus in particular
on intelligent navigation and interaction
with the digital dossier.
Due to the wish and need to
support the use of the dossier from a variety of perspectives,
the presentation environment must offer a multitude of
potential interaction scenarios, and accomodate the needs
of the variety of users in an intelligent fashion.
Our research will focus in particular on the question of how to support guided tours
in an adequate manner, with regard to the information need of the user.
As we have demonstrated in the prototype implementations, a guided tour
may take the form of a walk-through in virtual space, an automated story-line
in a concept map, or allow the (professional) to experiment with the(de-)construction of
for example an art installation, or to experiment with exhibition parameters, such as light and space.
For adapting a guided tour to the
personal information need of the user, there is a variety of solutions,
as for example explored in the CATCH/CHIP project.
In the I-GUARD project, we will investigate the use of filtering on concept node types
in the concept map, according to the interests indicated by the user.
In addition, we will explore how to utilize the navigation behavior of the user
as an indication of his/her interests, to construct a personalized tour.
For our research, we will take the digital dossier developed for Marina Abramovic,
in the sequel referred to as the abramovic dossier,
as a starting point, since it provides the most clear and innovative solution to
the problems of navigation and presentation:
- navigation -- by means of a concept graph
- presentation -- by means of a presentation gadget that allows for the simultaneous
inspection of multiple media items
Apart from allowing the presentation of 2D information (text, images, and video),
the abramovic dossier also allows for the incorporation of 3D models
of artworks and installation in a seamless (immersive) way.
6a) Scientific aspects
Given the need to
represent information of one or multiple works
of art in a digital dossier,
that allows for the presentation of that information in a
rich media presentation environment,
we address, from a research perspective, the following
issues:
research issues
- navigation using a concept graph
- presentation of 2D and 3D information in a unified fashion
- interaction with support for (personalized) guided tours
The presentation of information in a 3D environment
will not be the central focus of our research, although our
experience in this area is an essential pre-requisite for our research.
Instead we will concentrate on navigation, presentation issues, and interaction with
support for guided tours.
In addition, we will tackle more broad issues of representation of information
and investigate a generic approach to the construction of
digital dossiers.
Navigation:
Given the complexity of the information space related to a work
of art, interactive facilities must be provided
for the user to gain a quick overview of the information
available as well as the means to deepen the inspection rapidly,
avoiding needless and cumbersome detours.
At the same time, the user must be allowed to inspect
the (multimedia) material related to the work of art,
such as a collection of images or video recordings.
To accommodate the complex structure of the information space
we have developed a 3D representation of a concept graph,
a semantic device well-known from artificial intelligence,
which allows the user to expand a node representing
a particular concept or aspect into a collection of related
nodes.
Moreover, for the presentation of (multimedia) information
we have developed a (generic) presentation gadget,
that displays images or video and textual information simultaneously.
The actual content of the presentation gadget depends
on the node of the concept graph from which it is
expanded.
The presentation gadget allows for limited browsing
following (textual) links, within the material presented.
At each moment, the presentation gadget can be collapsed
to continue navigation in the concept graph.
Using the concept graph in conjunction with the
presentation gadget resolves the duality of information
and presentation that is inherent for (multimedia) digital dossiers,
and thus promotes what may be called the immersive
experience of digital artist's dossiers.
Presentation:
Apart from the presentation gadget, which allows for the simultaneous
inspection of multiple media items, in a variety of formats,
we explored the use of guided tours as a means to present the information in a
story-like way, relieving the user of the often cumbersome task
to interact.
Guided tours, in the digital dossier, may take one of the following forms:
- automated (viewpoint) navigation in virtual space,
- the (narrative) presentation of a sequence of concept nodes,
- an animation explaining, for example, the construction of an artwork.
In practice, an actual guided tour may be constructed as a
combination of these elements, interweaving, for example,
the explanation of concepts, or biographic material of the artist,
with the demonstration of the positioning of an artwork in
an exhibition space.
The research issues, here, encompass:
- the selection of items in the guided tour, as well as
- the presentation of these items in an immersive manner.
Immersion, here, means that we strive for a seamless transition
between the presentation of conceptual material and visual material,
such as video registrations or 3D models of the artworks.
Interaction:
Our aim is to arrive at a general framework for artist's digital
dossiers, that provide intelligent guidance
to both the expert user, responsible for the
future re-installation of the work(s),
and the interested layman, that wishes to get acquainted
with a particular work or collection of works.
In general, there are two techniques that we can apply to provide such
guidance:
- filtering the information space according to the user's
perspective, and
- intelligent agents, that (pro) actively aid the user in searching
the information space.
Filtering the information space
may be used
to restrict the concept graph that defines the navigation
structure,
by stating assumptions with respect to the relevance
of particular categories from
a user's perspective.
Intelligent agents is an approach stemming from artificial intelligence
which allows for providing guidance in a variety of ways,
possibly even in an embodied form using
a face or humanoid figure to give suggestions
to the user on what interactions to perform.
In [Hoorn et al. (2004) ] we have investigated the use of embodied
agents in a digital dossier for the artist Marinus Boezem.
In our current research, however, we will very likely
not use embodied agents.
Nevertheless, we will investigate to what extent we can use an agent model,
possibly with learning capabilities as explored in [Hildebrand et al. (2003) ],
to provide guidance and support interaction.
Our goal is to arrive at an advice function, that offers the user at any
navigation point a choice of continuations and/or a selection of guided tours,
focussing on a topic of interest.
For selecting the items to be presented in a guided
tour, the most obvious way is to pre-define a sequence based
on user profiles.
Very likely this can be done in a more flexible way in a rule-based
manner, applied to a template tour.
More interesting, however, is to investigate whether guided tours
can be generated dynamically based on tracking actual user interaction
of (expert) users, using techniques from prediction theory, as
explained in section 6b.
To allow for meaningful interaction with 3D models,
allowing to view for example information about the materials used
or its installation procedure, we must find a way
to connect that information to user actions in a generic way.
In other words, there is an information representation problem,
namely, how to relate contextual information in a generic fashion
to elements of a 3D model representing an artwork.
Although such interactions can be realized by embedding
invisible) action/event objects in the model, a more generic way of
representing such relations is desirable, to avoid the need for the
time-consuming hand-crafting for which in practice there may not even be
the necessary (human) resources.
6b) Method of research
The method of research will primarily be explorative develepment
of (digital dossier) applications, in cooperation with ICN and
possibly other institutes of cultural heritage.
Although by some denounced as a non-scientific method, we believe that
the approach of explorative design and development is most fruitful in
this area of research. See for a discussion also [Grau (2003)].
Given the variety of information material related to artworks, and the need
to present this material in a visually compelling way, we will apply techniques as used in the development of
interactive narrative games, as described in [Hawkins (2005)].
For the selection of items in guided tours and the generation of interesting
sequences, we will explore the use of prediction theory.
As explained in [Cesa-Bianchi and Lucosi (2006)], prediction theory uses a model of prediction
based on expert advice.
However, instead of the traditional loss function, used in a stochastic
approach, prediction theory uses a regret function,
which expresses the difference between an actual prediction and the advice of a collection of experts.
An expert, in this context, is an abstract entity, that may be either
embodied by an algorithm, a random selection, or an actual expert.
We will investigate, for the construction of guided tours, whether it is
possible to generate interesting sequences by using a (sequence of) prediction(s)
that minimizes the regret function, which respect to the navigation sequence(s)
recorded from actual expert users.
In particular, we will strive for implementing the advice function, in a generic way,
by means of a learning mechanism that extracts recommended continuations and
guided tours from tracking expert user navigation.
6c) Scientific interest
Although digital archives or digital libraries
are by no means a new phenomenon,
our concept of digital dossiers contains a number
of innovative elements.
A digital dossier provides a unified information and
presentation space.
In this sense it differs significantly from
a digital archive with a traditional web interface,
where navigation and presentation are distinct.
Digital dossiers allow to a much greater extent for an immersive
experience of the information related to works of art.
As such it is reminiscient to explorations in
virtual archeology.
As concerns intelligent guidance,
again, although agent technology itself is by no means new,
we firmly believe that our use of agent technology offers
exciting new ways in which intelligent guidance may be realized.
Generating guided tours based on (expert) user tracking is no doubt an interesting
issue from a scientific point of view, eventhough we may not be able to guarantee
that such tours will be interesting for the target audience.
6d) Relation to other research
In contrast with the majority of research (proposals)
in the CATCH-framework,
which are concerned with making a great mass of artworks
accesible to a wide audience using (fairly)
traditional techniques of presentation,
our research addresses the more narrow area of contemporary artworks,
which require the use of more advanced presentation technologies.
Our work has started as an extension to the
INCCA project,
which provides a digital archive of meta-information indicating
sources of information on (the conservation of) contemporary art.
However, as also indicated in the
Culture 2000 project proposal, more advanced
practices and techniques are needed to adequately
capture the requirements for the re-installation
of contemporary art works.
Our research aims at developing the (information) technologies
needed to implement such practices and techniques,
and more in particular to develop a framework for realizing
digital dossiers with intelligent guidance,
adapted to the particular perspective of its users,
which may be conservation experts or intelligent laymen.
The framework will be built on open standards and may eventually be donated
to the (traditional) cultural heritage community.
6e) Relation to current research
The background of our research
is provided
by our previous research on intelligent multimedia,
as described in [IMVU].
Our intelligent multimedia platform supports interactive
3D graphics, multi-user virtual environments, as well
as embodied agents.
See [Eliens et al. (2002) ], [Community] and [Huang et al. (2003) ].
It uses distributed logic programming, [ Eliens (1992) ],
and is built on object-oriented programming technologies, [Eliens (2000) ].
It has been applied
in student projects,
done in cooperation with ICN, for developing digital
dossiers for respectively Marinus Boezem,
Marina Abramovic and Jeffrey Shaw.
As a complement to the theoretical and technical research we also
plan to continue our empirical
human-computer interaction studies, as reported in [Hoorn et al. (2004) ],
to assess the viability of our approach.
The contribution of the cultural heritage institutes,
in particular ICN, will be to provide the material,
assist in finding relevant categorizations and, inevitably,
function as the touchstone against which our efforts
will be evaluated.
Our current interest focusses on the use of game technology for immersive
(serious) applications.
This research provides the necessary background for the presentation
of multimedia material as well as the construction of naaratives. See
[Eliens and Bhikharie (2006)], [Eliens (2006b) ].
7) Work Programme
A brief summary of the
issues that will be tackled and the
deliverables that we expect to produce
during the four years of the project looks as follows.
We distinguish, over the four years, between a theoretical/study track
and a practical/application track.
theorectial track
- year 1: identification of major issues -- study of prediction theory
- year 2: adaptive guided tour -- study of personalisation and user profile issues
- year 3: guided tour based on user tracking -- study of agent technology and narrative theory
- year 4: writing of the thesis
In the first stage of the research,
the work to be done by the proposed candidate,
Chris van Riel,
will be a follow-up on his master thesis,
which describes a first exploration in the development of guided tours
for digital dossiers.
target application(s)
- year 1: extension of abraomiv dossier with user navigation tracking
- year 2: prototype realization of guded tour based on user tracking
- year 3: implementation of an advice function offering continuations and (limited) guided tours
- year 4: development of a generic digital dossier construction kit
The candidate researcher will work in close cooperation
with the other members of the intelligent multimedia
research group, in particular dr A. Eliëns.
to explore the topics mentioned and to
apply the intelligent multimedia
technology that is being developed to digital dossiers.
education track
The advanced education of the candidate researcher will
mainly be taken care of by the standard courses offered
by the SIKS research school. (See www.siks.nl)
In addition, the candidate is supposed to become familiar
with the literature in multimedia and other projects in
the domain of cultural heritage, in particular CATCH, with which there will be close
cooperation.
In the education track of the candidate, there will be specific
focus on the mathematical
background of decision and prediction theory, personalization and user-profile issues,
and the application of agent technology for learning and the realization of the
advice function.
8) Literature
Our papers, mentioned below, are avaliable online at:
As`our five most important papers, in the area of this research, we would like to mention:
[Ballegooij and Eliens (2001)], [Eliens et al. (2002) ], [Eliens et al. (2006)], [Hildebrand et al. (2003) ] and [Hoorn et al. (2004) ].
- [Remediation] Bolter J.D and Grusin R. (2000),
- Remediation -- Understanding New Media,
MIT Press
- [Query] Ballegooij A. van and Eliens A. (2001),
- Navigation by Query in Virtual Worlds,
Web3D 2001 Conference, Paderborn, Germany, 19-22 Feb 2001
- [Prediction] Cesa-Bianchi N. and Lucosi G. (2006),
- Prediction, Learning, and Games,
Cambridge University Press
- [DLP] Eliens A. (1992),
- DLP -- A language for Distributed Logic Programming,
Wiley
- [OO] Eliens A. (2000),
- Principles of Object-Oriented Software Development,
Addison-Wesley Longman, 2nd edn.
- [Platform] Eliens A., Huang Z., and Visser C. (2002),
- A platform for Embodied Conversational Agents based on Distributed Logic Programming,
AAMAS Workshop -- Embodied conversational agents - let's specify and evaluate them!, Bologna 17/7/2002
- [Navigate] Eliens A., van Riel C., Wang Y. (2006),
- Navigating media-rich information spaces using concept graphs -- the abramovic dossier,
accepted for: International Conference on Multidisciplinay Information Sciences and Technologies (InSciT2006) October, 25-28th 2006, Merida, Spain www.instac.es/inscit2006
- [VULife] Eliens A. and Bhikharie S.V. (2006),
- game @ VU -- developing a masterclass for high-school students using the Half-life 2 SDK,
accepted for: GAME'ON-NA'2006, September 19-21, 2006 - Naval Postgraduate School, Monterey, USA
- [Odyssee] Eliens A. (2006b),
- Odyssee -- explorations in mixed reality theatre,
accepted for: GAME'ON-NA'2006, September 19-21, 2006 - Naval Postgraduate School, Monterey, USA
- [VirtualArt] Grau O. (2003),
- Virtual Art -- From Illusion to Immersion,
MIT Press
- [Cinema] Hawkins B. (2005),
- Real-time cinematography for games,
Charles River Media
- [Interactive] Hildebrand M., Eliens A., Huang Z. and Visser C. (2003),
- Interactive Agents Learning their Environment,
Proc. Intelligent Virtual Agents 2003, Irsee, September 15-17, 2003 J.G. Carbonell and J.Siekmann (eds.), LNAI 2792, Springer, pp. 13-17
- [Empathic] Hoorn J., Eliens A., Huang Z., van Vugt H.C., Konijn E.A., Visser C.T.,
- Agents with character: Evaluation of empathic agents in digital dossiers,
Emphatic Agents, AAMAS 2004 New York 19 July - 23 July, 2004
- [Communities] Huang Z., Eliens A., Visser C. (2002),
- 3D Agent-based Virtual Communities,
In: Proc. Int. Web3D Symposium, Wagner W. and Beitler M.( eds), ACM Press, pp. 137-144
- [STEP] Huang, Z., Eliens, A., and Visser, C.,
- STEP: a Scripting Language for Embodied Agents,
in: Helmut Prendinger and Mitsuru Ishizuka (eds.), Life-like Characters, Tools, Affective Functions and Applications, Springer-Verlag, 2003.
- [HalfReal] Juul J. (2005),
- Half Real -- Video Games between Real Rules and Fictional Worlds,
MIT Press
- [Guide] Riel C. van, Eliens A. and Wang Y (2006),
- Exploration and guidance in media-rich information spaces: the implementation and realization of guided tours in digital dossiers,
accepted for: International Conference on Multidisciplinay Information Sciences and Technologies (InSciT2006) October, 25-28th 2006, Merida, Spain www.instac.es/inscit2006
- [ConceptMaps] Riel C. van, Wang Y. and Eliens A. (2006b) ,
- Concept map as visual interface in 3D Digital Dossiers: implementation and realization of nthe Music Dossier,
CMC2006, Costa Rica, Sept 5-8 2006
- [Present] Wang Y., Eliens A. and van Riel C. (2006),
- Content-oriented presentation and personalized interface of cultural heritage in digital dossiers,
accepted for: International Conference on Multidisciplinay Information Sciences and Technologies (InSciT2006) October, 25-28th 2006, Merida, Spain www.instac.es/inscit2006
9) Requested Budget
| personell | period | euro |
|---|
| OIO | 4 year | 172.371 |
| benchfee | 5.000 |
| | 177.373 |
As indicated,
the amount of euro 177.373 includes the 5000 euro benchfee for travel expenses and other support.
(C) Æliens
04/09/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
