topical media & game development

talk show tell print

representation & interaction


  
  <recordList>
   <record creation="1070031329" modification="1107262576">
    <priref>1001000025</priref> 
    <input.time>10:22:05</input.time> 
    <edit.time>13:55:55</edit.time> 
    <related>1001000019</related> 
    <edit.name>Tatja.Scholte</edit.name> 
    <input.name>bb-05</input.name> 
    <edit.date>2005-02-01</edit.date> 
    <input.date>2002-10-09</input.date> 
    <contributor.domain>organisation</contributor.domain> 
    <publisher.domain>organisation</publisher.domain> 
    <creator.domain>organisation</creator.domain> 
    <title>Conservation record for Antony Gormley's 'Untitled (For Francis)', 1985</title> 
    <subject>Gormley, Antony</subject> 
    <subject>mixed media</subject> 
    <subject_type>artist</subject_type> 
    <subject_type>artist_material</subject_type> 
    <publisher.role>museum</publisher.role> 
    <rights>TATE, Conservation Department. Please contact Publisher</rights> 
    <relation>www.tate.org.uk/search/default.jsp</relation> 
    <contributor.role>museum</contributor.role> 
    <creator.role>conservation department</creator.role> 
    <publisher>TATE, Conservation Department</publisher> 
    <language>English</language> 
    <identifier>Tate</identifier> 
    <identifier>T05004</identifier> 
    <form.type>Report</form.type> 
    <format>Word</format> 
    <format>Print</format> 
    <format>Photograph (black-and-white)</format> 
    <description>The Tate's conservation record for Antony Gormley's 'Untitled (For Francis)', 1985,
     includes a record of examination, a structure and condition report, 
     handling and installation notes, a record of treatment, 
     information concerning display and installation, as well as photographs of the work.
     </description> 
    <date>2002-10-08</date> 
    <coverage>UK</coverage> 
    <content.type>Artist's technique</content.type> 
    <content.type>Conservation</content.type> 
    <content.type>Display/Installation</content.type> 
    <content.type>Artist's material</content.type> 
    <contributor>TATE, Conservation Department</contributor> 
    <creator>TATE, Conservation Department</creator> 
    </record>
  </recordList>
  

further research

Contemporary art is an intrinsic part of our cultural heritage. Installations, performances, video and other forms of media art, as for example web art, have the interest of a small group of adherents, but are in comparison with more traditional art forms, far more difficult to present to a general audience. Another problem presents itself, due to the type of materials used and the context-specific aspects of these art forms, in the conservation of the works.

In our research we address the issue of providing access to these contemporary art forms from a wide variety of perspectives, ranging from the interested layman to the expert that has to deal with archiving, conserving and the possible re-installation of the art works.

Our research is centered around the notion of digital dossier, that is the collection of information needed to (properly) understand one or more works of art in the context of the oeuvre of the artist. The information consists of descriptive text, images depicting the work(s), and possibly video registrations as well as interviews with the artist. From the perspective of the expert, in charge of conserving the work(s), the dossier must contain all the information needed for (proper) re-installation.

For the presentation of the digital dossier we deploy a rich media presentation environment with which we gained ample experience in our research in intelligent multimedia  [IMVU], among others in previous pilots with ICN, in which we realized such environments for the Dutch artist Marinus Boezem and the Serbian-Dutch artist Marina Abramovic.

In our present research we will focus in particular on intelligent navigation and interaction with the (presentation environment of the) digital dossier. Due to the wish and need to support the use of the dossier from a variety of perspectives, the presentation environment must offer a multitude of potential interaction scenarios, and accomodate the needs of the variety of users in an intelligent fashion.

An essential part of intelligent interaction is provided by the possibility to give assignments in loosely structured natural language, to query for particular aspects of the work, as for example properties of the materials that are used, or to specify desires with respect to the presentation of the information offered.

In summary, from a research perspective we address the following issues:

research issues


  • representation of information of one or multiple works of art in a digital dossier,
  • presentation of that information in a rich media presentation environment,
  • intelligent navigation and interaction, and
  • support for interaction with loosely-structured natural language.
The results of our research will on the one hand contribute to making contemporary art forms accessible to a larger audience and on the other hand are explicitly meant to support the complex task of the conservation and re-installation of works of art in an effective manner.

scientific aspects

The presentation of information in a 3D environment will not be the central focus of our research. Instead we will concentrate on navigation and interaction, and support for interaction with natural language. In addition, we will tackle issues of representation of information and investigate a generic approach to the construction of digital dossiers. We will now discuss possible solutions and the scientific aspects related to the topics of, respectively, navigation, natural language and intelligent guidance seperately.

navigation:

Given the complexity of the information space related to a work of art, interactive facilities must be provided for the user to gain a quick overview of the information available as well as the means to deepen the inspection rapidly, avoiding needless and cumbersome detours. At the same time, the user must be allowed to inspect the (multimedia) material related to the work of art, such as a collection of images or video recordings.

To accommodate the complex structure of the information space we have developed a 3D representation of a concept graph, a semantic device well-known from artificial intelligence, which allows the user to expand a node representing a particular concept or aspect into a collection of related nodes. Moreover, for the presentation of (multimedia) information we have developed a (generic) presentation gadget, that displays images or video and textual information simultaneously. The actual content of the presentation gadget depends on the node of the concept graph from which it is expanded. The presentation gadget allows for limited browsing following (textual) links, within the material presented. At each moment, the presentation gadget can be collapsed to continue navigation in the concept graph.

Using the concept graph in conjunction with the presentation gadget resolves the duality of information and presentation that is inherent for (multimedia) digital dossiers, and thus promotes what may be called the immersive experience of digital artist's dossiers.

natural language:

Having a concept graph as a generic navigation device, it still remains a problem how to fill the concept graph with meaningful content, and how to indicate meaningful relations between the concepts and aspects covered by the nodes of the concept graph. In the prototype digital dossier for Marina Abramovic this was done by hand, based on information derived from a transcripion of an interview with the artist (provided to us by ICN).

Interviews with artists is one of the means ICN deploys to gain knowledge needed for the conservation of contemporary artworks. Such interviews provide a rich source of textual information, that includes both general viewpoints on the artist's oeuvre as well as specific constraints that adhere to the (re) installation of the work(s) of art.

In our research we will investigate the use of techniques from formal linguistics,  [Grammar], as a means to derive both structure and content of the concept graph for a particular dossier (semi) automatically. Using a basic lexicon of terms and phrases related to contemporary art we should be able to generate a representation of the textual information that may serve as a basis for construction of the concept graph. This representation must contain an enumeration of the concepts, the relation between occurrences of concepts, as well as a reference to the work(s) of art to which the concepts apply.

Natural language processing technology may not only serve for the static analysis of the material, when the digital dossier is created, but also dynamically when the dossier is being used, to aid the user in finding relevant information. Research issues here are, on the one hand, the interpretation of user input (that is, loosely structured natural language), and on the other hand, filtering the concept graph representing the information space in such a way that it adequately reflects the user's interest or perspective.

intelligent guidance:

Our aim is to arrive at a general framework for artist's digital dossiers, that provide intelligent guidance to both the expert user, responsible for the future re-installation of the work(s), and the interested layman, that wishes to get acquainted with a particular work or collection of works. In general, there are two techniques that we can apply to provide such guidance:
  • filtering the information space according to the user's perspective, and
  • intelligent agents, that (pro) actively aid the user in searching the information space.
Filtering the information space may be done, as indicated above, using techniques from formal linguistics to restrict the concept graph that defines the navigation structure, by stating assumptions with respect to the relevance of particular linguistic categories from a user's perspective.

Intelligent agents is an approach stemming from artificial intelligence which allows for providing guidance in a variety of ways, possibly even in an embodied form using a face or humanoid figure to give suggestions to the user on what interactions to perform. In  [AWC] we have investigated the use of embodied agents in a digital dossier for the artist Marinus Boezem. In our current research, however, we will very likely not use embodied agents. Nevertheless, we will investigate to what extent we can use an agent model, possibly with learning capabilities as explored in  [Interactive], to provide guidance and support interaction.

the background of our research

is provided by our previous research on intelligent multimedia, as described in  [IMVU]. Our intelligent multimedia platform supports interactive 3D graphics, multi-user virtual environments, as well as embodied agents. See  [Platform],  [Community] and  [STEP]. It uses distributed logic programming,  [DLP], and is built on object-oriented programming technologies,  [OO]. It has been applied in student projects, done in cooperation with ICN, for developing digital dossiers for respectively Marinus Boezem and Marina Abramovic.

A natural language interface for an agent capable of learning has been explored, in cooperation with the formal linguistics group of the University of Utrecht, by our candidate researcher in his master thesis project, which has resulted in a publication on the Interactive Virtual Agents conference,  [Interactive]. This application used the Grail parser, which is based on the notion of parsing as deduction,  [Grammar]. In the proposed research we wish to continue this cooperation to further incorporate language processing capabilities in our framework.

As a complement to the theoretical and technical research we also plan to continue our empirical human-computer interaction studies, as reported in  [AWC], to assess the viability of our approach.

The contribution of the cultural heritage institutes, in particular ICN, will be to provide the material, assist in finding relevant categorizations and, inevitably, function as the touchstone against which our efforts will be evaluated.

innovation

Although digital archives or digital libraries are by no means a new phenomenon, our concept of digital dossiers contains a number of innovative elements.

A digital dossier provides a unified information and presentation space. In this sense it differs significantly from a digital archive with a traditional web interface, where navigation and presentation are distinct. Digital dossiers allow to a much greater extent for an immersive experience of the information related to works of art. As such it is reminiscient to explorations in virtual archeology.

As concerns intelligent guidance, again, although agent technology itself is by no means new, we firmly believe that our use of techniques from formal linguistics in combination with agent technology offers exciting new ways in which intelligent guidance may be realized.

The type-logical perspective of our linguistic framework will enable us to exploit commonalities between the reasoning facilites underlying visual information processing, and language processing,  [Categorial]. Hence, fundamentally, we address issues of the relation between visual and verbal/linguistic structures. Pragmatically, this will be reflected in our explorations of how textual relations can best be presented in a tangible way to the user of the digital dossier.

On a longer term, our research opens the prospect of having an information representation scheme that may, with minimal additional annotation, be used to generate a fully functional digital dossier.

relevance for cultural heritage

In contrast with the majority of research (proposals) in the CATCH-framework, which are concerned with making a great mass of artworks accesible to a wide audience using (fairly) traditional techniques of presentation, our research addresses the more narrow area of contemporary artworks, which require the use of more advanced presentation technologies.

Our work has started as an extension to the INCCA project, which provides a digital archive of meta-information indicating sources of information on (the conservation of) contemporary art. However, as also indicated in the Culture 2000 project proposal, more advanced practices and techniques are needed to adequately capture the requirements for the re-installation of contemporary art works.

Our research aims at developing the (information) technologies needed to implement such practices and techniques, and more in particular to develop a framework for realizing digital dossiers with intelligent guidance, adapted to the particular perspective of its users, which may be conservation experts or intelligent laymen. The framework will be built on open standards and may eventually be donated to the cultural heritage community.

research directions -- media art

In a recent symposium on the preservation of contemporary media art, a number of institutions presented their projects, ranging from more technical topics, such as the conservation of videotapes and the mass storage of digital material to the conceptual issues in capturing new media, the variety of media formats and the need to record and maintain meta data about the artworks and related information.

To get an idea what the phrase media art encompasses, have a look at the circumscription given in the WikiPedia: new media art is a generic term used to describe art related to, or created with, technology invented or made widely available since the mid-20th Century, including technology stemming from telecommunications, mass media and digital modes of delivery the artworks Below, the disciplines that belong to this form of art are listed, together with their entries in the WikiPedia, in an abbreviated form:

(new) media art


  • audio art -- no definition available
  • computer art -- any art in which computers played a role in production or display of the artwork.
  • digital art -- art created on a computer in digital (that is, binary) form.
  • electronic art -- entry to game producer, should be Leonardo.
  • generative art -- art or design generated, composed, or constructed through computer software algorithms, or similar mathematical or mechanical autonomous processes
  • hacktivism -- the writing of code, or otherwise manipulating bits, to promote political ideology
  • interactive art -- a piece of art that involves the spectator in some way.
  • internet art -- art or, more precisely, cultural production which uses the Internet as its primary medium and, more importantly, its subject.
  • performance art -- art where the actions of an individual or a group at a particular place and in a particular time, constitute the work.
  • robotic art -- page does not exist
  • software art -- is an intersection of two almost non-overlapping realms: software and art.
  • video art -- is a subset of artistic works which relies on "moving pictures" and is comprised of video and/or audio data.
  • video game art -- involves the use of a computer game for the creation of a digital artwork.
By the nature of the WikiPedia, to which every user can contribute entries, this list nor the defining entries are by any means authorative. Nevertheless, it does provide an overview and may serve as a starting point for further research.

(C) Æliens 23/08/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.