topical media & game development
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

talk show tell print

representation & interaction

In re-thinking the abramovic dossier, we first needed to re-establish what are our goals in developing this application and what are our primary data sources. The goal, first of all, is to support conservators in their task of preserving contemporary art, and to assist them with the re-installation of such artworks.

...
Our primary data sources are meta-information, coming from the INCCA initiative, and video-recorded artist interviews, which were initiated by ICN as a means to record information about contemporary art that would otherwise be lost. In addition we provide media-material, including images and video, that may be regarded as recordings of the works of art, as well as the textual interpretations and classifications that exist, or may be constructed from this material.

At this point, I may remark that one of the pitfalls in creating a dossier is to get trapped in the visually salient features of the dossier, the presentation of the artworks themselves, and forget about the primary focus of the dossier, to make all information accessible in an adequate manner. For many cultural heritage applications, which aim to present art to the layman, presenting the artwork is the primary focus, and giving access to the information context generally comes second.

For our next generation of digital dossiers, we decided to take the following steps:

next generation dossier(s)


  1. adaptation of representation to Dublin Core (+ annotation needed for presentation)
  2. XML-based content management, with php forms (extending what we have now)
  3. there should also be a possibility to present the information and material in a 'plain' web format
  4. as well as in (a new version of) 3D dossiers
  5. we should think about the proper presentation parameters.
Dublin Core is the standard used in the INCCA initiative, to record meta-information about existing information sources. See section 3.3 for a description of the Dublin Core element set and the Resource Description Framework (RDF) on which it is based.

For the abramovic dossier, a collection of record-like structures was developed, together with a simple content-management tool, written in PHP. This content-management system must be adapted to be compatible with the Dublin Core-based resource descriptions.

Further, we decided that, along with the 3D presentation of the dossier, it would be worthwhile to develop a conversion tool that produces standard web-technology based presentations as well. This approach allows us to assess the usability merits of the 3D dossiers in a comparative way.

Finally, as I indicated before, an important issue that must be resolved concerns the proper presentation parameters. What do we present to the user? And how do we allow the user to interact with the material presented?

content management and data representation

For developing the abramovic dossier, we have a fixed number of record-like structures:

structures


  • Video -- to display video fragment, including interviews
  • Picture -- to present pictures of the artwork
  • Artwork -- contains all information connected to a work of art
  • TextItem -- to present text, from the interview or any other source
  • MaterialItem -- to present information about material(s) used
  • GroupNode -- to combine nodes in the concpet graph
  • Information -- acts as the outer container for all nodes
All these structures support a set of common attributes, including shortName, longName, ID, connectedNodesIDs, and description. In addition the Video, Picture and Picture have fields allowing to show a preview image. And the Video, Picture and TextItem, also have a url field giving access to the actual information item.

The Information and GroupNode structures are used for creating the top-levels of the concept graph, whereas the other structures, such as the Video and TextItem give access to for example a fragment of an interview and its transcription.

Below an example is given of the data underlying the concept graph of the abramovic dossier:

concept graph



  Information {
   informationNodes [
    GroupNode {
  	ID "MAIN"
  	shortName "Main"
  	longName "Main"
  	urlModel "models/conceptGraph/main/modelMain.wrl"
  	description [ "Central information node" ]
  	connectedNodesIDs [ "ARTWORKS", "KEYWORDS", 
  	                         "INTERVIEWS", "REPORT" ]
  	}
    GroupNode {
  	shortName "Artworks"
  	longName "Artworks"
  	description [ "Node that connects to all the artworks" ]
  	ID "ARTWORKS"
  	connectedNodesIDs [ "MAIN", "TRANSITORY", 
  	              "ULAY", "VIDEOINSTALLATION", "ABRAMOVIC" ]
  	urlModel "models/conceptGraph/artworks/artworksGroup.wrl"
  	}
  	## ...
   ]
  }
The Information node collects all available nodes, and takes care of connecting the individual nodes, based on the information specified for each node.

As an example of an Artwork node, that is an element of the list of nodes in the Information node, look at: 


  Artwork {
    shortName "Terra degla Dea Madre"
    longName "Terra degla Dea Madre"
    description ["15:40 min, colour, sound."]
    ID "AV24"
    connectedNodesIDs ["VIDEOINSTALLATION", "DTV24", 
               "TTV24", "PV24", "CV24", "VV24", "G0"]
    urlPreviewImage "images/previewImages/AV24.jpg"
    widthPreviewImage 479
    heightPreviewImage 349
  }
This node is connected to many other nodes, giving access to the information items that belong to it, such as the video clips of the interview, shown below. 

  Video {
    ID "CV24"
    shortName "Interview clip Terra degla Dea Madre"
    longName "Interview clip showing Terra degla Dea Madre"
    url "interviewclips/interview_terra_degla.avi"
    width 320
    height 360
    urlPreviewImage "images/previewImages/interview_terra_degla.jpg"
    widthPreviewImage 320
    heightPreviewImage 240
    description [""]
    connectedNodesIDs ["CLIP", "AV24"]
  }
In the url field of this declaration, the actual video file is indicated, which should be displayed at a resolution of 320x360, as specified in the width and height fields.

And finally, as an example of a TextItem, consider: 


  TextItem {
    shortName "Instruction"
    longName "Green Dragon Lying instructions for the public."
    description ["Text explaining the way the public has to interact with the artwork."]
    ID "ITO05"
    connectedNodesIDs ["AO05", "INTERACTION"]
    url "text/AO05_instruction.txt"
  }
For constructing the abramovic dossier, Tim Verweij developed the content management tool, that allows the user to browse and edit existing nodes, and to insert new nodes into the graph.

integration with the Dublin Core

 The Dublin Core is a general resource description formalism, that allows for specifying resources in a variety of domains. See section 3.3. For INCCA the Dublin Core was chosen, not because it is the most suitable formalism, but because it may serve as the least common denominator, and agreement on anything else simply seemed to be impossible. As a reminder, the Dublin Core provides the following elements:

Dublin Core


  • title -- name given to the resource
  • creator -- entity primarily responsible for making the content of the resource
  • subject -- topic of the content of the resource
  • description -- an account of the content of the resource
  • publisher -- entity responsible for making the resource available
  • contributor -- entity responsible for making contributions to the content of the resource
  • date -- date of an event in the lifecycle of the resource
  • type -- nature or genre of the content of the resource
  • format -- physical or digital manifestation of the resource
  • identifier -- unambiguous reference to the resource within a given context
  • source -- reference to a resource from which the present resource is derived
  • language -- language of the intellectual content of the resource
  • relation -- reference to a related resource
  • coverage -- extent or scope of the content of the resource
  • rights -- information about rights held in and over the resource
Descriptions of items in the digital dossier should incorporate these elements, together with the attributes needed for the insertion of items in the concept graph and the presentation parameters, that are necessary for displaying the (media) material. Technically, the namespaces supported by RDF does allow for merging these different types of annotations. However, the challenge here is to derive the presentation attributes automatically, and to come up with a reasonable default for inserting these items in the concept graph.

...
location of Tower of Babel project

1

intelligent guidance -- I-GUARD

 Although digital archives or digital libraries are by no means a new phenomenon, our concept of digital dossiers contains a number of innovative elements. A digital dossier provides a unified information and presentation space. In this sense it differs significantly from a digital archive with a traditional web interface, where navigation and presentation are distinct. Digital dossiers allow to a much greater extent for an immersive experience of the information related to works of art. As such it is reminiscient to explorations in virtual archeology, our to our notion of virtual context, presented in section 8.1.

Working out the issues indicated above, that is the integration with the Dublin Core and providing suitable content management, is a matter of diligent software engineering. But what can we further do to support the construction of digital dossiers and improve the usability of such dossiers? And what are the scientific issues, worth to be investigated?

To indicate the research issues, let me first expand the cope of our project and re-define the goal of our research:

I-GUARD


Contemporary art is an intrinsic part of our cultural heritage. Installations, performances, video and other forms of media art, as for example web art, have the interest of a small group of adherents, but are in comparison with more traditional art forms, far more difficult to present to a general audience. Another problem presents itself, due to the type of materials used and the context-specific aspects of these art forms, in the conservation of the works.

In our research we address the issue of providing access to these contemporary art forms from a wide variety of perspectives, ranging from the interested layman to the expert that has to deal with archiving, conserving and the possible re-installation of the art works.

The acronym I-GUARD stands for Intelligent Guidance in Artist's Digital Dossiers, and refers to a project the aim of which is to arrive at a general framework for artist's digital dossiers, that provide intelligent guidance to both the expert user, responsible for the future re-installation of the work(s), and the interested layman, that wishes to get acquainted with a particular work or collection of works. In general, there are two techniques that we can apply to provide such guidance:

intelligent guidance


  • filtering the information space according to the user's perspective, and
  • intelligent agents, that (pro) actively aid the user in searching the information space.
Filtering the information space may be done by using techniques from formal linguistics to restrict the concept graph that defines the navigation structure, that is by stating assumptions with respect to the relevance of particular (linguistic) categories or elements from a user's perspective. Intelligent agents is an approach stemming from artificial intelligence which allows for providing guidance in a variety of ways, possibly even in an embodied form using a face or humanoid figure to give suggestions to the user on what interactions to perform. With the latter type of guidance we have already experimented in the Marinus Boezem dossier, as described in section 9.2. So let's look at what natural language technology has to offer.

natural language:

 Having a concept graph as a generic navigation device, it still remains a problem how to fill the concept graph with meaningful content, and how to indicate meaningful relations between the concepts and aspects covered by the nodes of the concept graph. In the abramovic dossier this was done by hand, based on information derived from a transcripion of an interview with the artist. (provided to us by ICN). Interviews with artists is one of the means ICN deploys to gain knowledge needed for the conservation of contemporary artworks. Such interviews provide a rich source of textual information, that includes both general viewpoints on the artist's oeuvre as well as specific constraints that adhere to the (re) installation of the work(s) of art.

What we should strive for is to derive both structure and content of the concept graph for a particular dossier (semi) automatically. Using a basic lexicon of terms and phrases related to contemporary art we should be able to generate a representation of the textual information that may serve as a basis for constructing the concept graph. This representation must contain an enumeration of the concepts, the relation between occurrences of concepts, as well as a reference to the work(s) of art to which the concepts apply.

Natural language processing technology may not only serve for the static analysis of the material, when the digital dossier is created, but also dynamically when the dossier is being used, to aid the user in finding relevant information. Research issues here are, on the one hand, the interpretation of user input (that is, loosely structured natural language), and on the other hand, filtering the concept graph representing the information space in such a way that it adequately reflects the user's interest or perspective.

In summary, from a research perspective, digital dossier(s) concern the following issues:

digital dossier(s)


  • representation of information of one or multiple works of art,
  • presentation of that information in a rich media presentation environment,
  • intelligent navigation and interaction, and
  • support for interaction with loosely-structured natural language.
And to conclude, digital dossiers will on the one hand contribute to making contemporary art forms accessible to a larger audience and on the other hand are explicitly meant to support the complex task of the conservation and re-installation of works of art in an effective manner.

...
outside view of Tower of Babel project

2

example(s) -- Tower of Babel

 In the Tower of Babel project, shown above, multimedia material was projected from within buildings, on the windows, to the outside. Local citizens in a neighbourhood in Amsterdam where approached to submit material that expressed their emotions of daily life, with the question what moves you. The text and photograms could be submitted either by email or SMS. Also workshops were held, during which particpants could develop material. This material was then edited and prepared for projection, using 40 carroussel dia-projectors, taking about 2000 images, and six beamers projecting images and video. Also sound material, that was collected in the same manner, was being used during the projection.

...
Inside view of Tower of Babel project.

3

The centre of the location, a somewhat impoverished neighbourhood near the centre of Amsterdam, is a building dating from 1926, originally an antroposofic temple, that once served as a cinema for avant-garde movies, and is now being used as a library. The buildings surrounding it are, if not split up into apartments, being used as a local youth centre, a city archive and another library.

research directions -- media art

 In a recent symposium on the preservation of contemporary media art, a number of institutions presented their projects, ranging from more technical topics, such as the conservation of videotapes and the mass storage of digital material to the conceptual issues in capturing new media, the variety of media formats and the need to record and maintain meta data about the artworks and related information.

To get an idea what the phrase media art encompasses, have a look at the circumscription given in the WikiPedia: new media art is a generic term used to describe art related to, or created with, technology invented or made widely available since the mid-20th Century, including technology stemming from telecommunications, mass media and digital modes of delivery the artworks Below, the disciplines that belong to this form of art are listed, together with their entries in the WikiPedia, in an abbreviated form:

(new) media art


  • audio art -- no definition available
  • computer art -- any art in which computers played a role in production or display of the artwork.
  • digital art -- art created on a computer in digital (that is, binary) form.
  • electronic art -- entry to game producer, should be Leonardo.
  • generative art -- art or design generated, composed, or constructed through computer software algorithms, or similar mathematical or mechanical autonomous processes
  • hacktivism -- the writing of code, or otherwise manipulating bits, to promote political ideology
  • interactive art -- a piece of art that involves the spectator in some way.
  • internet art -- art or, more precisely, cultural production which uses the Internet as its primary medium and, more importantly, its subject.
  • performance art -- art where the actions of an individual or a group at a particular place and in a particular time, constitute the work.
  • robotic art -- page does not exist
  • software art -- is an intersection of two almost non-overlapping realms: software and art.
  • video art -- is a subset of artistic works which relies on "moving pictures" and is comprised of video and/or audio data.
  • video game art -- involves the use of a computer game for the creation of a digital artwork.
By the nature of the WikiPedia, to which every user can contribute entries, this list nor the defining entries are by any means authorative. Nevertheless, it does provide an overview and may serve as a starting point for further research.
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.