topical media & game development
1
a multimedia semantic web?
To finish this chapter, let's reflect on where we are
now with 'multimedia' on the web.
Due to refined compression schemes and standards
for authoring and delivery, we seemed to have made great
progress in realizing
networked multimedia.
But does this progress match what has been achieved for the
dominant media type of the web, that is text
or more precisely textual documents with markup?
web content
- 1st generation -- hand-coded HTML pages
- 2nd generation -- templates with content and style
- 3rd generation -- rich markup with metadata (XML)
Commonly, a distinction is made between successive generations
of web content, with the first generation being simple
hand-coded HTML pages.
The second generation may be characterized as HTML pages
that are generated on demand, for example by filling
in templates with contents retrieved from a database.
The third generation is envisaged to make use
of rich markup, using XML, that reflects the (semantic)
content of the document more directly,
possibly augmented with (semantic) meta-data
that describe the content in a way that allows
machines, for example search engines, to process it.
The great vision underlying the third generation
of web content is commonly refered to as the
the semantic web.
which enhances the functionality of
the current web by deploying knowledge representation
and inference technology from Artificial Intelligence,
using a technology known as the
Resource Description Framework
(RDF).
As phrased in
[CWI], the semantic web will bring
structure to the meaningful content of web pages,
thus allowing computer programs,such as search engines
and intelligent agents, to do their job more effectively.
For search engines this means more effective information retrieval,
and for agents better opportunities to provide meaningful
services.
A great vision indeed.
So where are we with multimedia?
As an example, take a shockwave or
flash presentation showing the various musea
in Amsterdam.
How would you attach meaning to it, so that it might
become an element of a semantic structure?
Perhaps you wonder what meaning could be attached to it?
That should not be too difficult to think of.
The (meta) information attached to such a presentation
should state (minimally) that the location is Amsterdam,
that the sites of interest are musea,
and (possibly) that the perspective is touristic.
In that way, when you search for touristic information
about musea in Amsterdam, your search engine should
have no trouble in selecting that presentation.
Now, the answer to the question how meaning can be attached
to a presentation is already given, namely by specifying
meta-information in some format
(of which the only requirement is that it is machine-processable).
For our shockwaveor flash
presentation we cannot dothis in a straightforward manner.
But for MPEG-4 encoded material, as well as
for SMIL content, such facilities
are readily available.
Should we then always duplicate our authoring effort
by providing (meta) information, on top
of the information that is already contained
in the presentation?
No, in some cases, we can also
rely to some extent on content-based search or
feature extraction,
as will be discussed in the following chapters.
2
Resource Description Framework -- the Dublin Core
The Resource Description Framework, as the W3C/RDF site informs us
integrates a variety of applications from library
catalogs and world-wide directories to syndication and aggregation of news,
software, and content to personal collections of music, photos,
and events using XML as an interchange syntax.
The RDF specifications provide, in addition a lightweight ontology system to support the
exchange of knowledge on the Web.
The Dublin Core Metadata Initiative
is an open forum engaged in the development of
interoperable online metadata standards that support a broad range of purposes and business models.
What exactly is meta-data?
As phrased in the RDF Primer
meta data
Metadata is data about data.
Specifically, the term refers to data used to identify, describe, or locate information resources,
whether these resources are physical or electronic. While structured metadata processed by computers
is relatively new, the basic concept of metadata has been used for many years in helping manage
and use large collections of information. Library card catalogs are a familiar example of such
metadata.
The Dublin Core proposes a small number of elements, to be used to give information
about a resource, such as an electronic document on the Web.
Consider the following example:
Dublin Core example
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:dcterms="http://purl.org/dc/terms/">
<rdf:Description rdf:about="http://www.dlib.org/dlib/may98/miller/05miller.html">
<dc:title>An Introduction to the Resource Description Framework</dc:title>
<dc:creator>Eric J. Miller</dc:creator>
<dc:description>The Resource Description Framework (RDF) is an
infrastructure that enables the encoding, exchange and reuse of
structured metadata. rdf is an application of xml that imposes needed
structural constraints to provide unambiguous methods of expressing
semantics. rdf additionally provides a means for publishing both
human-readable and machine-processable vocabularies designed to
encourage the reuse and extension of metadata semantics among
disparate information communities. the structural constraints rdf
imposes to support the consistent encoding and exchange of
standardized metadata provides for the interchangeability of separate
packages of metadata defined by different resource description
communities. </dc:description>
<dc:publisher>Corporation for National Research Initiatives</dc:publisher>
<dc:subject>
<rdf:Bag>
<rdf:li>machine-readable catalog record formats</rdf:li>
<rdf:li>applications of computer file organization and
access methods</rdf:li>
</rdf:Bag>
</dc:subject>
<dc:rights>Copyright © 1998 Eric Miller</dc:rights>
<dc:type>Electronic Document</dc:type>
<dc:format>text/html</dc:format>
<dc:language>en</dc:language>
<dcterms:isPartOf rdf:resource="http://www.dlib.org/dlib/may98/05contents.html"/>
</rdf:Description>
</rdf:RDF>
Items such as title, creator, subject and description,
actually all tags with the prefix dc,
belong to the Dublin Core and are used to give information about
the document, which incidentally concerns an introduction to the
Resource Description Framework.
The example also shows how rdf constructs can be used together
with the Dublin Core elements.
The prefixes rdf and dc are used to distinguish between
the distinct namespaces of respectively RDF and the Dublin Core.
The Dublin Core contains the following elements:
Dublin Core
- title -- name given to the resource
- creator -- entity primarily responsible for making the content of the resource
- subject -- topic of the content of the resource
- description -- an account of the content of the resource
- publisher -- entity responsible for making the resource available
- contributor -- entity responsible for making contributions to the content of the resource
- date -- date of an event in the lifecycle of the resource
- type -- nature or genre of the content of the resource
- format -- physical or digital manifestation of the resource
- identifier -- unambiguous reference to the resource within a given context
- source -- reference to a resource from which the present resource is derived
- language -- language of the intellectual content of the resource
- relation -- reference to a related resource
- coverage -- extent or scope of the content of the resource
- rights -- information about rights held in and over the resource
In section 10.3 we discuss an application in the domain
of cultural heritage, where the Dublin Core elements are used to
provide meta information about the information available for the conservation
of contemporary artworks.
3
research directions -- agents everywhere
The web is an incredibly rich resource of information.
Or, as phrased in [IR]:
information repository
The Web is becoming a universal repository of human knowledge
and culture, which has allowed unprecedented sharing of
ideas and information in a scale never seen before.
Now, the problem (as many of you can acknowledge) is to
get the information out of it.
Of course, part of the problem is that we often do
not know what we are looking for.
But even if we do know, it is generally not so easy
to find our way.
Again using the phrasing of [IR]:
browsing & navigation
To satisfy his information need,
the user might navigate the hyperspace of web links
searching for information of interest.
However, since the hyperspace is vast and almost unknown,
such a navigation task is usually inefficient.
The solution of the problem of getting lost in hyperspace
proposed in [IR] is information retrieval,
in other words query & search.
However, this may not so easily be accomplished.
As observed in [IR],
The main obstacle is the absence of a well-defined
data model for the Web, which implies that information
definition and structure is frequently of low quality.
Well, that is exactly the focus of the semanics web initiative,
and in particular of the Resource Description Framework discussed above.
Standardizing knowledge representation and reasoning about
web resources is certainly one (important) step.
Another issue, however, is how to support the user
in finding the proper resources and provide
the user with assistance in accomplishing his task
(even if this task is merely finding suitable entertainment).
What we need, in other words, is a unifying model
(encompassing both a data model and a model of computation)
that allows us to deal effectively with web resources,
including multimedia objects.
For such a model, we may look at another area
of research and development, namely intelligent agtents,
which provides us not only with a model
but also with a suitable metaphor and the technology,
based on and extending object-oriented technology,
to realize intelligent assistance, [OO].
For convenience, we make a distinction between two kinds
of agents, information agents
and presentation agents.
information agent
- gather information
- filter and select
Information agents are used to gather information.
In addition, they filter
the information and select those items that are relevant
for the user.
A key problem in developing information agents, however,
is to find a proper representation of what the user considers to be
relevant.
presentation agent
- access information
- find suitable mode of presentation
Complementary to the information agent is a
presentation agent (having access to the information gathered)
that displays the relevant information in a suitable way.
Such a presentation agent can have many forms.
To appetize your phantasy, you may look at the
vision of angelic guidance presented in [Angelic].
More concretely, my advice is to experiment with embodied
agents that may present information in rich media 3D.
In section [7-3], we will present
a framework for doing such experiments.
4
navigating information spaces
Having agents everywhere might change our perspective
on computing.
But, it may also become quite annoying to be bothered
by an agent each time that you try to interact with
with your computer (you know what I mean!).
However, as reported by Kristina Höök,
even annoyance can be instrumental
in keeping your attention to a particular task.
In one of her projects, the PERSONAS
project, which stands for
PERsonal and SOcial NAvigation through information spaceS
the use of agents commenting on people navigating information
space(s) is explored.
As a note, the plural form of spaces is mine,
to do justice to the plurality of
information spaces.
As explained on the PERSONAS web site,
which is listed with the acronyms,
the PERSONAS project aims at:
PERSONAS
investigating a new approach to navigation through information spaces, based on a personalised and social navigational paradigm.
The novel idea pursued in this project is to have agents
(Agneta and Frieda)
that are not helpful, but instead just give comments,
sometimes with humor, but sometimes ironic or even sarcastic comments
on the user's activities, in particular navigating
an information space or (plain)
web browsing.
As can be read on the PERSONAS web site:
Agneta & Frieda
The AGNETA & FRIDA system seeks to integrate web-browsing and narrative
into a joint mode. Below the browser window (on the desktop) are placed two
female characters, sitting in their livingroom chairs, watching the browser during the
session (more or less like watching television). Agneta and Frida (mother and
daughter) physically react, comment, make ironic remarks about and develop
stories around the information presented in the browser (primarily to each other),
but are also sensitive to what the navigator is doing and possible malfunctions of the
browser or server.
In one of her talks, Kristina Höök observed
that some users get really fed up with the comments
delivered by
Agneta and Frieda.
So, as a compromise, the level of interference
can be adjusted by the user,
dependent on the task at hand.
Agneta & Frieda
In this way they seek to attach emotional, comical or
anecdotal connotations to the information and happenings in the browsing session. Through an activity slider, the navigator can
decide on how active she wants the characters to be, depending on the purpose of the browsing session (serious information
seeking, wayfinding, exploration or entertainment browsing).
As you may gather, looking at the presentations accompanying
this introduction to multimedia
and [Dialogs],
I found the PERSONAS approach rather intriguing.
Actually, the PERSONAS approach is related
to the area of affective computing, see [Affective],
which is an altogether different story.
The
Agneta and Frieda
software is available for download at the
PERSONAS web site.
(C) Æliens
04/09/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
