Following NOAG-i 2001-2005,
the research proposed is related to the following areas:
NOAG-i 2001-2005

In the online version links are provided to information about these
areas (in Dutch) taken from NOAG-i 1997.
As an estimate in percentages, the contribution of the respective fields
might be summarized as: 50 (IS), 30 (MM), 20 (SE).
NWO subdisciplines Informatica (1996)
Based on the rather outdated classification of
(sub)disciplines of Computer Science, our research
could be classified as belonging (to some extent) to:
(1.2) distributed systems;
(2.3) information retrieval and presentation;
(2.7) expert systems;
(3.1) software architecture;
(3.3) object technology; and
(3.9) interoperability.
4) Composition of the Research Team:
name | expertise | affiliation | hours/week |
Dr. A. Eliens | multimedia | VU/IMSE | 8 (coordination) |
Dr. Z. Huang | agents | VU/AI | 8 (WASP/RIF) |
Prof. dr. J.C. van Vliet | architecture | VU/IMSE | 2 (promotor) |
Prof. dr. J. Treur | intelligent systems | VU/AI | 2 (promotor) |
Drs. C. Visser | DLP | VU | 18 (programmer) |
vacature | AI/CS | VU | 32-36 (OIO) |

The research will be executed within the intelligent multimedia
group at VU, under the supervision of
Dr. A. Eliëns and dr. Z. Huang.
Drs. C. Visser will provide programming support during the full four years
of the project, of which two years will be covered by the requested funding
(see 10).
5) Research School: SIKS
The partners are members of SIKS
(the Dutch Research School for Information and Knowledge Systems,
www.siks.nl).
6) Description of the Proposed Research
research
The AVID project aims at a unified framework for agents and
virtual environments, built on the distributed logic programming
language DLP.

Before stating our main research goals more explicitly
a brief characterization will be given of the work that led
to this proposal.
At the end of this section, it will be sketched how this work
fits in with our educational activities and (intelligent)
multimedia research.
background
The AVID project is a continuation of research done in
two NWO projects:

In the online version, links are given to more detailed
information about these projects.
Although the WASP proposal was written more than three years
before the RIF proposal, dr. Z. Huang started as a post-doc on the
WASP project when the RIF project ran already for more than
six months.
Since we then felt that the WASP project proposal was slightly
outdated, we made an effort to merge the WASP and
RIF projects, by focussing on agents in 3D virtual
environments which resulted in a paper presenting
a taxonomy of Web agents, [Taxonomy].
However, after about a year the continuity of the RIF project
was endangered due to a mutation of personell at CWI.
So, NWO was asked for permission to utilize the RIF funds
for prolonging the WASP project.
This was granted, provided that the research goals of the RIF
project were sufficiently covered within WASP.
In the RIF project we used the blaxxun
Community Server and VRML
to realize information retrieval and delivery im multi-user 3D
environments. See [Navigation].
Agents were then conceived as extensions on the server-side
using blaxxun's native agents enriched with embedded logic.
However, at the same time that the RIF project funds were
transferred to WASP, the first prototype of DLP in Java
became available, and we decided, somewhat radically, to drop
the more low-level blaxxun technology in favor of a unified
approach in DLP.
Thus, we extended DLP with primitives for manipulating VRML
worlds, using the Java External Authoring Interface that is
part of VRML. The DLP+VRML framework proved
to be surprisingly effective, as testified by the following references:
DLP+VRML

Now, the language DLP itself has quite a long history, [DLP].
It has also been described in [OO].
As a language, DLP offers an object-oriented extension
of (traditional, Edinburgh-style) Prolog, with multi-threaded
objects, non-logical instance or state variables,
communication by rendex-vous and (distributed) backtracking.
After preliminary prototypes in C++,
we focussed on an implementation in Java, to be able to use
DLP for Web programming, [Web].
The first Java implementation became available relatively late,
but just in time to create the DLP+VRML extension when needed.
As concerns the acceptance of our approach within the Web3D
community, we wish to point to the acceptance of our [Community]
paper for the highly competetive international Web3D Conference
2002 (acceptance: 1 out of 13), and the tutorial that we will be giving
there based on the material of our Multimedia Authoring II course,
see [Intelligent].
the research
Given the background of our research, we can state
our research goal quite simply:
problem
- starting point: Java-based DLP+VRML framework
- end point: fully integrated agent and virtual environment framework in DLP

So, what needs to be done is to develop more fully
the concepts and models that appeared to be fruitful
in our initial explorations of deploying
the DLP+VRML framework for the realization of agents in 3D
(multi-user) environments.
As a first characterization of how to achieve this we
may indicate the following steps:
steps
- realization of agent taxonomy, as a hierarchy of agent objects;
- tight integration of DLP with Web3D (VRML), that is an integration of the object model of VRML in DLP to allow for the dynamic creation of (shared) Web3D content;
- realization of extensible Agent Communication Language (ACL) in DLP;
- application of ACL to realize multi-user virtual environments in DLP+VRML;
- target applications.

Below, we will indicate in more detail the research topics
that need to be addressed to take these steps.
Target applications will result from other research in our
group in the area of persuasive technology as for example [Dialogs],
from cooperation with other research group at CWI as in [Context],
and from student projects.
As outlined below both VRML and DLP+VRML technology is part
of the students' curriculum and there have already been two master
thesis projects in the area of virtual environments.
Target application areas may include, but are not limited to,
e-commerce, edutainment , agent-aided 3D design and social marketing.
With regard to the integration of DLP and VRML,
let me explain that the approach we have chosen is one that
allows for treating an arbitrary VRML world as a black box,
with handles to which events can be sent or from which events
can be received.
Events are the primary mechanism of scenegraph-based 3D formats,
such as VRML and Java3D, for controlling dynamic, behavioral
properties.
A tight integration does not mean giving up this clean
separation, which allows for using existing VRML worlds,
but instead to allow also for the dynamic creation of
3D content from within DLP.
Also, the manipulation of existing
or dynamically created 3D content, as for example the animation
of an avatar (the humanoid presentation of a user or agent)
should be made more easy by offering suitable primitives in DLP.
For a further explanation of our approach and how it is related
to other approaches, see the paragraph on related work below.
research topics
Our research goals may be stated more succinctly
as follows (the acronyms will be explained below):
research goals
- suitably rich model of Web Agents and corresponding realization (WAF)
- sufficiently expressive Agent Communication Language, to be used for interaction between agents, as well as for the realization of shared objects and events (MU-ACL)
- framework for developing multi-user, multi-agent applications in Rich Media and Web3D (RM3D-DLP).

To realize our research goals, we have to address a variety
of topics.
topics
- representation and presentation model of agents
- ACL syntax and semantics
- multi-user virtual environments
- applications

representation and presentation model of agents
Our agent model is based on an extension of the BDI cognitive
model with sensors and effectors, that allow agents to
perceive events that occur in the virtual world as well
as to operate on the virtual world by sending events.
In this way agents can not only control the 3D world
they live in, but also their own presentational characteristics,
that is their appearance and attributes they possess.
In other words, we need to augment the agent model with
characteristics of a virtual actor for effectively deploying
the agent in 3D virtual environments.
ACL syntax and semantics
In our multi-user soccer prototype [Community],
the agents (footballers and goalkeepers) did
not communicate amongst themselves.
Rather they took notice of the state of the game,
that is their own position and the position of the ball,
the opposite goal, and their fellow players.
However, to allow for multiple users to join the game,
we used an invisible agent that communicates the game
information to the (invisible) agents of other users,
so that these agents may update the actual appearance
of the game as presented to the user.
In this way we realized a shared environment using
communication between agents based on the model of agents
that we also used for the visible (footballing) agents.
Clearly, we must more fully develop the syntax and semantics
of such an ACL to allow for inter-agent communication
as well as shared objects and events in a more generic fashion.
See [ACL] for an overview of Agent Communication Languages.
We also need to investigate the proper architecture to realize
a distributed multi-agent system.
In the current prototype, we used mediator DLP objects
on the server, which were needed to deal with the security
restrictions imposed by the Java-based architecture of
our DLP+VRML framework.
multi-user virtual environments
Current systems for creating multi-user virtual environments
are rather complex, and require a variety of authoring tools
and programming languages, ranging over C, Java, Javascript,
and proprietary scripting languages.
Our explorations indicate that we can develop a unified
framework for dealing with the various aspects
of rich media 3D environments, which include
3D models (static and dynanic),
intelligent agent avatars (WAF),
shared objecs and events (MU-ACL).
For an early account of what the architecture of multi-user environments
in VRML look like,
see [Environments].
applications
As indicated before, there is a large variety of potential
applications for RM3D-DLP technology, including
information retrieval and delivery,
presentation and navigation agents,
user assistance.
Assuming that such applications require intensive knowledge
programming,
they may clearly benefit from our approach.
related work
Our work is related to or even shows resemblance with
a number of other projects.
We will discuss:
related work
- [Logic] -- logic programming for the internet
- [Scripts] -- merging VRML with logic scripts
- [Jinni] -- client-side logic with VRML
- [DIVE] -- distributed interactive virtual environment
- [Parlevink] -- agents in VRML worlds
- [Alice] -- 3D interactive programming environment

An overview of logic programming languages for the
Internet is given in Andrew Davison's [Logic].
Due to its late arrival on the Web scene, DLP is not mentioned
in this overview. It was, however, mentioned in a previous
overview of the same author [Survey], in the category
of object-oriented extensions.
For your information, the principal investigator organized
a workshop at WWW6 in 1997 entitled
Logic Programming for the Web, [Workshop].
There have been a number of attempts to improve on
the scripting capabilities of VRML.
One example of interest, from our perspective,
is the extension proposed in [Script],
which allows for logic programming in VRML script nodes.
This approach affects the VRML files themselves, and hence
the VRML97 standard.
As a drawback, existing VRML content cannot be (re)used
without modification.
More importntly, however, such an approach does not allow
for migrating to newer or other scenegraph-based formats
such as X3D or Java3D.
In effect, Jinni (developed by Paul Tarau)
was a source of inspiration for our
first DLP+VRML prototype, [Jinni].
Nevertheless, our DLP language is semantically more suited
to deal with the complexities of a more tight coupling
of VRML and logic programming.
In particular, multi-threaded objects in DLP allow
for receiving events in an asynchronous manner,
which is impossible to do in ordinary sequential Prolog.
Distributed applications in Jinni are possible using
a Linda-style shared tuple space.
In our approach we rely on rendez-vous communication
and server-side mediator objects.
As the soccer prototype shows,
it is rather easy to create a shared repository of
state information using agent communication primitives
in DLP.
The DIVE project at the Swedish Institute of Computer Science
is an example of a complex heterogeneous system for
developing multi-user virtual communities, [DIVE].
It is offers support for a variety of graphic
formats, including VRML,
and allows for scripting behaviors and interactions
using Tcl.
DIVE applications and activities include virtual battlefields, spatial models of interaction, virtual agents, real-world robot control
and multi-modal interaction.
DIVE is supported on a variety of platforms.
The first DIVE version appeared in 1991.
Our approach clearly builds on later developments, in particular
Java, to achieve cross-platform portability.
In contrast to the heterogeneous nature of the DIVE platform,
we provide a unified language framework as a programmatic
interface for the creation of 3D content and behavioral properties.
The Dutch Parlevink project, at the University of Twente,
has developed a virtual theater, which demonstrates
that agents may usefully be employed in (VRML-based)
virtual environments, [Parlevink]
The focus of the Parlevink project, however, is more
on cognitive models and natural language processing.
In contrast, our goal is to improve virtual environment
technology itself, by
providing a unified framework for developing agents
and virtual environments.
Finally, we cannot conclude this brief overview of
related work without mentioning Alice,
a 3D interactive programming environment for virtual worlds,
developed at Carnegie-Mellon University, [Alice].
One interesting feature of Alice is that it allows for programming
generic behavioral properties in the object-oriented scripting
language Python.
Nevertheless, we believe that due to the knowledge-intensive
nature of the programming tasks required for
developing real applications in rich media virtual
environments, a logic-based language such as DLP
will in the long end be much more effective.
embedding in education: focus on multimedia
focus on multimedia
The research will have a clear impact on the educational activities
for the specialisation multimedia, in particular the Multimedia
Authoring II course, as indicated below.
It is to be expected that student projects will deploy the technology
developed within the AVID project.
In the first year students
start with a general Introduction to Multimedia.
This course centers around three themes:
the convergence between media, platforms and delivery technology,
the availability of
broadband communication and its impact on the development of
standards such as MPEG-4, and multimedia information retrieval
as an essential ingredient of the growing multimedia information repository
on the Web.

There are two follow-up courses, which are given in respectively the
second and third year:
courses

The first of these courses deals with the technology for creating 3D scenes and
worlds, whereas the second is
more focused on providing intelligent services in virtual environments.
Students use the DLP+VRML framework for their assignment in the second course. See [Intelligent].
embedding in research: (intelligent) multimedia
SIKS report (2001)
Over the past six years, our research efforts
have focussed on developing models and software
architectures for multimedia and hypermedia
applications.
(A full version of this description, including publications, is available online in the SIKS report (2001)).
This research has resulted in two Ph.D. theses:
- 10/4/2001 -- Structured hypermedia -- a matter of style
- 8/5/2001 -- Diva: Architectural Perspectives on Information Visualization

For both theses, prof. dr. J.C. van Vliet was promotor,
and the principal investigator had daily supervision and acted as co-promotor.
The hypermedia work was done in collaboration with
dr. L .Hardman and dr L. Rutledge from the CWI Multimedia
group.
This cooperation resulted in the formalization of the
Amsterdam Hypermedia Model, an extension of the Dexter
Hypertext Reference Model.
The hypermedia project also resulted in a software
framework for developing web-based hypermedia applications,
the hush library and its music and video extensions.
The visualisation project concerned the use of
animations and visualisation to display business process
simulation results in a hypermedia context.
During the project the focus shifted towards visualisation,
in particular business visualisation.
Also, explorations were done to investigate interactive
visualisation in 3D.
Our current research efforts are directed towards
developing a high-level platform for rich media 3D virtual
environments.
Our goal is to study aspects of the deployment and
architecture of virtual environments as an interface to multimedia information systems.
intelligent multimedia
The intelligent multimedia research theme may
be regarded as continuing the hypermedia and visualisation
projects described before.
Our efforts are directed towards realizing the technology
needed for developing intelligent multimedia applications.
In particular, we aim for developing demonstrators in the area
of persuasive technology, and thus explore design methods
and patterns for the realization of emotionally charged virtual worlds.
This work is being done in cooperation with dr. C. Dormann
and dr. Z. Ruttkay from CWI.

cooperation
The list below summarizes the cooperations that are due to
either the position of the research group within the department
or contacts resulting from previous research, as sketched above.
cooperation
- multi-agent research group at VU/AI (dr. C. Jonker)
- HCI at VU/IMSE (dr. G. van der Veer)
- persuasive technology VU/IMSE (dr. C. Dormann)
- Facial and Avatar Animation Group at CWI & Epictoid (dr. Z.Ruttkay)
- Multimedia Authoring Group at CWI (dr. J. van Ossenbruggen & dr. L. Rutledge)

7) Work Programme
With reference the our discussion of the steps that must be
taken to realize our research goals and the
topics that need to be addressed we suffice by giving
a brief summary of the deliverables that we expect to produce
during the four years of the project.
deliverables:
- year 1: realization of Web Agent Framework (WAF)
- year 2: realization of ACL for multi-user environments (MU-ACL)
- year 3: integration of WAF and MU-ACL
- year 4: thesis

8) Instrumentation
Not relevant.
9) Literature
The online version has links to online copies of most
of the cited references.
- Angelic
- Wolfgang Broll, Leonie Shäfer, Tobias Höllerer, Doug Bowman,
Interface with Angels: the future of VR and AR interfaces, IEEE Computer Graphics, November/December 2001, pp. 14-17
- Web3D
- Web3D Consortium,
VRML97 Standard, http://www.web3d.org/fs_specifications.htm
- Java3D
- Javasoft,
Java3D API, java.sun.com/products/java-media/3D
- Taxonomy
- Zhisheng Huang, Anton Eliëns, Alex van Ballegooij, Paul De Bra,
A Taxonomy of Web Agents, IEEE Proceedings of the First International Workshop on Web Agent Systems and Applications (WASA '2000), 2000.
- Navigation
- A. van Ballegooij and A. Eliëns,
Navigation by Query in Virtual Worlds, Web3D 2001 Conference, Paderborn, Germany, 19-22 Feb 2001
- Architecture
- Zhisheng Huang, Anton Eliëns, and Paul De Bra,
An Architecture for Web Agents, Proceedings of the Conference EUROMEDIA 2001, 2001.
- Avatars
- Zhisheng Huang, Anton Eliëns, and Cees Visser,
Programmability of Intelligent Agent Avatars, Proceedings of the Agent'01 Workshop on Embodied Agents, June 2001, Montreal, Canada
- Community
- Zhisheng Huang, Anton Eliëns, and Cees Visser,
3D Agent-based Virtual Communities, Proc. Web3D 2002 Conference, Tempe Arizona
- DLP
- A. Eliëns,
DLP -- A language for Distributed Logic Programming, Wiley, 1992
- OO
- A. Eliëns ,
Principles of Object-Oriented Software Development (2nd edn), Addison-Wesley (2000)
- Web
- Cees Visser, Anton Eliëns,
A High-Level Symbolic Language for Distributed Web Programming, Internet Computing 2000, June 26-29, Las Vegas
- Intelligent
- Zhisheng Huang, Anton Eliëns, Cees Visser,
Intelligent Multimedia Technology: An Approach to Combine Agent Technologies with Multimedia, in preparation
- Dialogs
- C. Dormann & A. Eliëns,
Exploring the design space for emotive dialogues, submitted to: Third Int Conf on Emotion and Design
- Context
- Lloyd Rutledge, Alex van Ballegooij and Anton Eliëns,
Virtual Context - relating paintings to their subject, Culture Track of WWW9 in Amsterdam, The Netherlands, Tuesday, May 16th, 2000
- ACL
- Yannis Labrou, Tim Finin and Yung Peng,
The current landscape of Agent Communication Languages, IEEE Intelligent Systems, Vol. 14, no. 2, 1999
- Environments
- Wolfgang Broll,
VRML and the Web: A basis for Multi-user Virtual Environments on the Internet, In Proceedings of WebNet96, World Conference of the Web Society (San Francisco, CA, Oct. 16-19, 1996), H. Maurer (ed.), AACE, Charlottesville, VA (1996), 51-56.
- Logic
- A. Davison,
Logic Programming Languages for the Internet, Chapter in Computational Logic: From Logic Programming into the Future, Antonis Kakas, Fariba Sadri (eds.), Springer Verlag (2001). available from: fivedots.coe.psu.ac.th/~ad
- Scripts
- A. Davison,
Enhancing VRML97 Scripting, Euromedia'2001, Valencia, Spain, April 18-20. available from: fivedots.coe.psu.ac.th/~ad
- Jinni
- P. Tarau,
Jinni: Intelligent Mobile Agent Programming at the Intersection of Java and Prolog, Proc. of PAAM'99, London, UK, April, see also www.binnetcorp.com/Jinni
- DIVE
- Distributed Interactive Virtual Environment,
Interactive Colloborative Environments, SICS, www.sics.se/dive
- Parlevink
- University of Twente,
Language Engineering Group, parlevink.cs.utwente.nl
- Alice
- Carnegie-Mellon University,
3D Programming Environment, www.alice.org
- Survey
- A. Davison,
A survey of Logic Programming-based Object-Oriented Languages, In Research Directions on Concurrent Object-Oriented programming, G. Agha, P. Wegner, A. Yonezawa (eds.), MIT Press (1993)
- Workshop
- A. Eliëns (chair),
Logic Programming and the Web, www.cs.vu.nl/~eliens/online/workshops/www6
10) Requested Budget
personell | period | euro |
OIO | 4 year | 129.879 |
programmer | 2 year | 107.248 |
| | 241.665 |
Remarks:
- the amount of euro 129.879 includes the benchfee for travel expenses and other support.
- the VU will provide an additional 2 years of programmer support.
[]
readme
titles
system
demo
dlp
vr
slides
resources