topical media & game development
navigation by query
Virtual worlds form (in itself) a rich repository
of multimedia information.
So, when working on the musical feature detector,
sketched in sectin 6.3, the thought occurred
to ask funding for a research project on information retrieval
in virtual worlds.
This project is called RIF, which stands for
RIF
Retrieval of Information in Virtual Worlds using Feature Detectors
For the RIF project, we decided to develop a small
multi-user community of our own, using the blaxxun
Community Server.
Then, during the development of our own virtual environment,
the question came upof how to present the results of
a query to the user.
The concept we came up with was navigation by query,
and in this section we will look at the prototype
we developed to explore this concept.
On the left is the 2D map of the third floor of CWI, on the right the model generated from it.
1
case study -- CWI
For our prototype, we took one of the worlds of our
virtual environment, the third floor of the CWI.
The reason for this is that we were (at the time)
doing our research there, and so there were plenty
locations of interest, such as the rooms of our collegues,
the printer room, and not to forget, the coffee corner.
the map
CWI
We started out by taking a map of the third floor,
and developed a model of it, using a tool developed by
a student, who needed such a tool for realizing his
game Out of the Dark.
the model
When dwelling around in (this part of) our virtual environment,
the user may pose (arbitrary) queries, for example
where is the coffee machine.
2
the query
C O F F E E
Remind, that after a few hours of research, coffee might be
needed to get fresh ideas!
navigation
explore
discover
As a result, the user is then so to speak taken by the hand
and led to one of the coffee machines that can be found
on the third floor.
In effect, with knowledge of the layout of the building
a viewpoint transformation is executed, in a tempo
that allows the user to
explore and discover
the (model of the) third floor of the CWI.
The idea is rather straightforward.
Some have asked us why navigation by query
might be useful.
Well, simply, it seems to offer an interesting alternative
to navigation by explicit interaction and navigation
in the formof a guided tour.
Our primary goal in developing the prototype, however,
was to see whether navigation by query is feasible,
and under what conditions.
3
information in virtual worlds
Developing the prototype has forced us to think
more explicitly about what information
is available in virtual worlds,
and (perhaps more importantly) how to gain access to it.
So the question we asked ourselves was
what are we searching for?
Now, in a virtual world, such as the ones built with VRML,
we can distinguish between the following types of information:
viewpoints, that is positions in the world from where
interesting things can be looked at or accessed in any other way;
areas of interest, where those intersting things are located;
objects, that may provide information or offer particular
kinds of functionality;
persons, that is other users thatare visiting the world; and even
text, which might be on billboards or slides.
what are we searching for?
types of information
- viewpoints
- areas of interest
- objects
- persons
- text
Some of this information is, so to speak, hard-wired
in the model and may be accessed anytime,
in some cases even by scanning the VRML file.
availability
- static -- always
- shared -- users
- dynamic -- runtime
- temporal -- events
- hidden -- actions
Other information, however,
is of a more dynamic nature,
since it might be due to the presence of multiple
users,
the execution of scripts, or events that happen
in response to user interaction.
Some information may even be explicitly hidden,
such as for example
the actions one should take in solving a puzzle
or playing a game.
scanning the scenegraph
- annotations
- node types
- textual content
- materials
- textures
- geometry
When the virtual world is loaded, all the information
(or at least most of it) is present in the so-called
scenegraph, the structure thatis built to render the world.
Using the software interface to access the scengraph
(which is usually browser-specific),
we can look for annotations, node types
and textual content
to extract information from the world.
This information may then be stored in a database,
and be reused later for other users and queries.
In principle, more advanced techniques could be used
to extract information from the materials used,
and even from textures and geometry.
presentation issues
In our prototype, we aimed at solving the question how
to present the results of a query, using navigation.
First of all, we had to
choose a metaphor
for navigation.
Dependent on the object of interest
a viewpoint can be selected.
choose a metaphor
get a viewpoint
- viewpoints
- areas of interest
- objects and persons
For a viewpoint, it is just that viewpoint.
For an area of interest, the viewpoint selected
must enable the user to view the area,
and when objects or persons are chosen, care must be taken
not to block the users' view by some obstacle.
Now answering a query then comes down to planning a suitable
route and apply a series of viewpoint transformations
along that route.
answer the query
- route planning
- viewpoint transformation
walking
Not surprisingly, the navigation metaphor we chose
was walking
as the preferred mode of viewpoint transformtions.
the prototype
The structure of the prototype is depicted in the figure below.
In realizing the prototype,we made the following (simplifying)
assumptions.
assumptions
- explicit annotation
- map for route planning
- keyword matching
We avoided a number of difficulties
by choosing for explicit annotations
(which indicate locations and areas of interest),
and by avoiding the intricacies of route planning
and advanced text processing.
The requirements laid down before hand just stated that
we would have a database and that we would avoid
superfluous user interface elements.
requirements
- database -- annotations & map
- 3D (pseudo-immersive) interface
Instead, we used control and input panels written in VRML,
in order to provide a 3D(pseudo-immersive) interface.
the system
Now, our assumptions may in principle be relaxed.
For example, annotation might be done incrementally by users
that visit the world or to
some extent even automatically, by using feature extractors.
Instead if explicit maps, we may dynamically create
maps based on users' navigation patterns.
And, instead of simple keyword matching, we may apply more
advanced text retrieval techniques.
relaxing the assumptions
- annotation -- incremental and/or automatic
- (explicit) maps -- based on user navigation
- (keyword) matching -- text retrieval
...
But this is left as future work.
Anyway,
we were satisfied
that we could state the following conclusions:
- navigation by query is feasible and may help users to find locations and objects
- determining suitable navigation routes without an explicitly defined map is hard
As is often the result with good research,
you solve one problem
and a number of other problems come up.
So, one of the questions that remains was: how can we improve
on navigation?
What additional navigation support can we provide?
future work
- shift in focus -- intelligent agents
- DLP + VRML -- distributed logic programming
Web Agent Support Program
no database, no walking
research directions -- extended user interfaces
Is desktop VR a suitable candidate as an
interface technology for multimedia information systems?
And if so, what needs to be done to apply this technology effectively?
At first sight, our vision of applying VR as an interface
to multimedia systems seems to be doomed to fail.
As Ben Schneiderman, in a keynote for the Web3D Symposium 2002,
observes:
3D GUI
Wishful thinking about the widespread adoption of
three-dimensional
interfaces has not helped spawn winning applications.
Success stories with
three-dimensional games do not translate into broad
acceptance of
head-tracking immersive virtual reality. To accelerate
adoption of advanced
interfaces, designers must understand their appeal and
performance benefits
as well as honestly identify their deficits. We need to
separate out the
features that make 3D useful and understand how they help
overcome the
challenges of dis-orientation during navigation and
distraction from
occlusion.
Ben Shneiderman
So, even if advanced (3D) user interfaces might be useful,
there are a number of questions to raise.
Again, following Ben Schneiderman:
Does spatial memory improve with 3D layouts? Is
it true that 3D
is more natural and easier to learn? Careful empirical
studies clarify why
modest aspects of 3D, such as shading for buttons and
overlapping of windows
are helpful, but 3D bar charts and directory structures are
not. 3D
sometimes pays off for medical imagery, chemical molecules,
and
architecture, but has yet to prove beneficial for performance
measures in
shopping or operating systems.
Ben Shneiderman
In particular, according to Schneiderman, we must beware of
tacky 3D, gadgets in 3D space that are superfluous
and only hindering the user to perform a task.
Well-spoken and based on adequate observations!
Nevertheless, at this stage, we should (in my opinion)
adopt a slightly more liberal attitude and explore in what ways
the presentation of (multimedia) information could be augmented
by using (desktop) VR.
But enough about augmentation.
Let's discuss technology, and investigate what is required
for the effective deployment of VR from
the point of view of intelligent agents!
(C) Æliens
04/09/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
