topical media & game development
The concept of hypermedia
Due to the rapidly evolving software
and hardware technology,
including the mass storage provided by CD-ROM,
hypermedia have become accessible to the general
public.
Hypermedia provide the technology to manage large
amounts of information of various kinds
by means of the computer.
From a historical perspective,
hypermedia may be understood as the combination
of hypertext and (interactive) multimedia.
Hypertext was conceived shortly after
the second world war as a means by which to support
mechanically the storage and retrieval
of large amounts of information.
Interactive multimedia have been developed
as the result of integrating computing
with media (notably audio and video display).
Nodes and links -- presentation and navigation
The basic model underlying hypertext
may be characterized as (text)
nodes
and
links between nodes or components thereof.
For the user, the conception of
text
as consisting of nodes and links results
in what may be called
non-linear text.
According to
[Bush], who was the first to
put the idea into print, the interpretation
of text as a network is a natural way for
humans to think about and deal with information.
The importance of
hypertext
is thus not the idea of an associative organization
of information (that allows one to travel
from one node to another by traversing a link),
but rather the realization of {\em
machine-supported links}.
Machine-supported links give great flexibility
in dealing with text as a network.
For example, instead of walking to a library
to consult a reference on infections diseases
when reading about a virus, one may simply
traverse a link by clicking
on an item to have the material available
for consultation.
Multimedia comes into the picture when we,
once more,
generalize the notion of
text
to comprise ordinary text as well as
graphics, audio and video.
Combining the notions of hypertext and multimedia
results in what we know as
hypermedia,
allowing the user flexible access to a variety
of information.
Underlying the notion of hypermedia, however,
is the assumption that the information is
somehow
available in electronic form,
and that this information may be managed
by means of computation.
Hypermedia systems, in other words,
require
programmable media.
The restriction to plain text of the earlier
systems may be explained by the observation
that text processing and display technology
is relatively easy and cheap.
Graphics, and in particular audio and video,
require considerably more extensive resources
for storage and computation.
Hypertext -- nodes and links
- non-linear text -- As we may think
[Bush]
Machine-supported links -- text as a network
Hypermedia systems -- programmable media
- components -- text, graphics, audio, video
- links -- relations between components
- presentation -- structured display
slide: The concept of hypermedia
A hypermedia system may, in a general way,
be conceived of as consisting of a
collection of stored components
(containing text, graphics, audio or
video fragments),
a collection of links
(which allow the user to retrieve
components in an associative manner)
and
presentation facilities
(to enable a structured display of the
components selected by the user).
The ease of use of a hypermedia system
depends to a great extent upon the
organization of the material (components)
and the presentation of the associative
relations (links) between the components.
In particular, a powerful user interface is
required to allow the user
to navigate through the information
without losing orientation.
Direct manipulation interfaces
Graphical user interfaces
with buttons and menus,
and windows for the display of
text and graphics,
have become the standard
for a variety of applications,
including text processing, spreadsheets
and CAD systems.
Originally menu-based,
user interfaces have developed
into what are often called
object-oriented user interfaces,
or more appropriately
direct manipulation interfaces.
Direct manipulation interfaces
allow for the association of
actions with the items displayed.
For example, clicking on an icon
may
result
in opening the window associated
with that icon.
Conversely, clicking on a specific
part of the window may result
in iconifying the window into a small
graphical item on the screen.
The reason for calling such interfaces
object-oriented, obviously,
is the analogy with sending
a message or command to an object
considered as the target.
However, direct manipulation interfaces
need not necessarily be implemented
or designed in an object-oriented way,
although in my opinion they
had better be.
Direct manipulation interfaces -- object-oriented
Requirements
WYSIWYG
- documents -- open, close
- blocks -- select, copy, paste
- links --
slide: Direct manipulation interfaces
Hypermedia systems may be used
for browsing or authoring,
or a combination of both.
Browsing hypertext requires
an interface resembling the interface
of a WYSIWYG (What You See Is What
You Get) text processor,
in that it must support a readable
display of the material and the facility
to open and close documents.
In addition, it must enable the selection
of specific parts of the document,
which may be blocks or single words,
to activate links that lead to another
block.
See slide [12-direct].
Often, as in the Intermedia system
described in [Meyro86],
the user is allowed to copy parts
into a private notebook, containing
the material selected while
browsing the system.
See slide [12-navigation].
Naturally, the display facilities
of hypermedia as opposed to hypertext
systems must be far more encompassing.
These facilities and the issues involved
in authoring will be discussed in
section [hypermedia-model].
Navigation
One of the major problems in
hypermedia interface design is
the support for navigation.
Experiments indicate that users
have considerable trouble in maintaining
a sense of orientation, and often rely
on using a keyword index or the
traditional hierarchical structure
of the document instead of the associative access allowed by following links.
See [McK91].
Navigation -- maps
Intermedia
Retrieval by attributes -- selection
- block -- to apply filters
- link -- conditional traversal
To support navigation based
on content characteristics or
personal preference, the Intermedia
system allows for the creation of
webs.
A web is a particular collection
of documents and links, which is a
subset of the total collection
satisfying user-specific criteria,
such as field of interest
or level of aptitude.
Moreover, the Intermedia system
supports the graphical display
of the structure of a web by means of
maps, showing the blocks and the
links between them.
An example of a web is the private notebook mentioned above, which contains
the blocks and links satisfying the
condition selected.
To support the selection of (parts of)
documents and links, the Intermedia
system maintains attributes for both
blocks and links.
The value of block attributes determine
whether a particular block is selected
or filtered out,
and the value of link attributes determine
whether activating a link results
in traversal or not.
The latter facilities, by the way,
suggest that hypermedia offer
significantly more than merely
user interface gadgets.
My opinion in this respect,
following [Wood90],
is that hypermedia technology
may be regarded as a unifying paradigm
allowing the integration of
disparate elements in a common
presentation framework.
Hypermedia applications
Hypermedia technology may best be thought
of as technology that may be embedded
in a variety of applications.
This is reflected in the classification
of hypertext and hypermedia systems
given in [Conklin87], pictured in slide [12-classification].
Classification of hypermedia systems
[Conklin87]
- macro-literary systems -- publishing, reading, criticism
- problem exploration tools -- authoring, outlining, programming
- browsing systems -- teaching, references, information
- general hypermedia technology -- authoring, browsing, collaboration
Applications -- embedded
- CASE, decision support, catalogs
slide: Hypermedia systems -- classification
[Conklin87] makes a distinction between
macro-literary systems
which support publishing, reading
and criticism
and may serve as online annotated libraries,
problem exploration tools
which support authoring, outlining and programming
and may be characterized as
idea processors,
browsing systems
which support the display and content-based
retrieval of information
and may be used for teaching,
reference or online help,
and systems that explore general hypermedia
technology
which offer a combination of authoring,
browsing and collaboration facilities
(of which the Intermedia system is an example).
In general, for systems that primarily support browsing, the presentation facilities will be most important,
whereas for authoring systems
the functional or programming capabilities
will count most.
Applications for which hypermedia technology
is an essential ingredient encompass
CASE tools, decision support systems
and product catalogs.
All these applications have in common
that a large amount of material,
that may come in various media formats,
needs to be related, to allow the user
flexible access both in a structured
and associative manner.
Despite the differences due to the particular application domains,
we may discern
a common hypermedia model underlying
the associative structure of these
applications.
It is interesting to note that
when looking at successful applications
of object-oriented programming,
such as those reported in [Harmon93],
these almost invariably include
an embedded hypermedia component,
as well as functionality that
is derived from a rule-based reasoning
facility.
Structure versus presentation
Hypermedia technology supports the organization of a variety of material into an associative structure.
In addition, hierarchical structuring facilities may be supported.
The basic notions underlying the structuring
facilities of hypertext have been expressed
in the Dexter hypertext model.
See [Halasz94].
Hypertext model -- documents
Dexter
- components, links and anchors
Component
- content -- text, graphics, video, program
- attributes -- semantic description
- anchors -- links to other documents
- presentation -- display characteristics
Compound
- children -- subcomponents
slide: The Dexter hypertext reference model
The Dexter model explains the structure
of hypertext documents in terms
of components, links and
anchors.
The notion of anchors is introduced
to explain how to attach a link
to either a source or destination
component.
An anchor is the indication
of a particular spot in the document,
or rather a component thereof, usually
identified by some coordinate.
The Dexter model distinguishes
between three layers in a hypertext system,
namely the document layer
(defining the content and structure
of documents),
a storage layer
(handling the storage and retrieval of
components and links)
and a presentation layer
(handling the display of documents
and the interaction with the user).
A component,
which is a part of a document is
characterized by the following features:
content
(which may be text, graphics, audio,
video or even a program),
attributes
(which give a semantic description
that may be used for retrieval or
selective display),
anchors
(which identify the places to which
a link is attached), and
presentation characteristics
(which determine the display of the
component).
In addition, for compound components,
a feature children may be defined, for
storing the list of subcomponents.
Multimedia
The original Dexter hypertext model is
strongly oriented towards text,
despite the provision for multimedia
content.
Multimedia, in particular audio and video,
are intrinsically time-based
and require temporal primitives to synchronize
the presentation of material from different
sources.
In the CMIF multimedia model
described in [Hardman94], channels
(which are abstract output devices) are
introduced to allow for the specification
of abstract timing constraints
by means of synchronization arcs between channels.
Multimedia model -- composition
CMIF
- data block -- atomic component
- channel -- abstract output device
- synchronization arc -- specifying timing constraints
- event -- actual presentation
Views -- authoring
- structure -- sequential and parallel composition
- channels -- presentation
slide: The CMIF multimedia model
The notion of channels provides the abstraction
needed to separate the contents of a presentation
from the actual display characteristics.
For example, text may be output through
a text channel while, simultaneously,
video may be output through a video channel.
The screen layout and allocation of these
channels may be determined independently.
The actual presentation is determined by
events,
that may either arise as the result of a user
action or as the result of the activation of
a synchronization arc.
For example, a synchronization constraint
may specify that an audio fragment containing
speech must be started 10 seconds after the beginning
of a video sequence.
Then, after 10 seconds, the video channel will issue
an event that causes the audio channel to start
presenting its contents.
The CMIF model has been developed to allow for
portable multimedia documents.
In particular, the notion of channels allows
for a platform-independent characterization
of presentation characteristics and timing constraints.
An important characteristic of the model,
from an authoring perspective, is that it supports
a compositional approach to authoring,
since it allows us to compose a channel
(specifying a sequential composition of components)
with arbitrary many other channels, in parallel.
In [Hardman94], an extension of the CMIF multimedia
model is developed to incorporate the associative
structuring facilities defined by the Dexter
hypertext model.
Hypermedia model -- components
- contents -- data block
- attributes -- semantic information
- anchors -- $(id, value)
- presentation -- channel, duration, ...
Compound
- children -- $( component, start-time
- synchronization -- $()
slide: A hypermedia reference model
In the combined model, a single component
consists of contents
containing the actual data blocks of the component,
attributes
that specify semantic information,
a list of anchors
(each specifying a symbolic name and a value,
which in the case of an audio or video
fragment is its time measured from the start),
and presentation characteristics,
which include
the specification of a channel and the
duration of the component.
As in the Dexter model, compound components
may have children attributes,
specifying for each child a component and its
start-time,
and a number of synchronization arcs,
each
specifying
a source (component and anchor)
and destination (component and anchor).
Synchronization arcs may cross channel boundaries.
The reader is encouraged to specify a
more detailed object model, based on the outline
given above.
Evidently, the incorporation of a variety
of content types and display channels is a serious
challenge.
In particular, the notion of time-based active objects
will probably be difficult to handle.
For an abstract characterization of active time-based
(media) object, the reader is referred to [Nierstrasz].
On the notion of links -- active documents
Hypermedia documents are
often referred to as
hyperdocuments,
because of their associative structure
imposed by (hyper) links.
Links, in general, may be characterized
as a possibly conditional connection
between a source anchor
and destination anchor.
There has been an ongoing discussion as to
whether links must lead from byte to byte
or whether they must be defined
at some higher level.
On closer inspection, there appear
to be a number of choices
with respect to the kind of links
that may be supported.
See, for example, Halasz (1988, 1991).
Links -- anchors
- << source, conditions, destination >>
World Wide Web -- distributed hypermedia
- information retrieval -- HTML
Active documents
slide: Links and activation
Perhaps the most important
distinction is that between
hard-wired links that act as a goto
in programming languages and
what may be called virtual links,
the destination of which is computed
when activating the source anchor.
This distinction is exemplified
in the World Wide Web (WWW)
distributed hypermedia system,
which was
initiated
by CERN (Switzerland).
The World Wide Web supports HTML
(HyperText Markup Language), a semi-official
hypermedia markup language in the SGML tradition.
The World Wide Web allows the user
to locate and retrieve documents
worldwide across the Internet.
However, a document may either be
stored physically somewhere on a
node in the network or may be generated
on the fly by some information retrieval
server producing HTML output.
The production of HTML documents
by some external program
as the result of accessing a link
somehow blurs the distinction between
programs and documents.
One step further in this direction is to allow
documents, whether stored or generated, to contain
embedded code that is executed
when the document is viewed.
Such documents may be characterized
as active documents.
Active documents are, for example,
proposed as an extension to the
MIME (Multipurpose Internet Mail)
standard, to allow
for `live mail'.
Embedding code in documents would allow
for synchronization facilities that
are currently beyond the scope of HTML
and MIME.
However, a standard (in development)
that provides features for synchronization
is the HyTime markup language,
which is another offspring of the
SGML family.
Summarizing,
active documents
are documents that result in
programmed actions
by being displayed.
From a systems programming point of view,
we may regard active documents as
program scripts that are executed
by a (hypermedia) interpreter.
(A well-know example of a
script-based hypermedia programming language is HyperTalk.)
Hypermedia programming, using scripts,
relies intrinsically on an event-driven
control mechanism.
In the following section, we will
explore how we may combine script-based
(event-driven) programming
with (more traditional) object-oriented
development (in C++).
(C) Æliens
04/09/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
