towards an aesthetics for interaction
learning objectives
After reading this chapter you should have an understanding
of the model underlying game playing, and the role
of narratives in interaction.
Furthermore, you might have an idea of how to define aesthetic meaning
in a cultural context, and apply your understanding to the creative
development of meaningful interactive systems.
As in music, the meaning of interactive applications is
determined, not only by its sensory appearance,
but to a high extent by the structure and functionality of the application.
This observation may, also, explain, why narratives become more and more important
in current video games, namely in providing a meaningful context
for possible user actions.
In this chapter, we take an interactive game-model
extended with narrative functionality as a starting point
to explore the aesthetics of interactive applications.
In section 12.1, we will introduce a model for interactive video games, and in section 12.2
we will present a variety of rules for the construction of narratives
in a game context.
Finally, in section 12.3, we will characterize the notion of meaning
from a traditional semiotics perspective, which we will then apply
in the context of games and interactive multimedia applications.
game theory
- system -- (formal) set of rules
- relation -- between player and game (affectionate)
- context -- negotiable relation with 'real world'
classic game (reference) model
- rules -- formal system
- outcome -- variable and quantifiable
- value -- different valorisation assignments
- effort -- in order to influence the outcome
- attachment -- emotionally attached to outcome (hooked)
- consequences -- optional and negotiable (profit?)
rules vs fiction
game fiction is ambiguous, optional and imagined by
the player in uncontrollable and unpredictable ways, but the emphasis
on fictional worlds may be the strongest innovation of the video game.
theory of interaction
are games relevant for a theory of interaction?
effective service management game(s)
- rules -- service management protocols
- outcome -- learning process
- value -- intellectual satisfaction
- effort -- study procedures
- attachment -- corporate identity
- consequences -- job qualification
additional criteria
- scenario(s) - problem solving in service management
- reward(s) - service level agreements
game play
... structure of interaction with game system and other player(s)
component framework
- holistic -- playing games as an undividable activity
- boundary -- limit the activities of people playing games
- temporal -- describe the flow of the game (interaction)
- structural -- physical and logical elements of the game system
pattern(s)
- resource management -- resource types, control, progress
- communication and presentation -- information, indicators
- actions and events -- control, rewards and penalties
- narrative structures and immersion -- evaluation, control, characters
- social interaction -- competition, collaboration, activities
- mastery and balancing -- planning, tradeoffs
- meta games and learning -- replayability, learning curve(s)
intimate media object(s)
- glow tags -- a subtle way to trigger the person who has placed it or who sees it
- living scrap book -- to capture and collect information and media digitally
- picture ball -- as an object of decoration and a focus for storytelling
- lonely planet listener -- enabling people to listen to a real time connection to another place
intimate media experience(s)
- sensorial -- experience is visual, audible, tactile, olfaric
- personalized -- objects embody meaning and memories
- analogue -- people relate to physical objects
- enhancement -- people already have extensive intimate media collections
- serendipity -- it supports unstructured and flexible usage
- longevity -- objects may exist over generations
experience as meaning
- experience occurs during the interaction between the user(s) and the
interactive system(s) in the lived environment
- designers convey meaning (consciously or unconsciously)
through the appearance, interaction and function of the system
- user(s) construct a coherent whole that is a combination of
sensual, cognitive, emotional and practical forms of experience
film as art
by still being read, the little treatise seems to prove that in spite of all the changes that have
taken place in their form, content and function,
films are still most genuinly effective when they rely on the basic properties
of the visual medium.
illusion
... in film or theatre, so long as the essentials of any event are shown,
the illusion takes place
patterns of light
... we can perceive objects and events as living and at the same time imaginary,
as real objects and as simple patterns of light on the
projection screen, and it is this fact that makes film art possible.
frames of reference
it is one of the most important formal qualities of film
that every object that is reproduced appears simultaneously in two
entirely different frames of reference, namely the two-dimensional
and the three-dimensional, and that as one identical object it fulfills
two different functions in the two contexts.
principles of montage
- cutting -- unit length, whole scenes, cuts within scenes
- time relations -- synchronized, before/after, neutral
- space relations -- same place (different time), different place
- subject matter -- similarity and/or contrast
film technique
- camera -- position, focus, movement
- transitions -- fading in/out, dissolving, stills
- arrangement -- light/shade, color, sound
cinematographic motion
- movement of objects
- effect of perspective
- motion of camera
- montage of scenes
narrative implication(s)
- objects -- the items in the image
- vectors -- (imaginary) lines suggesting interaction
- gaze -- inward (offer) / outward (demand)
composition
composition, ..., relates the representational
and interactive meanings of the picture to eachother,
through three interrelated systems.
representation(s)
- information value -- left/right, top/bottom, centre/margin
- salience -- foreground/background, relative size, contrast
- framing -- connecting or dissolving lines
information value
- left/right -- given versus new
- top/bottom -- ideal versus real
- centre/margin -- important versus marginal
scientific context
- mathematics -- matrix algebra, transforms
- physics -- game physic, particle systems
- computer science -- technological infra-structure
- information theory -- compression and delivery
- media theory -- history of communication
- semiotics -- theory of meaning
societal context
- cultural heritage -- digital dissemination of art
- education & communication -- presentation of concepts and examples
technological context
- modelling -- objects, characters
- interaction -- game programming
- architecture -- game engine design
- rendering -- programming the graphics hardware
creative context
- visual design -- style, models and attributes
- story telling -- narrative structure
learning/meaning
- actionary level -- action and movements
- sensory/iconic -- images and impressions
- symbolic -- language and mathematics
basic geometrical shapes
... basic geometrical shapes have always been a source
of fascination, even of religious awe.
And our scientific age is no exception.
nervous system
(basic geometrical shapes) have been thought to have the power
to directly affect our nervous system, for instance by the
constructivist artist Gabo: the emotional force of an absolute shape is unique
and not replaceable by any other means ...
semiotics -- a theory of meaning
- signifier -- sign/symbol
- signified -- what is referred to
semiotic modes
... is the move from the verbal to the visual
a loss, or a gain?
complexity
... it has to be handled visually,
because the verbal is no longer adequate?
multimedia
the multi-modality of written texts has, by and large, been ignored,
whether in educational contexts, in linguistic theorizing,
or in popular common sense.
Today, in the age of multimedia, it can suddenly be perceived
again.
quotes
- myth of transparency -- visual communication is always coded!
- literacy -- standards for semiotic order
- semiotic modes -- text, visual, auditive, ...
- computer technology -- central to semiotic landscape
- semiotic activities -- production, transformation, development
semiotic landscape
the place of visual communication in a given society can
only be understood in the context of, on the one hand,
the range of forms or modes of public communication available
in that society, and, on the other hand, their uses and valuations.
sonic act(s)
what is the meaning of meaning in apparently
meaningless expressions
modality
one of the crucial issues in communication is the question of
the reliability of messages.
Is what we see or hear true, factual, real, or is it a lie,
a fiction, something outside reality?
To some extent the form of the message itself suggests the answer.
coding orientation
- technical/scientific -- effectiveness, blueprint
- sensory -- pleasure principle is dominant
- abstract -- used by socia-cultural elite
- naturalistic -- dominant common sense paradigm of realism
aesthetics of shock
it is within the realm of probability that the shock,
which Walter Benjamin diagnosed as being film's aesthetic innovation,
will undergo renewal and intensification with far more sophisticated means.
voyeurism
the most obvious symptom of this loss of distance will be a voyeuristic,
dissecting penetration of representations of objects and bodies.
TV
for the first time in the history of man's striving for understanding,
simultaneity can be experienced as such, not merely
translated as a succession in time.
sensory stimulation
although the new victory over time and space represents an
impressive enrichment of the perceptual world, it also favors
a cult of sensory stimulation which is characteristic of the
cultural attitude of our time.
direct experience
proud of our inventions -- photography, film, radio, ... --
we praise the educational virtues of direct experience.
communication
when communication can be achieved by pointing with the finger, however,
the mouth grows silent, the writing hand stops and the mind shrinks.
channels
... the decisive questions remain: who controls the channels,
who distributes right of access, and who exercises economic and political
authority over the networks?
visions
... the history of technological visions is the history of our dreams,
our vagaries and our errors.
Media utopias fluctuate, often occurring in a magical or occult ambience.
synopsis course(s)
... the curriculum should emphasise basic principles,
and to the extent possible employ open standards and open source.
Practical assignments must be centered on local culture,
and stimulate the young talent to explore innovative
applications for cultural heritage, serious games and artistic
expression.
where, what & why
- where -- Ethiopia & VU
- what -- introduction multimedia
- why -- to develop curriculum
environment
- low end computers -- windows, linux
- elementary skills -- programming, design
assumptions
- open source -- flex 2 sdk, Delta3D
- open standards -- XML, X3D
- basic principles -- exploratory development
targets
- local -- present local cultural heritage
- serious -- develop serious game(s)
- benefits -- promote local culture and commerce
presentation
- philosophy -- pathos, ethos, logos
- trailer -- drama, apocalyptic, appeal to player
- climate star -- scientific issues & game play
- game development -- architecture and project plan
phrase(s)
- art -- rethorics of the material
- technology -- solving problem(s)
- science -- establish a theory
dimensions of aesthetic awareness
- spatial -- topological relations, layout of image
- temporal -- order, rhythm, structure
- dynamic -- interaction, reflection, involvement
concepts
technology
projects & further reading
As a project, explore the ways narratives may be constructed from
a collection of images.
Deploy the various editing facilities for providing
flashbacks, flashforwards, and other (temporal) relations
within storytelling.
You may implement this using flash, VRML, or even try to embed
such a narration facility in a game level developed with the
Delta3D or the
Source SDK.
For further reading I suggest you to take a look at
more theoretical material from media theory, such as [Remediation].
Also there is a large collection of books from MIT Media Press
that is of relevance for our new visual culture.
- einzelganger -- walking man of Alberto Giacommeti,
taken from an aanouncement of the Ives Ensemble, Amsterdam.
- game component framework, from [GamePatterns].
- diagram MIME
- diagram experience as meaning
- Roy Lichtenstein, 1962,
(a) Kiss II, (b) Masterpiece,
(c) Forget it, forget me.
- edgecodes -- showing George Lucas and his editoroid.
- El Lissitzky, suprematist works
- El Lissitzky, suprematist works
- Roy Lichtenstein, 1999, Still lifes with brushstrokes
- Les Demoisselles dAvignon, Picasso, 1908, regarded as the start
of Cubism, and Le Goutier, Jean Metzinger, 1911, often referred to as the Mona Lisa of Cubism.
- poster for exhibition of dutch china work.
- signs -- abstract, [Signs], p. 228, 229.
