learning objectives
After reading this chapter you should have an understanding
of the model underlying game playing, and the role
of narratives in interaction.
Furthermore, you might have an idea of how to define aesthetic meaning
in a cultural context, and apply your understanding to the creative
development of meaningful interactive systems.
As in music, the meaning of interactive applications is
determined, not only by its sensory appearance,
but to a high extent by the structure and functionality of the application.
This observation may, also, explain, why narratives become more and more important
in current video games, namely in providing a meaningful context
for possible user actions.
In this chapter, we take an interactive game-model
extended with narrative functionality as a starting point
to explore the aesthetics of interactive applications.
In section 12.1, we will introduce a model for interactive video games, and in section 12.2
we will present a variety of rules for the construction of narratives
in a game context.
Finally, in section 12.3, we will characterize the notion of meaning
from a traditional semiotics perspective, which we will then apply
in the context of games and interactive multimedia applications.
...
1
a game model
Games present challenges, invoke involvement, and are essentially interactive.
Although it might seem far-fetched to regard game playing as a
paradigm for interaction, it is definitely worthwhile to have
a closer look at game theory for inspiration.
According to [Juul (2005)], from a theoretical perspective
games may be said to have the follwoing properties:
game theory
- system -- (formal) set of rules
- relation -- between player and game (affectionate)
- context -- negotiable relation with 'real world'
In particular, relation(s) and context determine the meaning
of the game for the player,
both on an individual/existential level and in relation to a societal context.
...
2
To characterize the defining characteristics of games in a more precise
way, [Juul (2005)] presents a classic game model that may act
as a reference for the description and evaluation of games:
classic game (reference) model
- rules -- formal system
- outcome -- variable and quantifiable
- value -- different valorisation assignments
- effort -- in order to influence the outcome
- attachment -- emotionally attached to outcome (hooked)
- consequences -- optional and negotiable (profit?)
For current day video games, [Juul (2005)] observes that there is
a tension between rules and the fictional or narrative component of
the game:
rules vs fiction
game fiction is ambiguous, optional and imagined by
the player in uncontrollable and unpredictable ways, but the emphasis
on fictional worlds may be the strongest innovation of the video game.
In some cases it might even not be clear what the rules of the game are,
as for example in Second Life, where presence
and expecience seem to be prevalent.
In general, role playing games seem to be less constrained than skill-based games.
Nevertheless, in both cases does the visual environment augment the experience,
adding to the narrative context.
...
3
So, returning to our original question:
theory of interaction
are games relevant for a theory of interaction?
our tentative answer is yes!
In an attempt to formulate criteria for effective service management games,
developed in cooperation with Getronics-PinkRoccade, [Eliens & Chang (2007)],
we gave a characterization in terms of the reference game model,
as outlined below:
effective service management game(s)
- rules -- service management protocols
- outcome -- learning process
- value -- intellectual satisfaction
- effort -- study procedures
- attachment -- corporate identity
- consequences -- job qualification
There is no need to emphasize that this is only a first approximation, and for that
matter a rough one.
What we must keep in mind, however, is that the model is not only applicable on a
macro-level, to characterize an entire game, but more importantly may also
be applied on a micro-level, to establish the criteria for each (relevant) step
in the game play.
To emphasize the relevance particular aspects of
service management games, we added
two more criteria to the model:
additional criteria
- scenario(s) - problem solving in service management
- reward(s) - service level agreements
After all, the goal of playing a service management game is
to be trained in, as stated above,
problem solving in service management situations,
and reaching acceptable service level agreement(s)!
game (interaction) design pattern(s)
Game play is an experience that requires active involvement of the player and
may stand as a model for interaction, that is interaction with high emotional load.
Types or patterns of interaction that may occur in game playing are analysed in
[Björk & Holopainen (2005)], which characterizes game play as:
game play
... structure of interaction with game system and other player(s)
[Björk & Holopainen (2005)] explicitly avoid giving a definition of either game(s) or
game play, but instead focus on interaction patterns, that
is how players interact with the game system (and other players) to
effect a change in the state of the game.
For the description of interaction patterns, [Björk & Holopainen (2005)] introduce a component framework,
that distinguishes between the following types of components:
component framework
- holistic -- playing games as an undividable activity
- boundary -- limit the activities of people playing games
- temporal -- describe the flow of the game (interaction)
- structural -- physical and logical elements of the game system
The various components, each affect game playing in a particular manner,
either by the way the game presents itself through an interface (structural component),
or by the rules that determine the game (boundary component).
An overview of the aspects or elements that belong to each component
is given in the figure, taken from [Björk & Holopainen (2005)], below.
...
4
For narrative structure(s), that we will discuss in the next section,
obviously the temporal component is most important,
containing elements such as closures as well as actions and events
that may occur during game play.
Referring to [Björk & Holopainen (2005)] for details, an impression of what (types of)
interaction patterns may exist is given in the following list:
pattern(s)
- resource management -- resource types, control, progress
- communication and presentation -- information, indicators
- actions and events -- control, rewards and penalties
- narrative structures and immersion -- evaluation, control, characters
- social interaction -- competition, collaboration, activities
- mastery and balancing -- planning, tradeoffs
- meta games and learning -- replayability, learning curve(s)
For example, with respect to actions and events that may occur during game play,
there are various ways rewards and punishments may be dealt with.
Also, as we mentioned in section 10.4
when discussing interaction markers,
there exists a variety of patterns by which to present information
and indicate (opportunities for) interaction.
...
5
example(s) -- intimate media
From the company that used the slogan "let's make things better",
and now advertises its products with "sense and simplicity",
there is the MIME project, not to be confused with the multipart internet mail
standard, which focusses on Multiple Intimate Media Environments.
As
concepts embodying their ideas they propose, among other:
intimate media object(s)
- glow tags -- a subtle way to trigger the person who has placed it or who sees it
- living scrap book -- to capture and collect information and media digitally
- picture ball -- as an object of decoration and a focus for storytelling
- lonely planet listener -- enabling people to listen to a real time connection to another place
On a more abstract level,
seven core qualities are identified which capture the essence of the
intimate media experience:
intimate media experience(s)
- sensorial -- experience is visual, audible, tactile, olfaric
- personalized -- objects embody meaning and memories
- analogue -- people relate to physical objects
- enhancement -- people already have extensive intimate media collections
- serendipity -- it supports unstructured and flexible usage
- longevity -- objects may exist over generations
As can be read on their website:
intimate media describes the things that people create and collect to
store and share their personal memories, interests and loves.
And:
intimate media is central to how people make sense of their world by
representing roots, heritage and a sense of belonging, achievement and
connection.
research directions -- experience as meaning
For the design and appreciation of the new category of digital systems,
including games, we may,
with (forward) reference to our discussion of the history of thought in section 12.4,
well take pragmatist aesthetics as a common ground, since it does justice to
the existential dimension of aesthetic awareness, and allows for
a process of aesthetic literacy, that is becoming sensible to aesthetic awareness
and reflection. You may wonder though, how we get to this conclusion.
In [Hallnäss and Redström (2002)] it is observed that the aesthetic potential
of the narrative space centered on the consumer product
has received surprisingly little attention.
The authors then argue that, motivated by insights
from phenomenology, there should be a shift of attention
from use to presence,
where presence does not merely mean appearance
but a more complex dialectic process of appearance
and gradual disappearance dependent on the role
the object plays in the life of the user/subject.
The notion of expressional is then introduced,
to convey the expressive meaning of objects, and in particular
interactive objects, in our surroundings.
For the design of presence, aesthetics
is then considered as a logic of expressions,
in which expressions act as
the presentation of a structure in a given space
of design variables.
...
6
So far, this makes perfect sense.
We may further mention here the
framework set up by Dhaval Vyas, [ Vyas and van der Veer (2006) ],
which characterizes the user's experience a the result of
the process of constructing meaning.
In diagrammatic form, the process of constructing meaning may be depicted as above.
In more detail, the validity of the experienc as meaning framework
may be substantiated by the foloowing observations:
experience as meaning
- experience occurs during the interaction between the user(s) and the
interactive system(s) in the lived environment
- designers convey meaning (consciously or unconsciously)
through the appearance, interaction and function of the system
- user(s) construct a coherent whole that is a combination of
sensual, cognitive, emotional and practical forms of experience
In other words, an interactive system is determined by
function, interaction and appearance.
As such the framework may be called pragmatist, and has indeed
been nfluenced by [Dewey (1931)].
Returning to our argumentation,
for objects that are not designed for usability in the functional sense
the notion of use is too strict and is, using a dialectic argument, subject
to the dialectics of presence, as argued in [Hallnäss and Redström (2002)].
Conversely, using a similar dialectic argument, for new categories of
objects, presence requires use, or getting used to, in other
words a process in which the user becomes interested and familiar with the object.
We may even speak of aesthetic affordance, with the realization that
the notion of affordance, explaining an ecology of behavior, originally stems
from the late-idealist phenomenology expounded in [Heidegger (1927)].
But, however appealing the notion of expressional,
in the light of our discussion in section 12.4,
where we distinguish between aesthetic awareness
as a given, or a priori, sensibility and aesthetic judgement
as being of a more empirical nature, we would prefer
to consider aesthetics as a logic of sensibility,
which includes a dimension of self-reflection
in the sense of its being aware of its own history.
Put differently, to characterize the contextual aspect
of aesthetics, as it certainly applies to art,
we may speak of aesthetic literacy,
that is aesthetic awareness that is self-reflective by nature.
guidelines for narrative construction(s)
Storytelling is vital to human existence.
Whether to fall asleep during childhood,
to dream about heroism in adolescence,
to escape from the daily routine of work as an adult, ...,
storytelling is ubiquitous,
be it in books, movies, or (even) games.
To think about storytelling, I have taken [Arnheim (1957)] as a guideline, since:
film as art
by still being read, the little treatise seems to prove that in spite of all the changes that have
taken place in their form, content and function,
films are still most genuinly effective when they rely on the basic properties
of the visual medium.
...
With reference to our discussion in section 11.3, entitled
immersion is not illusion, we may remark that we actually do not need visual realism
to be taken in by a story:
illusion
... in film or theatre, so long as the essentials of any event are shown,
the illusion takes place
As to the question of why film may be regarded as art, [Arnheim (1957)] observes:
patterns of light
... we can perceive objects and events as living and at the same time imaginary,
as real objects and as simple patterns of light on the
projection screen, and it is this fact that makes film art possible.
This, by extrapolation, also holds for video games,
which may be regarded as a first-person variant/successor of films.
As a consequence, both film and games may play with the relation between
two distinct frames of reference.
We repeat the characterization given in section 11.3:
frames of reference
it is one of the most important formal qualities of film
that every object that is reproduced appears simultaneously in two
entirely different frames of reference, namely the two-dimensional
and the three-dimensional, and that as one identical object it fulfills
two different functions in the two contexts.
Taking film as an example on which to model
storytelling in games, we may distinguish between
the following principles of montage:
principles of montage
- cutting -- unit length, whole scenes, cuts within scenes
- time relations -- synchronized, before/after, neutral
- space relations -- same place (different time), different place
- subject matter -- similarity and/or contrast
Technology plays an essential role in the production of movies.
In particular,
montage is enabled by technical properties of film, including:
film technique
- camera -- position, focus, movement
- transitions -- fading in/out, dissolving, stills
- arrangement -- light/shade, color, sound
Film, or movies, being a dynamic medium, essentially involve motion.
Following [Arnheim (1957)], we may characterize motion by making a distinction between:
cinematographic motion
- movement of objects
- effect of perspective
- motion of camera
- montage of scenes
Nowadays, with the rise of digital technology,
editing film has changed significantly, and is within the reach of (almost) everybody.
However, the basic principles of what was previously called motion pictures
still seem to apply, even for CG movies or video games.
the meaning of composition
Where montage employs dynamic characteristics of the (moving) image,
the static image itself may be said, following [Kress and van Leeuwen (1996)], to have
narrative implications as well:
narrative implication(s)
- objects -- the items in the image
- vectors -- (imaginary) lines suggesting interaction
- gaze -- inward (offer) / outward (demand)
In other words, images may suggest a story, and the persona within an image
may express a relation to us, either
transactional, directly approaching us, or non-transactional,
without the on-looker being ivolved directly.
Image composition plays an essential role in storytelling,
since, as [Kress and van Leeuwen (1996)] observe:
composition
composition, ..., relates the representational
and interactive meanings of the picture to eachother,
through three interrelated systems.
Composition results in visual effects,
since representational elements may be put together
according to particular mechanisms
or systems as they are called in [Kress and van Leeuwen (1996)]:
representation(s)
- information value -- left/right, top/bottom, centre/margin
- salience -- foreground/background, relative size, contrast
- framing -- connecting or dissolving lines
The effects of compositional mechanisms are partly due to physiological properties
of the image and partly due to a codified interpretation
in a particular social context.
For example, information value is clearly
determined by social context:
information value
- left/right -- given versus new
- top/bottom -- ideal versus real
- centre/margin -- important versus marginal
In section 5.4, we discussed how we applied
these interpretations in the design of information flows in PANORAMA,
with, we must admit, with surprising effect!
...
7
example(s) -- edgecodes
The edgecodes
documentary film by Phillip Daniels gives
an inside account of film editing,
a history of the evolution of editing
conventions, as well as an
account of the technological innovations of the late
20th century and their impact on film editing.
It was shown at the documentary film festival
IDFA 2004, in Amsterdam.
Movies were, as Daniels staes, the new artform of
the 20th century, which distinguishes itself from
other artforms by ... editing!
The film begins with the statement such as
the concept that a film is shot is
entirely false, a film is not shot, its is built,
continuing with the statements that
the message of the movie medium is that of
transition, and that
a movie must have a beginning, middle and ending,
but not necessarily in that order.
The documentary is highly visual, after all it is an editor's
movie, and contains many fragments from wellknown movies
and interviews with famous directors, among which
George Lucas, who introduced the editoroid
in the eighties, an editing machine built with at the time modern
computing technology.
George Lucas, image left above,
explained the introduction of his editing machine
by saying that he wanted to have a system, ... that is
intuitive, obvious, ... and hihgly malleable, ...
visual ....
He wanted a machine that allowed him to use his moterskills,
without the intervention of an engineer.
But in the interview he admitted that they were
on the bleeding edge in those days.
Nowadays, real-time editing, with computer graphics (CG) support
is (finally) feasible.
See chapter 4.
research directions -- multimedia in context
In 1998, I organized a multimedia course for PhD. students,
together with Lynda Hardman (CWI), entitled:
multimedia in context.
When defending the position of multimedia & game development
in the computer science curriculum, it bacame once more necessary to reflect on the relation of
multimedia to its (various) context(s).
First and for all, we may identify the scientific context of multimedia,
which includes many seemingly unrelated areas of science:
scientific context
- mathematics -- matrix algebra, transforms
- physics -- game physic, particle systems
- computer science -- technological infra-structure
- information theory -- compression and delivery
- media theory -- history of communication
- semiotics -- theory of meaning
Secondly, we have the societal context.
Given the (explosive) rise of the creative industry
the list given here is rather poor, and should include
at least the various communities that are coming into existence.
societal context
- cultural heritage -- digital dissemination of art
- education & communication -- presentation of concepts and examples
Obviously, there is a strong technological context of multimedia.
Without the technology, multimedia and game development
would not qualify to be seen as an academic discipline.
technological context
- modelling -- objects, characters
- interaction -- game programming
- architecture -- game engine design
- rendering -- programming the graphics hardware
Finally, there is a creative context.
However, from my perspective this is intimately tied to a technological context,
since at this stage there is still a great need for explorative development,
to discover new applications and the aesthetics governing these apllications.
creative context
- visual design -- style, models and attributes
- story telling -- narrative structure
There should be no need to emphasize that all these contexts are itself one way or another
stronlgly related.
In other words, multimedia & game development is intrinsically a multi-disciplinary
affair, which even cannot be so easlily taken out of a societal context,
in that the value of new media applications is ultimately determined by its
adoption in the new digital culture.
the definition of meaning
As indicated in section 1.1
meaning, or for that matter learning,
takes place, according to
[Bruner (1972)], at various levels:
learning/meaning
- actionary level -- action and movements
- sensory/iconic -- images and impressions
- symbolic -- language and mathematics
Learning by doing, or learning by games may be effective,
precisely because all these levels are involved.
...
8
A cognitaive approach, however, may not be adequate to explain
the meaning of multimedia or visual art.
As observed in [Kress and van Leeuwen (1996)]:
basic geometrical shapes
... basic geometrical shapes have always been a source
of fascination, even of religious awe.
And our scientific age is no exception.
Artistic speculation may lead to an absolutist interpretation
of geometric shapes, or as stated in [Kress and van Leeuwen (1996)]:
nervous system
(basic geometrical shapes) have been thought to have the power
to directly affect our nervous system, for instance by the
constructivist artist Gabo: the emotional force of an absolute shape is unique
and not replaceable by any other means ...
...
9
A well-established theory of meaning is provided by semiotics,
which distinguishes between signifier(s) and signified(s):
semiotics -- a theory of meaning
- signifier -- sign/symbol
- signified -- what is referred to
According to [Kress and van Leeuwen (1996)], meaning is the relation between signifier and signified.
Semiotic theory may be applied to the domain of language and speech, which is its original domain
of application, as well as to the visual domain.
Hovever, for the visual domain there seem to some obstacles.
One question is whether semiotic theory applies to the visual domain.
But another issue is, according to [Kress and van Leeuwen (1996)], that in our culture
the visual domain seems to be inferior to the linguistic domain.
[Kress and van Leeuwen (1996)] raise the question:
semiotic modes
... is the move from the verbal to the visual
a loss, or a gain?
Yet, in a brilliant turnover, where they establish the viablity of
semiotics for the visual domain, they invert the possible inferiority of
the visual domain into a position where it may potentially be superior to language, asking whether
complexity
... it has to be handled visually,
because the verbal is no longer adequate?
To strengthen their argument, they even point to the hidden multi-modality
of written texts, that is the use of images and sensorial impressions
hidden in the written word(s):
multimedia
the multi-modality of written texts has, by and large, been ignored,
whether in educational contexts, in linguistic theorizing,
or in popular common sense.
Today, in the age of multimedia, it can suddenly be perceived
again.
In this way the not only establish a firm position for
visual semiotics, but at the same time
raise the issue of literacy, visual literacy that is,
as an essential skill related to the place of visual
communicatin in the semiotic landscape:
quotes
- myth of transparency -- visual communication is always coded!
- literacy -- standards for semiotic order
- semiotic modes -- text, visual, auditive, ...
- computer technology -- central to semiotic landscape
- semiotic activities -- production, transformation, development
semiotic landscape
the place of visual communication in a given society can
only be understood in the context of, on the one hand,
the range of forms or modes of public communication available
in that society, and, on the other hand, their uses and valuations.
As recognized by politicians and educators, visual literacy or media literacy
is to be regarded an essential skill in our media-driven (information) society,
not only to be able to cope with information,
but also to determine what is right or wrong,
to be able to distinghuish between being manipulated and being informed!
...
10
Even in art expressions, for which there is no clear notion of signified(s),
we may given an answer to the question of meaning.
sonic act(s)
what is the meaning of meaning in apparently
meaningless expressions
The meaning of such expressions is goverend by the relation between the signifiers,
or in other words, the formal properties of the artefact.
Such formal properties also play a role in what has been called modality
in section 11.3.
According to [Kress and van Leeuwen (1996)], modality is strongly related to the reliability of (visual)
messages:
modality
one of the crucial issues in communication is the question of
the reliability of messages.
Is what we see or hear true, factual, real, or is it a lie,
a fiction, something outside reality?
To some extent the form of the message itself suggests the answer.
In orther words, modality may be taken as a veracity marker,
that is an indication of how a message is meant, as serious, ironic, sarcastic, or realistic ...
Modality may also be expressed by a choice of what [Kress and van Leeuwen (1996)] call a
coding orientation, which determines, as we have discussed in section 11.3,
what counts as real:
coding orientation
- technical/scientific -- effectiveness, blueprint
- sensory -- pleasure principle is dominant
- abstract -- used by socia-cultural elite
- naturalistic -- dominant common sense paradigm of realism
Coding orientation, in other words, subtly implies a judgement
on what is considered real, that us relevant!
the politic(s) of meaning
With the introduction of new media, including film, television,
and more recently computer games, concern will be expressed with the
loss of tradional values.
As an example, film, as well as its succecor television may be criticised:
aesthetics of shock
it is within the realm of probability that the shock,
which Walter Benjamin diagnosed as being film's aesthetic innovation,
will undergo renewal and intensification with far more sophisticated means.
A fortiori, television, and the habit of recording everything, making
our private lives public is subject to criticism:
voyeurism
the most obvious symptom of this loss of distance will be a voyeuristic,
dissecting penetration of representations of objects and bodies.
[Arnheim (1957)], for example, comments on television as a great achievement,
and yet seems to experience it as a loss:
TV
for the first time in the history of man's striving for understanding,
simultaneity can be experienced as such, not merely
translated as a succession in time.
sensory stimulation
although the new victory over time and space represents an
impressive enrichment of the perceptual world, it also favors
a cult of sensory stimulation which is characteristic of the
cultural attitude of our time.
What the loss entails becomes clear, when convenience and accessibility
is compared with the effort required by more tradional ways of thinking:
direct experience
proud of our inventions -- photography, film, radio, ... --
we praise the educational virtues of direct experience.
communication
when communication can be achieved by pointing with the finger, however,
the mouth grows silent, the writing hand stops and the mind shrinks.
In other words, [Arnheim (1957)] fears that our traditional intellectual values,
and perhaps even skills, will be lost.
Although written more than half a century ago, these precautions are reminiscent
to the concerns expressed by parents and educators nowadays.
...
11
%:@- new media
It is not easy to either do away with these concerns, nor to fully agree with them.
However, given the developments in the new media, the community web sites,
the availability of online games, the potential danger of addiction to games,
and perhaps even the use of wikipedia to do schoolwork,
there seems to be some reason for concern.
From a slightly more political perspective, we may ask ourselves, following
[Grau (2003)]:
channels
... the decisive questions remain: who controls the channels,
who distributes right of access, and who exercises economic and political
authority over the networks?
Not to be led astray by false media dreams, but having confidence in our autonomy
and the autonomy of our children.
As observed in [Grau (2003)] our vision(s) may be wrong:
visions
... the history of technological visions is the history of our dreams,
our vagaries and our errors.
Media utopias fluctuate, often occurring in a magical or occult ambience.
Yet, our visions represent our dreams, and
the new media do provide us with
a new creative realm of reality.
Principally, multimedia and game technologies are enabling technologies,
enabling the development of the (digital) culture of the 21th century.
Not only in our western world, for our own children, but also
in developing countries, which should not fall victim to
a digital divide.
example(s) -- multimedia in africa
End of 2006, we were asked to participate in a joint project with ethiopian
universities to develop a multimedia curriculum for these
universities. In developing such a curriculum we should stick
to the principles outlined below:
synopsis course(s)
... the curriculum should emphasise basic principles,
and to the extent possible employ open standards and open source.
Practical assignments must be centered on local culture,
and stimulate the young talent to explore innovative
applications for cultural heritage, serious games and artistic
expression.
where, what & why
- where -- Ethiopia & VU
- what -- introduction multimedia
- why -- to develop curriculum
environment
- low end computers -- windows, linux
- elementary skills -- programming, design
assumptions
- open source -- flex 2 sdk, Delta3D
- open standards -- XML, X3D
- basic principles -- exploratory development
targets
- local -- present local cultural heritage
- serious -- develop serious game(s)
- benefits -- promote local culture and commerce
These statements express thye situation beforehand.
At the time of writing the project has not been started, due to a delay
caused by one of the candidate tutors from the ethiopian dropping out.
research directions -- humour in games
For presenting Clima Futura to the jury of the scientific communication contest,
we decided to have three central presenters (anchors) and an expert-panel (choir),
that may comment on detailed scientific or technical issues. The presentation,
stressed the multi-disciplinary approach,
covers the following topics, in the order of listing:
presentation
- philosophy -- pathos, ethos, logos
- trailer -- drama, apocalyptic, appeal to player
- climate star -- scientific issues & game play
- game development -- architecture and project plan
Although it too early to look back, we may on reflection
ask attention for another potential pitfall, that endangers any educational game,
once aptly expressed by Sartre in his criticism of l'esprit de serieux.
Indeed, we may become too serious!
As a potential line of research that may support the design and development of Clima Futura,
we refer to an ontology of humour, [Dormann et al. (2007)],
that may be taken as a guideline to avoid the common pitfall of serious games.
In brief, [Dormann et al. (2007)] distinguishes between three theories of humour, that each denote
a particular function of humour:
relief theory, which explains humour as a reduction of stress,
superiority theory, which asserts that humour has a social function,
as a means to enforce the norm of a group or culture,
and incongruity theory, which relates humour to the discovery
of hidden meanings.
We leave it to the imagination of the reader to establish in what way
the various types of humour may be put to effect in the climate issue,
or for that matter in any serious game.
...
12
development(s) -- philosophy and beyond
In teaching multimedia & game devlopment
I sometimes point out to my students what the
diffence is between an artist, an engineer and a (real) scientist.
In a nutshell:
phrase(s)
- art -- rethorics of the material
- technology -- solving problem(s)
- science -- establish a theory
Leaving aside whether computer science is a science,
or for that matter what role the computer plays in it,
such a comparison is worthwhile, to make clear the mixture
of abilities and skills that a multimedia student must develop.
Not an essential skill, or even required background,
some knowledge of the
history of thought
is recommended.
However,
philosophy is not a very popular subject, and
many seem to easily do away with philosophical abstractions
and apparently tedious theory, even though these same
philosophical abstractions may provide better understanding
of the forces that are at work.
Concluding our manuscript, we will in this section briefly trace the evolution of the
notion of aesthetics to our current day understanding,
starting with the idealist transcendental conception of
aesthetics as the abstract a priori form of experience,
ending with semiotic theory that emphasizes the social
determinants of aesthetic experience.
Our discussion, is based on our studies
in the past, [Eliens (1979)], almost forgotten.
The outline given below
includes the references to the material we originally studied.
However, for reference, links to relevant online
material are also included.
perspective(s)
- transcendental -- abstract form of experience, [Kant (1781)]
- speculative -- criteria for beauty, [Kant (1781)]
- phenomenological -- self-conscious subjectivity, [Hegel (1807)]
- psychoanalytical -- sub-conscious meaning, [Freud (1958)]
- pragmatical -- art as experience, [Dewey (1931)]
- hermeneutical -- understanding of the senses, [Hermeneutics]
- semiotics -- social construction of meaning, [Kress and van Leeuwen (1996)]
To my mind, the epistemological understanding of
aesthetics as the pure form of sensuousness, as expressed
in [Kant (1781)], is most fundamental in understanding
the notion of aesthetics in the context of interactive systems,
since it allows us to characterize the dimensions of
sensuous awareness delimiting our experience of art,
architecture and interactive systems.
The epistemological or transcendental characterization
of aesthetics describes, in other words,
the a priori principles of sensuousness,
that determine our perception of reality,
by imposing organisation and form on the chaotic
multitude of appearances.
As phrased in [Kant (1781)], appearances consist
of material, which is a posteriori given,
and form, determined by the a priori nature
of our mind.
As dimensions of pure sensuousness, or aesthetic awareness,
Kant distinguishes between space and time.
In [Kant (1781)], the notion of aesthetic judgement is
introduced.
Our ability for aesthetic awareness allows us to recognize
and appreciate beauty, however Kant emphasizes that
any attempt to conceptualize the judgement of beauty
is doomed to fail, or may at best be determined
empirically, in an ad hoc manner.
Later thinkers in the idealist school took over
Kants conception of aesthetic awareness as the receptive
side of our mind, in search for knowledge,
and emphasized the relation between truth and beauty, [Schiller (1977)].
In particular [Hegel (1807)] characterized beauty
as the sensuous presence of Idea,
and he identifies our need for truth and beauty
with the intrinsic movement of self-consciousness.
In other words, aesthetic awareness in not a dis-interested a
priori ability that allows us to organise our perceptions
and to recognize and appreciate pure form, rather
it is intentional and through self-reflection
subject to recurrent improvement and change,
continuously looking for truth and beauty,
that is meaning.
We may note here that psychoanalytic theory has contributed
to understanding the hidden dimensions of meaning, [Freud (1958)].
Hegels conception of aesthetic awareness is
surprisingly close to the idea of pragmatic aesthetics
as expressed by [Dewey (1931)],
a representative of the anglo-saxon school of empiricist
philosophy which is in many ways irreconcilable with
the German idealist/phenomenologist school of thinking.
Essential in Dewey's thinking is the notion
of qualitative immediacy and
the unification of awareness and judgement in
the experience of art, where Dewey stresses the
re-creating role of the subject/recipient in experiencing art.
In this way, the experience of art is instrumental,
according to Dewey, to reconcile the individual
with his environment.
A similar concern with the existential role of the experience
of art, and consequently aesthetic awareness, may be
found in hermeneutic thinking of the 20th century, where
for example [Hermeneutics] speaks of beauty
bridging the gap between the ideal and reality.
However, by that time art is no longer pure but must
as aesthetic art be appreciated with
a certain degree of distance,
that is its judgement is no longer direct,
governed by pure sensuousness, but regulated by reflection
and to a certain extent disciplined appreciation.
This position may, however, be attributed to the role of
the arts in the 19th and 20th century, and even, as
argumented by [Grau (2003)], be seen as an
opposition to the mass media of the 19th century, which strived for
direct sensuous immersion, for example in life-like panoramas.
The influence of convention and social context
has been studied in semiotic theory, [Kress and van Leeuwen (1996)],
and in our time, where we are concerned with
the influence of the old and new media, and
media literacy is (again) one of the urgent
topics on our political agenda, the relation
between sensuousness and reflection is again of interest.
We believe that the semiotic perspective is of particular
importance for the design of interactive systems.
Nevertheless, to summarize this section, for
our epistemological understanding of aesthetics
the original notion of sensuousness as
the receptive side of our faculty of knowledge
still seems to provide a good starting point.
However, both an analytic view of aesthetic awareness,
which for example forces us to think about the
difference between aesthetic experience and a
drug-induced state of mind, [Saw (1971)],
and a recognition of the moral dimension of beauty,
[Meditations], may serve us in establishing the value
of aesthetics for the design and appreciation of
interactive systems.
Assuming a notion of aesthetics as a logic of sensibility,
we may, in summary, distinguish between three dimensions of form,
extending Kant's original proposal, as indicated below:
dimensions of aesthetic awareness
- spatial -- topological relations, layout of image
- temporal -- order, rhythm, structure
- dynamic -- interaction, reflection, involvement
The dimension of dynamics clearly is the great
unknown, and more in particular it is the dimension we
have to explore in the context of interactive
systems, not in isolation but in relation to the other
dimensions,
not so much to establish definite criteria,
but to understand the forces at work, or in other words
the relevant parameters of design.
[Sartre (1936)] gives an existential foundation
for the dimension of dynamics, by observing
that the human body is instrumental in gaining awareness,
as the centre of both obscurity and reflection from
which consciousness emerges, through selection and action.
It is in the existential dimension of aesthetic awareness that we come most close to the
experience of the new digital artefacts, since it concerns both involvement and human action.
Interestingly, and in apparent contradiction with [Hallnäss and Redström (2002)],
cited previously, to establish
a foundation for the aesthetics of interactive systems [Interaction]
seek refuge with pragmatist aesthetics
as it promotes aesthetics of use rather than aesthetics of appearance.
Again, although we agree with the gist of [Interaction],
we wish to emphasize that the contribution of pragmatist
aesthetics is not its focus on use, but the role
of experience in understanding and appreciating
aesthetic artefacts, that is the active role
of the subject in becoming aware of its meaning.
...
13
questions
concepts
technology
projects & further reading
As a project, explore the ways narratives may be constructed from
a collection of images.
Deploy the various editing facilities for providing
flashbacks, flashforwards, and other (temporal) relations
within storytelling.
You may implement this using flash, VRML, or even try to embed
such a narration facility in a game level developed with the
Delta3D or the
Source SDK.
For further reading I suggest you to take a look at
more theoretical material from media theory, such as [Bolter and Grusin (2000)].
Also there is a large collection of books from MIT Media Press
that is of relevance for our new visual culture.
- einzelganger -- walking man of Alberto Giacommeti,
taken from an aanouncement of the Ives Ensemble, Amsterdam.
- game component framework, from [Björk & Holopainen (2005)].
- diagram MIME
- diagram experience as meaning
- Roy Lichtenstein, 1962,
(a) Kiss II, (b) Masterpiece,
(c) Forget it, forget me.
- edgecodes -- showing George Lucas and his editoroid.
- El Lissitzky, suprematist works
- El Lissitzky, suprematist works
- Roy Lichtenstein, 1999, Still lifes with brushstrokes
- Les Demoisselles dAvignon, Picasso, 1908, regarded as the start
of Cubism, and Le Goutier, Jean Metzinger, 1911, often referred to as the Mona Lisa of Cubism.
- poster for exhibition of dutch china work.
- signs -- abstract, [ van Rooijen (2003)], p. 228, 229.
The walking man is one of my favorate sculptures,
for over a long time.
It is also associated to the motto of part iv:
a journey of a thousand miles begins with the first step .
As an autobiographical note, the walking man, with
einzelganger superposed (in translation loner),
reflects the writing of topical media.
In particular, the image put in sequence, reminds me of
the repetitive complaints of my (former) superior at the faculty,
who over and over again told me that I was always
alone in my room, isolated, on an island.
I must admit there is a truth in this, as I felt that the
disciplines of software engineering and multimedia are widely divergent,
and in that sense I was on my own.
This book has undergone many rewritings, due partly to a clash
between the expectations of others and my own vision on multimedia.
And with a superior who emphasizes that he is "the boss",
but has no intellectual authority nor any inspirational leadership
whatsoever,
at least not in the area of multimedia and gane development,
there is really no other way than to go your own way.
So I did it my way, indeed, quoting Paul Anka's song,
made 'unforgettable' by Frank Sinatra.
In other words, after this brief autobiographical digression,
the visual theme of this chapter on the aesthetics of interactive
systems is on individual judgement, as exemplified among others
by the suprematist works of El Lissitzky, the amplification
of cartoons as art by Roy Lichtenstein, and the pioneers of Cubism.
After all, individual judgement is what you need, when you
wish to be involved in multimedia and/or game development.
(C) Æliens
23/08/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
