topical media & game development
PDF
A theoretical foundation for the aesthetics of interaction and awareness -- making sense of the senses
Anton Eliëns, Dhaval Vyas
resource(s)
abstract
To clarify the notion of aesthetics in the context of interactive
systems and to arrive at a foundation for interaction aesthetics,
we start by looking at the history of thought from a philosophical
stance.
We distinguish between two complementary aspects of aesthetics,
namely awareness and judgement.
We identify space, time and dynamics as the dimensions
constituting aesthetic awareness, and tentatively propose a model
based on
interactive game playing to clarify the meaning of interaction.
From semiotic theory we derive a grammar of visual design
that may aid our understanding of how interactive systems
are perceived and what affordance they offer.
Combined, these perspectives clarify the dynamics
of the aesthetic experience of interactive systems,
as an interplay between awareness and judgement,
informed by art, architecture and gaming.
Here we provide an outline of the dialectics
of aesthetic awareness, that aids our understanding
of the adoption and use of the new category of
fun or awareness-oriented interactive systems,
which we will illustrate by discussing our own work,
in particular PANORAMA,
a system supporting social awareness.
keywords:
aesthetics, interaction, semiotics, affordance, emotive dialogs, augmented reality, digital dossier, social awareness
Introduction
When we think of
media as an extension of our senses, [Zielinski (2006)], we may
reformulate the question of interaction aesthetics as the
problem of clarifying the aesthetics of media rich interactive applications.
However, what do we mean exactly by the phrase aesthetics.
The Internet Enceclopedia of Philosophy
discusses under the heading of aesthetics topics such as
Aesthetics
- intentions -- motives of the artist
- expression -- where form takes over
- representation -- the relation of art to reality
These topics obviously does not cover what we want,
so we took the call for contributions to the
aesthetics of interaction as a good chance
to dust of our old books, and rekindle our interest
in this long forgotten branch of philosophy, aesthetics.
It may come as a shock to realize how many perspectives
apply to the notion of aesthetics.
First of all, we may take an analytical approach, as we do in section 2, to see in what ways
the phrase aesthetics is used, and derive its meaning from its usage in a variety of contexts.
However, we find it more worthwhile to delve into the history of thought
and clarify the meaning of aesthetics from an epistemological point of view,
following [Kant (1781)], as an abstract a priori form of awareness, which is in later
phenomenological thinking augmented with a notion of self-consciousness.
In this line of thinking we also encounter the distinction between aesthetic awareness
and aesthetic judgement, the dialectic relationship of which becomes evident
in for example the occurrence of aestheticism in avant-garde art, [Burger (1981)].
When writing this paper, we came along a report of how the Belgium curator
Jan Hoet organized the Documenta IX, a famous yearly art event in Germany,
and we were struck by the phrase art and the public sharing accomodation, [Hoet (1992)],
which in a way that we have yet to clarify expresses some of our intuition
we have with respect to the role the new interactive systems may play in our lives.
What can we hope to achieve when taking a more philosophical look at interaction aesthetics?
Not so much, at first sight.
According to [Körner (1973)], aesthetic theory ... will not be able to provide
aesthetic guidance even to the extent to which moral theory can give moral guidance.
The reason is that aesthetic experience and creation defy conceptualization, or in other
words they defy the identification, classification and evaluation of aesthetic objects
by means of non-aesthetic attributes.
However, as [Körner (1973)] observes, in a paradoxical way aesthetic experience not
only defies but also invites conceptualization, and therefore it seems worthwhile
to gain a better understanding in what underlies the experience and creation of (aesthetic)
interactive systems.
In a mere analytical sense, our paper may
be regarded as giving a clarification of some of the
terminology used, and in a synthetic sense it may be regarded
as another contribution to the rethorics of interaction aesthetics.
However, our intention is to arrive at a deeper understanding of the notion of
aesthetics that allows us to formulate a model that provides us with the concepts
that we need not only to make sense of what we experience using interactive applications, but
also make sense when we approach the problem of designing such systems.
Afterall, what is the real question?
Ultimately to design implements or artefacts that embody the
computing technology of the future.
In this paper we will take a little intellectual detour, after a brief analytical exercise in section 2,
by sketching how the notion of aesthetics evolved in the history of thought in section 3,
to arrive at a classification of the dimensions of awareness in section 4,
followed by the description of a model for interaction in section 5.
Our model is based on a model for interactive game playing and allows for degrees of interaction,
that is a certain indirectness of interaction.
In section 6 we discuss the dialectics of awareness and introduce our framework for the design of
interactive systems, based on our conception that the users' experience determines
the meaning of the system.
In section 7 we extrapolate this framework in a tentative grammar of design by discussing
the meaning of composition from the perspective of semiotic theory.
In section 8 we briefly discuss these notions in the context of our own work,
and in section 9 we draw our conclusions and give indications for future exploration and research.
An analytical approach
Let us take a more close look at in what phrases the term aesthetics is used.
The list of phrases below is collected from papers discussed in a workshop
on the Aesthetics of Computing, CHI 2006, [Hallnäss et al. (2006)].
Taking a philosophical stance, a mere analytical approach
shows rightout confusion and a blurred understanding of
notions central to any theory of aesthetics.
For example, we encountered the definition of form
as the way in which material build things.
Here are some more:
usage of aesthetics
- design aesthetics is what relates to notions of form and expression in design practice. (3)
- design practice where aesthetic qualities are emphasied. (4)
- the aesthetics of an interactive artefact evolves in the relationship with the user.
- experimental design aesthetics differ very little from art.
- when pro-active technology goes home, pragmatic aesthetics is needed. (5)
- there is a need for a specific basic interaction design course and
knowledge about the aesthetics of interactions (7)
- recent trends call for a stronger focus on the aesthetics of
user experiences (6)
- this intention is aesthetics, and aesthetics for its own sake,
and even goes beyond an interest of meaning.
- to use software is to perceive, to grasp and to apply. We are
right in the middle of aesthetics! (9)
- by turning to aesthetics the critical approach to
computing thus includes an emancipatory aspect.
- very few of these actually hold any artistic or aesthetic quality,
which is not surprising at all with such a new media. (10)
- aesthetics have their root in philosophy, which defines
aesthetics as the (perceived) sense of beauty. (11)
There obviously is a bewildering variety of ways in which the term aesthetics is used.
When we make a first selection we may reduce the list above to
phrases such as
aesthetic qualities of objects,
aesthetics of design process,
aesthetics of user experiences,
aesthetics as critical judgemente, and
aesthetics as sense of beauty
Still, there is confusion, for example is sense
in sense of beauty meant as appearance
or as faculty of perception or both?
So,
finally, in our most drastic reduction we arrive at the following forms of usage:
usage of aesthetics (3)
- aesthetics of experience
- aesthetics of appearance
- aesthetics of use
Let us be clear, although we definitely do cherish the insights
expressed in the phrases and sentences above, we refuse to accept
the terminology used without further ado, and propose instead to look for definitions and explanations
in some of the classics of philosophy.
The notion of aesthetics in the history of thought
Philosophy is not a very popular subject, and
some seem to easily do away with philosophical abstractions
and apparently tedious theory, even though these same
philosophical abstractions may provide better understanding
of the forces that are at work.
In this section, we will briefly trace the evolution of the
notion of aesthetics to our current day understanding,
starting with the idealist transcendental conception of
aesthetics as the abstract a priori form of experience,
ending with semiotic theory that emphasizes the social
determinants of aesthetic experience.
Our discussion, in this section, is based on our studies
in the past, [Eliens (1979)], and the outline given below
includes the references to the material we originally studied.
However, for reference, links to relevant online
material are also included.
perspective(s)
- transcendental -- abstract form of experience, [Kant (1781)]
- speculative -- criteria for beauty, [Kant (1781)]
- phenomenological -- self-conscious subjectivity, [Hegel (1807)]
- psychoanalytical -- sub-conscious meaning, [Freud (1958)]
- pragmatical -- art as experience, [Dewey (1931)]
- hermeneutical -- understanding of the senses, [Gadamer (1977)]
- semiotics -- social construction of meaning, [Kress and van Leeuwen (1996)]
To our mind, the epistemological understanding of
aesthetics as the pure form of sensuousness, as expressed
in [Kant (1781)], is most fundamental in understanding
the notion of aesthetics in the context of interactive systems,
since it allows us to characterize the dimensions of
sensuous awareness delimiting our experience of art,
architecture and interactive systems.
The epistemological or transcendental characterization
of aesthetics describes, in other words,
the a priori principles of sensuousness,
that determine our perception of reality,
by imposing organisation and form on the chaotic
multitude of appearances.
As phrased in [Kant (1781)], appearances consist
of material, which is a posteriori given,
and form, determined by the a priori nature
of our mind.
As dimensions of pure sensuousness, or aesthetic awareness,
Kant distinguishes between space and time.
In [Kant (1781)], the notion of aesthetic judgement is
introduced.
Our ability for aesthetic awareness allows us to recognize
and appreciate beauty, however Kant emphasizes that
any attempt to conceptualize the judgement of beauty
is doomed to fail, or may at best be determined
empirically, in an ad hoc manner.
Later thinkers in the idealist school took over
Kants conception of aesthetic awareness as the receptive
side of our mind, in search for knowledge,
and emphasized the relation between truth and beauty, [Schiller (1977)].
In particular Hegel, [Hegel (1807)], characterized beauty
as the sensuous presence of Idea,
and he identifies our need for truth and beauty
with the intrinsic movement of self-consciousness.
In other words, aesthetic awareness in not a dis-interested a
priori ability that allows us to organise our perceptions
and to recognize and appreciate pure form, rather
it is intentional and through self-reflection
subject to recurrent improvement and change,
continuously looking for truth and beauty,
that is meaning.
We may note here that psychoanalytic theory has contributed
to understanding the hidden dimensions of meaning, [Freud (1958)].
Hegels conception of aesthetic awareness is
surprisingly close to the idea of pragmatic aesthetics
as expressed by Dewey, [Dewey (1931)],
a representative of the British school of empiricist
philosophy which is in many ways irreconcilable with
the German idealist/phenomenologist school of thinking.
Essential in Dewey's thinking is the notion
of qualitative immediacy and
the unification of awareness and judgement in
the experience of art, where Dewey stresses the
re-creating role of the subject/recipient in experiencing art.
In this way, the experience of art is instrumental,
according to Dewey, to reconcile the individual
with his environment.
A similar concern with the existential role of the experience
of art, and consequently aesthetic awareness, may be
found in hermeneutic thinking of the 20th century, where
for example [Gadamer (1977)] speaks of beauty
bridging the gap between the ideal and reality.
However, by that time art is no longer pure but must
as aesthetic art be appreciated with
a certain degree of distance,
that is its judgement is no longer direct,
governed by pure sensuousness, but regulated by reflection
and to a certain extent disciplined appreciation.
This position may, however, be attributed to the role of
the arts in the 19th and 20th century, and even, as
argumented by [Grau (2003)], be seen as an
opposition to the mass media of the 19th century, which strived for
direct sensuous immersion, for example in life-like panoramas.
The influence of convention and social context
has been studied in semiotic theory, [Kress and van Leeuwen (1996)],
and in our time, where we are concerned with
the influence of the old and new media, and
media literacy is (again) one of the urgent
topics on our political agenda, the relation
between sensuousness and reflection is again of interest.
Nevertheless, to summarize this section, for
our epistemological understanding of aesthetics
the original notion of sensuousness as
the receptive side of our faculty of knowledge
still seems to provide a good starting point.
However, both an analytic view of aesthetic awareness,
which for example forces us to think about the
difference between aesthetic experience and a
drug-induced state of mind, [Saw (1971)],
and a recognition of the moral dimension of beauty,
[Cheng (2006)], may serve us in establishing the value
of aesthetics for the design and appreciation of
interactive systems.
Dimensions of aesthetic awareness
In [Hallnäss and Redström (2002)] it is observed that the aesthetic potential
of the narrative space contered on the consumer product
has received surprisingly little attention.
The authors then argue that, motivated by insights
from phenomenology, there should be a shift of attention
from use to presence,
where presence does not merely mean appearance
but a more complex dialectic process of appearance
and gradual disappearance dependent on the role
the object plays in the life of the user/subject.
The notion of expressional is then introduced,
to convey the expressive meaning of objects, and in particular
interactive objects, in our surroundings.
For the design of presence, aesthetics
is then considered as a logic of expressions,
in which expressions act as
the presentation of a structure in a given space
of design variables.
However appealing the notion of expressional,
in the light of our discussion in section 3,
where we distinguish between aesthetic awareness
as a given, or a priori, sensibility and aesthetic judgement
as being of a more empirical nature, we would prefer
to consider aesthetics as a logic of sensibility,
which includes a dimension of self-reflection
in the sense of its being aware of its own history.
Put differently, to characterize the contextual aspect
of aesthetics, as it certainly applies to art,
we may speak of aesthetic literacy,
that is aesthetic awareness that is self-reflective by nature.
Assuming a notion of aesthetics as a logic of sensibility,
we may distinguish between three dimensions of form,
extending Kant's original proposal, as indicated below:
dimensions of aesthetic awareness
- spatial -- topological relations, layout of image
- temporal -- order, rhythm, structure
- dynamic -- interaction, reflection, involvement
The dimension of dynamics clearly is the great
unknown, and more in particular it is the dimension we
have to explore in the context of interactive
systems, not in isolation but in relation to the other
dimensions,
not so much to establish definite criteria,
but to understand the forces at work, or in other words
the relevant parameters of design.
In [Sartre (1936)], Sartre gives an existential foundation
for the dimension of dynamics, by observing
that the human body is instrumental in gaining awareness,
as the centre of both obscurity and reflection from
which consciousness emerges, through selection and action.
A model of interaction
Interestingly, and in apparent contradiction with [Hallnäss and Redström (2002)],
cited in the previous section, to establish
a foundation for the aesthetics of interactive systems [Graves-Petersen et al. (2004)]
seeks refuge with pragmatist aesthetics
as it promotes aesthetics of use rather than aesthetics of appearance.
Again, although we agree with the gist of [Graves-Petersen et al. (2004)],
we wish to emphasize that the contribution of pragmatist
aesthetics is not its focus on use, but the role
of experience in understanding and appreciating
aesthetic artefacts, that is the active role
of the subject in becoming aware of its meaning.
As we will further discuss in section 6, where we
introduce a framework for the design of interactive systems
based on our (pragmatist) conception of
experience as meaning,
the dynamics of interaction may be considered
to be an important dimension of aesthetic awareness
when we realize that
interaction constitutes the object.
Although nowadays art may also be recognized to offer
interaction in various degrees, for a model
of interaction we prefer,
following a suggestion in [Grau (2003)],
to look for inspiration at game playing, and
in particular we propose the game model
introduced in [Juul (2005)] as a first explanation
of the dynamics of interaction.
Later, in sections 6 and 7, we will
further explore the differences between games,
art, architecture and interactive applications, and discuss
possible refinements to the model.
Following [Juul (2005)], we may characterize a game
as a system, or a formal set of rules.
In addition, we can identify a relation between
the player and a game, a relation that can be of
a rather affectionate or involved nature,
and we may consider the context of playing, which is
in a broad sense a negotiable relation with the real
world, which may go as far as
becoming rich or famous in the real world.
In a more formal way, still following [Juul (2005)],
we can define a classic game model by considering
the following aspects or elements:
classic game model
- rules -- formal system
- outcome -- variable and quantifiable
- value -- different valorisation assignments
- effort -- in order to influence the outcome
- attachment -- emotionally attached to outcome
- consequences -- optional and negotiable
Where an arbitrary interactive system may differ from a game played
for entertainment is obviously the actual outcome,
the value attributed to that in
the real world, and probably
the effort required and the possible consequences.
You would not like to run the risk to die a virtual
death when answering your email, would you?
However, when interactive systems replace task-bound
functionality with fun, the difference becomes less clear.
One element not sufficiently captured by the
classic game model is the narrative aspect of the game play.
To quote [Juul (2005)]:
rules vs fiction
Game fiction is ambiguous, optional and imagined by
the player in uncontrollable and unpredictable ways, but the emphasis
on fictional worlds may be the strongest innovation of the video game.
We may observe that many games already have a
strong relation to reality in what narrative context
they supply, or else in the realities of the media industry,
in particular Hollywood.
For serious interactive systems, we may assume an even stronger
and in some sense more straightforward relation
with reality, by the use of media content
that is relevant for the life of the individual.
interaction markers
Given the large variety of games, including first
person shooters, role-playing games, strategy games
and decision-making simulation games, we can distinguish
between a range of degrees of interaction,
direct interaction, on the one hand, as for example in first person shooter and
indirect interaction, on the other hand, as for example
in simulation games, or role-playing games
where the individual actions may contribute to a plot
such that the effects will become visible at a later time.
Where in game playing the variety of interaction modes
seems to be well understood within each community
of game players, for the develepment of more
general interactive systems we will have to think seriously
whether the target user will be able to learn the
various modes of interaction,
either by explicit instruction or during play.
And as designers we must be concerned with
issues of visualisation and
interaction mappings, that is in other words which
affordances the application offers for a particular
group of users.
To tackle this design problem, we introduce
the notion of interaction markers,
that in a similar way to veracity markers indicating
believability in communication, cover the potential
for interaction with a vocabulary of visual and
other sensory cues, which may partly be derived
from interaction conventions common
in game communities.
Experience as meaning
In [Vyas and van der Veer (2006)], we introduced a framework for the design
of interactive systems, based on the observation that
the user's experience of an artefact or application
may be considered to consist of
the meaning s/he constructs.
These observations concerned both the appreciation of
musical scores and movies, and actual interactive applications.
Our framework is captured in the following postulates, which
entail that
an interactive system is determined by
function, interaction and appearance:
framework
- experience occurs during the interaction between the user(s) and the
interactive system(s) in the lived environment
- designers convey meaning (consciously or unconsciously)
through the appearance, interaction and function of the system
- user(s) construct a coherent whole that is a combination of
sensual, cognitive, emotional and practical forms of experience
As reported in [Vyas and van der Veer (2006)],
to validate the framework we carried out several
design experiments, in which we instructed the designers
to enact the envisaged experience of the potential user.
In later work we deployed cultural probes
to gain deeper understanding of the world
of experience of potential users and their
expectations with regard to the system.
the dialectics of awareness
In the course of our field study for the PANORAMA system,
that we will discuss in more detail in section 8,
we tried to establish what relation users would have
to the system, not only in the way they interact
with it, but also in terms of what role the system plays
in their lives, and when and how they would be aware of
the system.
Due to the intrinsic properties of the PANORAMA
system, which is a system meant to support social awareness
in a work environment, we could not assume
the direct focussed attention that characterized
the applications we had in mind when formulating
the design framework above.
Instead, we must adapt the model to take
the various forms of awareness or attention into account.
Our thoughts in this direction were triggered
by a lecture of Linda Stone (former vice-president
of Microsoft) at the Crossmedia Week
September 2006 in Amsterdam,
entitled Attention -- the Real Aphrodisiac.
In that lecture Linda Stone made a distinction between
applications popular before 1985,
applications which were in general meant for
self-improvement, for example language-learning,
applications that were popular between 1985 and 2005,
applications that she characterized as supporting
continuous partial awareness,
such as email and news-feeds, and applications
of the period thereafter, from now into the future,
which may be characterized as applications
that allow the user to be creative, take part in a community,
and are in other words more focussed and less dependent
on the external environment.
Admittedly, it takes a few more steps to formulate
a theory of the dialectics of awareness.
However, with the function of the PANORAMA
system in mind, we may make, following [Benjamin (1936)],
some interesting distinctions between
the experience of art and architecture.
Where art is usually experienced in a delimited
time span, and is similarly delimited in space,
that is the position of the observer,
architecture is everywhere and always there.
As a consequence, art receives focussed attention
and may be appreciated with reflective distance,
whereas architecture is often not perceived
consciously, but merely present and subject to
an almost sub-conscious sensibility,
which is only brought to the focus of attention
when it is either aesthetisized, for example
when taking photographs, or when something
surprising is sensed, for example in the change
of skyline in New York.
As argued in [Hallnäss and Redström (2002)],
many of the new interactive systems,
whether in the category of ambient media,
ubiquitous computing or calm technology,
will fall somewhere inbetween the spectrum spanned
by art and architecture, or more likely even
alternate between the forms of awareness associated
with respectively art and architecture.
The meaning of composition
Having an understanding of the dimensions of aesthetic awareness,
can we isolate the relevant design parameters and formulate
rules of composition that may help us in developing
interactive applications?
According to our philosophical credo, no!
However, the history of art clearly shows the impact
of discoveries, such as the discovery of perspective,
as well as conventions in the interpretation of
art, as for example in the iconic representation
of narrative context in 17th century Dutch painting.
Moreover, the analysis of the visual culture of mass media may also
give us better understanding of the implied meaning
of compositional structures.
The notion of perspective, described in [Alberti (1435)],
is an interesting notion in itself,
since it describes both the organisation of the image
as well as the optimal point of view of the viewer.
The normal perspective as we know it is the central
perspective.
However, there are variants of perspective that force the
viewer in an abnormal point of view,
as for example with anamorphisms.
Perspective had an enormous impact on (western) art
and visual culture.
It defines our notion of naturalist realism, and allowed
for the dvelopment of the panorama as
a mass medium of the 19th century, [Grau (2003)].
Art that deviated from central perspective, such
as cubism or art from other cultures, was often
considered naive.
Photography and its pre-cursors had a great impact
on the perfection of perspectivist naturalism,
and what is called photorealism
became the touchstone of perfection for early
computer graphics, [Bolter and Grusin (2000)].
Apart from perspective, other conventions regulate
the composition of the 2D image,
in particular, following [Kress and van Leeuwen (1996)],
the information value related to where
an object is placed in the image,
and the salience of the object,
determined by its relative size, being foreground or background,
and visual contrast.
Also framing is used to emphasize meaning,
as for example in the close-up in a movie shot.
In analysing a large collection of image material,
[Kress and van Leeuwen (1996)], somewhat surprisingly found
that lef/right positioning usually meant
given versus new,
top/bottom positioning ideal versus real,
and centre/margin positioning important
versus marginal.
It is doubtful whether these meaning relationships
hold in all cultures, but as a visual convention
it is apparently well-rooted in western visual culture.
For 2D images, [Kress and van Leeuwen (1996)] further identify
narrative elements, that is relations between
objects in the image that suggest a story,
such as a diagonal line from a person to a door,
or a relation of an object to the viewer,
such as a gaze towards the viewer,
a technique that has been used only since late
renaissance painting.
More than paintings or 2D images, film is the medium
for conveying narrative structures.
The art of storytelling in film has been perfected
in such a way that Hollywood films may seem more real
than life.
However, as emphasized in [Remdediation], this is not due
to any inherent form of naturalism, but to
the fact that we have got accustomed to the conventions
applied, that is the techniques of cutting, montage, camera
movements, close-ups, etcetera.
In a highly recommended book, [Arnheim (1957)], Rudolf Arnheim
gives an extensive analysis of the
principles of montage and film technique, and he explains why film is such
an effective medium:
frames of reference
It is one of the most important formal qualities of film
that every object that is reproduced appears simultaneously in two
entirely different frames of reference, namely the two-dimensional
and the three-dimensional, and that as one identical object it fulfills
two different functions in the two contexts.
Due to the subtle play between these two frames of reference
film may be considered an art form,
and as such perhaps the dominant art form of the 20th century.
As a mass medium, film may be characterized by
what Arnheim, following Benjamin, called the aesthetics of shock,
replacing reflective distance with immersive thrill.
As an art form, however, it is the dominant paradigm
for aesthetic awareness, lacking however still one dimension,
interactive dynamics.
As observed in [Bolter and Grusin (2000)], interaction is what
distinguishes video games from film.
Current day technology allows for high-resolution
photorealist graphics, that make video games or
virtual applications almost indistinguishable from film.
Virtual reality technology as applied in video games
adds arbitrary choice of perspective,
as exemplified in first-person shooters or fly-overs,
as well as an arbitrary mix of
the imaginary and real, as in CG movies, in an interactive
fashion.
Now, should we take the aesthetics of interactive video games
as the standard for interactive applications?
Not necessarily, since the naturalism strived for
in most games may at best be characterized
as naive realism, mostly photorealism.
As observed in [Kress and van Leeuwen (1996)], realism is a social construct,
and hence the program for developing an aesthetics for
interactive applications should perhaps
include the development of appropriate realisms.
Again with an eye to the history of art,
where we have for example impressionism,
cubism, expressionism,
as a guideline in the
design of interactive systems, it might be even better
to look for appropriate interaction-isms,
styles of developing interactive systems
from a particular perspective.
Applications to design
Much of our own work may be best characterized as explorative
development using virtual reality and game technology
for (more or less) serious applications.
In [Eliens et al. (2003)], we describe a framework for mixed media,
based on agent technology, [Eliens et al. (2002)], supporting
emotive dialogs by humanoid agents, in rich media
virtual environments.
In [Eliens et al. (2006)], we describe an immersive digital dossier
that allows navigating the concept space surrounding the
work of the artist/performer Marina Abramovic,
and to look at her work, including video recordings of
performances and 3D models of art work installations.
Immersive, in this context, means that there is no
disruptive break between navigation and presentation,
for example by pop-up windows.
In [Eliens (2006)], we explored the combination of live
video with 3D technology to achieve
visual effects in a transparent manner,
that we called see-through aesthetics,
giving emotive distortions and images affecting
the interpretation of the perceived scene.
In [Eliens and Bhikharie (2006)], we created a realistic multi-user game,
featuring our facul;ty building, offering a simple puzzle
for the player to obtain a hidden treasure,
in effect the power to use weapons.
Although each of these projects had their implied
aesthetics, that is a particular perspective on reality,
as reflected in the system, our approach to design
in these cases has been rather intuitive.
In our explorative efforts, we were primarily led
by what Brancusi called the rethorics of the material.
The PANORAMA system
In developing the PANORAMA system we could, however,
not follow this naive approach,
since it is rather more
complex in what we hope to achieve with it.
The PANORAMA system is meant to support social awareness,
in non-work related ways, using a large screen display
in a public room in our faculty.
To achieve social awareness, we ask the staff to
contribute items of self-reflection,
such as holiday postcards or birth announcements.
In order to reflect the liveliness of the workplace,
we monitor places where occasional encounters
may take place,
for example during a break at the coffee machine
or in the printer room, waiting for the printer queue.
Encounters in such places are often of an informal,
personal nature, but may be mixed with work-related
interests.
As an experimental feature, we consider to allow for
direct interaction using the system, for example,
to play a game, possibly with a mobile phone as an
input device.
In summary, the PANORAMA system is determined by the following
contributions of its users,
contributions that are not necessarily direct
or even do require explicit activity.
PANORAMA
- self-reflection(s) -- e.g. picture/postcard(s)
- casual encounter(s) -- at coffemachine or printer
- occasional battle(s) -- optional direct interaction
For a deeper understanding of what
role the system would play in the
(working) life of the staff,
we engaged in several field studies (in progress)
and used cultural probes
to determine what could be valuable contributions
to ask for and how to display these on
the PANORAMA screen.
In this stage, PANORAMA is still in a design phase.
We have developed, however, a first prototype implementation
using ViP technology, based on the system described in [Eliens (2006)].
In this realization, we deploy a moving virtual gallery,
containing video and image feeds.
The gallery acts like a moving scroll,
displaying information in a continuous manner, in a panorama-like way.
The images in the gallery are fed by channels, containing
information that is either due to explicit
contributions (self-reflections) or ongoing activity
in the work place (casual encounters), monitored
by camers.
Obviously, the PANORAMA system itself will be subject to
a dialectic of awareness, that is it will be present, but the staff will
only occasionally pay atention to it, dependent
on their interests and also on what visual cues and effects
the system presents to draw attention to ongoing activity.
Although we would like the system to be autonomous
in the decision how to present information, we cannot hope
to do this by computational means only, [Eliens (1988)], and
hence we need to provide
interaction markers to
invite the users to contribute actively to the system,
or influence the way information is displayed
according to their preference.
Conclusions
In this paper, we have sketched a foundation for
the aesthetics of interaction by, in summary,
making a distinction between aesthetic awareness
and aesthetic judgement.
From an epistemological perspective,
we characterized aesthetic awareness as the
pure sensuousness of mind, imposing form
on our experience and perceptions, motivated by a desire
for meaning and understanding.
As dimensions of aesthetic awareness we distinguished between
space, time and dynamics, and we provided a tentative
model for interaction dynamics based on a model
of interactive game playing.
Looking at semiotic theory, we explored whether
we could identify any rules of composition,
that could guide us in the process of design.
Finally, we briefly discussed our own work in the
area of media-rich interactive systems.
Our paper is based on a relatively extensive study
of philosophical literature,
including works of modern analytical philosophy.
In discussing our work and speculating
about possible guidelines for design,
we may well have fallen victim to our own analytical sword.
However, given our interest in the
the forces at work in media and aesthetic literacy,
we feel justified not only in presenting our
philosophical findings, but also
in our transgression of philosophical rigor,
to clarify how we make sense of our senses,
and how to apply this understanding to
the design of interactive systems.
References
- [Perspective] Alberti L.B. (1435),
- On painting and on sculpture,
Phaidon, edited by C. Grayson, 1972
- [Film] Arnheim R. (1957),
- Film as Art,
University of California Press
- [Reproduction] Benjamin W. (1936),
- The Work of Art in the Age of Mechanical Reproduction,
Online Archive -- http://www.marxists.org
- [Remediation] Bolter J.D and Grusin R. (2000),
- Remediation -- Understanding New Media,
MIT Press
- [Avantgarde] Burger P. (1981),
- Theorie de Avantgarde,
Edition Suhrkamp
- [Meditations] Cheng F. (2006),
- Cinq meditations sur la beaute,
Editions Albin Michel
- [Pragmatics] Dewey J. (1931),
- Art as Experience,
Ref ???
- [Creativity] Eliens A. (1979),
- Creativity: reflection and involvement,
Ninth Int Conf of Aesthetics, Dubrovnic, August 1979
- [Computational] Eliens A. (1988),
- Computational Art,
First Int Symposium on Electronic Art, Leonardo Supplementary Issue, Pergamon Press 1988, pp. 21-26
- [Platform] Eliens A., Huang Z., and Visser C. (2002),
- A platform for Embodied Conversational Agents based on Distributed Logic Programming,
AAMAS Workshop -- Embodied conversational agents - let\'s specify and evaluate them!, Bologna 17/7/2002
- [Media] Eliens A., Dormann C., Huang Z., Visser C. (2003),
- A framework for mixed media -- emotive dialogs, rich media and virtual environments,
Proc. TIDSE03, 1st Int. Conf. on Technologies for Interactive Digital Storytelling and Entertainment, Göobel S. Braun N.,n Spierling U., Dechau J. and Diener H. (eds>), Fraunhofer IRB Verlag, Darmstadt Germany, March 24-26, 2003
- [Topical] Eliens A. (2005),
- Topical media & game development,
Syllabus Introduction Multimedia, Vrije Universiteit, Amsterdam, http://www.cs.vu.nl/~eliens/media
- [VULife] Eliens A. and S.V. Bhikharie (2006),
- Game @ VU -- developing a masterclass for high-school students using the Half-life 2 SDK,
In Proc. GAME'ON-NA 2006, P. McDowell ed., Eurosis-ETI, pp. 49-53, Sept 19-21, Monterey, USA
- [Odyssee] Eliens A. (2006),
- Odyssee -- explorations in mixed reality theatre,
In Proc. GAME'ON-NA 2006, P. McDowell ed., Eurosis-ETI, pp. 62-64, Sept 19-21, Monterey, USA
- [Dossier] Eliens A., van Riel C., Wang Y. (2006),
- Navigating media-rich information spaces using concept graphs -- the abramovic dossier,
Proc. InSciT2006, 25-28 Oct. 2006, Merida, Spain
- [Witz] Freud S. (1958),
- Der Witz und seine Beziehung zum Unbewussten,
Fischer Verlag
- [Hermeneutics] Gadamer H-G. (1977),
- Die Aktualität des Schönen,
Reclam, Stuttgart
- [VirtualArt] Grau O. (2003),
- Virtual Art -- From Illusion to Immersion,
MIT Press
- [Interaction] Graves-Petersen M., Iversen O.S., Gall Krogh P.,
- Ludvigsen M. (2004), Aesthetic Interaction -- A Pragmatist's Aesthetics of Interactive Systems,
Proc. DIS2004, Cambridge Mass. USA, pp. 269-276
- [Presence] Hallnäss L. and Redström J. (2002),
- From Use to Presence: On the Expressions and Aesthetics of Everyday Computational Things,
ACM Trans. on Computer-Human Interactions, Vol. 9, No 2, pp. 106-124
- [Workshop] Hallnäss L., Graves Petersen M., Gall Kroch P.,
- and Pold S. (2006), Aesthetics as Critical Computing -- A Workshop on the Role of Aesthetics within the field of Information Technology,
The Fourth Decennial Aarhus Conference Aarhus, Denmark 20-24 August 2005
- [Phenomenology] Hegel G.W.F. (1807),
- Phänomenologie des Geistes,
Felix Meiner Verlag, ed. 1952
- [Sein] Heidegger M. (1927),
- Sein und Zeit,
Niemeyer Tubingen, 1993
- [Documenta] Hoet J. (1992),
- Jan Hoet -- On the way to Documenta IX,
Edition Cantz
- [HalfReal] Juul J. (2005),
- Half Real -- Video Games between Real Rules and Fictional Worlds,
MIT Press
- [Kritik] Kant E. (1781),
- Kritik der reinen Vernunft,
Felix Meiner Verlag, ed. 1976
- [Urteil] Kant E. (1781),
- Kritik der Urteilskraft,
Reclam, Stuttgart, 1976
- [Fundamental] Körner S. (1973),
- Fundamental Questions of Philosophy,
Penguin Books
- [Semiotics] Kress G. and van Leeuwen T. (1996),
- Reading Images: The Grammar of Visual Design,
Routledge
- [Imagination] Sartre J.P. (1936),
- L'imagination,
Presse Universitaire de France, ed. 8, 1971
- [Aesthetics] Saw R.L. (1971),
- Aesthetics -- An Introduction,
Anchor Books, New York
- [Erziehung] Schiller F. (1977),
- Uber die ästhetische erziehung des Menschen,
Reclam, Stuttgart
- [Meaning] Vyas, D., van der Veer, G. C. (2006),
- Experience as Meaning: Some Underlying Concepts and Implications for Design,
Proc. of 13th European Conf. on Cognitive Ergonomics. ACM Press: NY, (2006), 81-91.
- [Affordance] Vyas, D., Chisalita, C., van der Veer, G. C. (2006),
- Affordance in Interaction,
Proc. of 13th European Conference on Cognitive Ergonomics. ACM Press: NY, (2006), 92-99.
- [Panorama] Vyas D., van de Watering M., Eliens A.,
- van der Veer G. (2007), Engineering Social Awareness in Work Environments,
accepted for HCI International 2007, 22-27 July, Beijing, China
- [DeepTime] Zielinski S. (2006),
- Deep Time of the Media -- Towards an archaeology of Hearing and Seeing by Technical Means,
MIT Press
(C) Æliens
27/08/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
