topical media & game development
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

talk show tell print

A theoretical foundation for the aesthetics of interaction and awareness -- making sense of the senses


Anton Eliëns, Dhaval Vyas

resource(s)

abstract

To clarify the notion of aesthetics in the context of interactive systems and to arrive at a foundation for interaction aesthetics, we start by looking at the history of thought from a philosophical stance. We distinguish between two complementary aspects of aesthetics, namely awareness and judgement. We identify space, time and dynamics as the dimensions constituting aesthetic awareness, and tentatively propose a model based on interactive game playing to clarify the meaning of interaction. From semiotic theory we derive a grammar of visual design that may aid our understanding of how interactive systems are perceived and what affordance they offer. Combined, these perspectives clarify the dynamics of the aesthetic experience of interactive systems, as an interplay between awareness and judgement, informed by art, architecture and gaming. Here we provide an outline of the dialectics of aesthetic awareness, that aids our understanding of the adoption and use of the new category of fun or awareness-oriented interactive systems, which we will illustrate by discussing our own work, in particular PANORAMA, a system supporting social awareness.
keywords: aesthetics, interaction, semiotics, affordance, emotive dialogs, augmented reality, digital dossier, social awareness

Introducion

When we think of media as an extension of our senses,  [Zielinski (2006)], we may reformulate the question of interaction aesthetics as the problem of clarifying the aesthetics of media rich interactive applications.

However, what do we mean exactly by the phrase aesthetics. The Internet Enceclopedia of Philosophy discusses under the heading of aesthetics topics such as

Aesthetics


  • intentions -- motives of the artist
  • expression -- where form takes over
  • representation -- the relation of art to reality
These topics obviously does not cover what we want, so we took the call for contributions to the aesthetics of interaction as a good chance to dust of our old books, and rekindle our interest in this long forgotten branch of philosophy, aesthetics.

It may come as a shock to realize how many perspectives apply to the notion of aesthetics, as listed below:

perspectives


  1. analytical
  2. transcendental
  3. speculative
  4. phenomenological
  5. psychological
  6. psychoanalytical
  7. pragmatical
  8. hermeneutical
  9. semiotics
  10. artistic/design
In an analytical sense, our paper may be regarded as giving a clarification of some of the terminology used, and in a synthetic sense it may be regarded as another contribution to the rethorics of interaction aesthetics. However, our intention is to arrive at a deeper understanding of the notion of aesthetics that allows us to formulate a model that provides us with the concepts that we need not only to make sense of what we experience using interactive applications, but also make sense when we approach the problem of designing such systems. Afterall, what is the real question? Ultimately to design implements or artefacts that embody the computing technology of the future.

An analytical approach

Taking a philosophical stance, a mere analytical approach shows rightout confusion and a blurred understanding of notions central to any theory of aesthetics.

For example, we encountered the definition of form as the way in which material build things.

Let us be clear, although we definitely do cherish the insights expressed in the sentences below, we refuse to accept the terminology used without ado.

usage of aesthetics (1)


  1. design aesthetics is what relates to notions of form and expression in design practice. (3)
  2. design practice where aesthetic qualities are emphasied. (4)
  3. the aesthetics of an interactive artefact evolves in the relationship with the user.
  4. experimental design aesthetics differ very little from art.
  5. when pro-active technology goes home, pragmatic aesthetics is needed. (5)
  6. there is a need for a specific basic interaction design course and knowledge about the aesthetics of interactions (7)
  7. recent trends call for a stronger focus on the aesthetics of user experiences (6)
  8. this intention is aesthetics, and aesthetics for its own sake, and even goes beyond an interest of meaning.
  9. to use software is to perceive, to grasp and to apply. We are right in the middle of aesthetics! (9)
  10. by turning to aesthetics the critical approach to computing thus includes an emancipatory aspect.
  11. very few of these actually hold any artistic or aesthetic quality, which is not surprising at all with such a new media. (10)
  12. aesthetics have their root in philosophy, which defines aesthetics as the (perceived) sense of beauty. (11)
There is a bewildering variety of ways in which the term aesthetics is used.

usage of aesthetics (2)


  • aesthetic qualities of objects
  • aesthetics of design process
  • aesthetics of user experiences
  • aesthetics as critical judgemente
  • aesthetics as sense of beauty
Still, there is confusion, for example is sense in sense of beauty meant as appearance or as faculty of perception or both?

usage of aesthetics (3)


  • aesthetics of experience
  • aesthetics of appearance
  • aesthetics of use

The notion of aesthetics in the history of thought

Some seem to easily do away with philosophical abstractions and apparently tedious theory, even though these same philosophical abstractions may provide better understanding of the forces that are at work.

perspectives

perspectives


  1. analytical -- context of usage
  2. transcendental -- abstract form of experience
  3. speculative -- criteria for beauty
  4. phenomenological -- self-conscious subjectivity
  5. psychological -- perception of form
  6. psychoanalytical -- sub-conscious meaning
  7. pragmatical -- experience of meaning
  8. hermeneutical -- understanding of the senses
  9. semiotics -- social construction of meaning
  10. artistic/design -- production of meaning through form

Dimensions of aesthetic awareness

perspective(s)


  • transcendental -- a priori sensibility
  • empirical -- a posteriori judgement
  • phenomenological -- speculative (inter) subjectivity

aesthetic awareness


  • spatial -- topological relations, layout of image
  • temporal -- order, rhythm, structure
  • dynamic -- interaction, reflection, involvement

A model of interaction

interaction constitutes the object

game theory perspectives
  • system -- (formal) set of rules
  • relation -- between player and game (affectionate)
  • context -- negotiable relation with 'real world'

dictionary


classic game model
  • rules: formal system
  • outcome: variable and quantifiable
  • value: different valorisation assignments
  • effort: in order to influence the outcome
  • attachment: emotionally attached to outcome (hooked)
  • consequences: optional and negotiable (profit?)

rules vs fiction


Game fiction is ambiguous, optional and imagined by the player in uncontrollable and unpredictable ways, but the emphasis on fictional worlds may be the strongest innovation of the video game.

degrees of interaction

degree(s) of interaction


  • direct interaction -- first person shooter
  • indirect interaction -- contributing to a plot
playful -- learnability -- visualization -- mapping -- affordance

veracity markers -- interaction markers

Experience as meaning

framework set up by Dhaval Vyas  [Vyas and van der Veer (2006)].

experience as meaning


user's experience = meaning s/he construct

framework


  • experience occurs during the interaction between the user(s) and the interactive system(s) in the lived environment
  • designers convey meaning (consciously or unconsciously) through the appearance, interaction and function of the system
  • user(s) construct a coherent whole that is a combination of sensual, cognitive, emotional and practical forms of experience
In other words, an interactive system is determined by function, interaction and appearance.

the dialectics of awareness

dialectics of awareness


  • ...-1985: self-improvement -- self for self
  • 1985-2006: continuously partially aware -- others for others
  • 2006-...: creative contributions -- self for others

The meaning of composition

narrative implications


  • objects -- the items in the image
  • vectors -- (imaginary) lines suggesting interaction
  • gaze -- inward (offer) / outward (demand)
transactional or non-transactional

composition


Composition, then, relates the representational and interactive meanings of the picture to eachother, through three interrelated systems.

representations


  • information value -- left/right, top/bottom, centre/margin
  • salience -- foreground/background, relative size, contrast
  • framing -- connecting or dissolving lines

information value


  • left/right -- given versus new
  • top/bottom -- ideal versus real
  • centre/margin -- important versus marginal

frames of reference

frames of reference


It is one of the most important formal qualities of film that every object that is reproduced appears simultaneously in two entirely different frames of reference, namely the two-dimensional and the three-dimensional, and that as one identical object it fulfills two different functions in the two contexts.

realism versus naturalism

realism


A realism is produced by a particular group as an effect of the complex of practices which define and constitute that group.

naturalism


Each realism has its naturalism, that is a realism that is a definition of what counts as real, a set of criteria for the real, and it will find its expression in the right, the best, the most natural form of representing that kind of reality, be it a photograph or a diagram.

dominant paradigm


The dominant standard by which we judge visual realism (and hence visual modality) remains for the moment, naturalism as conventionally understood, photorealism.

 [Kress and van Leeuwen (1996)]: realism is a social construct

the aesthetics of shock or surprise

ecstatic transport


Using contemporary image techniques, immersive art very often visualizes elements that can best be described as Dionysian: ecstatic transport and exhilaration.

collective memory


It is an apparent feature of the concept of immersion that it engages with the spatial and pictorial concentration of the awareness of one's own people, the formation of collective identity through powerful images that occupy the function of memory.

aesthetics of shock


It is within the realm of probability that the shock, which Walter Benjamin diagnosed as being film's aesthetic innovation, will undergo renewal and intensification with far more sophisticated means.

principles of montage


  • cutting -- unit length, whole scenes, cuts within scenes
  • time relations -- synchronized, before/after, neutral
  • space relations -- same place (different time), different place
  • subject matter -- similarity and/or contrast

film technique


  • camera -- position, focus, movement
  • transitions -- fading in/out, dissolving, stills
  • arrangement -- light/shade, color, sound

Applications to design

The notion of perspective is an interesting notion itself, since it describes both
  • the organisation of the image, as well as
  • the (optimal) point of view of the viewer.
This intricate relation between viewer and image, dependent on perspective, implies that when looked at from the 'wrong point of view', there will be a distortion of the image. The 'normal' perspective, as we know it, is the 'central' perspective. However, there are variants of perspective that force the viewer into an abnormal point of view, as for example with anamorphismes.

In a multi-dimesnional space often a change of perspective, that is a change of point of view, suffices for the correction of a reducing or distorting projection. Just image how a plane is projected as a line on an orthogonal surface, and a line as a point.

application(s)


  • digital dossiers -- immersive concept spaces
  • PANORAMA -- a system to promote social awareness
  • ViP -- augmented reality presentation technology

art versus architecture

art versus architecture


  • art -- delimited in space and time, focussed attention, reflective distance
  • architecture -- everywhere and always, sub-consious sensibility, ambient immersion

Conclusions

References
[Perspective] Alberti L.B. (1435),
On painting and on sculpture, Phaidon, edited by C. Grayson, 1972
[Film] Arnheim R. (1957),
Film as Art, University of California Press
[Reproduction] Benjamin W. (1936),
The Work of Art in the Age of Mechanical Reproduction, Online Archive -- http://www.marxists.org
[Remediation] Bolter J.D and Grusin R. (2000),
Remediation -- Understanding New Media, MIT Press
[Avantgarde] Burger P. (1981),
Theorie de Avantgarde, Edition Suhrkamp
[Meditations] Cheng F. (2006),
Cinq meditations sur la beaute, Editions Albin Michel
[Pragmatics] Dewey J. (1931),
Art as Experience, Ref ???
[Creativity] Eliens A. (1979),
Creativity: reflection and involvement, Ninth Int Conf of Aesthetics, Dubrovnic, August 1979
[Computational] Eliens A. (1988),
Computational Art, First Int Symposium on Electronic Art, Leonardo Supplementary Issue, Pergamon Press 1988, pp. 21-26
[Platform] Eliens A., Huang Z., and Visser C. (2002),
A platform for Embodied Conversational Agents based on Distributed Logic Programming, AAMAS Workshop -- Embodied conversational agents - let\'s specify and evaluate them!, Bologna 17/7/2002
[Media] Eliens A., Dormann C., Huang Z., Visser C. (2003),
A framework for mixed media -- emotive dialogs, rich media and virtual environments, Proc. TIDSE03, 1st Int. Conf. on Technologies for Interactive Digital Storytelling and Entertainment, Göobel S. Braun N.,n Spierling U., Dechau J. and Diener H. (eds>), Fraunhofer IRB Verlag, Darmstadt Germany, March 24-26, 2003
[Topical] Eliens A. (2005),
Topical media & game development, Syllabus Introduction Multimedia, Vrije Universiteit, Amsterdam, http://www.cs.vu.nl/~eliens/media
[VULife] Eliens A. and S.V. Bhikharie (2006),
Game @ VU -- developing a masterclass for high-school students using the Half-life 2 SDK, In Proc. GAME'ON-NA 2006, P. McDowell ed., Eurosis-ETI, pp. 49-53, Sept 19-21, Monterey, USA
[Odyssee] Eliens A. (2006),
Odyssee -- explorations in mixed reality theatre, In Proc. GAME'ON-NA 2006, P. McDowell ed., Eurosis-ETI, pp. 62-64, Sept 19-21, Monterey, USA
[Dossier] Eliens A., van Riel C., Wang Y. (2006),
Navigating media-rich information spaces using concept graphs -- the abramovic dossier, Proc. InSciT2006, 25-28 Oct. 2006, Merida, Spain
[Witz] Freud S. (1958),
Der Witz und seine Beziehung zum Unbewussten, Fischer Verlag
[Hermeneutics] Gadamer H-G. (1977),
Die Aktualität des Schönen, Reclam, Stuttgart
[VirtualArt] Grau O. (2003),
Virtual Art -- From Illusion to Immersion, MIT Press
[Interaction] Graves-Petersen M., Iversen O.S., Gall Krogh P.,
Ludvigsen M. (2004), Aesthetic Interaction -- A Pragmatist's Aesthetics of Interactive Systems, Proc. DIS2004, Cambridge Mass. USA, pp. 269-276
[Presence] Hallnäss L. and Redström J. (2002),
From Use to Presence: On the Expressions and Aesthetics of Everyday Computational Things, ACM Trans. on Computer-Human Interactions, Vol. 9, No 2, pp. 106-124
[Workshop] Hallnäss L., Graves Petersen M., Gall Kroch P.,
and Pold S. (2006), Aesthetics as Critical Computing -- A Workshop on the Role of Aesthetics within the field of Information Technology, The Fourth Decennial Aarhus Conference Aarhus, Denmark 20-24 August 2005
[Phenomenology] Hegel G.W.F. (1807),
Phänomenologie des Geistes, Felix Meiner Verlag, ed. 1952
[Sein] Heidegger M. (1927),
Sein und Zeit, Niemeyer Tubingen, 1993
[Documenta] Hoet J. (1992),
Jan Hoet -- On the way to Documenta IX, Edition Cantz
[HalfReal] Juul J. (2005),
Half Real -- Video Games between Real Rules and Fictional Worlds, MIT Press
[Kritik] Kant E. (1781),
Kritik der reinen Vernunft, Felix Meiner Verlag, ed. 1976
[Urteil] Kant E. (1781),
Kritik der Urteilskraft, Reclam, Stuttgart, 1976
[Fundamental] Körner S. (1973),
Fundamental Questions of Philosophy, Penguin Books
[Semiotics] Kress G. and van Leeuwen T. (1996),
Reading Images: The Grammar of Visual Design, Routledge
[Imagination] Sartre J.P. (1936),
L'imagination, Presse Universitaire de France, ed. 8, 1971
[Aesthetics] Saw R.L. (1971),
Aesthetics -- An Introduction, Anchor Books, New York
[Erziehung] Schiller F. (1977),
Uber die ästhetische erziehung des Menschen, Reclam, Stuttgart
[Meaning] Vyas, D., van der Veer, G. C. (2006),
Experience as Meaning: Some Underlying Concepts and Implications for Design, Proc. of 13th European Conf. on Cognitive Ergonomics. ACM Press: NY, (2006), 81-91.
[Affordance] Vyas, D., Chisalita, C., van der Veer, G. C. (2006),
Affordance in Interaction, Proc. of 13th European Conference on Cognitive Ergonomics. ACM Press: NY, (2006), 92-99.
[Panorama] Vyas D., van de Watering M., Eliens A.,
van der Veer G. (2007), Engineering Social Awareness in Work Environments, accepted for HCI International 2007, 22-27 July, Beijing, China
[DeepTime] Zielinski S. (2006),
Deep Time of the Media -- Towards an archaeology of Hearing and Seeing by Technical Means, MIT Press
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

(C) Æliens 23/08/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.