OCR Document

~=------------------------------------------ '"C::'~"': " """

Computational Art

What value has the use of a computer for the

visual arts and music? The ultimate answer to this question must come from those practicing the arts. For each form of art the answer might be different. Although I feel inclined to state my opinion right away, based upon my experience with electronic music, I would rather tackle this query by taking a step back, reflecting on the possible uses of the ~omputer in the arts.

Evidently, in many branches of scientific endeavor the u~e of the computer has known significant growth. Can a similar growth be expected for the use of computers in the arts? At first sight there are many differences between the use of the computer in science and the use of the computer in art. Art or artistic experiments are not as likely to be put into numbers as, for instance, the experiments of the exact sciences. 'Moreover, whereas the goal or specification of the problem is usually clear in a scientific enterprise, one might not always be able to state a goal or criterion that must be met for an artistic enterprise.

I will not consider all possible uses of the computer in science, but will concentrate on a specific branch of Computer Science: Artificial Intelligence. Artificial Intelligence is relevant to our question since it is concerned with modelling and implementing functions that are thought to be intelligent. With this preference I state my first presupposition: artistic behavior is intelligent behavior. Although some of the results of Artificial Intelligence are controversial, this discipline of science has known some generally recognized successes, for instance in the field of computer chess.

Artificial Intelligence differs from other branches of Computer Science in that it is expressly concerned with 'symbolic computing'. This is exemplified in the research dealing with automated reasoning or computational logic, which involves investigating to what extent and how proof procedures can be effectively mechanized. The example of computational logic is of interest since, although it never attained its goal of providing procedures for discovering theorems, it has resulted in effective proof-verification programs and logic-based programming languages. Another

well-known and significant application of automated rea

soning techniques can be found in expert systems, which are increasingly becoming of interest in real-life situations.

Returning to our question, ''What value has the use of the computer in the visual arts and music?", I note that there are several ways to phrase thisquestion. For instance, it can be understood as ''What possible uses does the computer have in the arts?" But an inventory is not what I am primarily interested in. Rather, I would like to take it as querying the possibility of computational art, stressing the analogy with

A. Eliens, Centre for Mathematics and Computer Science, P.O. Box 40i9, 1009 AB Amsterdam, The Netherlands

Received 19 April 1988.

@19881SAST

Pergamon Press plc. Printed in Great Britain. 0024-094X/88 $3.00+0.0G

...

ABSTRACT

A. Eliiins

The author conducts a simple

thought experiment investigating the existence and scope of 'computational art': the utilization of the computer in the visual arts and music. In the experiment he sets the task of constructing an artifact that is capable of producing works of art. Since it appears that the artifact needs at least the capability of imagination, he queries the nature of images and imagery and argues that imagination is strongly intentional. Next he introduces the concept of notational systems, since they seem to govern the artistic activity of (not exclusively) machines. Confronted with the question of whether we are able to develop a computational analogue for taste, he finds that notational systems prove to be necessary for mediating the method of production of an artwork and the appraisal of its artistic value. Furthermore, the author shows that there are certain epistemological limits to the creativity of an imaginative device. Although the outcome of this hypothetical construction task clearly denies the possibility of an autonomously creative artifact, there seems to be no reason to worry about the opportunities for computational art: the computer appears to be a unique tool in exploring the possibilities of artistic production, guided by artists.

computational logic: to what extent can artistic behavior be automated? Answering this question in its full depth is almost impossible. Therefore, I have chosen to follow a very particular method--constructing a creative artifact, a machine that is autonomously capable of producing art. This hypothetical engineering task is not of a practical nature, though. I will not deal with the pragmatics of constructing an artistic device, but rather with the philosophical issues involved: those concerning imagination and taste. In other words, this thought experiment will function as a vehicle for developing the argument concerning the possibility and scope of computational art: the approaches to the visual arts and music that involve thee use of a computer in some essential way.

The plan of this essay is as fol

lows: I investigate the possibiiity of mechanizing the process of imagination by using techniques from Artificial Intelligence. Then I introduce notational systems as a means to formalize the production of art. I will here raise the question whether notational systems are appropriate for the visual arts. Finally, I will assess whether our device is creative. To this end I will consider the possibility of implementing taste, since I regard the task of mechanizing creativity to be dependent on the mechanization of taste.

No knowledge of Artificial Intelligence or the philosophy of art is presupposed, although it would certainly aid in appreciating the argument.

THE CONSTRUCTION OF AN IMAGINATIVE ARTIFACT

I have set the task of constructing an artifact that has the ability to imagine things, people or perhaps other artifacts and is also capable, as an artist, of producing images that can be appreciated by other people or artifacts. More specifically, I ask the question "How do we program a computer to behave like an artist?"

The reason for choosing a computer for our engineering task instead of any other mechanical device is that the computer is a device with universal computational power. If art can be automated, then it can be automated by using a computer. The physical nature of the device we intend to pro

LEONARDO, Ewc/ronic Art Suppi£mentallssue, pp. 21-25, 1988

",,~___ =':'"C' "'

gram as an artist is not of interest. What is of importance, however, is the kind of function we are trying to implement: artistic behavior. It is obvious that a simple picture-processor is not what we are looking for.

Intentionality

The class of programs we are interested in is, because of the nature of our problem, the class of programs that show 'intentionality'. Intentional behavior in this context means goaldirected behavior; more specifically, behavior thaL is somehow driven by the goal to produce images. We must implement a behavioral [unction: a function that allows the machine to react to feedback and to enter into a dialogue about its images and representations [I].

ArtifiCial Intelligence has provided a computational model of human cognitive functioning. The strength of the model lies in the fact that it has enabled the development of a variety of intelligent programs, ranging from chess-playing programs to languageunderstanding systems. The working hypothesis underlying the model is that mental functioning can be mimicked by symbolic computation. Symbolic computation must be understood as the manipulation of symbols. Regarding computation as symbol

manipulation has the advantage of separatinK the interpretation~ of the symbols from their representation~ It enables us to manipulate formally the representations according to some formal rules Wi'thout having to worry about the semantic content of the representations, provided that the rules are well chosen.

The success of the computational model of t~e human mind indicates that the image-the product of imagining-need not resemble the intermediary representations which led to it. Imagery, for instance as occurring in dreams or delusions, is more likely to be the result of a chain of symbol manipulations [2]. To quote Cohen: ". . . representation of the visual world is certainly not exclusively in visual terms. . . . actually Jrepresentations] might better be regarded as transcripts. . ." [3].

Learning.

Assume that we are able to construct a machine that is capable of performing 'ordinary' intelligent functions such as perceiving and solving simple problems. Moreover, we assume that the

A. Eliiins, Computational Art

machine is equipped with the hardware to display images. The device we have in mind, however, must be able

not only to perform these functions, but ~lso to improve on its skills. To this end, the machine must be endowed ~th die capability to learn, whether from brute experience, from intentionally experimenting with its environmentor by being given the right eX3fllples.

Le.arning by Doing. The oldest example of a ma~hine th~t is able to form concepts of perceptual regulflrities is based on a 'constructive' vi~ of perception. Perception is regarded as a process of ana!ysis-by-synthesis. This view is of particular relevance" to our task because of the assumption of an image-generating process: the incoming information is matched with the generated information. By randomly varying the generated image, one can find the right concept without prior instruction [4]. Imagery, in this view, is simply constructive activity without any input to be matched. As an explanation of the adaptive power of the imagination, the notion of randomness as an information-generating principle is somewhat unsatisfactory. Obviously, some mechanism is needed to attune the preference of the mechanism for finding regularities and structure.

Learning"by Discovery. AM, a program that discovers concepts of number-theory; is an example of how

heuristics can guide the process of learning. Starting with some primitive built-in concepts, each concept that is discovered is evaluated in terms of its interestingness by means of heuristic rules. The measure of interestingness determines where the concept will be placed on the agenda for further exploration. For example, the likeliness of discovering the concept prime is enhanced by raising the interestingness of numbers having only two factors

[5]. At the time this program was developed, this 'induction' -principle meant a significant step forward in constructing learning programs. A possible objection to this approach, however, is that the range of discovery is limited by the built-in heuristics. The generalization of this approach, applying heuristics to improve on heuristics, has curcrently not been

achieved. Clearly, though, this use of heuristics demonstrates that it is possible, in principle, to endow our device with the intention to learn and to improve on its imaginative skills.

Learning By Example. In reality, the intention to learn is not always sufficient. Significant advances, also among students of art, are often achieved by presenting the right examples-in other words by teaching. Winston [6] describes a learning program that adapts its conceptual representation of a class of objects by reacting to examples presented by its teacher. When presenting the object, the teacher tells whether the object is typical for the class of objects or how it deviates. The assumption here is that the program has the intention to learn a specific concept. The process of learning is governed by the presen

~ tation of paradigmatic examples and

counter-examples.

Motivation

The prospects for our hypothetical engineering task look good. We are able to build a machine that can form abstract percepwal categories, that can find interesting concepts, that can identify objects and, moreover, that is capable of consttUctire image-generating activity. What is SIill lacking, however, is a motiv.nional or emotional component.

A motivational system can be computationally realized as an amplification mechanism of innare, built-in drives, such as the dIM fur seIf-preservation, a cognitire infunnation-seeking drive, etc.. ~~- MO£eO""'cr, we may grant the derice me pleasure of inspecting its inner life ~ 2llowing it to take its own s&ne 2IS a snnhol of itself [8]. Thus, we bare consnuaed an artifact that is caprabIe of imagining in a nontrivial way- 11. does DOl. merely re

produce stored ~ '::imigbi be an artist. Is it crearire. ~

NOTATIOXAL SYSTEMS

In the previous section ~ bare investigated thej ~ of implementing imaginat:iwe behario£- To decide, however, wbether- ~ ...i:. succeed in constructing a c:lewice War is mJly creative, we have ID G!le a closer look at the relationship beIJ';~ ihe perceptual experience of;om ~ and the symbolic represencilloos War mediated its consuua:ioo

Depiction ver:sus Description

The depictive qualin" of an image does not depend mereh- 011. lis congruence with visual reafuy bm: 2lso on the organization of pmpenies om of which

~--"-'

\ \

the image emerges. This is even more obvious when nothing is represented in the referential sense. From this point of view, resemblance to visual reality can be best understood as the similarity between the experience of perceiving the image of the object and the experience of perceiving the object itself.

We might model the experience of an image mechanically by taking the exploratory activity as a description of the experience of an image. For instance, the visual exploration of an image can be expressed in terms of transition probabilities between the elements ofthe matrix of pixels.

Assuming the validity of this approach, does there exist a dual method of image synthesis? Can an artifact, having the experience of an image, infer its construction rules? As a historical note, Paul Klee made what he called 'Rezeptiv-Bildem' (reception

images) using his visual scanning as a construction principle.

Production Methods

To avoid the referential problem, let us take music as an example. According to Sartre, in hearing a melody we practice an 'imaginative reduction' and make an 'ideal' object of the music by de-temporalizing it to its thematic configuration [9]. The other side of this process of imaginative reduction clearly is the process of composition. Related to the processes of experience and production, the role of notation in music is that of an intermediary: it allows one to identify a piece of music as a conceptual entity, apart from its history of production.

Music is the prime example of an art with a notation. From the point of view of music history, a score allows one to make a distinction between the constitutive properties or conceptual structure of the work and its accidental, contingent properties that differ from interpretation to interpretation. From the perspective of compositional practice, systems of notation provide a method of production.

Although I shall not attempt to give a precise formal account of notational systems, following Goodman [10] I will try to delineate what I understand by notational systems in a sufficiently precise way. A notational system consists of a symbol scheme and an interpretation that defines the extension of the symbols and their combinations. The notion of extension can be explained simply as follows: if a score

contains an F-sharp, then the intended meaning is that an F-sharp will be played on the appropriate instrument. However, not all systems of symbols are notational. A notational system must adhere to certain restrictions. It must be unambiguous, in that one sym

bol does not denote several things at a

time. It may, however, be redundant, in that one particular event is denoted by several distinct symbols. On a syntactic level and in a mathematical

sense the system must be discrete. A

nondiscrete or dense system allowing arbitrarily small differences between its symbols will lead to confusion. In summarizing, a notational system can be characterized as a system that is definite about its intended interpretation by being unambiguous and sufficiently differentiated. This does not exclude all freedom of interpretation, though, since for instance an instrumentalist may further differentiate between the indications given in a score. The early history of electronic music, as made in the analog studios, shows how the pursuit of exact control and density of sound led to an abolishment of notation as a vehicle for composition. In a sense, software sound synthesis reintroduced notation, although in a non-standard way, in the form of computer programs and input. Obviously, programs share with scores the property of being definite and discrete and hence repeatable. But one must note a shift in meaning here from a product-oriented to a more processoriented interpretation of notation. In effect, if computer music is to be taken seriously, it is partly for overthrowing

the monopoly of standard musical notation through the introduction of non-standard notation in the form of programs [11].

The Role of Notation in the Visual Arts

In the history of the visual arts there is no parallel to the development and use of notation in music. A sketch cannot be taken as the analogue of a score, since, in particular for non-representational paintings, none of the pictorial properties can be dismissed as irrelevant. Obviously, there is a problem of density; although we might digitize the image, we still have no conceptual abstraction of it. However, if we take a process-oriented view, we might be able to specify the method of production of the visual image in a sufficiently abstract way and thus cre

ate the opportunity for developing a notational system [12].

To find a notational system for the visual arts we must above all conceptualize the way an image is produced. In this respect, computational art forms a natural extension of the development of art in this century. Kandinsky, for instance, searched for a 'notation for painting', with which he could compose the score for an image-"correlating colors with musical sensations to depict the inner space of subjectivity". Cubism provides another example, as it achieved a certain independence between the 'representational' and the 'presentational' aspects of painting. Somewhat

-Over-generalizing, one can say that reflecting on the method of production has given a constructivist turn to modem painting, thus preparing the way for computational art [13].

Any notational system for the visual arts unavoidably will have a strongly process- or action-oriented flavor. The use of the computer actually creates the opportunity for employing such systems in a definite and repeatable way. A notational system for the visual arts is a promise that only the computer has in store.

THE ALGORITHMIC GENIUS

An art-producing artifact must have aesthetic sensibility. If the device we envisage is going to count as a genius, it must have taste. The concept of notational systems allows us to describe the productive activity as the manipulation of the symbols of a formal system. To establish if what it produces satisfies its intentions, the device must have the capability of judgement.

The Notion of Artificial Taste

Gips and Stiny [14] have provided a computa!i°nal solution to the problem of artificial taste. They propose taking as the measurement of the aesthetic value of an image the ratio of

. visual complexity to specificational simplicity. Aesthetic rating will be high with this method if a maximum of evocative effect is produced by as efficient means as possible. They obtain these measures by matching the image with the results of a generative system consisting of a number of primitive shapes and rules for composing more complex shapes out of those previously generated. .Some refinements

A. Eliens, Computational Art

they built in include selection rules to determine what shapes are chosen and painting rules to govern the construction of compound shapes and the means to control the variability among the shapes that constitute the image. By selecting suitable shapes and applying the appropriate rules, one can generate an image that is sufficiently

similar to the original image. As a measure of specificational simplicity we can then use, for instance, the num

, ber of rul~s used to derive the image.

The scheme proposed by Gips and Stiny °relates the' appearance of an image to the constructive intentionality from which it originated. Can this scheme be applied in practice? There is clearly a trade-off here between generality and feasibility. To put it differently, one can allow a very large range of possible images, but then the search space will likely be too large for all possibilities to be generated and

tested. In addition, there may be a

more fundamental defect to the solution proposed by Gips and Stiny. Their working hypothesis is, in effect, that

( one can identifY basic elements, rules of construction and organizational principles governing the selection of rules and elements that uniquely determine the appearance of an image. However, I must note that the priqcipal difficulty'for developing a notation for the visual arts-density-may also preclude the mechanization of aesthetic sensibility: almost imperceptible changes in the basic elements might effect a completely different configuration.

Can the Machine Be Creative?

We can without doubt make our machine creative in the sense of its being able to produce novelty. In the theory of creativity, the creative process is

often conceived of as consisting of a

stage of incubation in which, so to speak, the ingredients~of the work of art are being pre;>ared, and a stage of illumination, in which the final concept is formed. The recognition oC'a new idea as valid can be explained psychologically by assuming that the idea has some excitatory value for the 'prepared mind' [15].

In order to implement creativity we must further restrict the generative system developed to give a computational description of taste such that at each step the choice that is made contributes to the novelty and 'interestingness' of the final product. Novelty as such is easily obtained by randomiz

A. Eliiins, Computational Art

ing the choice. However, the use of stochastic processes, as for instance in serial music, is not very valuable unless the parameters over which they are varied are given a definite meaning and unleOSsAhe range of variation is

,delimited in an appropriate way [16]. So we must insist that the novelty that is produced satisfies our criteria of interestingness and validity.

Since we have assumed that the imagination underlies' any artistic activity, it seems necessary to reconsider this notion more carefully. In philo

sophical terms, \ imagination is a spe

cies oCthought that, is attuned to what is intrinsically meaningful [17]. Computational models that" reduce this activity of thought to "~ere representational activity in the absence ()f input" [18] clearly lack the valuational aspect of the process of imagining.

To incorporate this valuational aspect I propose installing some ,rules for assessing the interestingness of the image or idea. But this solution has sO-me intrinsic limitations. A problem arises similar to that guiding the discovery of mathematical concepts: sooner or later the built-in heuristics

for assessing the interestingness of an

idea are not able to cope with the complexity of the newly generated ideas. The inability of the device to adapt its notions of interestingness and meaning is of an epistemological nature. When a range of concepts is delimited by built-in rules, a machine can only fill up the gaps. It can explore, if given

sufficient time, all conceRts ~ithin this

range. ~t cannot, however, except to a minor extent, enlarge this range in a significant way. An artifact is not equipped to change its categorical framework because it cannot apperceive the meaning of such a framework in constituting possible reality.

Therefore, art cannot be auto

mated. As Harold Co hen states [19], "art presumes [such] a flux of categories". A machine simply cannot be the agent of such a reflection. In other words, art is not an objective a computer can have, nor is prog[ession in art an objective a!computer can have. To complete this argument, consider it from a s?ciological point of view. Since art might have as a theme not only the form of an art product, but also the function of a work of art in society, art by an-artifact can be fully appreciated only in a community of artifacts [20]. And what kind of community would that be?

CONCLUSION

We must admit that we have failed in our engineering task of constructing an artistic device. Our failure is due to the fact that we are unable to endow the machine with the taste and creativity necessary to an artist.

Nevertheless, we should not be disappointed, since we have encountered several valuable notions that clarifY the possible use of the computer in the visual arts and music. It appears that the computer is an excellent notational device. Although, in effect, the

computer can have no more than an

instrumental status, it provides a hitherto unknown amplification of the

'constructive and combinatorial powers of the imagination. Moreover, the formalization necessary to make full use of the opportunities offered seems toche in line with the development of the arts toward a reflection on their methods of-production. I have introduced the concept of notational systems to provide the means for descri bing an image in terms of its process of construction in an abstract but precise way. Taking conslTudivii)' (which includes the selection of the material and the procedures for manipulating that material) and conceptuality (which can be characterized as the awareness of such a choice as constituting artistic acti\ity), I conclude that we must give the maChine a chance. It lies in the hands of the artists to discover where this pursuit of a notation for the visual arts ",ill lead us.

References

1. H. Cohen, "Some ~cxes on ~lachine Generated Art", in R.. Goganh. eeL, A>ptlrlm: De Com. puter in de vistute KD5t [IkusseIs: ICSAC, 1981).

2. Z.W. PyI}'5hyn. .Wbatme ~find's Ere Tells the Mind's Body:ACriUqueoHlenlalImagery", Psyelwwgiad lJuIktm (July 1973].

3. Cohen [1].

4. D.MacKay, "£MEr' . 6' TllProblemforAutomala, Shannon and }lcCanhy,£ds.. AuIomala Stu. dies (Princeton, ~: l'rinceron Unn'efSity Press, 1956).

5. D. Lenat. "TIle L"biquitrofDisoJ\-ery", Artifi. cial Intelligma9, No.. 3197i',.

6. P.H. \\rmsron, .~ ~ (Reading, MA: Addison WesIef. Ig;-;).

7. Cf. W. Reitman, ~ As a Problem

Solving Coalition", in Tomlins aDd Yessick, eds.,

ComputerSimulatimt.t(lI ~~ofPsyclwl. ogical Theory (Ne..- YOri:: joIm \\'iIer and Sons, 1963).

8. JO. WISdom., ~km2Jiq- in M2drines", Proe. ArisL Sot:. SuppL 2& 1952:,.

9. JP. Sartre. L~ "'Paris: Presses Univ. de France, 1936j.

10. Goodman :1!r6;,.

;

1 I