topical media & game development
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

talk show tell print

print / present

part ii. delivery & presentation

if you linger for a long time in one place you'd almost think there must be something there

wittgenstein

delivery & presentation


...

2

reading directives

 In this part we will look at the issues involved in delivery and presentation, primarily from a technical perspective. We will argue the importance of codecs (read compression), and we will discuss the criteria for selecting a particular codec, as well as the standards that have been developed for packaging multimedia content in an effective way. In chapter 4, we will discuss multimedia presentation platforms, and we will look at the Microsoft DirectX 9 platform in somewhat greater detail.

Essential sections are section 3.1, which introduces codecs, 3.2, which discusses the MPEG-4 and SMIL standards and section 4.1, with puts the development of high-end multimedia platforms in a historical perspective. Sections 3.3 and 4.3 can safely be skipped on first reading.

perspectives

As you can see below, the topics introduced in this part are not only relevant from a technical perspective. Other perspectives are equally valid:

perspectives -- delivery & presentation


  • technical -- codec selection
  • political -- market vs. consortium
  • sociological -- digital services
  • legal -- copyright protection
  • scientific -- experience design
  • computer science -- computational support
  • futuristic -- global & personal information
  • commercial -- WMV, Quicktime, RealONE
For example, the issues of copyrights and copyright protection are hot topics, since the rise of the internet is obviously a threat to the tradional industries of music and film distribution.

essay topics

Since many of the interesting topics will only be hinted, you may select on or more topics for further investigation and study. As essay titles I would suggest:
  • multimedia standards -- MPEG4
  • XML-based multimedia -- SMIL
  • multimedia technology -- the DirectX 9 toolbox
When you write the essay, then assess first from which perspective you will tackle the subject. When you approach the material from a technical perspective, then make sure that you do understand the technical issues in sufficient detail.

...

3

the artwork

  1. logo -- a drawing by Soutine, it is (almost) my personal logo, and also decorates the cover of  [OO].
  2. signs -- property marks,  [Signs], p. 76, 77.
  3. photographs -- Jaap Stahlie, commissioned work.

print / present / tag(s)

3

codecs and standards

without compression delivery is virtually impossible

learning objectives

After reading this chapter you should be able to demonstrate the necessity of compression, to discuss criteria for the selection of codecs and mention some of the alternatives, to characterize the MPEG-4 and SMIL standards, to explain the difference between MPEG-4 and MPEG-2, and to speculate about the feasibility of a semantic multimedia web.

Without compression and decompression, digital information delivery would be virtually impossible. In this chapter we will take a more detailed look at compression and decompression. It contains the information that you may possibly need to decide on a suitable compression and decompression scheme (codec) for your future multimedia productions.

We will also discuss the standards that may govern the future (multimedia) Web, including MPEG-4, SMIL and RM3D. We will explore to what extent these standards allow us to realize the optimal multimedia platform, that is one that embodies digital convergence in its full potential. Finally, we will investigate how these ideas may ultimately lead to a (multimedia) semantic web.

...

1

...

18

questions

3. codecs and standards

concepts


technology


projects & further reading

As a project, you may think of implementing for example JPEG compression, following  [Li and Drew (2004)], or a SMIL-based application for cultural heritage.

You may further explore the technical issues on authoring DV material, using any of the Adobe, mentioned in appendix E. or compare

For further reading I advice you to take a look at the respective specifications of MPEG-4 and SMIL, and compare the functionality of MPEG-4 and SMIL-based presentation environments. An invaluable book dealing with the many technical aspects of compression and standards in  [Li and Drew (2004)].

the artwork

  1. costume designs -- photographed from Die Russchische Avantgarde und die Buhne 1890-1930
  2. theatre scene design, also from (above)
  3. dance Erica Russel,  [Wiedermann (2004)]
  4. MPEG-4 -- bits rates, from  [Koenen (2000)].
  5. MPEG-4 -- scene positioning, from  [Koenen (2000)].
  6. MPEG-4 -- up and downstream data, from  [Koenen (2000)].
  7. MPEG-4 -- left: scene graph; right: sprites, from  [Koenen (2000)].
  8. MPEG-4 -- syntax, from  [Koenen (2000)].
  9. MIT Media Lab web site.
  10. student work -- multimedia authoring I, dutch windmill.
  11. student work -- multimedia authoring I, Schröder house.
  12. student work -- multimedia authoring I, train station.
  13. animation -- Joan Gratch, from  [Wiedermann (2004)].
  14. animation -- Joan Gratch, from  [Wiedermann (2004)].
  15. animation -- Joan Gratch, from  [Wiedermann (2004)].
  16. animation -- Joan Gratch, from  [Wiedermann (2004)].
  17. Agneta and Frieda example.
  18. diagram (Clima Futura) game elements
  19. signs -- people,  [ van Rooijen (2003)], p. 246, 247.
Both the costume designs and theatre scene designs of the russian avantgarde movement are expressionist in nature. Yet, they show humanity and are in their own way very humorous. The dance animation by Erica Russell, using basic shapes and rhythms to express the movement of dance, is to some extent both solemn and equally humorous. The animations by Joan Gratch use morphing, to transform wellknown artworks into other equally wellknown artworks.

print / present / tag(s)

4

multimedia platforms

with DirectX 9 digital convergence becomes a reality

learning objectives

After reading this chapter you should be able to characterize the functionality of current multimedia platforms, to describe the capabilities of GPUs, to mention the components of the Microsoft DirectX 9 SDK, and to discuss how to integrate 3D and video.

Almost 15 years ago I bought my first multimedia PC, with Windows 3.1 Media Edition. This setup included a video capture card and a 4K baud modem. It was, if I remember well, a 100 Mhz machine, with 16 Mb memory and a 100 Mb disk. At that time, expensive as it was, the best I could afford. Some 4 years later, I acquired a Sun Sparc 1 multimedia workstation, with a video capture card and 3D hardware accelerator. It allowed for programming OpenGL in C++ with the GNU gcc compiler, and I could do live video texture mapping at a frame rate of about one per second. If you consider what is common nowadays, a 3Ghz machine with powerful GPU, 1 Gb of memory, a 1.5Mb cable or ADSL connection and over 100 Gb of disk space, you realize what progress has been made over the last 10 years.

In this chapter, we will look in more detail at the capability of current multimedia platforms, and we will explore the functionality of the Microsoft DirectX 0 platform. In the final section of this chapter, I will then report about the work I did with the DirectX 9 SDK to implement the ViP system, a presentation system that merges video and 3D.

...

1

...

14

questions

4. multimedia platforms

concepts


technology


projects & further reading

As a project, I suggest the development of shader programs using Rendermonkey or the Cg Toolkit, or a simple game in DirectX.

You may further explore the possibilities of platform independent integration of 3D and media, by studying for example OpenML. For further reading, among the many books about DirectX, I advice  [Luna (2003)],  [Adams (2003)] and  [Fay et al. (2004)].

the artwork

  1. dutch light -- photographs from documentary film Dutch Light.
  2. ViP -- screenshot, with morphing shader, see section 4.3.
  3. impasto -- examples, see section 4.1
  4. impasto -- after a painting of van Gogh, using Cg shaders,
  5. 3D vision, from  [Sullivan (2005)], see example(s) section 4.2.
  6. idem.
  7. photographs of DirectX and multimedia books, by the author.
  8. DirectX -- diagram from online documentation.
  9. ViP -- screenshot, with the news and animations.
  10. DirectX -- diagram from online documentation.
  11. DirectX -- diagram from online documentation.
  12. ViP -- screenshot, featuring Abramovic.
  13. Peter Frucht -- Reality of TV news, see section 4.3.
  14. Clima Futura -- architecture diagram.
  15. signs -- people,  [ van Rooijen (2003)], p. 248, 249.
The theme of the artwork of this chapter is realism. In the documentary dutch light, it was investigated whether the famous dutch light in 17th century painting really existed. The photographs shown here are a selection of shots that were taken on particular locations over a period of time. However, as an art historian formulated it in the documentary: dutch light is nothing but a bag of tricks shared by dutch 17th century painters. The examples from impasto demonstrated that, after all, realism is an arbitrary notion.
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

(C) Æliens 18/6/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.