At the basis of rendering lies traditional computer graphics technology. That is, the transformation of vertices (points in 3D space), rasterization (that is determining the pixel locations and pixel properties corresponding to the vertices), and finally the so-called raster operations (determining whether and how the pixels are written to the framebuffer). OpenGL, developed by SGI was the first commonly available software API (application programmers interface) to control the process of rendering. Later, Microsoft introduced Direct3D as an alternative for game programming on the PC platform.

The process outlined above is called the graphics pipeline. You put models, that is collections of vertices, in and you get (frames of) pixels out. This is indeed a simplification in that it does not explain how, for example, animation and lighting effects are obtained. To gain control over the computation done in the graphics pipeline, Pixar developed Renderman, which allows for specifying transformations on the models (vertices) as well as operations on the pixels (or fragments as they are called in  [Cg]) in a high level language. As vertex operations you may think of for example distortions of shape due to a force such as an explosion. As pixel operations, the coloring of pixels using textures (images) or special lighting and material properties. The languages for specifying such vertex or pixel operations are collectively called shader languages. Using offline rendering, almost anything is possible, as long as you specify it mathematically in a computationally feasible way.

The breakthrough in computer graphics hardware was to make such shading languages available for real-time computer graphics, in a way that allows, as  [Cg] phrase it, 3D game and application programmers and real-time 3D artists to use it in an effective way.

Leading to the programmable computer graphics hardware that we know today,  [Cg] distinguish between four generations of 3D accellerators. The phrase GPU was introduced by NVIDIA to indicate that the capabilities of the GPU far exceed those of the VGA (video graphics array) originally introduced by IBM, which is nothing more than a dumb framebuffer, requiring updates from the CPU.

4 generations of GPU

• Before the introduction of the GPU, there only existed very expensive specialized hardware such as the machines from SGI.
• The first generation of GPU, including NVIDIA TNT2, ATI Rage and 3dfx Voodoo3, only supported rasterizing pre-transformed triangles and some limited texture operations.
• The second generation of GPUs, which were introduced around 1999, included the NVIDIA GeForce 2 and ATI Radeon 7500. They allowed for both 3D vertex transformations and some lighting, conformant with OpenGL and DirectX 7.
• The tird generation GPUs, including NVIDIA GeForce 3, Microsoft Xbox and ATI Radeon 8500, included both powerful vertex processing capabilities and some pixel-based configuration operations, exceeding those of OpenGL and DirectX 7.
• Finally, the fourth generation of GPUs, such as the NVIDIA GeForce FX and ATI Radeon 9700, allow for both complex vertex and pixel operations.

  texture tex;
  float4x4 wvp;	// World * View * Projection matrix
  
  sampler tex_sampler = sampler_state
  {
      texture = /;    
  };
  

  struct vsinput {
      float4 position : POSITION; 
      float3 normal : NORMAL; 
      float2 uv : TEXCOORD0;
  };
  struct vsoutput {
      float4 position   : POSITION;   // vertex position 
      float4 color    : COLOR0;     // vertex diffuse color
      float2 uv  : TEXCOORD0;  // vertex texture coords 
  };
  

  vsoutput vs_id( vsinput vx ) {
      vsoutput vs;
    
      vs.position = mul(vx.position, wvp);
      vs.color = color;
      vs.uv = vx.uv; 
      
      return vs;    
  }
  

  struct psoutput
  {
      float4 color : COLOR0;  
  };
  
  
  psoutput ps_id( vsoutput vs ) 
  { 
      psoutput ps;
  
      ps.color = tex2D(tex_sampler, vs.uv) * vs.color;
  
      return ps;
  }
  

  technique render_id
  {
      pass P0
      {          
          VertexShader = compile vs_1_1 vs_id();
          PixelShader  = compile ps_2_0 ps_id(); 
      }
  }
  

     float3 spherePos = normalize(vx.position.xyz);
     float3 cubePos = 0.9 * vx.position.xyz;
  
     float t = frac(speed * time);
     t = smoothstep(0, 0.5, t) - smoothstep(0.5, 1, t);
  
     // find the interpolation factor
     float lrp = lerpMin + (lerpMax - lerpMin) * t;
  
     // linearly interpolate the position and normal
     vx.position.xyz = lerp(spherePos, cubePos, lrp);
     vx.normal = lerp(sphereNormal, cubeNormal, lrp);
  
     // apply the transformations
     vs.position = mul(wvp, vx.position);
  

      float4 x = tex2D(tex_sampler, vs.uv);
      if (x.r > x.g && x.r > x.b) { x.r *= xi; x.g *= xd; x.b *= xd; }
      else  if (x.g > x.r && x.g > x.b) { x.g *= xi; x.r *= xd; x.b *= xd; }
      else  if (x.b > x.r && x.b > x.g) { x.b *= xi; x.r *= xd; x.g *= xd; }
      ps.color = x;
  

  shape descriptor: XLB=10; XUB=60; YLB=3; YUB=50   (rectangle)  
  property descriptor: pixel(14,7): R=5; G=1; B=3 
  

  property: (bwcolor,{b,w},bwalgo) 
  

           d(x,y) = d(y,x)
  	 d(x,y)  d(x,z) + d(z,y)
  	 d(x,x) = 0
  

   $to\_1,...,to\_r$ -- translation, rotation, scaling
  

   rotate(image-id,dir,angle)
   segment(image-id, predicate)
   edit(image-id, edit-op)
  

  precision = ( returned and relevant ) / returned  
  recall = ( returned and relevant ) / relevant 
  

  video v, frame f 
  f has associated objects and activities 
  objects and activities have properties
  

  property: name = value 
  

   (fd,fi) -- frame-dependent and frame-independent properties 
  

   { giver : Person, receiver : Person, item : Object } 
   giver = Jane, receiver = Dennis, object = briefcase 
  

  which videos are in the library 
  what constitutes the content of each video
  what is the location of a particular video
  

  SELECT -- v:[s,e] 
  FROM -- video:<source><V> 
  WHERE -- term IN funcall 
  

  SELECT  vid:[s,e]
  FROM video:VidLib
  WHERE (vid,s,e) IN VideoWithObject(Dennis) AND
  	object IN ObjectsInVideo(vid,s,e) AND
  	object != Dennis AND
  	typeof(object) = Person