Robotic Pandas

On abstractions for building next generation user interfaces

Designing a GPU-oriented geometry abstraction – Part Three

Posted by mcdirmid on 24/11/2009

In this post, I want to propose an geometry abstraction along with an example on how it works. But first I’ll clarify the goals of the abstraction:

  • Should support functional composition and transformation. This basically means that it will support operations like a + b where a and b are geometry values and the result is a geometry value or f(a) where f is a pure function from geometry value to geometry value.
  • A geometry value can be rendered by one call without any other companion values.
  • Very limited magic and hard coded properties. Only properties relevant for rendering, vertex position, pixel color, and topology, are semi-magical in that a render call will look for them. All other properties and all calculations, such as those that specify layout or apply lighting, are non-magical and expressed explicitly by manipulating/transforming geometry values.

The above goals ensure that the abstraction is easy to use and that we can easily modularize geometry-building code using standard in-language constructs such as procedures, objects, or functions (including high-order functions). Now, let’s introduce the abstraction via an example using a neutral syntax:

// recipe for face
// create topology with 4 vertices and 2 primitives
face = new_topology<D3>(4, 2);
// set indices of topology primitives to (0,1,2) and (0,2,3)
// per_primitive creates a function applied to all primitives by their integer id.
face = set_indices(face, per_primitive(id => (0, 1+id, 2+id)));

The above code defines a face as a primitive triangle topology (D3 = 3 points) with four vertices and two triangles. The code then defines an index buffer that differs according to primitive ids (per_primitive…). As a result, there are two primitives formed from vertices (0,1,2) and (0,2,3). Now we define the spatial properties of a face:

// set vertex positions to 0:(-1,-1,0), 1:(-1,+1,0), 2:(+1,+1,0), 3:(+1,-1,0)
// per_vertex creates a function applied to all vertices by their vertex id.
face = make_position(face, per_vertex(id => (id/2*2-1,(id+1)%4/2*2-1,0)));
// face texture coordinates are similar to its positions, except in 2D and from 0,0 to 1,1.
face = make_texcoord(face, per_pixel((face.position.xy + 1) / 2));
// make face normal, just point it along the -z axis.
face = make_normal(face, (0,0,-1));
// give face a transform, initially this transform is identity.
face = make_transform(face, identity);

Positions are defined per vertex using a formula over vertex ids, which could have also been expressed as a table indexed by vertex ids if we wanted (formulas are more compact). Texture coordinates are defined according to the 2D position and shifted so that they are from (0,0) to (1,1) rather than (-1,-1) to (+1,+1). Additionally, texture coordinates are defined on a per-pixel basis, so that expressions that depend on them are never interpolated a the pixel shader. One normal is defined for the face and used as the normal for all of the face’s vertices and pixels. Finally, the face is given a matrix transform property with an initial value of the identity matrix. This call also ensures that the position and normal of the face are transformed appropriately according to the transformation matrix (normal transformation will ignore the matrix’s translation components). Now to form a cube from the face:

// duplicate the face 6 times to create a cube.
cube = duplicate(6, face);
// each face is translated out to (0,0,-1) and rotated according to its id
// note: we use open gl row major transform matrices, so prepend the matrices.
// transforms update position and normals,
// for normals the translate component is ignored when the transform is applied, of course.
// face 0 is rotated 0.0pi around (0,1,0)
// face 1 is rotated 0.5pi around (1,0,0)
// face 2 is rotated 0.5pi around (0,1,0)
// face 3 is rotated 1.0pi around (1,0,0)
// face 4 is rotated 1.5pi around (0,1,0)
// face 5 is rotated 1.5pi around (1,0,0)
cube = prepend_transform(cube, per_duplicate1(id =>
  rotate((0+id%2,1-id%2,0), (i/3+(id%3).Min(1))*.5pi) * translate(0,0,-1));

The first statement duplicates the face six times to form the beginnings of a cube. The next statement prepends a transform that translates/rotates the position and normal of each face according to its ID, which is simply its duplicate ID. Now…we have a complete cube! Incidentally, we could just encapsulate all of this code into a simple make_cube function, or better yet, just use the cube value as a prototype anytime we need a cube value. Continuing on, we color and light the cube:

// makes the cube ready for rendering by giving it a color
// property, color is initially transparent
cube = make_color(cube, transparent);
// prepare cube for lighting, causes color to be computed via lighting equations.
cube = prepare_lighting(cube);
// prepare materials.
cube = add_material(cube, specular(power=3));
cube = add_material(cube, diffuse(knob=100%,ambient=100%));
// set color of cube according to face id, so each face has a different color.
cube = set_material_color(cube, per_duplicate1(id => color_table[id]));
// light cube with an ambient light at 50% of white.
lights = ambient_light(color = .5 * white);
// directional light 50% white at a weird angle
lights = add_light(lights, directional_light(color = .5*white, direction = (-.5,-.5,+1)));
// spot light 50% white directed straight on.
lights = add_light(lights, spot_light(color = .5*white, direction=(0,0,+1),
                                      position=(0,0,-10),outer=4dg, inner=2dg));
// set the lights for this cube, same lights can be applied to other geometries.
cube = apply_lights(cube, lights);

This code adds a color property to the cube (before it had none!) allowing it to be rendered, or for our purposes, prepared for lighting. The prepare_lighting call adds a bunch of properties related to lighting as well as ensuring that the cube’s color is now computed via a lighting algorithm (configured by lighting properties). The next line’s adds diffuse and specular materials to the cube, set a material color for each face, and creates lights to light the cube. The lights value is used to light the cube geometry value but could also be applied light other geometry values in the same scene.

Stepping back, lighting is a fairly complicated algorithm and there are many different was to do it. For this reason, lighting is not built into the geometry abstraction, all of the lighting related functions (prepare_lighting, apply_lights, add_material) are all explicitly defined outside of the geometry abstraction and alternative lighting schemes can be defined in the same way. To finish up the rendering:

// rotation provided by world matrix, can be updated via user input.
cube = prepend_transform(cube, world);
// finally, a camera...
camera = perspective_camera(position = (0,0,-4), fov = .25pi);
cube = apply_camera(cube, camera);
// we get a cube, multi color faces, lit with three lights.

The world matrix provides for rotation and scaling, which will trickle down to the matrix that is used to transform face normal’s and vertex positions. A camera is created and then applied to the cube to provide for standard 3D view and projection matrices, and finally the cube is rendered. As a bonus, let’s apply normal mapping to the cube faces by updating the normal of each face to include the results of a texture lookup:

map = convert_to_normal_map(folded_paper_texture);
// extract normals from textures
// per-pixel texture coordinates to sample per-pixel normals.
cube = set_normal(cube, map[cube.texcoord]).rotate(cube.normal.angle_between((0,0,-1)));

Finally, you might get something like this:


I haven’t implemented this geometry abstraction yet, so the above rendering was developed from the way Bling is today.

In the above code, we are building up the type of the geometry value along with its structure. The type of the value indicates what transformations the geometry value supports (e.g., setting an existing property’s value) and prevents invalid transformations from occurring (e.g., applying normal-based lighting without specifying a normal!). However, I’m not aware of any static type system that allows us to mix and match independent types, so I’ll probably implement this through dynamic typing in C#.

Another issue that I need to address in more detail is that of what value one is referring to during a transformation. There is the value embedded in the value we are building from and the final value after the value is completed and when it undergoes processing (obtained from an implicit self parameter). In the above code, we mostly use the former semantics, but some operations obviously depend on the latter semantics; e.g., the normal I refer to in a lighting calculation is the final transformed version of the normal and not the current normal of a lighting calculation. Ah, kind of a headache.

I promised to talk about geometry shading in my last post, but it will have to wait. Basically, the constituent values used to build up a geometry value are not necessarily constants and can be dynamic values from the external environment; e.g., a slider thumb’s state, a texture sample, or a noise value. When the cube value is processed, if a dynamic value is encountered that affects the topology of the geometry, then a geometry shader is needed. We’ll still have to add some operations to transform the topology (possibly with a different primitive type) and to “step” the geometry when obvious state is involved. I’ll definitely talk about this next time.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: