Robotic Pandas

On abstractions for building next generation user interfaces

Archive for the ‘Bling’ Category

Posts about the Bling advanced UI toolkit.

What is a programmable abstraction?

Posted by mcdirmid on 30/07/2009

In a recent blog post I described the principles behind Bling’s design. The primary principle is preferring programmable over fixed abstractions. But I feel that my definition of a programmable abstraction is so far unsatisfactory and undeveloped. To converge on a better definition, let’s first start at the beginning and discuss abstraction.

Read the rest of this entry »

Posted in Bling | Tagged: | Leave a Comment »

Announcing Bling 3!

Posted by mcdirmid on 27/07/2009

I’d like to announce a newly rewritten release of Bling with many improvements and exciting new features. Bling is a novel experiment in how WPF/UI programming can be enhanced via a lightweight domain-specific language hosted completely within C#. Bling basically changes the meaning of statements and expressions to do more that is otherwise possible with normal values in C#; e.g., a + b no longer means add the value of “a” to the value of “b,” but rather it creates tree that we can then transform into a databinding relationship, HLSL code, and so on. We exploit this to reduce or completely UI boilerplate code; e.g., no more value converters, no more HLSL code, no more C# shader classes, etc…Bling 3 goes farther with this style of programming through the addition of lifted functions (so f(a) also creates a tree!). For more details, see my blog.

Read the rest of this entry »

Posted in Bling | 3 Comments »

Bling Manifesto

Posted by mcdirmid on 22/07/2009

User interfaces toolkits are becoming increasingly complicated through frameworks with rigid architectures and numerous high-level abstractions that must be customized down to meet application requirements. In contrast, modern low-level 3D graphics toolkits are moving en masse away from similar fixed function APIs to flexible programmable shader models. For example, rather than call a point light on diffuse material fixed function, one would instead encode the math for this operation in a shader. Custom forms of lighting are easily expressed through new math. Can similar model work for user interfaces? Consider layouts: a few fixed kinds of panels (stack, flow, etc…) can be configured and composed to implement various kinds of layouts. The number of fixed layouts combined with their customization options make such APIs very complicated and difficult to learn and use. However, the ability to express layout math can accommodate any kind of layout, and since the layout is being expressed as math their are no customization options to learn and use. This brings us to the first principle of Bling’s design: to optimize for flexibility and simplicity, support bottom up construction of “programmable” abstractions as opposed to the top-down customization of high-level fixed abstractions.

The drawback of such programmable abstractions is convenience and repetition—no one wants to reinvent the wheel while inventing an effectively round wheel requires complex processes. Consider implementing a custom layout algorithm in WPF, boilerplate make expressing programmable layouts difficult and repetitive: one must define a new panel, define minimum, maximum, and preferred sizes, and deal manually with layout changing events. Later on, this panel can only be reused in its entirety: parts of the algorithm must be re-implemented in a new panel. Likewise, boilerplate in Direct3D makes it difficult to setup even simple scenes, while boilerplate to interface with shaders prevents the modularization of lighting math that can run either on the CPU or GPU. The solution to the boilerplate problem is to eliminate boilerplate. For example in expressing layouts, convenient and concise databinding to arbitrary expressions makes it practical to encode arbitrary layout constraints that automatically react to dynamic changes. Likewise, the direct expression of shader code regardless of whether on the CPU or GPU allows for the modularization of lighting code into simple functions. This brings us to the second principle of Bling’s design: avoid inconvenience and repetition through boilerplate-hiding abstractions such as those concerning databinding or function composition. Code can then simply be organized into methods or functions.

Other Bling principles:

  • To reduce confusion and keep the API as small and simple as possible, avoid redundant abstractions. For example, Bling does not support WPF storyboards because they overlap with databinding and explicit clocks to synchronize animations. Likewise, styles and triggers are easily replaced with constraints expressed via databinding.
  • Focus on math. Almost everything in UI is math heavy, be it layout, animation, gradient blending, lighting, or physics.  Therefore it makes a lot of sense to optimize our abstractions for the expression and modularization of math. Bling includes an extensive math library, support for composite tuple, matrix, and function operations, simple algebraic solving, and symbolic derivation.
  • Programmer friendliness through reasonable defaults. The days when a rectangle does not show up on the screen because the programmer forgot to set a stroke or color should be over! For programmer friendliness, Bling strives to ensure that all properties have default values, which then cause something noticeable to happen on the screen.
  • Avoid abstraction performance penalty through dynamic code generation. Layers of abstraction can be expensive in performance critical parts of a UI, such as in layout, lighting, animation, or physics. This expense can be eliminated by using abstraction-heavy code to generate unabstracted code at run-time. Furthermore, code can be generated to run on GPUs as well as CPUs, leading to even greater performance improvements while isolating programmers from the otherwise inconveniences of GPU programming.
  • To support code generation and symbolic analysis, manipulate expression trees, not values. Bling is like LINQ in that it is based on manipulating expression trees. Various tricks such as the use of operator overloading allows expression tree compositions to resemble comparable operations on values; e.g., a + b can form an addition tree node with a and b sub-trees.

Posted in Bling | 6 Comments »

Fun(ctional) graphics in C#!

Posted by mcdirmid on 20/07/2009

Graphics programming often involves customizing and combining well known techniques as mathematic formulas and algorithms related to geometry, lighting, physics, and so on. For performance and architecture reasons, realizing these formulas in a real programming involves drastic transformations that results in code that is hard to read and write. For example, multiple graphics techniques are encoded into low-level pixel and vertex shader code in way that renders them unrecognizable.

One solution is to encode graphic techniques in a high-level functional programming language, where functions allow us to effectively represent, combine, and manipulate graphic techniques. Take for example lighting computations that can be represented as a function Position3D –> Normal3D –> InputColor –> ResultColor, which can then be combined via addition: take two lighting computations l1 and l2, where a succinct l1 + l2 expands to p –> n –> c –> l1(p)(n)(c) + l2(p)(n)(c). As graphics is math-intensive, it benefits dramatically from function representations.

Unfortunately, their are a couple of drawbacks to this approach. First, the execution of a functional program is drastically slower than imperative code. However, as most graphics computations are moving to GPUs via memory-restricted shader code, this problem can be solved by generating shader code from functional programs. Such is the approach taken by Conal Elliott’s Haskell-hosted Vertigo library done at MSR. In Vertigo, geometries can be expressed as parametric surfaces, surface normals for lighting are computed via symbolic differentiation. The second drawback is more one of familiarity: the techniques seem to require abandoning our existing mainstream languages and tools for dramatically different and less familiar languages such as Haskell. However, while functional programming definitely benefits from functional languages, functional programming techniques can definitely be applied in more familiar languages. As a mainstream language, C# even provides a lambda construct to support programmers who want to dabble in functional programming; e.g., by using LINQ to query databases.

Read the rest of this entry »

Posted in Bling | 2 Comments »