Robotic Pandas

On abstractions for building next generation user interfaces

Bling Manifesto

Posted by mcdirmid on 22/07/2009

User interfaces toolkits are becoming increasingly complicated through frameworks with rigid architectures and numerous high-level abstractions that must be customized down to meet application requirements. In contrast, modern low-level 3D graphics toolkits are moving en masse away from similar fixed function APIs to flexible programmable shader models. For example, rather than call a point light on diffuse material fixed function, one would instead encode the math for this operation in a shader. Custom forms of lighting are easily expressed through new math. Can similar model work for user interfaces? Consider layouts: a few fixed kinds of panels (stack, flow, etc…) can be configured and composed to implement various kinds of layouts. The number of fixed layouts combined with their customization options make such APIs very complicated and difficult to learn and use. However, the ability to express layout math can accommodate any kind of layout, and since the layout is being expressed as math their are no customization options to learn and use. This brings us to the first principle of Bling’s design: to optimize for flexibility and simplicity, support bottom up construction of “programmable” abstractions as opposed to the top-down customization of high-level fixed abstractions.

The drawback of such programmable abstractions is convenience and repetition—no one wants to reinvent the wheel while inventing an effectively round wheel requires complex processes. Consider implementing a custom layout algorithm in WPF, boilerplate make expressing programmable layouts difficult and repetitive: one must define a new panel, define minimum, maximum, and preferred sizes, and deal manually with layout changing events. Later on, this panel can only be reused in its entirety: parts of the algorithm must be re-implemented in a new panel. Likewise, boilerplate in Direct3D makes it difficult to setup even simple scenes, while boilerplate to interface with shaders prevents the modularization of lighting math that can run either on the CPU or GPU. The solution to the boilerplate problem is to eliminate boilerplate. For example in expressing layouts, convenient and concise databinding to arbitrary expressions makes it practical to encode arbitrary layout constraints that automatically react to dynamic changes. Likewise, the direct expression of shader code regardless of whether on the CPU or GPU allows for the modularization of lighting code into simple functions. This brings us to the second principle of Bling’s design: avoid inconvenience and repetition through boilerplate-hiding abstractions such as those concerning databinding or function composition. Code can then simply be organized into methods or functions.

Other Bling principles:

  • To reduce confusion and keep the API as small and simple as possible, avoid redundant abstractions. For example, Bling does not support WPF storyboards because they overlap with databinding and explicit clocks to synchronize animations. Likewise, styles and triggers are easily replaced with constraints expressed via databinding.
  • Focus on math. Almost everything in UI is math heavy, be it layout, animation, gradient blending, lighting, or physics.  Therefore it makes a lot of sense to optimize our abstractions for the expression and modularization of math. Bling includes an extensive math library, support for composite tuple, matrix, and function operations, simple algebraic solving, and symbolic derivation.
  • Programmer friendliness through reasonable defaults. The days when a rectangle does not show up on the screen because the programmer forgot to set a stroke or color should be over! For programmer friendliness, Bling strives to ensure that all properties have default values, which then cause something noticeable to happen on the screen.
  • Avoid abstraction performance penalty through dynamic code generation. Layers of abstraction can be expensive in performance critical parts of a UI, such as in layout, lighting, animation, or physics. This expense can be eliminated by using abstraction-heavy code to generate unabstracted code at run-time. Furthermore, code can be generated to run on GPUs as well as CPUs, leading to even greater performance improvements while isolating programmers from the otherwise inconveniences of GPU programming.
  • To support code generation and symbolic analysis, manipulate expression trees, not values. Bling is like LINQ in that it is based on manipulating expression trees. Various tricks such as the use of operator overloading allows expression tree compositions to resemble comparable operations on values; e.g., a + b can form an addition tree node with a and b sub-trees.

6 Responses to “Bling Manifesto”

  1. John "Z-Bo" Zabroski said

    “15. Everything should be built top-down, except the first time.” ~ Alan Perlis, Epigrams on Programming

    I don’t disagree that this is how controls should be built, but usually in interfaces with volatile changes, the important thing is simply to avoid irrelevant detail.

    I dislike the suggestion triggers, storyboards, etc. in WPF are somehow higher level than Bling. They’re simply more rigid abstractions, at relatively equal abstraction levels. You should know from experience that some programmers struggle with less rigid abstractions.

    Thus, to modify your first principle:

    Sean: to optimize for flexibility and simplicity, support bottom up construction of “programmable” abstractions as opposed to the top-down customization of high-level fixed abstractions.

    Z-Bo: to optimize for flexibility and simplicity, support bottom up construction of “programmable” abstractions, eventually providing affordances for top-down customization of high-level problem domain specific abstractions (list controls, etc).

    • mcdirmid said

      I disagree.

      Fixed (“high-level/rigid”) abstractions are completely replaced by programmable abstractions + some domain unspecific way of modularizing the code/logic that builds up the programmable abstractions (e.g., classes, functions, rules…). The fixed domain-specific abstractions provide no/little benefit and make the library + language difficult to learn/use/master/customize/and so on. This is a hard brick wall, and its why all the 3D toolkits (DirectX 10, OpenGL ES 2.0) are basically dumping fixed abstraction en masse. Its not really the performance that is the issue, its the ability to express what the fixed abstractions are incapable of expressing. Legacy fixed abstractions are then supported by an emulation layer formed…by procedures and functions.

      Programmers struggle with programmable abstractions because they quite simply struggle with programming, while fixed abstractions make things easier at the beginning only if you don’t need to do much configuration (hence, less programming). The best example would be layout: take a typical flow layout manager and as long as you don’t need to configure it, its easier to use than to specify layout mathematically. However, as soon as you need to tweak the flow or combine it with other layouts, the complexity piles on pretty quickly. Now take the flow layout algorithm and package it into a function for a programmable layout manager. Its still as easy to specify flow, but you can break the function open if you want and specify your own custom layout pretty easily, then create a new function to make that layout reusable.

      What about a grid? Its not difficult to build up a grid from scratch (in Bling) using canvas. A basic grid can be modularized in a small function, but once you understand how easy it is to build the grid (maybe by looking at the source of the basic grid function), its much easier to imagine how to build heavily customized grids from scratch that are completely impossible to build using the WPF grid widget.

      For most of the fixed WPF abstractions, there are nice programmable alternatives (e.g., a trigger as a simple condition in a databinding clause!). The one exception might be template, which customizes child entities based on type-based rules. But then that is just dynamic overriding, or scoped rule-based dispatch, and is something that SuperGlue did easily without a separate fixed abstraction.

      • John "Z-Bo" Zabroski said

        I’m not sure what you are disagreeing to.

        I get the sense we are talking past each other.

        I always try to ask myself, “Am I advancing my argument, or am I advancing the other person’s understanding of my perspective?”

        I agree that standard WPF layout stinks. I have a good example where if you override a default setting of a control. all of a sudden the alignment changes (I’ll write this one on my blog when I remember what it is, but I haven’t used WPF’s layout model in so long that I’ve forgotten most of that ugliness).

        I agree that programmable abstractions give you a very *elastic* API. The abstractions can be stretched in all directions.

        @Programmers struggle with programmable abstractions because they quite simply struggle with programming, while fixed abstractions make things easier at the beginning only if you don’t need to do much configuration (hence, less programming).

        This is completely true, but I still see elastic abstractions vs. inelastic abstractions as attempting to solve problems on the same level (mostly). I do see some obscenely powerful potential in Bling for stuff we’re targeting on doing in Silverlight, but I mostly have prototypes right now and nothing that convinces me I’ve got something I can charge $12,000 a month to customers for. Actually, what I find I don’t like is the notion of a Grid at all… or even a list control. They’re special cases, not general cases, of a general query data model.

        On a related note, you may appreciate this quote from Autofac IoC contributor Rinat Abdullin, which I saved:

        Synergy effects do not accumulate just from the good things. Ceteris paribus, any fixed force in a complex system that constantly pushes in specified direction, makes its effects accumulate in geometrical progression. Le Bon Gustave in his “La psychologie des foules” talks about similar tendencies in the sociological systems, while Alexander Bogdanov in “Tektology: Universal Organization Science” applied these principles to any system.

        Simply put: constant denial of efficient development principles in a project leads to problems that stack up and slow the development till it is too expensive to go on.
        ” ~ Rinat Abdullin

      • mcdirmid said

        I’m promoting what you call elastic abstractions over inelastic ones. My point is that any inelastic abstraction can be expressed as an elastic one, so why have inelastic abstractions? For example, if a layout can be packaged into a function, then why not? Actually, my argument/perspective is continually evolving, I find myself re-implementing inelastic abstractions in Bling using elastic ones, just because they are more convenient. The only difference is that the architecture is much more simplified in the sense that everything can be boiled down to stateless function application. The open source nature of inelastic abstractions in Bling (since we need the math to be gray vs. black box) then allows us to copy/modify the implementation of the function if we want to create something new.

        For example, we’ve just gotten done re-implementing in Bling various kinds of lights (point, directional, spot, ambient), materials (diffuse, specular, glass, diffraction), and cameras (perspective) for use in 3D lighting computations. These are very much inelastic in the sense that the API resembles (or is the same as) the WPF 3D fixed-function API. Whats the difference? All of the Bling fixed abstractions encapsulate just math without any rigid architecture, and they are applied via function application. Otherwise, we can always break down and combine these abstractions in different ways by providing them with different function application; e.g., we can use a light without a material, or a material without a light. The positions and normals can come from a parametric surface, or they can be derived from an image-based height field and normal map.

        So I guess I’m not really disagreeing, I’m just had trouble understanding your point. I’m still evolving my argument, so this discussion is greatly appreciated.

      • John "Z-Bo" Zabroski said

        I’m still evolving my point, too. It’s mostly marketing toward my CTO (and partly to my CEO), and then finally to the dev team.

        Give some thought to my expression trees template selector hack/fix… I am pretty close to arriving at what I really desire… the ability to automagically match data structures, data types and algorithms based on some configuration.

        I showed my template selector fix to some MS employees, and the feedback I got was that they hadn’t used anything even approaching that modest level of programmable power.

        I mainly use Bling-like stuff right now, because I’ve gotten so frustrated with the standard model. However, I still use some WPF controls as “atoms” of functionality. e.g. TextBox is very useful thing I would not want to roll my own for.

  2. John "Z-Bo" Zabroski said

    Also, check out:

    I was going to do a whole series on this, but only did about three posts. I’m really busy this summer. The CEO loves the new project I’m working on, so features just don’t stop coming.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: