Menu
▾
▴
Pixel Bender Integration
Pixel Bender Integration - Functional and Design Specification
Glossary
Pixel Bender - Pixel Bender is a programming language designed for authoring hardware-independent image-processing algorithms. The Pixel Bender Toolkit, can be used to create custom filters for Flash and Flex.
\
\
shader - A shader used in the context of Pixel Bender, is essentially a compiled instance of a Pixel Bender kernel, that at runtime, executes on all the pixels of an image (or in the case of Flash, a DisplayObject), one pixel at a time.
Summary and Background
NOTE: This document assumes some working knowledge of Pixel Bender and the Flash 10 classes introduced to work with Pixel Bender shaders.
In the Gumbo time frame, a set of bindable bitmap filter classes (found in the mx.filters package) were introduced. These filters mirror those provided in Flash, but are modifiable at runtime. As changes are made to the filter instances, the filters that are applied to a display object are automatically re-applied.
Flash 10 introduced the ShaderFilter, which allows one to essentially create custom bitmap filters, by authoring a Pixel Bender shader, and then applying that shader as a filter.
The feature described in this document will introduce a new Flex equivalent to the Flash ShaderFilter, while also providing a more declarative-friendly interface to the underlying Pixel Bender shader instance (abstracting away many of the complexities of working with Pixel Bender shaders).
In addition, a new AnimateFilter Effect is introduced. The Blur and Glow effects in Flex today take care of automatically attaching and animating a blur and glow filter (respectively) to a target object. The AnimateFilter allows any filter (inclusive of the new ShaderFilter) to be applied and animated.
Usage Scenarios
Jane wishes to leverage a Pixel Bender shader she found on the Adobe Pixel Bender Exchange in her own Flex application as a declarative filter in her MXML document.
Joe has written his own Pixel Bender shader, and wants to apply it to his Flex component and animate the properties of the shader over time as part of an existing Effect sequence.
Detailed Description
ShaderFilter
The Flex ShaderFilter class is provided in order to abstract away some of the details of utilizing the Flash Shader, ShaderFilter, and ShaderData classes. It also simplifies setting and animating input parameters of generic Pixel Bender shaders, so that the class can be used easily in a declarative setting (MXML).
When applying a Pixel Bender shader as a bitmap filter using generic AS3 code in Flash 10, it looks something like this:
// myShader.pbj represents a compiled pixel bender shader's byte code.
<a href="Embed%28source%3D%26quot%3BmyShader.pbj%26quot%3B%2C%20mimeType%3D%26quot%3Bapplication/octet-stream%29">Embed(source="myShader.pbj", mimeType="application/octet-stream)</a>
var MyShaderClass:Class;
// Create new shader instance and initialize with embedded byte code.
var shader:Shader = new Shader();
shader.byteCode = new MyShaderClass();
// Configure desired input parameters of shader.
shader.data.radius.value = <a href="50">50</a>;
// Apply the shader to a DisplayObject's filter stack
// by wrapping it in a ShaderFilter instance.
myButton.filters = <a href="new%20ShaderFilter%28shader%29">new ShaderFilter(shader)</a>;
The flow is essentially:
- Embed the compiled Pixel Bender bytecode.
- Create a new Shader instance.
- Assign your embed data to its bytecode property.
- Configure properties of the shader by addressing each input parameter via the 'data' property of the shader, and assigning the new value as an array to the value property of the input parameter.
- Instantiate a new ShaderFilter instance with the created Shader.
- Insert the new filter into the filter stack of a display object.
The Flex ShaderFilter class helps to abstract away some of these details and takes care of most of the work for you, including re-applying the filter to the filter stack when any of the properties of the filter change (such as during an animation).
The Flex ShaderFilter class must be initialized with bytecode representing a Pixel Bender shader. After that the class will create (behind the scenes) a Flash Shader instance from the byte code. The ShaderFilter class also serves as a proxy directly to the underlying Shader, and provides a convenience mechanism for accessing both scalar and multi-dimensional Shader input parameters directly as named properties:
Take the example above, now using the Flex ShaderFilter class, paired with the new Flex compiler support for embedding Pixel Bender byte code as a Shader instance directly:
// MyShaderClass will be realized as a Shader instance pre-initialized
// with the myShader.pbj byte code.
<a href="Embed%28source%3D%26quot%3BmyShader.pbj%26quot%3B%29">Embed(source="myShader.pbj")</a>
var MyShaderClass:Class;
// Create new ShaderFilter instance and initialize with Shader.
var filter:ShaderFilter = new ShaderFilter(new MyShaderClass());
// Configure desired input parameters of shader.
filter.radius = 50;
// Apply the shader to a DisplayObject's filter stack.
myButton.filters = <a href="filter">filter</a>;
Proxied Shader Properties
Note in the previous code example, that the ShaderFilter greatly simplifies access to the input parameters, now simple scalar shader properties can be accessed directly (e.g. shader.radius). Multidimensional input parameters can be accessed by name as well:
// center is of type Float2
shader.center = <a href="10%2C20">10,20</a>;
In additional the class allows setting and animating individual components of multidimensional shader input parameters using a property suffix convention, for instance, the code below is equivalent to the above for an input parameter of type 'Float2' in Pixel Bender terms:
// center is of type Float2
shader.center_x = 10;
shader.center_y = 20;
The specific rules of how properties are proxied through to the underlying shader and its input parameters (ShaderData) are as follows:
- If the property identifier fits the pattern 'NAME_' and the value being set is a scalar, we look for a matching multidimension shader input parameter 'NAME' and interpret the provide suffix 'S' as a named dimension as detailed below.
- We interpret the property name 'as is' otherwise (if 'NAME' does not match an n-dimensional property on the shader, or if the value provided is of type array).
For shader input parameters of type BOOL2, BOOL3, BOOL4, FLOAT2, FLOAT3, FLOAT4, INT2, INT3, or INT4, we support either "r g b a", "x y z w", or "s t p q" as convenience suffixes, to access the 1st, 2nd, 3rd and 4th component respectively.
For shader input parameters of type MATRIX2×2, MATRIX3×3, or MATRIX4×4, and of 'a b c d e f g h i j k l m n o p" are supported as property suffixes, to access the 1st - 16th component of a given matrix.
Examples:
// translate is an input parameter of type MATRIX3x3
// This example sets the 3rd component of the given matrix.
myFilter.translate_c = 27;
// center is an input parameter of type FLOAT3
// This example sets the 3rd component of the given float array.
myFilter.center_z = 15;
// This example replaces the entire array.
myFilter.center = <a href="15%2C15%2C15">15,15,15</a>;
Built-in Properties of ShaderFilter:
The ShaderFilter class supports the following flash.filters.ShaderFilter properties, allowing the affect bits of the shader to extend beyond the physical bounds of the display object:
- leftExtension
- rightExtension
- topExtension
- bottomExtension
The ShaderFilter class supports the following flash.display.Shader properties, as documented in the Flash 10 Shader documentation:
- precisionHint
The ShaderFilter class also provides access directly to the underlying Shader instance via the shader property.
AnimateFilter Effect
The AnimateFilter effect is a more generic version of what the Blur and Glow effects do today. It allows any filter to be applied and optionally animated during the context of an Effect or Transition.
The effect itself extends the new Gumbo Animate effect, and provides an additional input property bitmapFilter of type IBitmapFilter.
The only real difference between the stock Animate effect and the AnimateFilter effect, is that the properties that the effect is animating apply not to the target of the Effect, but the associated filter instead. So for example over time, the extents of a drop shadow can be animated during an effect sequence.
The AnimateFilter effect currently applies the associated filter when the effect begins, and removes it when the effect finishes.
API Description
ShaderFilter
package mx.filters {
use namespace flash_proxy;
/**
*
*/
public dynamic class ShaderFilter extends Proxy
implements IBitmapFilter, IEventDispatcher
{
//--------------------------------------------------------------------------
//
// Constructor
//
//--------------------------------------------------------------------------
/**
* Constructor.
* @param shader Fully realized flash.display.Shader instance, or
* Class representing a Shader (such as from an Embed).
*/
public function ShaderFilter(shader:*=null);
//--------------------------------------------------------------------------
//
// Properties
//
//--------------------------------------------------------------------------
//----------------------------------
// shader
//----------------------------------
/**
* An object representing the Shader to use with this filter, either a Class
* or Shader instance is allowed.
*/
public function get shader():Shader;
public function set shader(value:*);
//----------------------------------
// bottomExtension
//----------------------------------
/**
* @copy flash.filters.ShaderFilter#bottomExtension()
*/
public function get/set bottomExtension:Number;
//----------------------------------
// topExtension
//----------------------------------
/**
* @copy flash.filters.ShaderFilter#topExtension()
*/
public function get/set topExtension:Number
//----------------------------------
// leftExtension
//----------------------------------
/**
* @copy flash.filters.ShaderFilter#leftExtension()
*/
public function get/set leftExtension:Number
//----------------------------------
// rightExtension
//----------------------------------
/**
* @copy flash.filters.ShaderFilter#rightExtension()
*/
public function get/set rightExtension:Number
//----------------------------------
// precisionHint
//----------------------------------
/**
* The precision of math operations performed by the shader.
* The set of possible values for the precisionHint property is defined
* by the constants in the ShaderPrecision class.
*
* @see flash.display.Shader
*/
public function get/set precisionHint:String
//--------------------------------------------------------------------------
//
// ShaderData Proxy
//
// getProperty and setProperty support special case conventions for
// targetting specific indexes of multi-dimensional shader properties.
//
// If the property identifier fits the pattern 'NAME_' and the value
// being set is a scalar, we look for a matching multidimension shader
// input parameter 'NAME' and interpret the provide suffix 'S' as a named
// dimension as detailed below (indexForDimension).
//
// We interpret the property name 'as is' otherwise (if 'NAME' does not
// match an n-dimensional property, or if the value provided is an array).
//--------------------------------------------------------------------------
/**
* @private
* Proxies all property 'gets' to the owned shader instance.
*/
override flash_proxy function getProperty(name:*):*;
/**
* @private
* Proxies all property 'sets' to the owned shader instance.
* If the shader bytecode has yet to be set or instanced, we
* queue the properties for later application.
*/
override flash_proxy function setProperty(name:*, value:*):void;
/**
* @private
* Proxies method calls to our shader instance.
*/
override flash_proxy function callProperty(name:*, ... args):*;
//--------------------------------------------------------------------------
//
// IBitmapFilter/BaseFilter
//
//--------------------------------------------------------------------------
/**
* @private
* Notify of a change to our filter, so that filter stack is ultimately
* re-applied by the framework.
*/
public function notifyFilterChanged():void;
/**
* @private
* Returns a native flash.filters.ShaderFilter instance suitable
* for application in a DisplayObject filter stack.
*/
public function clone():BitmapFilter;
//--------------------------------------------------------------------------
//
// IEventDispatcher
//
//--------------------------------------------------------------------------
/**
* @private
*/
public function addEventListener(type:String, listener:Function, useCapture:Boolean = false,
priority:int = 0, useWeakReference:Boolean = false):void;
/**
* @private
*/
public function dispatchEvent(event:Event):Boolean;
/**
* @private
*/
public function hasEventListener(type:String):Boolean;
/**
* @private
*/
public function removeEventListener(type:String, listener:Function,
useCapture:Boolean = false):void;
/**
* @private
*/
public function willTrigger(type:String):Boolean;
}
AnimateFilter Effect
package spark.effects
{
/**
* This effect applies an IBitmapFilter instance and allows you to animate
* an arbitrary set of properties of the filter between values, as specified
* by the propertyValuesList.
*/
public class AnimateFilter extends Animate
{
include "../core/Version.as";
//--------------------------------------------------------------------------
//
// Class constants
//
//--------------------------------------------------------------------------
/**
* @private
*/
private static var AFFECTED_PROPERTIES:Array = <a href="%20%26quot%3Bfilters%26quot%3B%20"> "filters" </a>;
//--------------------------------------------------------------------------
//
// Constructor
//
//--------------------------------------------------------------------------
/**
* Constructor.
*/
public function AnimateFilter(target:Object = null, filter:IBitmapFilter = null);
//--------------------------------------------------------------------------
//
// Properties
//
//--------------------------------------------------------------------------
//----------------------------------
// bitmapFilter
//----------------------------------
/**
* IBitmapFilter instance to apply and animate.
*/
public var bitmapFilter:IBitmapFilter;
//--------------------------------------------------------------------------
//
// Methods
//
//--------------------------------------------------------------------------
/**
* By default, the affected properties are the same as those specified
* in thepropertyValuesListarray. If subclasses affect
* or track a different set of properties, they should override this
* method.
*/
override public function getAffectedProperties():Array /* of String */
{
return AFFECTED_PROPERTIES;
}
}
}
AnimateFilter Effect Instance
package spark.effects.supportClasses
{
/**
* The AnimateFilterInstance class implements the instance class for the
* AnimateFilter effect. Flex creates an instance of this class when
* it plays a AnimateFilter effect; you do not create one yourself.
*/
public class AnimateFilterInstance extends AnimateInstance
{
//--------------------------------------------------------------------------
//
// Constructor
//
//--------------------------------------------------------------------------
/**
* Constructor.
*/
public function AnimateFilterInstance(target:Object);
//--------------------------------------------------------------------------
//
// Properties
//
//--------------------------------------------------------------------------
//----------------------------------
// bitmapFilter
//----------------------------------
/**
* IBitmapFilter instance to apply and animate.
*/
public var bitmapFilter:IBitmapFilter;
//--------------------------------------------------------------------------
//
// Methods
//
//--------------------------------------------------------------------------
/**
* @copy mx.effects.IEffectInstance#startEffect()
*
* Here the effect instance will apply the owned bitmap filter.
*/
override public function startEffect():void;
/**
* @copy mx.effects.IEffectInstance#finishEffect()
*
* Here the effect instance will remove the owned bitmap filter.
*/
override public function finishEffect():void;
/**
* Unlike Animate's setValue we assign the new value to the filter
* associated with our effect instance rather than the target of
* the effect.
*
* @private
*/
override protected function setValue(property:String, value:Object):void;
/**
* Unlike Animate's getValue we return the value of the property requested
* from the filter associated with our effect instance rather than
* the effect target.
*
* @private
*/
override protected function getCurrentValue(property:String):Number;
}
}
B Features
Not applicable.
Examples and Usage
ShaderFilter
To leverage the ShaderFilter class you must first embed the bytecode representing a compiled Pixel Bender kernel (as compiled with the Pixel Bender Toolkit), within your ActionScript class, e.g.:
/***********************************
* EMBEDDED 'SPHERIZE' SHADER DATA:
*
* parameter 'center' ==> type: float2, minValue: float2(-200,-200), maxValue: float2(800,500),
* defaultValue: float2(400,250), description: "displacement center"
*
* parameter 'radius' ==> type: float, minValue: float(.1), maxValue: float(400),
* defaultValue: float(200), description: "radius"
*/
<a href="Bindable">Bindable</a>
<a href="Embed%28source%3D%26quot%3Bshaders/spherize.pbj%26quot%3B%29">Embed(source="shaders/spherize.pbj")</a>
private static var SpherizeShader:Class;
Then, to leverage this Pixel Bender shader as a filter within an MXML document:
<mx:Label text="ABCDEF">
<mx:filters>
<ShaderFilter shader="{SpherizeShader}" radius="25" center_x="50" center_y="15" />
</mx:filters>
</mx:Label>
You may also use the @Embed MXML compiler directive directly to embed your pixel bender data.
<mx:Label text="ABCDEF">
<mx:filters>
<ShaderFilter shader="@Embed(source='shaders/spherize.pbj')" radius="25"
center_x="50" center_y="15" />
</mx:filters>
</mx:Label>
AnimateFilter Effect
To utilize an embedded Pixel Bender shader from within an animated Effect sequence, again, first embed the compiled bytecode:
/***********************************
* EMBEDDED 'PIXELATE' SHADER DATA:
*
* parameter 'size' ==> type: float, minValue: float(1), maxValue: float(200),
* defaultValue: float(50), description: "Hexagon Size"
*
* parameter 'base' ==> type: float2, minValue: float2(-200,-200), maxValue: float2(800,500),
* defaultValue: float2(400,250), description: "Base Point"
*/
<a href="Bindable">Bindable</a>
<a href="Embed%28source%3D%26quot%3Bshaders/pixelate.pbj%26quot%3B%29">Embed(source="shaders/pixelate.pbj")</a>
private static var PixelateShader:Class;
Then, declare a ShaderFilter instance and leverage one or more AnimateFilter instances as desired:
<Declarations>
<!-- Pixelate Filter -->
<ShaderFilter id="pixelator" shader="{PixelateShader}" base_x="{ticker.width/2}"
base_y="{ticker.height/2}" topExtension="50" bottomExtension="50"
rightExtension="50" leftExtension="50"/>
<!-- Pixelate Effect Sequence -->
<Linear id="linearEase"/>
<mx:Sequence id="pixelator_seq">
<AnimateFilter bitmapFilter="{pixelator}" duration="500" target="{myLabel}"
easer="{linearEase}" effectEnd="rotateHeadline()">
<PropertyValuesHolder property="size" values="<a href="0.5%2C%2030">0.5, 30</a>"/>
</AnimateFilter>
<AnimateFilter bitmapFilter="{pixelator}" duration="500" target="{myLabel}"
easer="{linearEase}" >
<PropertyValuesHolder property="size" values="<a href="30%2C%200.5">30, 0.5</a>"/>
</AnimateFilter>
</mx:Sequence>
</Declarations>
Now you can simply play the new effect sequence, which will, in this case, apply the animation to a component instance with the id, 'myLabel':
pixelator_seq.play();
Additional Implementation Details
Not applicable.
Prototype Work
The ShaderFilter and AnimateFilter classes are available in the Gumbo SDK trunk currently.
Compiler Work
Not applicable.
Web Tier Compiler Impact
Not applicable.
Flex Feature Dependencies
Effects and filters infrastructure.
Backwards Compatibility
Syntax changes
Not applicable.
Behavior
Not applicable.
Warnings/Deprecation
Not applicable.
Accessibility
Not applicable.
Performance
Not applicable.
Globalization
Not applicable.
Localization
Compiler Features
Not applicable.
Framework Features
Not applicable.
Issues and Recommendations
None.
Documentation
Not applicable.
QA
Suggested focus areas and test cases to consider (not exhaustive):
- Test a ShaderFilter instance on both a Gumbonent, as well as legacy UIComponent (and a container or two).
- Test that when modifying a ShaderFilter property dynamically at runtime, that the object the filter is attached to updates accordingly automatically (without having to reapply the filter).
- Test the ShaderFilter with various types of input parameters (float, float2, float3, matrices, etc.).
- Test the ShaderFilter's leftExtension, topExtension, rightExtension, and bottomExtension properties.
- Test the ShaderFilter's precisionHint property.
- Test creating a new ShaderFilter instance from AS3 only (using the shader parameter in the constructor of the newly created ShaderFilter for instance).
- Test the AnimateFilter effect with a ShaderFilter instance.
- Test the AnimateFilter effect with otehr types of Flex filters.
