How to use a rendering shader in a scene

Jean-Marc Le Roux

Writing vertex and fragment shaders is a very important part of any 3D application. It is even more important when Flash 11 and Stage3D are involved because anything that gets rendered to the screen is rendered using shaders.
When working with Minko, you don't need AGAL assembly or PixelBender3D.
With Minko, shaders are written in ActionScript.

Minko embeds a JIT compiler that compiles ActionScript shaders into AGAL bytecode at runtime. Your entire application is then written in ActionScript 3.0. This JIT compiler performs optimizations, memory allocations, buffers and textures settings and a lot of other things for you. Because it is AS3, you can use methods, loops, conditionals, classes...

The documentation will be available in the next few days. It will contain both tutorials and technical articles. But to help you getting started, I want to share a few examples.

How to use a rendering shader in a scene

Use a shader on the whole scene:


viewport.defaultEffect = new SinglePassRenderingEffect(new MyCoolShader())

Use a shader on a specific sub-scene:


var es : EffectGroup = new EffectGroup(new SinglePassRenderingEffect(new MyCoolShader()));

es.addChild(... // add anything you want to display with this shader
_scene.addChild(es);

If you want to see the output AGAL assembly code compiled by the JIT compiler, you can use the following debug option:

Minko.debugLevel = DebugLevel.SHADER_AGAL;

How to use a post processing shader in a scene

You just have to set the Viewport.postProcessingEffect property:


_viewport = new SinglePassPostProcessingEffect(new MyCoolPostProcessingShader());

Jean-Marc Le Roux

Simple texture


public class SimpleTextureShader extends ActionScriptShader
{
  // returns the vertex position in clipspace (normalized screenspace [-1 .. 1])
  override protected function getOutputPosition() : SValue
  {
    return vertexClipspacePosition;
  }

  override protected function getOutputColor() : SValue
  {
    // we have to use "interpolate" for every value coming from the vertex shader
    var uv : SValue = interpolate(vertexUV);

    return sampleTexture(BasicStyle.DIFFUSE, uv);
  }
}

Jean-Marc Le Roux

Solid Red

public class SimpleShader extends ActionScriptShader
{
  override protected function getOutputPosition() : SValue
  {
    return vertexClipspacePosition;
  }

  // returns red
  override protected function getOutputColor() : SValue
  {
    return float4(1., 0., 0., 1.);
  }
}

Jean-Marc Le Roux

Color transform


public class SimpleTextureShader extends ActionScriptShader
{
  private var _offset : Vector4 = null;
  private var _multiplier : Vector4 = null;

  public function ColorTransformShader(offset : Vector4, multiplier : Vector4)
  {
    _offset = offset;
    _multiplier = multiplier;
  }

  override protected function getOutputPosition() : SValue
  {
    return vertexClipspacePosition;
  }

  override protected function getOutputColor() : SValue
  {
    // we have to use "interpolate" for every value coming from the vertex shader
    var uv : SValue = interpolate(vertexUV);
    var diffuse : SValue = sampleTexture(BasicStyle.DIFFUSE, uv);

    diffuse = multiply(diffuse, multiplier);
    diffuse = add(diffuse, _offset);

    return diffuse;
  }
}

Jean-Marc Le Roux

LightCubeShader

This shader comes from the Gravity demo and requires the minko-lighting extension.


public class LightCubeShader extends ActionScriptShader
{
        override protected function getOutputColor() : SValue
        {
                var pointLights         : WorldDataList = getWorldDataList(LightData);
                var numLights           : int                   = pointLights.length;
                var illumination        : SValue                = float3(0., 0., 0.);

                for (var lightIndex : int = 0; lightIndex &lt; numLights; ++lightIndex)
                {
                        var lightPosition       : SValue        = getWorldParameter(3, LightData, LightData.LOCAL_POSITION, lightIndex);
                        var lightDiffuse        : SValue        = getWorldParameter(3, LightData, LightData.PREMULTIPLIED_DIFFUSE_COLOR, lightIndex);
                        var squareLocalDist     : SValue        = getWorldParameter(1, LightData, LightData.SQUARE_LOCAL_DISTANCE, lightIndex);

                        var lightToPoint        : SValue        = subtract(interpolate(vertexPosition), lightPosition);                         
                        var lightDirection      : SValue        = normalize(lightToPoint);
                        var lambertFactor       : SValue        = saturate(interpolate(vertexNormal).dotProduct3(lightDirection));

                        // hacky hacky...
                        var attenuation         : SValue        = saturate(multiply(squareLocalDist,
                                                                                                                   reciprocal(dotProduct3(lightToPoint, lightToPoint))));

                        // here goes the hack again...
                        lightDiffuse.scaleBy(attenuation)
                                                .scaleBy(.8);
                        illumination.incrementBy(lightDiffuse);
                }

                var uv          : SValue        = interpolate(vertexUV);
                var diffuse     : SValue        = sampleTexture(BasicStyle.DIFFUSE, uv);

                illumination = power(illumination, 1.2);

                return float4(multiply(diffuse.rgb, illumination), diffuse.a);
        }

        override protected function getOutputPosition() : SValue
        {
                return vertexClipspacePosition;
        }

        override public function getDataHash(styleData          : StyleData,
                                                 transformData  : TransformData,
                                                 worldData              : Dictionary) : String
        {
                return (worldData[LightData] as WorldDataList).length.toString(16);
        }
}

Jean-Marc Le Roux

Greyscale Post-Processing Shader


public class GreyscalePostProcessingShader extends PostProcessingActionScriptShader
{
	override protected function getFinalColor(outputColor : SValue) : SValue
	{
		return float4(
			float3(divide(add(outputColor.r, outputColor.g, outputColor.b), 3.)),
			1.0
		);
	}
}

AlexandroG

Please help to port from GLSL to Minko ActionScriptShader


// Tangent, Binormal, Normal - float3
// vNormal - float3 too.

// Transform the surface normal into world space (in order to compute reflection
// vector to perform environment map look-up):
float3x3 mTangentToWorld = transpose( float3x3( Tangent, Binormal, Normal ) );
float3   vNormalWorld    = normalize( mul( mTangentToWorld, vNormal ));

So question is:
how to define a 3x3 matrix by three float3 vectors and multiply this matrix on float3

ADDED
is multiply3x3 solution?

Regards.

Jean-Marc Le Roux

Hello,

how to define a 3x3 matrix by three float3 vectors and multiply this matrix on float3

If you are trying to perform normal mapping, you should consider transforming stuff into tangent space instead of transform the normal into world space. Mainly because their is no "transpose" method. This is because there is no "transpose" AGAL operation.

Yet, if you want to have the equivalent of the following GLSL code:

float3x3 mTangentToWorld = transpose( float3x3( Tangent, Binormal, Normal ) );
float3   vNormalWorld    = normalize( mul( mTangentToWorld, vNormal ));

it should look like this in AS3:


// compute the vertex bitangent (aka binormal)
var vertexBitangent : SValue    = cross(vertexNormal, vertexTangent);

// tangent space to local space
var a : SValue = float3(vertexTangent.x, vertexBitangent.x, vertexNormal.x);
var b : SValue = float3(vertexTangent.y, vertexBitangent.y, vertexNormal.y);
var c : SValue = float3(vertexTangent.z, vertexBitangent.z, vertexNormal.z);

var localNormal = float3(
  dotProduct3(normal, a),
  dotProduct3(normal, b),
  dotProduct3(normal, c)
);

localNormal = normalize(localNormal);

We have to do it that way because:
- We can't allocate matrices in the shader. This is a technical limitation to make sure the memory allocation is as efficient as possible. But this is not an actual problem because you can still use float3/float4 with dotProduct3/dotProduct4 operations.
- There is no transpose() method anyway, because there is no transpose AGAL operation. So we have to build the transposed matrix by defining the a, b, c vectors as demonstrated above.

Please note that it would be much more efficient to do the opposite and transform local values into tangent space:


private function toTangentSpace(vector : SValue) : SValue
{
  var vertexBitangent	: SValue	= cross(vertexNormal, vertexTangent);
			
  return float3(
    dotProduct3(vector, vertexTangent),
    dotProduct3(vector, vertexBitangent),
    dotProduct3(vector, vertexNormal)
  );
}

Regards,

AlexandroG

Great!

AlexandroG

New question. How to make quad aligned to screen?
In GLSL I do this like:


pos = size * (gl_Vertex.x * view_inverse_matrix[0] 
+ gl_Vertex.y * view_inverse_matrix[1]).xyz;
// other calculataions
gl_Position = gl_ModelViewProjectionMatrix * vec4(pos,1.0);

Regards.

Jean-Marc Le Roux

How to make quad aligned to screen?
pos = size * (gl_Vertex.x * view_inverse_matrix[0] + gl_Vertex.y * view_inverse_matrix[1]).xyz; // other calculataions gl_Position = gl_ModelViewProjectionMatrix * vec4(pos,1.0);

Here your are re-multiplying by the inverse of the view matrix to make sure the view matrix won't have any effect. It's a bit complicated.

I've been working on a particle engine for minko, and here is how I do it:
- I've a mesh with lots of quads (as many quads as I have particles)
- each quad has 4 vertices, each vertices has a 2D position and the ID of the particle (the vertex format is made of one component: PARTICLE_DATA = VertexComponent.create(["x", "y", "id"], VertexComponentType.FLOAT_3))
- in the vertex shader, something like this:


override protected function getOutputPosition() : SValue
{
  var p : SValue	= getVertexAttribute(PARTICLE_DATA);
  var xy : SValue	= p.xy;

  var id : SValue = p.z;
  // get the center of the quad in view space
  var center : SValue = multiply4x4(float4(0, 0, 0, 1), localToViewMatrix);
  // get the size of the quad
  var scale : SValue = multiply4x4(float4(1, 1, 0, 1), localToViewMatrix);
  var offset : SValue	= float4(multiply(xy, scale.xy), 0., 0.);

  return multiply4x4(add(center, offset), projectionMatrix);
}

The idea is to store the corners of the quad in the vertex stream, then you scale them, then your translate them. When you have the final position in view space, you apply the projection matrix.

Regards,

Lance Linder

Greetings,

I am trying to create a billboard effect using a custom shader. This example for aligning a quad to screen seems to be what I want however when trying to implement it I get "Bad AGAL source operands. Both are constants (this must be precomputed) at token 1 of vertex program."

Seems like "var center : SValue = multiply4x4(float4(0, 0, 0, 1), localToViewMatrix);" is producing "m44 vt0, vc0, vc1" which is causing the error.

Another example I have been studying that uses raw AGAL to accomplish this is here http://goo.gl/xrW7t

I am rather new to Stage3D and Minko so any help getting this working would be greatly appreciated :)

Lance

Jean-Marc Le Roux

Greetings,

"Bad AGAL source operands. Both are constants (this must be precomputed) at token 1 of vertex program."

AGAL forbids operations with both constant (vc/fc) registers. This is to make sure constant computations are done on the CPU. But it is sometimes too big of a constraint. This is why in Minko 2, the compiler detects those situations and do a "mov" of constant registers when it is required. This is why I forgot this kind of problems when I posted the code :)

To fix it, Minko 1 has the "copy" instruction: it turns constants into temporaries.
The code above can be fixed like this:


var center : SValue = multiply4x4(float4(0, 0, 0, 1), copy(localToViewMatrix));

I've been working on a little particles editor using the piece of code above. For now I'm trying to see if the shader lab could somehow be used to create those particles effects. And I think it just might... If we can't do it with the shader lab, we will add a new tool to create particles.

The interesting thing here with AS shaders is that you can create this kind of interface:


public interface IParticleSystem
{
  function getPosition(id : SValue, time : SValue) : SValue;
  function getRotation(id : SValue, time : SValue) : SValue;
  function getAlpha(id : SValue, time : SValue) : SValue;
  function getColor(id : SValue, time : SValue) : SValue;
}

Then, you can create particles effect by implementing this interface (and extendind ActionScriptShaderPart of course otherwise you won't have all the shader operations). The editor just output an object that can be used to initialized such IParticleSystem object.

And voilà, you just got yourself an hardware accelerated particles engine fully written in AS3 :)

I'll soon blog about this and Minko 2 JIT AGAL compiler new optimizations (constants detection, constants compression, temporary registers allocation...).

Live long and prosper,

Jean-Marc

Lance Linder

Thank you for taking the time to write such a detailed response.

The work you are doing on Minko 2.0 and the particle system sounds very exciting! Can't wait to experiment with the new version :)

As you suggested the copy() instruction did solved the compile error. Unfortunately I ended up with another even more ambiguous error but that doesn't matter... if Minko 2.0 is in the works I will just sit tight and try again when its released.

On a side note I have to say I have found Minko to be refreshingly easy to work with. From a programing and extensibility standpoint it is a work of art. Finally I can spend less time trying to figure out how to extend a 3d framework and more time actually creating new and interesting extensions. :D

Cheers,
Lance

Jean-Marc Le Roux

As you suggested the copy() instruction did solved the compile error.

What error do you get?

if Minko 2.0 is in the works I will just sit tight and try again when its released

Minko already works just fine so you don't need to wait. If you have questions/problems, post here and we will try to help :)

Happy new year!

Jean-Marc

Lance Linder

Jean-Marc,

Though I have got basic shaders to work in Minko I am far from an expert in 3d math so the error is most certainly due to my lack of understanding.

Two things in the example above that have me stumped are variables "offset" and "id" which are created but never used and the reference to variable "corner" that doesn't exist.

My understanding is that the variable "id" refers to the individual vertices that make up the quad but the example doesn't actually show it being used. In the example above line "return multiply4x4(add(center, corner), projectionMatrix);" references a variable called "corner" yet in the rest of the body this variable isn't defined and I am not sure what to supply here.

I think what I am missing is how to create the variable "corner" which I assume is somehow derived from "offset" and "id"?

As a feeble attempt I have tried using both "id" and "offset" in place of the missing "corner" but both endup with the following AGAL error:
Error #3614: The native shader compilation failed.enGL specific:

Thanks for the help and happy new year!
Lance

tecbot

Hello Jean-Marc,

i need your help for my shader. I want to make the following opcode:


dp4 v0.x, va0, vc0
dp4 v0.y, va0, vc1
mov v0.zw, va0.zw

This is for a uv transformation with offset, scale and rotation.
The va0 is the vertex UV.
The vc0-1 ist the UV transform and the data comes from a Matrix:


uvTransform[0] = matrix.a; uvTransform[1] = matrix.b; uvTransform[2] = 0; uvTransformData[3] = matrix.tx;
uvTransform[4] = matrix.c; uvTransform[5] = matrix.d; uvTransform[6] = 0; uvTransformData[7] = matrix.ty;

how can i make this with the ActionScriptShader?

Regards,
Thomas

How to use a rendering shader in a scene

How to use a post processing shader in a scene

Howdy, Stranger!

Categories

In this Discussion

Tagged