Shadows in WebGL Part 1December 12, 2011

Posted by Andor Salga in FSOSS, Open Source, point cloud, webgl, XB PointStream.

In my last blog I wrote about an anaglyph demo I created for my FSOSS presentation in October. It was part of a series of delayed blogs which I only recently had time to write up. So, in this blog I’ll be proceeding with my next fun experiment: Shadows in WebGL.

Shadows are useful since they not only add realism, but can also provide additional visual cues in a scene. Having never implemented any type of shadows, I started by performing some preliminary research and found that there are numerous methods to achieve this effect. Some of the more common techniques include:

• vertex projection
• projected planar shadows

I chose vertex projection since it seemed very straightforward. After a few sketches, I got a fairly good grasp of the idea. Given the position for a light and vertex, the shadow cast (for that vertex) will appear at the line intersection between the slope created by those points and the x-intercept. If we had the following values:

• Light = [4, 4]
• Vertex = [1, 2]

Our shadow would be drawn at [-2, 0]. Note that the y component is zero and would be equal to zero for all other vertices since we’re concentrating on planar shadows.

At this point, I understood the problem well; I just needed a simple formula to get this result. If you run a search for “vertex projection” and “shadows” you’ll find a snippet of code on GameDev.net which provides the formula for calculating the x and z components of the shadow. But if you actually try it for the x component:

$Sx = Vx - \frac{Vy}{Ly} - Lx$
$Sx = 1 - \frac{2}{4} - 4$
$Sx =-3.5$

It doesn’t work.

When I ran into this, I had to take a step back to think about the problem and review my graphs. I was convinced that I could contrive a working formula that would be just as simple as the one above. So I conducted additional research until I eventually found the point-slope equation of a line.

Point-Slope Equation

The point-slope equation of a line is useful for determining a single point on the slope give the slope and another point on the line. This is exactly the scenario we have!

$y - y1 = m(x - x1)$

Where:
m – The slope. This is known since we have two given points on the line: the vertex and the light.

[x1, y1] – A known point on the line. In this case: the light.

[x, y] – Another point on the line which we’re trying to figure out: the shadow.

Since the final 3D shadow will lie on the xz-plane, the y components will always be zero. We can therefore remove that variable which gives us:

$-y1 = m(x - x1)$

Now that the only unknown is x, we can start isolating it by dividing both sides by the slope:
$-\frac{y1}{m} = \frac{m(x - x1)}{m}$

Which gives us:
$-\frac{y1}{m} = x - x1$

And after rearranging we get our new formula, but is it sound?
$x = x1 - \frac{y1}{m}$

If we use the same values as above as a test:
$x = 4 - \frac{4}{\frac{2}{3}}$
$x = -2$

It works!

I now had a way to get the x component for the shadow, but what about the z component? What I did so far was create a solution for shadows in 2 dimensions. But if you think about it, both components can be broken down into 2 2D problems. We just need to use the z components for the light and point to get the z component of the shadow.

The shader code is a bit verbose, but at the same time, very easy to understand:

void drawShadow(in vec3 l, in vec4 v){
// Calculate slope.
float slopeX = (l.y-v.y)/(l.x-v.x);
float slopeZ = (l.y-v.y)/(l.z-v.z);

// Flatten by making all the y components the same.
v.y = 0.0;
v.x = l.x - (l.y / slopeX);
v.z = l.z - (l.y / slopeZ);

gl_Position = pMatrix * mVMatrix * v;
frontColor = vec4(0.0, 0.0, 0.0, 1.0);
}


Double Trouble

The technique works, but its major issue is that objects need to be drawn twice. Since I’m using this technique for dense point clouds, it significantly affects performance. The graph below shows the crippling effects of rendering the shadow of a cloud consisting of 1.5 million points—performance is cut is half.

Fortunately, this problem isn’t difficult to address. Since detail is not an important property for shadows, we can simply render the object with a lower level of detail. I had already written a level of detail python script which evenly distributes a cloud between multiple files. This script was used to produce a sparse cloud—about 10% of the original.

Matrix Trick

It turns out that planar shadows can be alternatively rendered using a simple matrix.

void drawShadow(in vec3 l, in vec4 v){
// Projected planar shadow matrix.
mat4 sMatrix = mat4 ( l.y,  0.0,  0.0,  0.0,
-l.x,  0.0, -l.z, -1.0,
0.0,  0.0,  l.y,  0.0,
0.0,  0.0,  0.0,  l.y);

gl_Position = pMatrix * mVMatrix * sMatrix * v;
frontColor = vec4(0.0, 0.0, 0.0, 1.0);
}


This method doesn’t offer any performance increase versus vertex projection, but the code is quite terse. More importantly, using a matrix opens up the potential for drawing shadows on arbitrary planes. This is done by modifying all the elements of the above matrix.

Future Work

Sometime in the future I’d like to experiment with implementing shadows for arbitrary planes. After that I can begin investigating other techniques such as shadow mapping and shadow volumes. Exciting! (:

Anaglyph Point CloudsNovember 18, 2011

Posted by Andor Salga in FSOSS, Open Source, point cloud, webgl, XB PointStream.

A couple of weeks ago I gave a talk at FSOSS on XB PointStream. For my presentation I wanted to experiment and see what interesting demos I could put together using point clouds. I managed to get a few decent demos complete, but I didn’t have a chance to blog about them at the time. So I’ll be blogging about them piecemeal for the rest of the month.

The first demo I have is an anaglyph rendering. Anaglyphs are one way to give 2D images a depth component. The same object is rendered at two slightly different perspectives using two different colors. Typically red and cyan (blue+green) are used.

The user wears anaglyph glasses, which have filters for both colours. A common standard is to use a red filter for the left eye and a blue filter for the right eye. These filters ensure each eye only sees one of the superimposed perspectives. The mind them merges these two images into a single 3D object.

Method

There are many ways to achieve this effect. One method which involves creating two asymmetric frustums can be found here. However, you can also create the effect by simply rotating or translating the object. It isn’t as accurate, but it’s very easy to implement:

// ps is the instance of XB PointStream
// ctx is the WebGL context

ps.pushMatrix();
// Yaw camera slightly for a different perspective
cam.yaw(0.005);
// Create a lookAt matrix. Apply it to our model view matrix.
ps.multMatrix(M4x4.makeLookAt(cam.pos, V3.add(cam.pos, cam.dir), cam.up));

// Render the object as cyan by using a colour mask.
ps.render(pointCloud);
ps.popMatrix();

// Preserve the colour buffer but clear the depth buffer
// so subsequent points are drawn over the previous points.
ctx.clear(ctx.DEPTH_BUFFER_BIT);

ps.pushMatrix();
// Restore the camera's position for the other perspective.
cam.yaw(-0.005);
ps.multMatrix(M4x4.makeLookAt(cam.pos, V3.add(cam.pos, cam.dir), cam.up));

// Render the object as red by using a colour mask.
ps.render(pointCloud);
ps.popMatrix();


Future Work

I hacked together the demo just in time for my talk at FSOSS, but I was left wondering how much better the effect would look if I had created two separate frustums instead. For this I would need to expose a frustum() method for the library. I can’t see a reason not to add it considering this is a perfect use case, so filed!

XB Awesome at FSOSS 2011October 28, 2011

Posted by Andor Salga in Arius3D, FSOSS, Open Source, point cloud, webgl, XB PointStream.
1 comment so far

Tomorrow I’ll be giving a talk at FSOSS 2011 titled “XB PointStream: Rendering Point Clouds with WebGL”. Okay, the name is a bit dry, but I’ve packed a lot of awesome into this talk. If you liked my recent post on turbulent point clouds, you should definitely come to my talk! (:

I’ll be in room S2169 at 2:00, hope to see you there!

XB PointStream 0.6 ReleasedMarch 24, 2011

Posted by Andor Salga in Open Source, webgl, Arius3D, XB PointStream, FSOSS, point cloud.

I’m happy to announce we just released XB PointStream 0.6! In this release we fixed a number of problems and added many features. We managed to tie together a number of tools which use point clouds including Arius3D, 3D Studio Max and Blender. Go ahead and upgrade to Firefox 4 and check out the demos page!

Change Log

The following are some of the major additions we added to the library:

• Created a PSI parser for native Arius3D point clouds – read the blog
• Created a PTS parser for 3D Studio Max point clouds – read the blog
• Created an ASC exporter for Blender – read the blog
• Added a reference testing framework using Sundae.jsread the blog
• Added a documentation generator
• Replaced setInterval with requestAnimationFrame
• Fixed coloring issues on Chrome on OSX 10.6

Tweening Point Clouds in XB PointStreamNovember 5, 2010

Posted by Andor Salga in Arius3D, FSOSS, Open Source, point cloud, Processing.js, webgl, XB PointStream.

Read a boring blog? Screw it, take me to the demo!

Last week Seneca College held its 9th annual Free Software and Open Source Symposium (FSOSS). This year was especially exciting since we had a slew of great WebGL talks and workshops.

Matthew Postill presented his work on SceneCreator, a Web app which enables users to build 3D environment. [Blurb]

Cathy Leung organized an Open Web Game Jam and presented the results [Blurb, Recording]

• I ran a Processing.js Game Development workshop [Blurb, Wiki]

Mickael Medel gave a talk on XB PointStream [Blurb, Recording]

Tips on Tweening

We were lucky to have developers from Arius3D Dan Arnold and Anu Rastogi attend Mike’s talk. Cathy, Mike and I spoke briefly with them after the talk about the project and the presentation. They mentioned we could add a new dimension to future presentations by adding morphing or tweening to the point clouds. Theoretically it should be easy since point clouds lack any sort of topology and thus can be freely deformed.

Tweening in Processing.js

I liked the idea of morphing point clouds, but before implementing it in 3D, I gave it a go in 2D using Processing.js. It turned out to be very simple, only requiring a few lines of code. Check out the demo.

Tweening in XB PointStream

Since the principles were the same, I figured I shouldn’t have issues extending it to 3-space. So I tried morphing two point clouds in XB PointStream, but that didn’t turn out well. Our library just locks up when trying to load more than one point cloud in a single canvas. We have a bug filed for this issue and I think it’s something we should fix.

Since I couldn’t morph two point clouds, I had to cheat and programatically create a second shape to morph into. I chose a sphere since I could just normalize a set of random vectors as the .ASC file streamed. This of course involved cracking open the library and hacking the JavaScript and shaders. But I just wanted to get it done to see if it could work.

The end result (see top) isn’t bad and the use case forces us re-think certain parts of the library. For example, while trying to tween two point clouds I first had to make sure they both contained the same number of points. This took some time. It could have been faster if I could specify to only render every other point. Also, since I had to change the shaders, allowing users to specify their own would be useful.

In the near future we’ll have Mickey morphing into Eggenburg. Now that will be slick!

Game Development using Processing.js Workshop @ FSOSS2010September 24, 2010

Posted by Andor Salga in FSOSS, Game Development, Games, Open Source, Processing.js, webgl.

Update: The Wiki page which has links to the demos, examples and games can be found here.

Last year at FSOSS, Al MacDonald gave a presentation on Processing.js. He talked about its potential, the community, and he demonstrated how to develop a neat Twitter widget. Don’t get me wrong, Processing.js was still impressive even then, but it had a while to go in terms of language parsing, 3D (using WebGL) and bugs.

During that time there was a group of students (including me) who had recently started to contribute to the project—to fix those bugs and add those features. Since then we have resolved hundreds of issues making Processing.js closer and closer to the Processing language. We had a lot of help as well. Developers joined the IRC channel we created or forked our repository and began contributing too.

Sometime between last FSOSS and this coming one, my supervisors (Dave Humphrey and Cathy Leung) and I spread the word about the project at conferences like WWW2010 and OCE2010. It was great telling everyone about the language—especially the story of Seneca’s involvement—but there was something we hadn’t quite tapped into…So we kicked around the idea of having a Processing.js workshop.

This is what we needed.

It seemed Processing.js had matured over the few months since we started contributing to it. Having a hands-on, guided class at FSOSS would really demonstrate to the open source community the power of this library. It would be informative, interactive and fun.

There was also talk about having a Game Jam, so I wrote up a proposal which leaned towards game development and titled it “Game Development using Processing.js“. But the material learned during the workshop can just as easily apply to data-visualization or other interactive graphics.

I’m excited about conducting the workshop and I already began jotting down notes and ideas which could be covered. There’s about a month until the workshop…But I also have quite a bit of work ahead of me. So for the next few weeks I’ll be putting together source code, simple demos, links, games and wikis. If you have any suggestions or comments, feel free to comment!

Hope to see you there!