Bug Fixes and Things

It turns out that using this background was actually masking some nasty bugs I had with not enough knots on my color and skeleton spline. Those issues, which looked pretty bad against a white background, have been fixed. The skeleton spline knot error that I had was causing the piecewise sinusoidal alpha attenuation function not to work properly (because it's defined over [o, 1], and some print statements informed me that I was only going to [0, 0.6], which is close to where alpha is a maximum.)

The blending seems decent enough, although I am sure it's possible to find some really pathological colors to mix together that look bad. Here, I have preserved the alpha the blending; in fact, the resulting alpha value for two splines that collide is equal to the greatest value of the attenuation function. I did this to make sure that colliding splines don't seem to "fade" when the center where one aurora is strongest collides with the edge of another spline, where that spline is supposed to fade away to an alpha of zero.

The next two projects I want to look at are getting some kind of a brightness adjustment, perhaps with a noise map, and looking at some notion of depth so that I can further refine the aurora blending.

To be honest, I think I am close to maximizing the potential of this kind of structure. In combination with different shapes, or small piecewise auroras, it could be effective. Or possibly another approach is needed entirely. One thing that does bother me, for example, is that we have no easy way of having pixels cast rays at the normal to the NURBS curve. It would be really nice if the color spread followed the actual normals instead of some arbitrary normal that I specify. For shapes that loop, fold, or spiral, like many found in nature, my structure is deficient.

Not really sure what to do but keep plugging away.

Blending Colliding Splines

To simulate more complex auroral structures, we will eventually want to be able to blend overlapping auroras. This is not an obvious process, and it is not without its flaws. What I have done is to create a weighted average when multiple splines collide. I let W be equal to the product of the thickness and endpoint interpolation (both are functions with a domain and range of [0, 1]). Then each color is put into the average as

Sum(for all i ) [Wi^2*Ci] / Sum(for all i) [Wi^2]

This removes the hard edges along the collision regions, but it also could make colors blend into one another too fast. I will have to do some experiments to figure out a sweet spot. The bottom picture is of three auroras that collide without any alpha attenuation, and with weights equal to 1. The top picture corresponds to the same splines, squared weights, and both skeleton endpoint and height interpolation.

I will try to do some comparisons tomorrow for reference.

Noise Alpha Attenuation

Something else we can do is use diamond-square noise height maps (due to Lucky) to create an organic kind of randomness to the aurora. Not really sure how much mileage we get out of it though... The alpha channel becomes

Alpha = 255 * Noise(s, t) * EndpointInterp(s) * HeightInterp(t);

with all the noise and interpolation functions having a domain and range of [0, 1]. Here, s represents the parameter moving along the skeleton spline and t represents the value of the distance from the pixel to the skeleton spline divided by maxdist(s).

Here, you can see the difference between having noise and not having noise. I might try doing a brightness adjustment with the noise instead.

Gaussian Alpha Attenuation

As a preface, I fixed the endpoint interpolating issue. The problem was that I was confusing the polynomial order with the degree. As a result of this confusion, I did not use enough redundant knots. Including them has fixed this issue.

I have now turned my attention to the issue of how to round off the boundaries of the spline itself, particularly near the base, left and right sides, where the aurora does not blend convincingly into the background image.

To make the blending more convincing, I have used Gaussian functions to attenuate the alpha channel. Right now, what I am doing is something like

Alpha = 255 * exp(-1 * c^2/(2*q^2) * exp(-16 * t^2)

where c = d / dmax - f, q = .1, f = .4, d is the point distance along the normal direction to the skeleton, and dmax is the result of computing the maximum distance spline function for the parameter value of t (which is also used to compute the spline shape and color).

While these mathematics are crude and convoluted, my point here is to convey the fact that we want the alpha to attenuate according to two factors. First, the alpha should be greatest at some "center" value of t in the range [0, 1]. This will help to make sure the ends of the aurora will not stick out. Second, for the same reason as before, we want the alpha to attenuate from some "center" defined as a fraction of the maximum distance spline function. This way, the aurora maintains the same shape as the skeleton, but eliminates the need to give the base special treatment.

First Attempt at Rendering

For the first time, I am rendering onto something other than a black background, and that reveals a few fundamental flaws. First, as I have previously mentioned, the sides (i.e., the gradient near the ends of the skeleton spline) do not blend well into the background. They look distinctly artificial. I am not sure how to fix this at the moment.

Second, the problem just revealed is that our B-splines are not endpoint interpolating. That means we have gaps at both ends where the curve goes off to zero. This is most obvious in the color spline (so, at the base of the aurora, the color goes to black when it should go to purple), and in the distance spline (so, at the top right end of the picture, the distance goes to zero, and the aurora tapers off to a single point). I am revisiting the code for the B-splines to fix this problem.

Third, although the aurora colors are interpolated in CIE Lab space, the alpha blending occurs in RGB space. One artifact I have noticed it causes is the pink near the base of the skeleton, where we should see a deep purple instead.

A C++ Port for Optimization

I was disappointed with the speed I was getting from C#, so I ported the code over to C++ to take advantage of the SSE instructions and other optimizations. At first, I didn't notice a major improvement, which suggests some slowness in my algorithm. I tracked it down to the BSpline intersection method.

The problem on there is that it involves bisection over the parameter space of the spline, which results in doing a lot of calculating. And in fact, a lot of points, particularly the endpoints, were being calculated once for each pixel in the image, which doesn't really work well, especially because I am going to want to sample with multiple rays later on.

To solve the problem, I noticed that I could precalculate a large table of points on the spline over the parameter space, say with a delta-t of dt=0.0001. Instead of calculating stuff over and over again, I could just compute the table once and simply reference the table in my intersection method. This solution was a huge win, because on a 500x500 image with a single aurora spline, I was able to do the optimized rendering in 0.48 seconds instead of the 6.00 seconds it was running before: the improvement is about 1200% for that situation alone!

Further Comparisons of Color Space Interpolation

Color interpolation works differently in various color spaces. I wanted to get a feel for how spline interpolation would work in the various color spaces, like CIE Lab and CIE Luv in particular. The "Color Transfer Between Images" paper [Reinhard et al. 2001] mentions an L-alpha-beta space, which I do not believe is actually related to the CIE standards. I also experimented with LCH(ab) and LCH(uv), but I am disenchanted with them because the hue coordinates are bent into a cylinder: the interpolation goes through colors we do not want to see.

One thing that is interesting is that YCbCr looks a bit grayed or washed out compared to the other color spaces, so perhaps it would not work well in our application. I have trouble discerning much of a difference between Luv and Lab.

In the first image, from top to bottom, the color spaces are RGB, YCbCr, L-alpha-beta, Luv, Lab.
In the second image, it goes RGB, Luv, Lab, LCH(uv), LCH(ab).