Hopefully I'll be able to render something similar in a while. At the same time, I eagerly devoured any paper I could find on rendering techniques from the past decade, some intended for real-time rendering, some old enough to be real-time today. Out of all this, I quickly settled on my goals:

• Represent spherical or lumpy heavenly bodies from asteroids to suns.
• With realistic looking topography and features.
• Viewable across all scales from surface to space.
• At flight-simulator levels of detail.
• Rendered with convincing atmosphere, water, clouds, haze.

For most of these points, I found one or more papers describing a useful technique I could use or adapt. At the same time, there are still plenty of unknowns I'll need to figure out along the way, not to mention significant amounts of fudging and experimentation.

The Spherical Grid

To get started I needed to build some geometry, and to do that I needed to figure out what geometry I should use. After reviewing some options, I quickly settled on a regular spherical displacement map (AKA a heightmap). That is, starting with a smooth sphere, move every surface point up or down, perpendicular to the surface, to create terrain on the surface.

If these vertical displacements are very small compared to the sphere radius, this can represent the surface of a typical planet (like Earth) at the levels of detail I'm looking for. If the displacements are of the same order as the sphere radius, you can deform it into very irregular potato-like shapes. The only thing heightmaps can't do is caves, tunnels, overhang and other kinds of holes, which is fine for now.

The big question is, how should the spherical surface be divided up and represented? With a sphere, this is not an easy question, because there is no single obvious way to divide a spherical surface into regular sections or grids. Various techniques exist, each with their own benefits and specific use cases, and I spent quite some time looking into them. Here's a comparison between four different tesselations:

Source

Note that the tesselation labeled ECP is just the regular geographic latitude-longitude grid.

The main features I was looking for were speed and simplicity, so I settled on the 'quadcube'. This is where you start with a cube whose faces have been divided into regular grids, and project every surface point out from the middle to an enclosing sphere. This results in a perfectly smooth sphere, built out of 6 identical round shells with curved edges. This arrangement is better known as the 'cube map' and often used for storing arbitrary 360 degree panorama views.

Here's a cube and its spherical projection:

The projected cube edges are indicated in red. Note that the resulting sphere is perfectly smooth and round, even though the grid has a bulgy appearance.

Cube maps are great, because they are very easy to calculate and do not require complicated trigonometry. In reverse, mapping arbitrary spherical points back onto the cube is even simpler and in fact natively supported by GPUs as a texture mapping feature.

This is important, because I'll be generating the surface terrain and texture dynamically and will need to index and access each surface 'pixel' efficiently. Using a cube map, I simply identify the corresponding face, and then index it using x/y coordinates on the face's grid.

The downside of cube maps is that the distance and area between points varies along the grid, which makes it harder to perform certain operations on a surface equally. However, these area distortions are much smaller than e.g. a lat-long grid, where the grid spacing actually approaches zero near the poles. Even more, the distortions made by a cube map are the exact opposite of those you get with a regular perspective projection. This makes it easy to render into cube maps, which will be useful for texture generation.

Level of Detail

There's another reason I picked the cube map approach, and that has to do with the level of detail requirements. My goal is to make a planet that can be viewed from the ground, the air as well as from space. It would be incredibly slow to always render everything at maximum detail, so I need to adaptively add and remove detail as the viewer gets closer to the surface.

However, increasing the level of detail uniformly across the entire sphere is not enough, because I only want to render detail where the viewer will see it. To a viewer on the ground, most of the planet is hidden by the horizon, and the engine should be able to effectively cut away the unseen pieces, so no wasteful processing takes place.

It is here that I get a huge benefit from the cube map layout of the sphere, because it lets me apply the well-researched realm of grid-based flat terrain rendering with only minor adjustments. Specifically, I am using a 'chunked LOD' approach. Every face of the cube map becomes a quadtree, with each level splitting four ways to form the next level with more detail:

Source

The chunks for the various levels of detail are all loaded into GPU memory, ready to be accessed at any time. When the terrain has to be rendered, the engine walks down the quad-tree, determines the appropriate level-of-detail for each section, and outputs the list of chunks to be rendered for a particular frame. Then, the GPU does its work, blasting through each chunk at a blistering pace, leaving the CPU to do other things.

Because all the data is already in memory, changing the level of detail just means rendering a different set of chunks. Each chunk has the same geometrical complexity, and performance is directly proportional to how many are rendered on screen. More detail means more chunks, but that usually also means you can cut away pieces of the terrain that are far away.

The chunked approach is also very easy to work with, because there is no data dependency between the different chunks. Each chunk has a copy of its own vertex data, which means individual chunks can be paged in and out of GPU memory at will. This is important for keeping memory usage down while still being able to scale to massive sizes.

Putting It All Together

At this point, I have all the pieces in place to render an adaptive sphere mesh. This is what it looks like (sorry, the video capture is a bit jerky):

The detail increases as the camera gets closer to the sphere and shifts around the surface as it moves.

Far from being a little coding experiment, it actually took me quite some time to get to this point, because I was learning OGRE, sharpening my C++ skills, as well as researching the techniques to use.

The next step is to look at generating heightmaps and textures for the surface.

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

• Creating Spherical Worlds, Maxis/Electronic Arts. (source).
• Rendering Massive Terrains using Chunked Level of Detail Control, Thatcher Ulrich. (source)

Now, whenever size is an issue, the best way to make a small program is to generate all data on the fly, i.e. procedurally. This saves valuable storage space. While this might seem like a black art, often it just comes down to clever use of (high school) math. And as is often the case, the best tricks are also the simplest, as they use the least amount of code.

To illustrate this, I'm going to break down my demo and show you all the major pieces and shortcuts used. Unlike the actual 1K demo, the code snippets here will feature legible spacing and descriptive variable names.

Initialization

JS1K's rules give you a Canvas tag to work with, so the first piece of code initializes it and makes it fill the window.

From then on, it just renders frames of the demo. There are four major parts to this:

• Animating the wires
• Rotating and projecting the wires into the camera view
• Coloring the wires
• Animating the camera

All of this is done 30 times per second, using a normal setInterval timer:

setInterval(function () { ... }, 33);

Drawing Wires

The most obvious trick is that everything in the demo is drawn using only a single primitive: a line segment of varying color and stroke width. This allows the whole drawing process to be streamlined into two tight, nested loops. Each inner iteration draws a new line segment from where the previous one ended, while the outer iteration loops over the different wires.

The lines are blended additively, using the built-in 'lighten' mode, which means they can be drawn in any order. This avoids having to manually sort them back-to-front.

To simplify the perspective transformations, I use a coordinate system that places the point (0, 0) in the center of the canvas and ranges from -1 to 1 in both coordinates. This is a compact and convenient way of dealing with varying window sizes, without using up a lot of code:

with (graphics) {
  ratio = width / height;
  globalCompositeOperation = 'lighter';
  scale(width / 2 / ratio, height / 2);
  translate(ratio, 1);
  lineWidthFactor = 45 / height;
  ...

I also add a correction ratio for non-square windows and calculate a reference line width lineWidthFactor for later.

Then there's the two nested for loops: one iterating over the wires, and one iterating over the individual points along each wire. In pseudo-code they look like:

For (12 wires => wireIndex) {
  Begin new wire
  For (45 points along each wire => pointIndex) {
    Calculate path of point on a sphere: (x,y,z)
    Extrude outwards in swooshes: (x,y,z)
    Translate and Rotate into camera view: (x,y,z)
    Project to 2D: (x,y)
    Calculate color, width and luminance of this line: (r,g,b) (w,l)
    If (this point is in front of the camera) {
      If (the last point was visible) {
        Draw line segment from last point to (x,y)
      }
    }
    else {
      Mark this point as invisible
    }
    Mark beginning of new line segment at (x,y)
  }
}

Mathbending

To generate the wires, I start with a formula which generates a sinuous path on a sphere, using latitude/longitude. This controls the tip of each wire and looks like:

offset = time - pointIndex * 0.03 - wireIndex * 3;
longitude = cos(offset + sin(offset * 0.31)) * 2
          + sin(offset * 0.83) * 3 + offset * 0.02;
latitude = sin(offset * 0.7) - cos(3 + offset * 0.23) * 3;

This is classic procedural coding at its best: take a time-based offset and plug it into a random mish-mash of easily available functions like cosine and sine. Tweak it until it 'does the right thing'. It's a very cheap way of creating interesting, organic looking patterns.

This is more art than science, and mostly just takes practice. Any time spent with a graphical calculator will definitely pay off, as you will know better which mathematical ingredients result in which shapes or patterns. Also, there are a couple of things you can do to maximize the appeal of these formulas.

First, always include some non-linear combinations of operators, e.g. nesting the sin() inside the cos() call. Combined, they are more interesting than when one is merely overlaid on the other. In this case, it turns regular oscillations into time-varying frequencies.

Second, always scale different wave periods using prime numbers. Because primes have no factors in common, this ensures that it takes a very long time before there is a perfect repetition of all the individual periods. Mathematically, the least common multiple of the chosen periods is huge (414253 units ~ 4.8 hours). Plotting the longitude/latitude for offset = 0..600 you get:

The graph looks like a random tangled curve, with no apparent structure, which makes for motions that never seem to repeat. If however, you reduce each constant to only a single significant digit (e.g. 0.31 -> 0.3, 0.83 -> 0.8), then suddenly repetition becomes apparent:

This is because the least common multiple has dropped to 84 units ~ 3.5 seconds. Note that both formulas have the same code complexity, but radically different results. This is why all procedural coding involves some degree of creative fine tuning.

Extrusion

Given the formula for the tip of each wire, I can generate the rest of the wire by sweeping its tail behind it, delayed in time. This is why pointIndex appears as a negative in the formula for offset above. At the same time, I move the points outwards to create long tails.

I also need to convert from lat/long to regular 3D XYZ, which is done using the spherical coordinate transform:

distance = f.sqrt(pointIndex+.2);
x = cos(longitude) * cos(latitude) * distance;
y = sin(longitude) * cos(latitude) * distance;
z = sin(latitude) * distance;

You might notice that rather than making distance a straight up function of the length pointIndex along the wire, I applied a square root. This is another one of those procedural tricks that seems arbitrary, but actually serves an important visual purpose. This is what the square root looks like (solid curve):

The dotted curve is the square root's derivative, i.e. it indicates the slope of the solid curve. Because the slope goes down with increasing distance, this trick has the effect of slowing down the outward motion of the wires the further they get. In practice, this means the wires are more tense in the middle, and more slack on the outside. It adds just enough faux-physics to make the effect visually appealing.

Rotation and Projection

Once I have absolute 3D coordinates for a point on a wire, I have to render it from the camera's point of view. This is done by moving the origin to the camera's position (X,Y,Z), and applying two rotations: one around the vertical (yaw) and one around the horizontal (pitch). It's like spinning on a wheely chair, while tilting your head up/down.

x -= X; y -= Y; z -= Z;

x2 = x * cos(yaw) + z * sin(yaw);
y2 = y;
z2 = z * cos(yaw) - x * sin(yaw);

x3 = x2;
y3 = y2 * cos(pitch) + z2 * sin(pitch);
z3 = z2 * cos(pitch) - y2 * sin(pitch);

The camera-relative coordinates are projected in perspective by dividing by Z — the further away an object, the smaller it is. Lines with negative Z are behind the camera and shouldn't be drawn. The width of the line is also scaled proportional to distance, and the first line segment of each wire is drawn thicker, so it looks like a plug of some kind:

plug = !pointIndex;
lineWidth = lineWidthFactor * (2 + plug) / z3;
x = x3 / z3;
y = y3 / z3;

lineTo(x, y);
if (z3 > 0.1) {
  if (lastPointVisible) {
    stroke();
  }
  else {
    lastPointVisible = true;
  }
}
else {
  lastPointVisible = false;
}
beginPath();
moveTo(x, y);

Coloring

Each line segment also needs an appropriate coloring. Again, I used some trial and error to find a simple formula that works well. It uses a sine wave to rotate overall luminance in and out of the (Red, Green, Blue) channels in a deliberately skewed fashion, and shifts the R component slowly over time. This results in a nice varied palette that isn't overly saturated.

pulse = max(0, sin(time * 6 - pointIndex / 8) - 0.95) * 70;
luminance = round(45 - pointIndex) * (1 + plug + pulse);
strokeStyle='rgb(' +
  round(luminance * (sin(plug + wireIndex + time * 0.15) + 1)) + ',' +
  round(luminance * (plug + sin(wireIndex - 1) + 1)) + ',' +
  round(luminance * (plug + sin(wireIndex - 1.3) + 1)) +
  ')';

Here, pulse causes bright pulses to run across the wires. I start with a regular sine wave over the length of the wire, but truncate off everything but the last 5% of each crest to turn it into a sparse pulse train:

Camera Motion

With the main visual in place, almost all my code budget is gone, leaving very little room for the camera. I need a simple way to create consistent motion of the camera's X, Y and Z coordinates. So, I use a neat low-tech trick: repeated interpolation. It looks like this:

sample += (target - sample) * fraction

target is set to a random value. Then, every frame, sample is moved a certain fraction towards it (e.g. 0.1). This turns sample into a smoothed version of target. Technically, this is a one-pole low-pass filter.

This works even better when you apply it twice in a row, with an intermediate value being interpolated as well:

intermediate += (target - intermediate) * fraction
sample += (intermediate - sample) * fraction

A sample run with target being changed at random might look like this:

You can see that with each interpolation pass, more discontinuities get filtered out. First, jumps are turned into kinks. Then, those are smoothed out into nice bumps.

In my demo, this principle is applied separately to the camera's X, Y and Z positions. Every ~2.5 seconds a new target position is chosen:

if (frames++ > 70) {
  Xt = random() * 18 - 9;
  Yt = random() * 18 - 9;
  Zt = random() * 18 - 9;
  frames = 0;
}

function interpolate(a,b) {
  return a + (b-a) * 0.04;
}

Xi = interpolate(Xi, Xt);
Yi = interpolate(Yi, Yt);
Zi = interpolate(Zi, Zt);

X  = interpolate(X,  Xi);
Y  = interpolate(Y,  Yi);
Z  = interpolate(Z,  Zi);

The resulting path is completely smooth and feels quite dynamic.

Camera Rotation

The final piece is orienting the camera properly. The simplest solution would be to point the camera straight at the center of the object, by calculating the appropriate pitch and yaw directly off the camera's position (X,Y,Z):

yaw   = atan2(Z, -X);
pitch = atan2(Y, sqrt(X * X + Z * Z));

However, this gives the demo a very static, artificial appearance. What's better is making the camera point in the right direction, but with just enough freedom to pan around a bit.

Unfortunately, the 1K limit is unforgiving, and I don't have any space to waste on more 'magic' formulas or interpolations. So instead, I cheat by replacing the formulas above with:

yaw   = atan2(Z, -X * 2);
pitch = atan2(Y * 2, sqrt(X * X + Z * Z));

By multiplying X and Y by 2 strategically, the formula is 'wrong', but the error is limited to about 45 degrees and varies smoothly. Essentially, I gave the camera a lazy eye, and got the perfect dynamic motion with only 4 bytes extra!

Addendum

After seeing the other demos in the contest, I wasn't so sure about my entry, so I started working on a version 2. The main difference is the addition of glowy light beams around the object.

As you might suspect, I'm cheating massively here: rather than do physically correct light scattering calculations, I'm just using a 2D effect. Thankfully it comes out looking great.

Essentially, I take the rendered image, and process it in a second Canvas that is hidden. This new image is then layered on the original.

I take the image and repeatedly blend it with a zoomed out copy of itself. With every pass the number of copies doubles, and the zoom factor is squared every time. After 3 passes, the image has been smeared out into an 8 = 2³ 'tap' radial blur. I lock the zooming to the center of the 3D object. This makes the beams look like they're part of the 3D world rather than drawn on later.

For additional speed, the beam image is processed at half the resolution. As a side effect, the scaling down acts like a slight blur filter for the beams.

Unfortunately, this effect was not very compact, as it required a lot of drawing mode changes and context switches. I had no room for it in the source code.

So, I had to squeeze out some more room in the original. First, I simplified the various formulas to the bare minimum required for interesting visuals. I replaced the camera code with a much simpler one, and started aggressively shaving off every single byte I could find. Then I got creative, and ended up recreating the secondary canvas every frame just to avoid switching back its state to the default.

Eventually, after a lot of bit twiddling, a version came out that was 1024 bytes long. I had to do a lot of unholy things to get it to fit, but I think the end result is worth it ;).

Closing Thoughts

I've long been a fan of the demo scene, and fondly remember Second Reality in 1993 as my introduction to the genre. Since then, I've always looked at math as a tool to be mastered and wielded rather than subject matter to be absorbed.

With this blog post, I hope to inspire you to take the plunge and see where some simple formulas can take you.

W00t, certainly one of the niftiest things I've built.

Engine tweaks

Along with the architecture changes, I implemented some engine tweaks, noted here for completeness.

In previous comments, Erlend suggested using displacement mapping, so I gave it a shot. Before, the mesh for every tile was calculated on the CPU once, then copied into GPU memory. However, this mesh data was redundant, because it was derived literally from the height map data. Instead I changed it so that now, the transformation of mesh points onto the sphere surface happens real-time on the GPU in a per-vertex program.

This saves memory and pre-calculation time, but increases the rendering load. I'll have to see whether this technique is sustainable, but overall, it seems to be performing just fine. As a side effect, the terrain height map can be changed real-time with very low cost.

Technical hurdles

I spent some time tweaking the engine to run faster, but there's still plenty of work and some technical hurdles to cover.

One involves the Ogre Scene Manager, which is the code object that manages the location of objects in space. In my case, I have to deal with both the 'real world' in space as well as the 'virtual world' of brushes that generate the planet's surface. I chose to use two independent scene managers to represent this, as it seemed like a natural choice. However, it turns out this is unsupported by Ogre and causes random crashes and edge cases. Argh. It looks like I'll have to refactor my code to fix this.

Another major hurdle involves the planet surface itself. Currently I'm still just using a single distored-crater-brush to create it, and the lack of variation is showing.

Finally, surfaces are being generated using 16-bit floating point height values, and their accuracy is not sufficient beyond a couple levels of zooming. This results in ugly bands of flat terrain. To fix this I'll need to increase the surface accuracy.

Future steps

With the basic planet surface covered, I can now start looking at color, atmosphere and clouds. I have plenty of reading and experimentation to do. Thankfully the web is keeping me supplied with a steady stream of awesome papers... nVidia's GPU Gems series has proven to be a gold mine, for example.

Random factoid: what game developers call a "cube map", cartographers call a "cubic gnomonic grid". It turns out that knowing the right terminology is important when you're looking for reference material...

Code

The code is available on GitHub.

References

Great ideas are best discovered when standing on the shoulders of giants:

• id Tech 5 Challenges, From Texture Virtualization to Massive Parallelization, J.M.P. van Waveren, id Software
• GPU Gems, nVidia

· • ·

Imagine a budget-starved teacher needs to hand out notes for class, but can only afford one copy. The document is 10 pages long, and there are 10 students who each need a complete copy.

The teacher could just give the notes to one student, and ask him to make all the copies, but that would only shift the burden, leaving him to pay for all 100 pages.

Instead, the teacher has an idea. She hands page 1 to student #1, page 2 to student #2, and so on, and tells each student to make 10 copies of their single page. The next week, the students can distribute them amongst themselves before class, and everyone gets a complete set. Nobody has to pay for more than their own 10 pages.

Everyone's happy: the teacher gets to share her knowledge cheaply, and the students don't mind paying for their own copies.

In the middle of the term, a new student joins. She could borrow someone else's big pile of notes, and copy the entire stack of paper, but that would mean she would have to pay for it all, and she's on a budget too.

So instead, she just goes around and asks each student to make a single copy of the pages they were assigned previously. The next week, she collects all the pages, and assembles a complete copy without even bothering the teacher.

She gets a free pass to catch up with the class, but the other students don't mind chipping in. That's because she immediately joins the game and can make copies too. The teacher can now hand out one page extra each week, or decide to give one student a free pass. If more students join, it works better and better.

Now instead, imagine that students join and leave the class every single day, and the teacher isn't quite so organized. She just puts her big stack of notes on the desk, and tells everyone they can take any page they want, as long as they promise to immediately make copies for anyone who asks. The students are all friendly, and make sure to keep each other in the loop about which pages everyone has. Both the originals and the copies are copied as many times as needed.

· • ·

That's BitTorrent in a nutshell. For any given class—i.e. a file that people are interested in—a cloud of students forms—i.e. the peers in the so called peer-to-peer network. The peers compare notes, see which pieces they are missing, and swap copies with each other. Eventually, the teacher (a.k.a. the seeder) can leave, taking her original copy with her, and the system will keep working. As long as there is at least one copy of every page in the room, the students can make more, and the document as a whole will live on.

This is pretty much the only way you can effectively distribute a massive archive of sensitive data to thousands or millions of people, without incurring massive bills. You can't use free or ad-supported services, as the material would get taken down instantly due to its sensitive nature. And you can't host it directly, as that would leave a trail pointing back to you.

With BitTorrent, your initial group of 'students' can be sworn to secrecy. After the initial round of copying, the teacher sneaks out, and the students just pin a notice on the bulletin board: "We have copies of The Forbidden Secrets by Dr. X. Come see us." Nobody claims to know who Dr. X is. Ideas and information flow freely, without censorship.

Polishing Bacon

But despite its minimalism, a big aspect of Facing.me is the effort and care we put into it. Our goal was to achieve a level of polish typically reserved for premium iPhone apps and bring it into the browser. We wrapped the whole thing in a crisp design, enhanced with tasteful web fonts. But most importantly, we sought to expose the app's functionality with as little interruption as possible. To do that, we layered on plenty of transitions driven by CSS3 and JavaScript, and stream in data and content as needed.

Based on previous work in custom animations—and bacon—we refined the approach of using jQuery as an animation helper for completely custom transitions. We tell jQuery to animate placeholder properties on orphaned proxy divs, and key off those animations with per-frame code to drive the fancy stuff.

As a result, we can have a photo grow a picture frame as you pick it up, and then flip it around to show a person's full profile. This careful choreography involves animating about a dozen CSS properties, including borders, shadows, margins and 3D transforms, all with custom expressions and hand-tuned animation curves. Similar transitions are used for lightbox dialogs.

Throughout all of this, the animations remain eminently manageable. We can interrupt and reverse them at any point, and run multiple copies at the same time, thanks to pervasive use of view controllers. Far from being a useless tech demo, it actually enables us to craft the user experience exactly the way we like it: being able to acknowledge user intentions with intuitive feedback no matter what's going on, and firing off new events and requests without worrying about the internal state. Gone are the fragile jQuery behavior soups of old.

The one downside is that only the newer browsers—i.e. Chrome, Safari and Firefox—get to see everything the way it was intended. And actually the performance in Firefox is still a bit disappointing. IE9 users will have to be satisfied with a crude 2D approximation until IE10 comes out.

Built in our spare time by just 3 guys in a virtual garage, we're pretty proud of the end result. We'd love for you to take it for a spin, so head over to facing.me and grab yourself an invite. There's a feedback form built-in, and any suggestions are welcome.

Discuss on Google Plus.

Arguments for abandoning the purity of vanilla HTML (James Pearce) were followed by a philosophical lesson on not throwing away the baby with the bathwater (John Allsopp) and I found myself agreeing wholeheartedly with both, cognitive dissonance notwithstanding. As someone who isn't a fan of the mobile app world, I had to admit I was ignorant on the difficulties of implementing offline web apps (Andrew Betts), and blissfully unaware of the absolute zoo of devices people really do try to access the web with (Anna Debenham), webological purity be damned.

We've barely scratched the surface of what browsers can do (Paul Kinlan), we need to chase the high of writing code and actually have it Just Work (Rebecca Murphey), and above all, we need to remember where it all came from (Chris Wilson), lest we repeat the mistakes of the past. If you weren't one of the lucky people who managed to snag tickets before it sold out, take some time out of your busy day to enjoy these sessions in video once they are posted online rather than just skipping through slides.

If there's one thing that stood out though, it's how little of what I heard on and off-stage is part of the daily discourse online in the tech world, in the news or on sites like HackerNews, Twitter and Reddit. We only see caricatures of these conversations. More than ever, I'm convinced I need to filter out these echochambers from my thoughts and seek out more substance. Particularly, the Silicon Valley-centric TechCrunch-driven worship of runaway success adds nothing, and only holds us back. It makes people think they need to chase something that only ever happens by accident, and diverts attention away from rolling up your sleeves and doing what actually needs to be done. This was emphasized all the more by the fact that the venue had no wi-fi, which meant everyone had their eyes away from their screens for a change.

You can read more about MathBox in the follow-up blog post.

The adventurous can go see how the sausage was made and check out the code for MathBox, the library I wrote to make it happen, as well as the HTML5 slide deck.

So, WebGL it was, because I needed 3D. Unfortunately that meant the promise of having it just work everywhere was tempered by a lack of browser support, but I would certainly hope that's something we can overcome sooner than later. Dear Apple and Microsoft: get your shit together already. Dear Firefox and Opera: your WebGL performance could be a lot better.

Shady Dealings

These days I don't really touch WebGL without going through Three.js first. Three.js is a wonderful, mature engine that contains tons of useful high-level components. At the same time, it also does a great job in just handling the boilerplate of WebGL while not getting in the way of doing some heavy lifting yourself.

Rendering vector-style graphics with WebGL is not hard, certainly easier than photorealistic 3D. Primitives like lines and points are sized in absolute pixels by default, and with hardware multisampling for anti-aliasing, you get somewhat decent image quality out of it. Though, as is typical for a Web API, we're treated like children and can only cross our fingers and request anti-aliasing politely, hoping it will be available. Meanwhile native developers have full control over speed and quality and can adjust their strategy to the specific hardware's capabilities. The more things change... And then Chrome decided to disable anti-aliasing altogether due to esoteric security issues with buggy drivers. Bah.

Now, when rendering with WebGL, you really have two options. One is to just treat it as a dumb output layer, loading or generating all your geometry in JavaScript and rendering it directly in 3D. With the speed of JS engines today, this can get you pretty far.

Viewports, Primitives and Renderables

At its core, Three.js matches pretty directly with WebGL. You can insert objects such as a Mesh, Line or ParticleSystem into your scene, which invokes a specific GL drawing command with high efficiency. As such, I certainly didn't want to reinvent the wheel.

Hence, MathBox is set up as a sort of scene-manager-within-a-scene-manager. It's a little sandbox that speaks the language of math, allowing you to insert various primitives like curves, vectors, axes and grids. Each of these primitives then instantiates one or more renderables, which simply wrap a native Three.js object and its associated ShaderGraph material. Thus, once instantiated, MathBox gets out of the way and Three.js does the heavy lifting as normal. You can even insert multiple mathboxen into a Three.js scene if you like, mixed in with other objects.

For example, a vector primitive is rendered as an arrow: it consists of a shaft and an arrowhead, realized as a line segment and a cone. An axis primitive is an arrow as well, but it also has tick marks (specially transformed line segments), and is positioned implicitly just by specifying the axis' direction rather than a start and end point.

To render curves and surfaces, you can either specify an array of data points or a live expression to be evaluated at every point. This turned out to be essential for the kinds of intricate visualizations I wanted to show, my slides being driven by timed clocks, shared arrays of data points, and live formulas and interpolations. I even fed in data from a physics engine, and it worked perfectly.

This is all tied together through Viewport objects, which define a specific mapping from a mathematical coordinate space into the 3D world space of Three.js. For example, the default cartesian viewport has the range [–1, 1] in the X, Y and Z directions. Altering the viewport's extents will shift and scale anything rendered within, as well as reflow grids and tick marks on each axis.

There are two more sophisticated viewport types, polar and spherical, which each apply the relevant coordinate transform, and can transition smoothly to and from cartesian. More viewport types can be added, all that is required is to define an appropriate transformation in JavaScript and GLSL. That said, defining a seamless transition to and from cartesian space is not always easy, particularly if you want to preserve the aspect-ratio through the entire process.

Interpolate all the things!

Finally, I had to tackle the problem of animation, keeping in mind a tip I learnt from the ever so mindbending Vihart: "If I can draw the point of a sentence, I don't actually need to say the sentence." This applies doubly so for animation: every time you replace a "before" and "after" with a smooth transition, your audience implicitly understands the change rather than having to go look for it.

Hence, each primitive can be fully animated. Each has a set of options (controlling behavior) and styles (controlling GLSL shaders), and there is a universal animator that can interpolate between arbitrary data types in a smart fashion.

For example, given a viewport with the XYZ range [[–1, 1], [–1, 1], [–1, 1]], you can tell it to animate to [[0, 2], [0, 1], [–3, 3]], and it just works. The animator will recursively animate each subarray's elements, and any dependent objects like grids and axes will reflow to match the intermediate values. This works for colors, vectors and matrices too. In case of live curves with custom expressions, the animator will invoke both the old and the new, and interpolate between the results.

However, executing animations manually in code is tedious, particularly in a presentation, where you want to be able to step forward and backward. So I added a Director class whose job it is to coordinate things. All you do is feed it a script of steps (add this object, animate that object). Then, as it applies them, it remembers the previous state of each object and generates an automatic rollback script. It also contains logic to detect rapid navigation, and will hurry up animations appropriately, avoiding that agonizing situation of watching someone skip through their slide deck, playing the same cheesy PowerPoint transitions over and over again.

Presenting Naturally

With MathBox's core working, it was time to build my slides for the conference. After a quick survey, I quickly settled on deck.js as an HTML5 slidedeck solution that was clean and flexible enough for my purposes. However, while MathBox can be spawned inside any DOM element, it wouldn't work to insert a dozen live WebGL canvases into the presentation. The entire thing would grind to a halt or at least become very choppy.

So instead, I integrated each MathBox graphic as an IFRAME, and added some logic that only loads each IFRAME one slide before it's needed, and unloads it one slide after it's gone off screen. To sync up with the main presentation, all deck.js navigation events were forwarded into each active IFRAME using window.postMessage. With the MathBox Director running inside, this was very easy to do, and meant that I could skip around freely during the talk, without any worries of desynchronization between MathBox and the associated HTML5 overlays.

In fact, I applied the same principle to this post. To avoid rendering all diagrams simultaneously and spinning up laptop fans more than necessary, each MathBox IFRAME is started as it scrolls into view and stopped once it's gone.

I've also found that having a handheld clicker makes a huge difference while speaking—as it allows you to gesture freely and move around. So, I grabbed the infrared remote code from VLC and built a simple bridge from to Cocoa to Node.js to WebSocket to allow the remote to work in a browser. It's a shame Apple's decided to discontinue IR ports on their laptops. I guess I'll have to come up with a BlueTooth-based solution when I upgrade my hardware.

Towards MathBox 1.0

In its current state, MathBox is still a bit rough. The selection of primitives and viewports is limited, and only includes the ones I needed for my presentation. That said, it is obvious you can already do quite a lot with it, and I couldn't have been happier to hear that all this effort had the desired response at the conference. I wasn't 100% sure whether other people would have the same a-ha moments that I've had, but I'm convinced more than ever that seeing math in motion is essential for honing our intuition about it. MathBox not only makes animated diagrams much easier to make and share, but it also opens the door to making them interactive in the future.

I plan to continue to evolve MathBox as needed by using it on this site and addressing gaps that come up, though I've already identified a couple of sore points:

• I used tQuery as a boilerplate and because I liked the idea of having a chainable API for this. However, this also means it's currently running off an outdated version of Three.js. I need to look into updating and/or dropping tQuery.
  MathBox has been updated to Three.js r52.
• Numeric or text labels are completely unsupported. It should be possible to use my CSS3D renderer for Three.js to layer on beautifully typeset MathJax formulas, positioning them correctly in 3D on top of the WebGL render.
  I've added labeling for axes. I've integrated MathJax, but it's tricky because the typesetting is painfully slow in the middle of a 60fps render. But it's automatically used if MathJax is present.
• All styles have to be specified on a per-object basis. Some form of stylesheet, default styles or class mechanism to allow re-use seems like an obvious next step.
• There are undoubtedly memory leaks, as I was focused first and foremost on getting it to work.
• Expressions that don't change frame-to-frame are still continuously re-evaluated, which is wasteful. There is a live: false flag you can set on objects, but it triggers a few bugs here and there.
• There needs to be a predictable, built-in way of running a clock per slide to sync custom expressions off of. In my presentation I used a hack of clocks that start once first invoked, but this lacks repeatability.
  I added a director.clock() method that gives you a clock per slide.

Finally, it doesn't take much imagination to imagine a MathBox Editor that would allow you to build diagrams visually rather than having to use code like I did. However, that's a can of worms I'm not going to open by myself, especially because the API is already quite straightforward to use, and the library itself is still a bit in flux. Perhaps this could be done as an extension of the Three.js editor.

You can see what MathBox is really capable of in the conference video. I invite you to play around with MathBox and see what you can make it do. Contributions are welcome, and the architecture is modular enough to allow its functionality to grow for quite some time.

Like Hands on a Clock

What does it mean to let numbers turn? Well, when making mathematical choices, we have to be careful. You could declare that $ 1 + 1 $ should equal $ 3 $, but that only opens up more questions. Does $ 1 + 1 + 1 $ equal $ 4 $ or $ 5 $ or $ 6 $? Can you even do meaningful arithmetic this way? If not, what good are these modified numbers? The most important thing is that our rules need to be consistent for them to work. But if all we do is swap out the symbols for $ 2 $ and $ 3 $, we didn't actually change anything in the underlying mathematics at all.

So we're looking for choices that don't interfere with what already works, but add something new. Just like the negative numbers complemented the positives, and the fractions snugly filled the space between them—and the reals somehow fit in between that—we need to go look for new numbers where there currently aren't any.

Pulling a Dragon out of a Hat

With a basic grasp of what complex numbers are and how they move, we can start making Julia fractals.

At their heart lies the following function:

$$ f(z) = z^2 + c $$

This says: map the complex number $ z $ onto its square, and then add a constant number to it. To generate a Julia fractal, we have to apply this formula repeatedly, feeding the result back into $ f $ every time.

$$ z_{n+1} = (z_n)^2 + c $$

We want to examine how $ z_n $ changes when we plug in different starting values for $ z_1 $ and iterate $ n $ times. So let's try that and see what happens.

Instead, we pretend the steel is solid throughout. Rather than being composed of atoms with gaps in between, it's made of some unknown, filled in material with a certain density, expressed e.g. as grams per cubic centimetre. Given any shape, we can determine its volume, and hence its total mass, and go from there. That's much simpler than counting and keeping track of individual atoms, right?

Unfortunately, that's not quite true.

The Shortest Disappearing Trick Ever

Like all choices in mathematics, this one has consequences we cannot avoid. Our beam's density is mass per volume. Individual points in space have zero volume. That would mean that at any given point inside the beam, the amount of mass there is $ 0 $. How can a beam that is entirely composed of nothing be solid and have a non-zero mass?

Bam! No more iron anywhere.

While Douglas Adams was being deliberately obtuse, there's a kernel of truth there, which is a genuine paradox: what exactly is the mass of every atom in our situation?

To make our beam solid and continuous, we had to shrink every atom down to an infinitely small point. To compensate, we had to create infinitely many of them. Dividing the finite mass of the beam between an infinite amount of atoms should result in $ 0 $ mass per atom. Yet all these masses still have to add up to the total mass of the beam. This suggests $ 0 + 0 + 0 + … > 0 $, which seems impossible.

If the mass of every atom were not $ 0 $, and we have infinitely many points inside the beam, then the total mass is infinity times the atomic mass $ m $. Yet the total mass is finite. This suggests $ m + m + m + … < \infty $, which also doesn't seem right.

It seems whatever this number $ m $ is, it can't be $ 0 $ and can't be non-zero. It's definitely not infinite, we only had a finite mass to begin with. It's starting to sound like we'll have to invent a whole new set of numbers again to even find it.

Still, it shows that even if we don't understand the whole picture, we can get a lot done. This article is in no way a formal introduction to infinitesimals. Rather, it's a demonstration of why we might need them.

What is happening when we shrink atoms down to points? Why does it make shapes solid yet seemingly hollow? Is it ever meaningful to write $ x = \infty $? Is there only one infinity, or are there many different kinds?

To answer that, we first have to go back to even simpler times, to Ancient Greece, and start with the works of Zeno.

Achilles and the Tortoise

Zeno of Elea was one of the first mathematicians to pose these sorts of questions, effectively trolling mathematics for the next two millennia. He lived in the 5th century BC in southern Italy, although only second-hand references survive. In his series of paradoxes, he examines the nature of equality, distance, continuity, of time itself.

Because it's the ancient times, our mathematical knowledge is limited. We know about zero, but we're still struggling with the idea of nothing. We've run into negative numbers, but they're clearly absurd and imaginary, unlike the positive numbers we find in geometry. We also know about fractions and ratios, but square roots still confuse us, even though our temples stay up.

Limits are the first tool in our belt for tackling infinity. Given a sequence described by countable steps, we can attempt to extend it not just to the end of the world, but literally forever. If this works we end up with a finite value. If not, the limit is undefined. A limit can be equal to $ \infty $, but that's just shorthand for the sequence has no upper bound. Negative infinity means no lower bound.

Breaking Away From Rationality

Until now we've only encountered fractions, that is, rational numbers. Each of our sums was made of fractions. The limit, if it existed, was also a rational number. We don't know whether this was just a coincidence.

It might seem implausible that a sequence of numbers that is 100% rational and converges, can approach a limit that isn't rational at all. Yet we've already seen similar discrepancies. In our first sequence, every partial sum was less than $ 1 $. Meanwhile the limit of the sum was equal to $ 1 $. Clearly, the limit does not have to share all the properties of its originating sequence.

We also haven't solved our original problem: we've only chopped things up into infinitely many finite pieces. How do we get to infinitely small pieces? To answer that, we need to go looking for continuity.

Generally, continuity is defined by what it is and what its properties are: a noticeable lack of holes, and no paradoxical values. But that's putting the cart before the horse. First, we have to show which holes we're trying to plug.

What we just did was a careful exercise in hiding the obvious, namely the digit-based number systems we are all familiar with. By viewing them not as digits, but as paths on a directed graph, we get a new perspective on just what it means to use them. We've also seen how this means we can construct the rationals and reals using the least possible ingredients required: division by two, and limits.

Drowning By Numbers

In school, we generally work with the decimal representation of numbers. As a result, the popular image of mathematics is that it's the science of digits, not the underlying structures they represent. This permanently skews our perception of what numbers really are, and is easy to demonstrate. You can google to find countless arguments of why $ 0.999… $ is or isn't equal to $ 1 $. Yet nobody's wondering why $ 0.000… = 0 $, though it's practically the same problem: $ 0.1, 0.01, 0.001, 0.0001, … $

Furthermore, in decimal notation, rational numbers and real numbers look incredibly alike: $ 3.3333… $ vs $ 3.1415…\, $ The question of what it actually means to have infinitely many non-repeating digits, and why this results in continuous numbers, is hidden away in those 3 dots at the end. By imagining $ π $ as $ 3.1415…0000… $ or $ 3.1415…1111… $ we can intuitively bridge the gap to the infinitely small. We see how the distance between two neighbouring real numbers must be so small, that it really is equivalent to $ 0 $.

That's not as crazy as it sounds. In the field of hyperreal numbers, every number actually has additional digits 'past infinity': that's its infinitesimal part. You can imagine this to be a multiple of $ \frac{1}{\infty} $, an infinitely small unit greater than $ 0 $, which I'll call $ ε $. The idea of equality is then replaced with adequality: being equal aside from an infinitely small difference.

You can explore this hyperreal number line below.

Note that $ ε^2 $ is also infinitesimal. In fact, it's even infinitely smaller than $ ε $, and we can keep doing this for $ ε^3, ε^4, …\,$ To make matters worse, if $ ε $ is infinitesimal, then $ \frac{1}{ε} $ must be infinitely big, and $ \frac{1}{ε^2} $ infinitely bigger than that. So hyperreal numbers don't just have inwardly nested infinitesimal levels, but outward levels of increasing infinity too. They have infinitely many dimensions of infinity both ways.

So it's perfectly possible to say that $ 0.999… $ does not equal $ 1 $, if you mean they differ by an infinitely small amount. The only problem is that in doing so, you get much, much more than you bargained for.

A Tug of War Between the Gods

That means we can finally answer the question we started out with: why did our continuous atoms seemingly all have $ 0 $ mass, when the total mass was not $ 0 $? The answer is that the mass per atom was infinitesimal. So was each atom's volume. The density, mass per volume, was the result of dividing one infinitesimal amount by another, to get a normal sized number again. To create a finite mass in a finite volume, we have to add up infinitely many of these atoms.

These are the underlying principles of calculus, and the final puzzle piece to cover. The funny thing about calculus is, it's conceptually easy, especially if you start with a good example. What is hard is actually working with the formulas, because they can get hairy very quickly. Luckily, your computer will do them for you:

That was differential and integral calculus in a nutshell. We saw how many people actually spend hours every day sitting in front of an integrator: the odometers in their cars, which integrate speed into distance. And the derivative of speed is acceleration—i.e. how hard you're pushing on the gas pedal or brake, combined with forces like drag and friction.

By using these tools in equations, we can describe laws that relate quantities to their rates of change. Drag, also known as air resistance, is a force which gets stronger the faster you go. This is a relationship between the first and second derivatives of position.

In fact, the relaxation procedure we applied to our track is equivalent to another physical phenomenon. If the curve of the coaster represented the temperature along a thin metal rod, then the heat would start to equalize itself in exactly that fashion. Temperature wants to be smooth, eventually averaging out completely into a flat curve.

Whether it's heat distribution, fluid dynamics, wave propagation or a head bobbing in a roller coaster, all of these problems can be naturally expressed as so called differential equations. Solving them is a skill learned over many years, and some solutions come in the form of infinite series. Again, infinity shows up, ever the uninvited guest at the dinner table.

Closing Thoughts

Infinity is a many splendored thing but it does not lift us up where we belong. It boggles our mind with its implications, yet is absolutely essential in math, engineering and science. It grants us the ability to see the impossible and build new ideas within it. That way, we can solve intractable problems and understand the world better.

What a shame then that in pop culture, it only lives as a caricature. Conversations about infinity occupy a certain sphere of it—Pink Floyd has been playing on repeat, and there's usually someone peddling crystals and incense nearby.
"Man, have you ever, like, tried to imagine infinity…?" they mumble, staring off into the distance.

"Funny story, actually. We just came from there…"

Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.

More like this: How to Fold a Julia Fractal.

Don't Touch Me

As late as the Nexus One, most Android phones weren't really multi-touch. They had separate X/Y sensors, capable of distinguishing two touches from one, but unable to tell whether they were rotating clockwise or counter-clockwise. As a result, multi-touch has always been tacked on as an extra: despite now being capable of tracking 5 or even 10 touches, most surfaces still only respond to one.

This all comes down to the concept of affordance: what do our devices suggest we can do with them, and what do they actually allow us to do? For example, based on the design of the handle, doors generally afford either pulling or pushing, giving strong hints as to how to open them. Instead of arguing over whether things should look skeuomorphic, how about making them act skeuomorphic?

It's For Your Own Good

Now, everyone knows headphones can damage your hearing. This is why the EU passed laws to help ensure consumers would be protected from themselves: devices like iPods are limited in the sound level they can emit. If you've ever sat next to someone and listened to the kacrunch-kacrunch that their hip-hop or dubstep sounded like to everyone on the outside, you may think this is a very good idea.

Unfortunately, it takes a politician to think you could legislate against people being stupid, and a lawyer to come up with the convoluted verbiage that makes it seem sane to the rest of us. Then you need a developer to do a really hare-brained job in implementing it. Hence, in line with EU legislation, Android 4.2 introduces a new feature, for all users everywhere:

The Wobbly Camera

Moving on, Android 4.2 includes a completely redesigned camera app, with fun features like Photo Sphere and a more minimal UI. It seems pretty slick and follows the crisp white-on-black style of Jelly Bean. However within 10 seconds of using it, it will eagerly show you how stupid it is:

When you rotate the device from portrait to landscape, it freezes the screen so it can do a completely useless transition. The image doesn't change, the widgets don't actually move anywhere, the controls just rotate out and back in again, and the focus ring trips out for a bit.

Somewhere, a Google developer probably realized this, thought about it, and said "Fuck it, I'll use com.google.android.paint.by.numbers" instead of handling the odd upside down case elegantly.

Switching a camera from portrait to landscape is one of its main affordances. Android manages to jank even this up. To make matters worse, this minimal UI misses the mark: there is only one thing you can do with just one tap, and that's taking a picture. You can argue this makes the camera easier to handle, but I bet there's more than one Nexus 4 user who's never used the front camera, something more important than adjusting the white balance.

The past decade, we've seen what was once just Apple's domain become mainstream, that is, bringing high-end UX design to software. But I can't help but feel some of the finesse has been lost. Gripes about iOS and OS X suggest the same is true for Apple as well.

You hear a lot about "first time user experience" for example. But it's not about wrapping up your product like a present. It's about creating a connection of trust through empowerment and a little bit of emotional appeal: "This is for you, you can do amazing things with this." And that means "first-time" shouldn't refer to the first time you turn on the device, but the first time you use a device for a particular purpose and context. Travelling to Another Country should definitely be treated as a "first time" experience, same with How Do I Work This Camera, I Don't Have an App For This, I Don't Have Data Right Now, I Dropped It Down The Stairs, I Should've Cached This Map But I Didn't, My Friend Has a Windows Phone, etc. Throwing in more obnoxious tutorials is not the answer, creating affordance is.

One of the most brilliant things Apple did for the iPhone was to make screenshots easy. People could show what they were doing with it, it empowered them to do so, popping up in articles and feeds. It took until Android 4 before you could do that without a USB debugger, so the only screenshots we got were taken by developers. Which platform has which image today? The iPhone isn't actually that intuitive, but most people already knew all about swiping and pinching by the time they picked one up for the first time. But the Android experience cannot come into its own when it's only chasing the walled gardens that already exist. It lacks agency for the user and pretends the cloud can pick up the slack.

The most annoying part is, the worst bugs are well known, and the war over services is obvious. The issue tracker has long been full of reports and me-too-s and dear-god-when-will-this-be-fixed. Owning an Android means you accept having something that's always a bit broken, which doesn't integrate as well with anything you do as an iPhone does with its family, and just substitutes Google for Apple more and more. I'll stay on this side, I like my phone to be hackable, but that means being able to shut off the stupid warnings too, and having an OS actually worth hacking for.

If the insides of this phone were as thoughtfully put together as the outside, Android could be to mobile what OS X was to the desktop in the 2000s. But so far, no dice.

Above all, it's true what Steve Jobs said: Android is a stolen product. After so many years, there is little about the platform that can be said to set it apart from direct competitors. It's now just a mostly well done version of the same thing. I'm one of those old fogeys who has resisted getting a tablet, simply because I don't want a gianter phone. I want a touch computer, with everything that implies, not just something to do email and watch movies on.

Quotes from: Android – Design Principles, Google Inc. (cc)