Zero to Sixty in One Second

August 23, 2013, 12:00 am

Fusing WebGL, CSS 3D and HTML

Ladies and gentlemen, this is your captain speaking. Today's flight on WebGL Air will take us high above the cloud. Those of you sitting in Chrome class, on the desktop side of the plane, will have the clearest view. Internet Explorer class passengers may turn to their in-seat entertainment system instead. Passengers are reminded to put away their iPads and iPhones during take off and landing.

Edit: You can now also watch the YouTube video kindly provided by Leland Batey.

Acko.net, the domain, just turned 13, so it's time for a birthday present in the form of a complete front-end rewrite. The last design was entirely based on CSS 3D: it was a fun experiment, but created at a time when browser implementations were still wonky. It ultimately proved impractical: the DOM is not a good place to store complicated geometry. It's too bulky and there is a huge difference between a styled <div> and a shaded quad. After adding WebGL to the mix with MathBox and typesetting with MathJax, it turned into a catastrophic worst case for loading, and the smoothness was often lost even on fast computers.

Since then, we've seen a big push for rendering performance, both from the native and JS side. Hardware-accelerated DOM compositing is better understood, requestAnimationFrame is now common-place and reliable, and we have excellent profiling tools down to the frame level.

Hence the goal: to fuse 3D elements into the page like before, but with full 60 fps rendering. Plus to use WebGL instead of CSS 3D where possible, and be free of the constraints of the DOM.

Like Voodoo.js I use a fixed full-screen canvas and sync up scrolling with a 3D camera. The scene is mapped to CSS pixels and CSS perspective is locked to the camera. Once HTML, CSS 3D and WebGL are all in sync there's a truckload of linear algebra and easing functions to keep you amused. The code is based on the platform I kludged together for the christmas demo, at times a mess of ad hoc demo formulas and spaghetti, though robust enough in the parts that count.

Daim

Procedural Wildstyle

The so Designers Republic kitsch of last time was starting to grate, and I wanted something more stylish. Taking inspiration from street art and graffiti, in particular long-time favorite Daim, Acko's gone wild, though with a math twist. This design was built procedurally using some nifty vector calculus and more difference equations than you can shake a stick at.

At its heart it's really a game of Snake, albeit a complicated one. Starting from a line-based skeleton of the model, the curves are traced out, smoothed, oriented and finally extruded on the fly into ribbons. The final pose shows 261 lines mapped to 2,783 curve segments, tessellated into 43,168 triangles, though that amount is just a knob that can be tuned. In other words, it's a scalable vector graphic. No images were harmed in the making of this header, or the rest of the window dressing for that matter. These pixels are local, organic and bespoke, though I'm pretty sure they're not vegan.

The ribbons need to animate, so I'm using a slightly exotic set up: a single Three.js BufferGeometry mesh is created with all the ribbons' topology in it. That is, it only contains the indexes for the vertices. The actual positions and normals are read from a texture that is updated on the fly by JavaScript using gl.texSubImage2D.

As a ribbon follows its path, only its head and tail change, growing new segments or collapsing older ones. Each ribbon scrolls through the texture, which alters only a fraction of the total data per frame. Both the head and tail can add and remove segments at will, which allows me to vary the geometrical detail along each path. As long as the number of drawn segments at any given time never exceeds the texture height, a ribbon could in theory follow an infinitely long trail, wrapping around the edge endlessly. They're basically semi-virtualized meshes, allocated out of a fixed memory space. The arrows are an easy upgrade: I allocate a couple additional segments at the end and shape them to the right widths.

To draw a ribbon, I draw one contiguous subset of the total mesh (or two, if wrapping around the edge) which maps into a rectangular area of the texture. Ribbon color is stored statically as a vertex attribute, which means the entire hero piece can be drawn using a single shader, texture and vertex buffer.

For the background, I added a static BufferGeometry clocking in at 3,344 triangles. It consists of rounded boxes laid out in a randomized hyperboloid pattern. Aside from breaking the uniform whiteness, it's also quite the parallax enhancer down here.

DJ Ambient

All of this is shaded with some custom GLSL. I use a basic Blinn-Phong lighting model tweaked for aesthetics, and there's some edge highlights and fog to top it off. But if I left it there, the result would still look pretty flat, without shadows. A common approach is to use shadow mapping, but that only works for point sources like the sun. The diffuse shadows that you see on an overcast day require an entirely different approach.

What's needed is ambient occlusion, a measure of how much skylight is obscured by the surrounding geometry at every point. The less sky you can see from a point, the darker it should be. This is slow to compute analytically for large scenes and hence is typically either faked or baked in. An easy hack is Screen-Space AO, which is really just crease darkening using the final image's depth buffer. It's clever but expensive, creating only local shadows. Quality can vary a lot between implementations, often creating distracting dark or light halos around silhouettes. They all share the same blind spot: only the currently visible pixels can cast any shadow. So while I added alteredq's SSAO shader, it's disabled by default and only rendered at quarter resolution, upscaled with a bilateral filter to hide this fact.

Instead, I used a technique from NVidia's GPU Gems 2. It uses a disc-based model to calculate occlusion and even indirect lighting on the fly. Rather than go real-time on the GPU, I decided to only do it statically in JS, and just do plain occlusion. The discs for all the ribbons are generated ahead of time, and the disc-to-disc shadowing is done once for the final model in two passes. The first pass overestimates due to overlapping shadows and thus is too dark. The second pass uses the first to assign lesser weights to discs in shadow to compensate, then runs the algorithm anew.

There's another simplification: instead of small one-sided discs based on the real geometry, I use large two-sided discs and treat each ribbon as flat. Each disc occludes in both directions, but accumulates shadow on its front and back separately. Below is the actual disc model. In this diagram, the two radii of each disc represent its front and back illumination, with smaller discs receiving more shadow. The crosshairs mark the real disc radius.

When generating a ribbon vertex, the four nearest discs on the ribbon are interpolated bilinearly, and the vertex's normal and position is used to mix the front and back shadow values appropriately. Thus it's effectively trilinear filtering a 2×2×N grid per ribbon, though technically the interpolation is not strictly linear. The resulting occlusion is stored inside the texture in the unused alpha channel and applied directly in the shader.

I'm very happy with how good it looks, even with very coarse divisions like this. Unlike SSAO it costs practically nothing once generated. If you wish, you can view the design entirely in white to examine the lighting—or in black and white, or in the seventies, tone mapping is fun.

To ensure smooth rendering, the resolution is scaled down if it drops below 45 fps for several frames. This is essential on slower GPUs, but can sometimes overcompensate, as WebGL is easily interrupted by other tasks on the machine. The browser doesn't tell you how much you're currently pushing it, so it's impossible to safely scale back up without causing more stutters. To mitigate this problem, I built in some strategic reset points, like when you focus the window or scroll to the top. In the end, 60fps happens more often than not, though it remains a goal rather than a guarantee. If you fullscreen your browser, you are now the bane of my existence, especially if you keep two dozen other windows open on the second monitor at the same time. On an underpowered retina MacBook.

Finally I also turned on MultiSample Anti-Aliasing to avoid the dreaded jaggies and make the vector style shine. It's substituted with Fast Approximate AA if it's unavailable. FXAA is also used if SSAO is enabled, as you can't post-process multisampled images with WebGL.

Preparing all of this would take ~400ms on my 4 year old laptop, but most of that is spent on the occlusion which is static. Hence, I saved the lighting data into a JSON file for speedier loading, which cuts it down to ~100ms. The data gzips into 18KB, about the size of a small PNG, so no worries there. Add to that the time required to fetch and run the rest of the page, and we can call it an even second.

Of course, it only works on a subset of browsers and leaves the vast majority of mobile devices out in the cold. It does work in Chrome Beta on Android, though performance and stability is still pretty crap and more fixes are needed, both in the browser and in my own code. Rather than try and emulate some of the bling for CSS 3D-only environments, it's all or nothing. Without WebGL, you get plain images.

Achievement Unlocked

Reading all that, you might mistake the header as a mere love letter to hackery. But there's a twist, quite a literal one. The twisting of each ribbon is not generated arbitrarily, but mathematically derived. It embodies the differential principle of parallel transport. The up direction changes parallel to each curve, which means the ribbons never rotate in place. They only turn when they naturally want to. Hence, the design kind of has a will of its own. I can set the initial up direction of a ribbon, but only affect it through its curved path. Arranging and tuning all the ribbons was a nice puzzle in itself, and it's a nice nod to the math that's all over the place, even if it is invisible.

As I started playing around with some screensaver-style shots, I realized it would make a pretty neat demo just by itself, so I built that in too, to the dulcet tones of Selah Sue—whose last name I hope is not indicative. Here parallel transport would ensure a perfectly smooth ride, but that's not exciting, so there's some springy exponential easing with adaptive lookahead instead. That's on top of the secondary demo, which features an audio visualizer and the smooth drum and bass of Seba. There's maybe a third one too.

The songs are used here entirely for educational purposes of course. Not that it matters, since they're all on YouTube anyway. Click the Growl-like notifications to find out more about the artists. There's also a handful of achievements to be gathered, eight to be precise. Some of these are experiments that were turned off in the final version. Others are... trickier. Oh hey, did you notice the JS console?

By the way, special mention also goes to Time and how some browsers keep terrible track of it. Like when you're trying to sync a demo to a stuttery <audio> clock, Firefox ಠ_ಠ. There's also the matter of what to do when the user switches to another tab, and the answer is.... it'll desync, because I don't want to be sapping resources in a background tab. Minor constraints of putting on a live act, doors close after the show starts.

Achievement Unlocked Information Music

Turn Right Past The Header

The refresh-less navigation is also back from last time, slightly cleaned up. It still works on the same basic principle of fetching static, full HTML documents. However the transition mechanism now carefully choreographs the necessary DOM manipulation to avoid stutters: any image, iframe or video inserted in the content would mean future paints at an unknown time, so they're postponed until after the transition is complete. I did it the dirty but fast way, with string manipulation before it hits the DOM: such is the luxury of being the only one, person or machine, writing the markup.

Which leaves of course the content. But don't fix what ain't broke: distraction-free publishing is easy enough with Jekyll and a responsive 960 pixel grid with sideburns, so things still look and work mostly the same. I only added a home for my math talks and expanded the front page to highlight some demos and experiments. I aim to keep focusing on quality rather than quantity, and to remind visitors what the web looks like when you take away everything social media's done to it.

It's also fun to observe that after many years, the unified front-end convergence really has arrived. There is little difference between this site and the games that use HTML for their UI, such as the latest Sim City. For dealing with typography, illustration and UI, you want the comfort of DOM and CSS. For real-time graphical content, you'll want to draw it yourself, either in 2D, or 3D. Combine the two, and you get what game developers have been doing for years. Only this lacks all the sharp C bits, which game devs been replacing with Lua for years anyhow. That's Portuguese for "JavaScript" by the way.

So there you have it. A fresh look with a pile of juicy hax, and hopefully not too many bugs in the wild. Thanks go to all those who came before and provided all these toys for me to play with. You can too, for the source is entirely on Github. Like I said, educational. Meanwhile I'll be over here, listening to the wipE'out" soundtrack on repeat in memory of the old one. Comments welcome on Google Plus.

PS: The previous version has been archived with its fallbacks disabled so you can see the current state of CSS 3D in browsers. Still pretty broken. Still a great test case. So is the disembodied head.

↧

Animate Your Way to Glory

September 13, 2013, 12:00 am

≫ Next: Animate Your Way to Glory - Part II

≪ Previous: Zero to Sixty in One Second

Math and Physics in Motion

“The last time that I stood here was seven years ago.”
“Seven years ago! How little do you mortals understand time.
Must you be so linear, Jean-Luc?”
– Picard and Q, All Good Things, Star Trek: The Next Generation

Note: This article is significantly longer than previous instalments. It features 4 interactive slideshows, each introducing a new tool as well as related concepts around it. In one way, it's just another math guide, but going much deeper. In another, it's a thesis on everything I know about animating. Their intersection is a handbook for anyone who wants to make things move with code, but I hope it's an interesting read even if that's not your goal.

Developers have a tough job these days. A seamless experience is mandatory, polish is expected. On touch devices, they are expected to become magicians. The trick is to make an electronic screen look and feel like something you can physically manipulate. Animation is the key to all of this.

Not just any animation though. Flash intros were hated for a reason. The <blink> tag is not your friend, and flashing banner ads only annoy rather than invite. If elaborately designed effects distract from the content, or worse, ruin smoothness and performance, it'll turn people off rather than endear. Animation can only add value when its fast and fluid enough to be responsive.

It's not mere polish either, a finishing touch. Animation–and UI in general—should always be an additional conversation with the user, not a representation of internal software or hardware state. When we press Play in a streaming music app, the app should respond immediately by showing a Pause control, even if the music won't actually start playing for another second. When we enable Airplane Mode on our phones, we don't care that it'll take a few seconds to say good bye to the cell tower and turn off the radio. The UI is there to respond to our wishes: it should act like a personal assistant, not a reluctant helper, or worse, a demanding master.

The OS X 'genie' effect. Ridiculed, but it leaves no question where the window went.

Hence animation is visual language and communicates both explicitly and implicitly. It establishes an unspoken trust and confidence between designer and user: we promise nothing will appear, change or disappear without explanation. It can show where to find things, like an application that minimizes into place in the dock, or a picture sliding into a thumbnail strip. It can tell miniature stories, like a Download button turning into a progress bar turning into a checkmark. More simply just the act of scrolling around a live document, creating the illusion of viewing an infinite canvas, persisting in space and time. Here, page layout is the use of placement and style to denote hierarchy and meaning in a 2D space.

As with any conversation, tone matters, in this case expressed through choreography. Items can fade into the background or pop to demand our attention, expressing calm or assertiveness. Elements can complement or oppose, creating harmony or dissonance. Animations can be minimalist or baroque, ordered or parallel, independent or layered. The proper term for this is staging, and research shows that it can significantly increase our understanding of diagrams and graphs when applied carefully. Whenever elements transition, preferably one at a time, it is easier to gauge changes in color, size, shape and position than when we are only shown a before and after shot.

This is important everywhere, but especially so for abstract topics like data visualization and mathematics. When we have no natural mental model of something, we build our understanding based on the interface we use to examine it. The more those interfaces act like real objects, the less surprising they are.

In doing so, we replace explicit explanations with implicit metaphors from the natural world: distance, direction, scale, shadow, color, contrast. These are the cues our brains evolved to be excellent at interpreting. By imbuing virtual objects with these properties, we make them more realistic and thus more understandable. Mind you, this is not a call for skeuomorphism, far from it. The properties we are seeking to mimic are far more basic, far more important, than some faux leather and stitching.

D3.js Force Directed Graph — Mike Bostock

Star Trek TNG PADD, aka the iPad. Arrived slightly before the 2360s.

The clearest example of this has to be inertial scrolling. Compared to an ordinary mouse wheel, scrolling on a tablet is actually much more complicated. We can flick and grab, go as fast or slow as we want. When skimming through a list, often we never wait for the page to stop moving, in theory requiring more effort to read. Yet everyone who's seen a toddler with an iPad can attest to its uncanny ease of use and efficiency, offering improved control and comprehension. Our brains are very good at tracking and manipulating objects in motion, particularly when they obey the laws of physics: moving with consistent inertia and force.

Which brings me to the actual topic of this post: how animation works on a fundamental level. I'd like to teach a mental model based on physics and math, and how to precisely control it. Along the way, we'll come to understand why Apple built a physics engine into iOS 7's UI, reveal some secrets of the demoscene, compose fluid keyframe animations, and defeat the final boss: seamless rotation in 3D. In doing so, we'll also go beyond just visual animation. The techniques described here work equally well for manipulating audio, processing data or driving meatspace devices. In a world of data, animation is just a different word for precise control.

A Matter of Time

An animation is something that changes over time. As it so happens, these three humble words are a veritable Pandora's box of mathematics. They open up to the strange world of the continuously and infinitely dividable, also known as calculus.

In a previous article, I covered the origins of calculus and how to approach the concept of infinity. In what follows, we won't be needing it much though. We'll be working with finite steps throughout, with discrete time. This makes it vastly easier to understand, and is an eminently useful stepping stone to the true theory of continuous motion, which you can find in any good physics textbook.

Math class hates it when we just punch numbers into our calculator instead of deducing the exact result: a decimal number is meaningless on its own. On that, I can agree. But when we punch in a couple thousand numbers and look at them in aggregate, it can tell us just as much. This page will be your calculator.

Let's start where Isaac Newton supposedly did, with an apple.

Gravity kicks in. The apple bounces off the ground, losing some energy in the process. After a few bounces, its kinetic energy (speed) and potential energy (height) have both dissipated, and the apple is at rest.

But analyzing motion by watching it in real-time is tricky. It's better to visualize time as its own dimension, here horizontal, and look at the entire animation as a whole.

$$ \class{blue}{p(t)} $$

The apple's position $ \class{blue}{p(t)} $ moves through space and time, along arcs of decreasing height and duration. Once at rest, it continues advancing through time, without moving in space. In common parlance, this is the animation's easing curve.

$$ \class{blue}{p_i}, \, t_i $$

It's worth pointing out they're not really arcs. This animation consists of individually numbered frames $ i $, switching 60 times per second. While a frame is displayed, the position $ \class{blue}{p_i} $ of the apple is constant. In between its value changes instantly, at times $ t_i $.

For convenience's sake, it's reasonable to consider this a curve, approximated by a series of straight lines. After all, that's the illusion that the animation successfully tricks us into seeing. The discrete nature of the curve will let us dissect it more easily. We're interested in the physics of this motion.

$$ \class{green}{v_{i→}} = \frac{\class{blue}{p_{i+1}} - \class{blue}{p_{i}}}{t_{i+1} - t_i} $$

To determine the speed of the apple, we find the slope of a line segment: vertical divided by horizontal. Dividing distance by time gives us speed, e.g. meters per second. But actually, we're dealing with its cousin velocity which has a direction too. Positive slope means going up, negative slope means going down. This operation is called a forward or backward difference, depending on whether you look forward ($ \class{green}{v_{i→}} $) or backward ($ \class{green}{v_{←i}} $) around a point.

$$ \class{green}{v_{i↓}} = \frac{\class{blue}{p_{i+1}} - \class{blue}{p_{i-1}}}{t_{i+1} - t_{i-1}} $$

Forward differences tell us about what's happening between two adjacent points. We're more interested in what's happening at the points themselves. To fix this, we can take a central difference $ \class{green}{v_{i↓}} $, spanning two frames instead. We now get a good approximation for the slope directly at a point of interest, and thus the velocity.

$$ \class{blue}{p_i}, \, \class{green}{v_{i↓}} $$

If we apply this procedure along the entire curve, we can graph the apple's velocity over time, in sync with its position. This is the discrete version of taking the derivative in calculus, or differentiation and shows these two quantities are intimately related.

While in the air, the apple's velocity decreases along a straight line, first positive, then negative. On impact, the velocity suddenly reverses, though only to a portion of its previous value. At the top of each arc, the velocity passes through zero, which means the apple essentially hangs motionless in the air for a fraction of a second.

$$ \class{blue}{p_i}, \, \class{green}{v_{i↓}}, \, \class{orangered}{a_{i↓}} $$

To further analyze this, we can repeat the procedure, and find the slope of the velocity. This is the change in velocity over time, better known as acceleration. It can be expressed in meters per second per second, that is, $ m / s^2 $. According to Newton, acceleration is force divided by mass: the heavier something is, the less effect the same force has.

What looked like a complicated animation at the position level is now revealed to be very simple: the apple undergoes a small constant acceleration downwards from gravity. It also experiences a short burst of much stronger acceleration upwards whenever it bounces. Once the upward force goes below a critical threshold, the apple stops moving. At the end, gravity is countered by the apple's resistance to being squished, and the net acceleration is zero.

Suppose we were given only the acceleration, and wanted to reconstruct the animation. Can we do that?

$$ \class{green}{v_{i+1→}} = \class{green}{v_{i→}} + \class{orangered}{a_{i→}} \cdot (t_{i+1} - t_i) $$

Yep, we just work our way back up. If the acceleration represents a difference in velocity over time, then we can track the velocity by adding these differences back, accumulating them one step at a time. Since we divided the differences by time initially, we'll now have to multiply each value by the time between frames. Technically we need forward differences ($ \class{orangered}{a_{i→}} $) for this, not central ones ($ \class{orangered}{a_{i↓}} $), but the error will be minor.

$$ \class{green}{v_{i+1→}} = \sum\limits_{k=0}^i\class{orangered}{a_{k→}} \cdot Δt $$

In calculus, this accumulation process is called integration. In our case, it's a sum ($ \sum $). As we are multiplying the vertical value $ \class{orangered}{a_{k→}} $ by the horizontal time step $ Δt $, each term represents the area of a thin rectangle. By adding up all these signed areas, positive for up and negative for down, we can approximate the integral and get velocity back. Integrals and areas under curves are very closely linked.

$$ \class{green}{v_{i+1→}} = \class{green}{v_{0→}} + \sum\limits_{k=0}^i\class{orangered}{a_{k→}} \cdot Δt $$ $$ \class{blue}{p_{i+1}} = \class{blue}{p_0} + \sum\limits_{k=0}^i\class{green}{v_{k→}} \cdot Δt $$

Similarly, we can integrate velocity into position by adding up strips of area under the velocity curve, recreating the original bounce. Note that for both sums, we needed to manually specify the starting point. If we didn't set it correctly, the apple would drift, bounce on thin air or penetrate the ground.

We've produced real physical behavior from raw forces like gravity. That means we've just described a real physics engine. It's a one-dimensional one, but a physics engine none the less. It implements Euler integration, a fast but generally inaccurate method. In this case, the reconstruction is not perfect due to the earlier mentioned usage of central rather than forward differences.

We only need one of the three in order to produce a plausible copy of the other two. That means we can control animations on any of the three levels. If we want full control, we specify position directly. For simple constrained motions, we can manipulate velocity and integrate once. For full-on physics, we set acceleration from physical laws and integrate twice. This is why the Newtonian model of motion is so important.

It also reveals smoothness. A smooth animation isn't just continuous in its path. Its velocity is continuous too, without sudden jumps. In some cases, we'll even want smooth acceleration too. An ordinary bounce effect is shown to involve a large acceleration, a sudden jerk. This is a noticeable visual disruption, the kind we generally want to avoid. If you've ever tried to ignore a bouncing icon, you'll know how hard this is.

In fact, jerk is what we call the slope of acceleration. That's three derivatives deep, and it's turtles all the way down. The next ones are imaginatively called snap, crackle and pop, though they signify little directly. A large jerk however implies a sudden, jarring change in force.

$$ \class{purple}{E_p} = m \cdot g \cdot h $$

There's more physics hiding in plain sight. Earlier on, I mentioned energy: kinetic and potential. The apple's available potential energy $ \class{purple}{E_p} $ comes from gravity and is proportional to its height $ h $ above the ground, as well as the mass $ m $ and the local strength of gravity $ g $.

$$ \class{cyan}{E_k} = \frac{1}{2} \cdot m \cdot v^2 $$

The kinetic energy $ \class{cyan}{E_k} $ comes from its motion. It's proportional to the velocity squared. That means each additional meter per second makes the previous ones more energetic, adding more kinetic energy the faster it's already going. To explain, we can imagine the force required to stop a moving object. By increasing the speed, you don't just add additional momentum: the impact also takes less time, concentrating it.

$$ \class{purple}{E_p} = m \cdot g \cdot h $$ $$ \class{cyan}{E_k} = \frac{1}{2} \cdot m \cdot v^2 $$

In a closed system, total momentum is conserved. As we are treating gravity as an outside force, this does not apply. Energy is conserved however. There's a vertical symmetry, where one energy level goes up as the other goes down, and vice versa. So we actually have a fourth level to control physics at: that of energy and potential. With some minor bookkeeping, we can create motion this way, called Hamiltonian mechanics.

$$ \class{royal}{E_t} = \class{purple}{E_p} + \class{cyan}{E_k} $$

The total energy, potential plus kinetic, is perfectly constant between bounces. On impact, a significant amount is lost. Note that the dips towards zero are a side effect of the finite approximation: if the bounce occurs between two frames, the apple appears to slow down for a frame, instantly falling down and bouncing back to where it was one frame earlier. Finite differences are oblivious to this.

The energy levels follow a decaying exponential curve. This is very typical: exponentials show up whenever a quantity is related to its rate of change. Hamiltonian models are useful for more complicated things like 3D roller coasters, where they allow you to abstract away complex interactions into a few concise relations like this.

In simple animation though, we'll generally stick to the direct Newtonian model. We can use it to analyze real use cases. Let's start with a common easing curve, cosine interpolation, used by default in jQuery.animate() and these slides too.

$$ lerp(\class{orangered}{a}, \class{green}{b}, f) = \class{orangered}{a} + (\class{green}{b} - \class{orangered}{a}) \cdot f $$

We animate the apple's position, changing its Y coordinate. In practice, that means we apply linear interpolation, lerping, between the start $ \class{orangered}{a} $ and end $ \class{green}{b} $. We take the starting point and add a fraction $ f $ of the difference $ \class{green}{b} - \class{orangered}{a} $ to it. Half the difference gets us halfway there, and so on. As long as $ f $ is between 0 and 1, we end up somewhere in the middle. When $ f $ reaches 1, the animation is complete.

$$ elerp(\class{orangered}{a}, \class{green}{b}, f) = \class{orangered}{a} + (\class{green}{b} - \class{orangered}{a}) \cdot \class{blue}{ease(f)} $$ $$ \class{blue}{ease(f)} = 0.5 - 0.5 \cdot \cos πf $$

The purpose of the easing curve is then to make the animation non-linear, not in space, but in time: in this case, the apple smoothly starts and stops. We can use any curve we like, e.g. half of a cosine wave of period 2. This eased lerp is the basic building block of any animation system.

$$ \class{blue}{p_i}, \, \class{green}{v_{i↓}}, \, \class{orangered}{a_{i↓}} $$

The effect of the easing curve is visible when we take central differences again, and look at velocity and acceleration. The acceleration has been divided by 3 to fit. This doesn't seem bad, all three quantities appear to change smoothly. This picture is deceptive though.

All curves continue before and after the animation. The smooth cosine ease turns out to be quite jarring in its acceleration: it's like flooring the accelerator from standstill then easing off gently. At the halfway point you start braking, more and more until you stop. It's one of the most responsive animations possible that's still smooth at both ends. Smoother easing curves have smoother accelerations, but respond slower.

$$ \class{blue}{ease(f)} = f^2 $$

A simpler example is the half-ease, here achieved with a quadratic curve $ \class{blue}{f^2} $. The velocity is a linearly increasing ramp. The acceleration is constant, except for a very large instant deceleration at the end. This is like flooring the accelerator from standstill, holding it down for the duration, and then crashing into a wall—the suicide ease. Due to this, half-easing is typically used for fading transitions, where the object is invisible–or the audio inaudible–at the start or end.

$$ \class{blue}{ease(f)} = \left\{ \begin{array}{ll} f^2 & \mbox{if } f \leq 1 \\ 2f - 1 & \mbox{if } f > 1 \end{array} \right. $$

But we can repurpose it quite easily. By tweaking this at the velocity level, we can maintain a constant speed at the end. This is the slow start, and can be expressed directly as an open-ended easing curve. In this case, we allow $ f $ to exceed 1, and the linear interpolation turns into extrapolation for free, no extra charge. We can scale the curve vertically to change the final speed, and scale it horizontally to control the delay. The slow start (and stop) is used throughout these slides.

$$ \class{blue}{ease(f)} = \frac{1}{4} \cdot (1 - \cos 2πf) + \left\{ \begin{array}{ll} f^2 & \mbox{if } f \leq 1 \\ 2f - 1 & \mbox{if } f > 1 \end{array} \right. $$

We can combine curves too. Here, we add a cosine wave to the slow start, creating perhaps the motion of a rising jellyfish. Adding up animations is an easy way to create variations on a theme, used often in the demoscene. The derivatives add straight up too, so all three curves shift up and down by a sine or cosine wave. You can see how a small shift in position can have a large effect on both velocity and acceleration.

The next example is a bit different. Any guesses as to what this is? The hint is in the vertical scale, now measured in pixels. This animation moves almost 1000 pixels in just over one second.

It's an inertial flick gesture, recorded on Mac OS X. We can plot velocity and acceleration again. There's a slight measurement error, visible as noisy ripples on the acceleration, even after smoothing out the data: derivatives are very sensitive to noise. The velocity and acceleration have also been scaled down to fit, as they are both quite large.

The first part of the curve is not an animation at all: it was tracking the direct motion of my finger. Fingers move very smoothly: the acceleration follows a curve up and down. This is more physics: of nerve signals causing muscle fibers to contract and digits to move. This work smoothly converts chemical potential into kinetic energy. The small jump in speed at time 0 is easy to explain: my finger was already moving when it touched the pad.

The second part is the actual inertial animation. It kicks in as soon as the finger leaves the pad. All three values follow an exponential curve past that point, disregarding the noise. But the important one is velocity: the animation starts with the last known velocity and smoothly decays it to zero. Where we end up depends on how fast we were going when the finger left the pad.

$$ \class{green}{v_{i+1→}} = \class{green}{v_{i→}} \cdot (1 - \class{royal}{f}) $$

Inertial scroll is easiest to control at the velocity level. We can measure the initial velocity by finding the position's slope, usually averaged over several frames. We then start at this velocity, but reduce it every frame by a fraction $ \class{royal}{f} $, which is a coefficient of friction. We don't need to care how far we'll go or how long it'll take: we can just keep animating until the velocity gets close enough to 0.

Suppose we do care where we end up. We might be showing a list of items, each 100 pixels tall. It could be good to control the animation so it always stops right at an item. We can't violate the principle of smooth motion, so we can't just change the position or velocity directly. We have to change the coefficient of friction.

$$ \class{green}{v_{i→}} = \class{green}{v_{0→}} \cdot (1 - \class{royal}{f})^i $$

As the velocity follows a simple curve, we don't have to track it manually. We can express it over time as a direct relation, based on the initial velocity $ \class{green}{v_{0→}} $. The exponential nature is clear, with the frame number $ i $ appearing as the exponent of a number between 0 and 1.

$$ \begin{array}{rl} \class{blue}{p_{i}} & = \class{blue}{p_0} + \sum\limits_{k=0}^{i} \class{green}{v_{0→} \cdot (1 - f)^k} \cdot Δt \\ & = \class{blue}{p_0} + \class{green}{v_{0→}} \cdot Δt \cdot \class{purple}{\sum\limits_{k=0}^{i} (1 - f)^k} \end{array} $$

The position at frame $ i $ is then the sum of all the previous velocities times the time step $ Δt $, just like before, relative to the initial position $ \class{blue}{p_0} $. As the time step and initial velocity are constant, we can move both outside the sum.

$$ \begin{array}{rl} \class{blue}{p_∞} & = \class{blue}{p_0} + \class{green}{v_{0→}} \cdot Δt \cdot \class{purple}{\sum\limits_{k=0}^{∞} (1 - f)^k} \\ & = \class{blue}{p_0} + \frac{\class{green}{v_{0→}} \cdot Δt}{\class{royal}{f}} \end{array} $$

To find the final resting position, we theoretically have to continue the animation all the way to infinity. This can be done using a limit. For now, we'll just look up the formula for this infinite sum, a geometric series. We end up dividing by the coefficient of friction: the lower it is, the further we go after all. If the coefficient were 0, there'd be no friction. We'd divide by zero, because there's no final resting position when you never slow down.

$$ \class{royal}{f} = \frac{\class{green}{v_{0→}} \cdot Δt}{\class{blue}{Δp}} $$

We can invert this relationship to find the coefficient of friction required to stop at a given target. We just need the initial distance to the target, $ \class{blue}{Δp} $. To apply this in practice, we determine the friction needed to reach the next couple of items, and pick the one which is closest to the default case. The user won't notice the subtle change in friction—the UI will just magically seem better.

The simulation works identical in all cases and the velocities are still continuous and exponential, which means: physical. This effect only requires one additional calculation at the start, which makes it all the more strange that developers have come up with increasingly jarring ways to achieve something similar.

Now let's try animating in 2D.

$$ x(t) = \sin t $$ $$ y(t) = \sin t $$

We can move the apple in 2D by animating its X and Y coordinates. Here we animate both in lockstep, using a sine wave: the apple moves diagonally, as X and Y are always equal. By adjusting their relative amplitudes, we can control the angle of motion.

$$ x(t) = \sin t $$ $$ y(t) = \sin \frac{7}{8}t $$

If we animate X and Y separately, we create arbitrary paths. Here they both follow a sine wave, but with different frequencies. The resulting path is called a Lissajous curve. The sine waves drift in and out of phase, going from a diagonal to an oval to a circle, and back again.

$$ \class{blue}{\vec p(t)} = \begin{bmatrix} \class{blue}{p_x(t)} \\ \class{blue}{p_y(t)} \end{bmatrix} = \begin{bmatrix} \sin t \\ \sin \frac{7}{8}t \end{bmatrix} $$

It makes more sense to picture the position as a 2D vector, an arrow. It has both a direction and a length, relative to the origin. While the calculation is equivalent—animating X and Y separately—the vector representation is more natural once we look at the derivatives.

$$ \class{green}{\vec v_{i→}} = \frac{\class{blue}{\vec p_{i+1}} - \class{blue}{\vec p_{i}}}{t_{i+1} - t_i} $$

What does slope and velocity mean in this context? The same principle applies: we take the difference in position between two frames, and divide it by the difference in time $ Δt $. In this case, all quantities except time are vectors.

As a single frame is very short, the velocity is quite large, and always tangent to the path. Its length directly represents speed.

If we center the velocity vector, it traces out its own Lissajous curve. This one is slightly different and doubles back on itself at regular intervals.

$$ \class{orangered}{\vec a_{i→}} = \frac{\class{green}{\vec v_{i+1→}} - \class{green}{\vec v_{i→}}}{t_{i+1} - t_i} $$

We can apply finite differences again to dissect velocity into acceleration. It follows yet another Lissajous curve, a scaled and rotated version of the position.

Finally, we can disentangle these curves by plotting them out over time. Position, velocity and acceleration dance around each other. Despite its artificial construction, even this motion is physical: it's what happens when you take an object and hang it off independently moving horizontal and vertical springs of different stiffness. With the right visualization, raw physics is quite beautiful in its own right.

We've seen how to examine an animation at multiple levels of change: position, velocity, acceleration. Differences approximate derivatives and let us to dissect our way down the chain. Accumulators approximate integration and let us construct higher levels from lower ones. Thus we can manipulate an animation at any level. By plugging in correct physical laws or arbitrary formulas, we can produce behavior that is as physical or unphysical as we like.

Customer is King

Everything we've done so far has been independent animation, without interaction. Even inertial scrolling has this luxury: whenever the user is touching, there is no inertia and the animation system is inactive. It's only when you let go that the surface coasts.

In many cases, this is not enough: animations need to be scheduled and executed while retaining full interactivity. Often the animation needs to continue despite its target changing midway. In order to handle such situations, we need to build adaptive models that remain continuous and smooth, no matter what.

We'll also need to drop the assumption that the frame rate—the time step—is constant. In the real world, the frame rate might drop here or there, or be variable altogether. In either case, we'd prefer it if the effect of this was minimal. If we're adding music to an animation, this is essential to prevent desynchronization. It will also have some nasty consequences for our physics engine, and we need to level it up significantly.

So far, we've assumed a constant frame rate.

If our animation is defined by an easing curve, we can look up its value at any point along the way.

It seems at first, variable frame rates are trivial: we can evaluate the curve at arbitrary times instead of pre-set intervals.

$$ \class{green}{v_{i→}} = \frac{\class{blue}{p_{i+1}} - \class{blue}{p_{i}}}{t_{i+1} - t_i} $$

$$ \class{blue}{p_{i+1}} = \class{blue}{p_0} + \sum\limits_{k=0}^i\class{green}{v_{k→}} \cdot Δt_i $$

If we take forward differences to measure slope, we still get a smooth velocity curve. We can accumulate—integrate—these differences back into position as long as we account for a variable time step $ Δt_i $. It seems our physics engine should be unbothered too. But there's a few problems.

$$ \class{green}{v_{i+1→}} = \class{green}{v_{i→}} \cdot (1 - \class{royal}{f}) $$

First, if we implemented inertial scrolling like we did before, multiplying the velocity by $ 1 - \class{royal}{f} $ every frame, we'd get the wrong curve. The amount of velocity lost per frame should now vary, we can no longer treat it as a convenient constant.

$$ \begin{array}{rcl} (1 - \class{purple}{f_i})^\frac{t}{Δt_i} & = & (1 - \class{royal}{f})^\frac{t}{Δt} \\ ⇔ \,\,\, \class{purple}{f_i} & = & 1 - e^{\frac{Δt_i}{Δt} \log_e (1 - \class{royal}{f})} \end{array} $$

If we do the math, we can find an expression for the correct amount of friction $ \class{purple}{f_i} $ per frame for a given step $ Δt_i $, relative to the default $ \class{royal}{f} $ and $ Δt $. Not pretty, and this is just one object experiencing one force. In more elaborate scenarios, finding exact expressions for positions or velocities can be hard or even impossible. This is what the physics engine is supposed to be doing for us.

There's another problem. If we integrate these curve segments to get position, we get an exponential curve, just as before. Did we achieve frame rate independence?

Well, no. If we change the time steps and run the algorithm again, it looks the same. However, the new curve and old curve don't match up. The difference is surprisingly large, as this animation is only half a second long and the average frame rate is identical in both cases. Such errors will compound the longer it runs, and make your program unpredictable.

Luckily we can have our cake and eat it too. We can achieve consistent physics and still render at arbitrary frame rates. We just have to decouple the physics clock from the rendering clock.

Whenever we have to render a new frame, we compare both clocks. If the render clock has advanced past the physics clock, we do one or more simulation steps to catch up. Then we interpolate linearly between the last two values until we run out of physics again.

This means the visuals are delayed by one physics frame, but this is usually acceptable. We can even run our physics at half the frame rate or less to conserve power. Though more error will creep in, this error will be identical between all runs, and we can manually compensate for it if needed.

When we implement variable frame rates correctly, we can produce an arbitrary number of frames at arbitrary times. This buys us something very important, not for the end-user, but for the developer: the ability to skip to any point in time, or at least fast-forward as quickly as your computer can manage.

But just because the simulation is consistent, doesn't mean it's correct or even stable. Euler integration fits our intuitive model of how pieces add up, but it's actually quite terrible. For example, if we made our bouncing apple perfectly elastic in the physical sense—losing no energy at all—and apply Euler, it would start bouncing irregularly, gaining height.

Which means the first bounce simulation wasn't using Euler at all. It couldn't have: the energy wouldn't have been conserved. All the finite differentiation and integration magic that followed only worked neatly because the position data was of a higher quality to begin with. We have to find the source of this phantom energy so we can correct for it, creating the Verlet integration that was used.

We're trying to simulate this path, the ideal curve we'd get if we could integrate with infinitely small steps. We imagine we start at the point in the middle, and would like to step forward by a large amount. The time step is exactly 1 second, so we can visually add accelerations and velocities like vectors, without having to scale them. Note that this is not a gravity arc, the downward force now varies.

$$ \class{green}{v_{i+1→}} = \class{green}{v_{i→}} + \class{orangered}{a_{i→}} \cdot Δt $$ $$ \class{blue}{p_{i+1}} = \class{blue}{p_{i}} + \class{green}{v_{i→}} \cdot Δt $$

Earlier, I said that if we used forward differences, we could get the velocity between two points. And that we could make a reconstruction of position from forward velocity by applying 'Euler integration'. While that's true, that's not actually what Euler integration is.

See, this is a chicken and egg problem. This velocity isn't the slope at the start or the end or even the middle. It's the average velocity over the entire time step. We can't get this velocity without knowing the future position, and we can't get there without knowing the average velocity in the first place.

$$ \class{green}{v_{i+1↓}} = \class{green}{v_{i↓}} + \class{orangered}{a_{i↓}} \cdot Δt $$ $$ \class{blue}{p_{i+1}} = \class{blue}{p_{i}} + \class{green}{v_{i↓}} \cdot Δt $$

The velocity that we're actually tracking is for the point itself, at the start of the frame. Any force or acceleration is calculated based on that single instant. If we integrate, we move forward along the curve's tangent, not the curve itself. This is where the extra height comes from, and thus, phantom gravitational energy.

For any finite step, there will always be some overshooting, because we don't yet know what happens along the way. Euler actually made the same mistake we made earlier: he used a central difference where a forward one was required, because the forward difference can only be gotten after the fact. The 'central difference' here is the actual velocity at a point, the true derivative.

$$ \class{green}{v_{i+1↓}} = \class{green}{v_{i↓}} + \class{orangered}{a_{i↓}} \cdot Δt $$ $$ \class{blue}{p_{i+1}} = \class{blue}{p_{i}} + \frac{\class{green}{v_{i↓}} + \class{green}{v_{i+1↓}}}{2} \cdot Δt $$

As the acceleration changes in this particular scenario, we could try applying Euler, and then averaging the start and end velocities to get something in the middle. It fails, because the end velocity itself is totally wrong. Though we get closer than Euler did, we now undershoot by half the previous amount.

$$ \class{green}{v_{←i}} = \frac{\class{blue}{p_{i}} - \class{blue}{p_{i-1}}}{Δt} $$

To resolve the chicken and egg, we need to look to the past. We assume that rather than starting with one position, we start with two known good frames, defined by us. That means we can take a backwards difference and now know the average velocity of the previous frame. How does this help?

Well, we assume that this velocity happens to be equal or close to the velocity at the halfway point. We also still assume the acceleration is constant for the entire duration. If we then integrate from here to the next halfway point, something magical happens.

$$ \class{green}{v_{i→}} = \class{green}{v_{←i}} + \class{orangered}{a_{i↓}} \cdot Δt $$

We get a perfect prediction for the next frame's average velocity, the forward difference. By always remembering the previous position, we can repeat this indefinitely. That this works at all is amazing: we're applying the exact same operation as before—constant acceleration—for the same amount of time. On just a slightly different concept of velocity. Without even knowing exactly when the object reaches that velocity. That's Verlet integration.

Euler integration failed on a simple constant acceleration like gravity and can only accurately replicate a linear ease $ f $. This motion is a cubic ease $ f^3 $, with linear acceleration that decreases. Verlet still nails it, even when leaping seconds at a time. Why does this work?

Euler integration applies a constant acceleration ahead of a point. If there's any decrease in acceleration, it overestimates by a significant amount. That's on top of stepping in the wrong direction to begin with. Both position and velocity will instantly begin to drift away from their true values.

$$ \class{blue}{p_{i+1}} = 2 \cdot \class{blue}{p_{i}} - \class{blue}{p_{i-1}} + \class{orangered}{a_{i↓}} \cdot Δt^2 $$

Verlet integration applies the same constant acceleration around a point. If the acceleration is a perfect line, the error cancels out: the two triangles make up an equal positive and negative area. By starting with a known good initial velocity and cancelling out subsequent errors, we can precisely track velocity through a linear force. If we simplify the formula, velocity even disappears: we can work with positions and acceleration directly.

As this captures the slope of acceleration, we only get errors if the acceleration curves. In this case, the left and right areas don't cancel out exactly. The missing area however smoothly approaches 0 as the time step shrinks, a further sign of Verlet's error-defeating properties. If we do the math, we find the position has $ O(Δt^2) $ global error: decrease the time step $ Δt $ by a factor of 10, and it becomes 100× more accurate. Not bad.

For completeness, here's the 4th order Runge-Kutta method (RK4), which is a sophisticated modification of Euler integration. It involves taking full and half-frame steps and backtracking. It finds 4 estimates for the velocity based on the acceleration at the start, middle and end.

The physics can then be integrated from a weighted sum of these estimates, with coefficients $ [\frac{1}{6}, \frac{2}{6}, \frac{2}{6}, \frac{1}{6}] $. We end up in the right place, at the right speed. This method offers an $ O(Δt^4) $ global error. Decrease the time step 10× and it becomes 10,000× more accurate. We have a choice of easy-and-good-enough (Verlet) or complicated-but-precise (RK4), at any frame rate. Each has its own perks, but Verlet is most attractive for games.

With physics under our belt, let's move on. Why not animate time itself? This is the variable speed clock and it's dead simple. It's also a great debugging tool: sync all your animations to a global clock and you can activate bullet time at will. You can tell right away if a glitch was an animation bug or a performance hiccup. On this site too: if you hold Shift, everything slows down 5×.

$$ \class{green}{v_{←i}} = \frac{\class{blue}{t_i} - \class{blue}{t_{i-1}}}{\class{blue}{t_i} - \class{blue}{t_{i-1}}} = \frac{Δt_i}{Δt_i} = 1 $$

First, we differentiate the clock's time backwards—because in real-time applications, we don't know what the future holds. This is time's velocity $ \class{green}{v_{←i}} $. As we have to divide by the time step too, the velocity is constant and equal to 1. Let's change that.

$$ \class{blue}{t'_i} = \sum\limits_{k=0}^i \class{green}{v'_{←k}} \cdot Δt_k $$

We can reduce the speed of time at will, by changing $ \class{green}{v_i} $. If we then multiply by the time step $ Δt_i $ again and add the pieces back together incrementally, we get a new clock $ t'_i $. By integrating this way, we only need to worry about slope, not position: time always advances consistently. This is also where variable frame rates pay off: going half the speed is the same job as rendering at twice the frame rate.

Using our other tricks, we can animate $ \class{green}{v_i} $ smoothly, easing in and out of slow motion, or speeding into fast-forward. If we didn't do this, then any animation cued off this clock would jerk at the transition point. This is the chain rule for derivatives in action: derivatives compound when you compose functions. Any jerks caused along the way will be visible in the end result.

If time is smooth, what about interruptions? Suppose we have a cosine eased animation. After half a second, the user interrupts and triggers a new animation. If we abort the animation and start a new one, we create a huge jerk. The object stops instantly and then slowly starts moving again.

One way to solve this is to layer on another animation: one that blends between the two easing curves in the middle. Here it's just another cosine ease, interpolating in the vertical direction, between two changing values. We blend across the entire animation for maximum smoothness. This has a downside though: if the blended animation itself is interrupted, we'd have to layer on another blend, one for each additional interruption. That's too much bookkeeping, particularly when using long animations.

We can fix this by mimicking inertial scrolling. We treat everything that came before as a black box, and assume nothing happens afterwards. We only look at one thing: velocity at the time of interruption.

After determining the velocity of any running animations, we can construct a ramp to match. We start from 0 to create a relative animation.

We can bend this ramp back to zero with another cosine ease, interpolating vertically. This time however, the first easing curve is no longer involved.

If we then add this to the second animation, it perfectly fills the gap at the corner. We only need to track two animations at a time: the currently active one, and a corrective bend. If we get interrupted again, we measure the combined velocity, and construct a new bend that lets us forget everything that came before.

By using a different easing curve for the correction, we can make it tighter, creating a slight wave at the end. Either way, it doesn't matter how the object was moving before, it will always recover correctly.

But what if we get interrupted all the time? We could be tracking a moving pointer, following a changing audio volume, or just have a fidgety user in the chair. We'd like to smooth out this data. The interrupted easing approach would be constantly missing its target, because there is never time for the value to settle. There is an easier way.

$$ \class{blue}{p_{i+1}} = lerp(\class{blue}{p_{i}}, \class{purple}{o_{i}}, \class{royal}{f}) $$

We use an exponential decay, just like with inertial scrolling. Only now we manipulate the position $ p_{i} $ directly: we move it a certain constant fraction towards the target $ \class{purple}{o_{i}} $, chasing it. Here, $ \class{royal}{f} = 0.1 = 10\% $. This is a one-line feedback system that will keep trying to reach its target, no matter how or when it changes. When the target is constant, the position follows an exponential arc up or down.

$$ \class{blue}{p_{i+1}} = lerp(\class{blue}{p_{i}}, \class{purple}{o_{i}}, \class{royal}{f}) $$ $$ \class{cyan}{q_{i+1}} = lerp(\class{cyan}{q_{i}}, \class{blue}{p_{i}}, \class{royal}{f}) $$

The entire path is continuous, but not smooth. That's fixable: we can apply exponential decay again. This creates two linked pairs, each chasing the next, from $ \class{slate}{q_{i}} $ to $ \class{blue}{p_{i}} $ to $ \class{purple}{o_{i}} $. Each level appears to do something akin to integration: it smooths out discontinuities, one derivative at a time. Where a curve crosses its parent, it has a local maximum or minimum. These are signs that calculus is hiding somewhere.

$$ \class{blue}{p_{i+1}} = lerp(\class{blue}{p_{i}}, \class{purple}{o_{i}}, \class{royal}{f}) $$ $$ \class{cyan}{q_{i+1}} = lerp(\class{cyan}{q_{i}}, \class{blue}{p_{i}}, \class{royal}{f}) $$ $$ \class{slate}{r_{i+1}} = lerp(\class{slate}{r_{i}}, \class{cyan}{q_{i}}, \class{royal}{f}) $$

That's not so surprising when you know these are difference equations: they describe a relation between a quantity and how it's changing from one to step to the next. These are the finite versions of differential equations from calculus. They can describe sophisticated behavior with remarkably few operations. Here I added a third feedback layer. The path gets smoother, but also lags more behind the target.

If we increase $ f $ to 0.25, the curves respond more quickly. Exponential decays are directly tuneable, and great for whiplash-like motions. The more levels, the more inertia, and the longer it takes to turn.

$$ \class{blue}{p_{i+1}} = lerp(\class{blue}{p_{i}}, \class{purple}{o_{i}}, \class{blue}{f_1}) $$ $$ \class{cyan}{q_{i+1}} = lerp(\class{cyan}{q_{i}}, \class{blue}{p_{i}}, \class{cyan}{f_2}) $$ $$ \class{slate}{r_{i+1}} = lerp(\class{slate}{r_{i}}, \class{cyan}{q_{i}}, \class{slate}{f_3}) $$

We can also pick a different $ f_i $ for each stage. Remarkably, the order of the $ \class{royal}{f_i} $ values doesn't matter: 0.1, 0.2, 0.3 has the exact same result as 0.3, 0.2, 0.1. That's because these filters are all linear, time-invariant systems, which have some very interesting properties.

If you shift or scale up/down a particular input signal, you'll get the exact same output back, just shifted and scaled in the same way. Even if you shift by less than a frame. We've created filters which manipulate the frequencies of signals directly. These are 1/2/3-pole low-pass filters that only allow slow changes. That's why this picture looks exactly like sampling continuous curves: the continuous and discrete are connected.

Exponential decays retain all their useful properties in 2D and 3D too. Unlike splines such as Bezier curves, they require no set up or garbage collection: just one variable per coordinate per level, no matter how long it runs. It works equally well for adding a tiny bit of mouse smoothing, or for creating grand, sweeping arcs. You can also use it to smooth existing curves, for example after randomly distorting them.

However there's one area where decay is constantly used where it really shouldn't be: download meters and load gauges. Suppose we start downloading a file. The speed is relatively constant, but noisy. After 1 second, it drops by 50%. This isn't all that uncommon. Many internet connections are traffic shaped, allowing short initial bursts to help with video streaming for example.

$$ \class{blue}{p_{i+1}} = lerp(\class{blue}{p_{i}}, \class{purple}{o_{i}}, \class{royal}{f}) $$

Often developers apply slow exponential easing to try and get a stable reading. As you need to smooth quite a lot to get rid of all the noise, you end up with a long decaying tail. This gives a completely wrong impression, making it seem like the speed is still dropping, when it's actually been steady for several seconds. The same shape appears in Unix load meters: it's a lie.

$$ p'_{i+1} = lerp(p'_{i}, \class{purple}{o_{i}}, \class{royal}{f}) $$ $$ \class{cyan}{q_{i+1}} = lerp(\class{cyan}{q_{i}}, p'_{i}, \class{royal}{f}) $$

If we apply double exponential easing, we can increase $ f $ to get a shorter tail for the same amount of smoothing. But we can't get rid of it entirely: the more levels of easing we add, the more the curve starts to lag behind the data. We can do much better.

We can analyze the filters by examining their response to a standard input. If we pass in a single step from 0 to 1, we get the step response for the two filters.

Another good test pattern is a single one frame pulse. This is the impulse response for both filters. The impulse responses go on forever, decaying to 0, but never reaching it. This shows these filters effectively compute a weighted average of every single value they've ever seen before: they have a memory, an infinite impulse response (IIR).

Doesn't this look somewhat familiar? It turns out, the step response is the integral of the impulse response. It's a position. Vice versa, the impulse response is the derivative of the step response. It's a velocity. Surprise, physics!

But it gets weirder. Integration sums together all values starting from a certain point, multiplied by the (constant) time step. That means that integration is itself a filter: its impulse response is a single step, the integral of an impulse. Its step response is a ramp, a constant increase.

It works the other way too. Differentiation takes the difference of neighbouring values. It's a filter and its step response is just an impulse, detecting the single change in the step. Its impulse response is an upward impulse followed by a downward one: the derivative of an impulse. When one value is weighed positively and the other weighed negatively, the sum is their difference.

$$ \sum p_i \cdot Δt \,\,\, ↑ $$

$$ ↓ \,\,\, \frac{Δp}{Δt} $$

This explains why exponential filters seem to have integration-like qualities: these are all integrators, they just apply different weights to the values they add up. Every step response is another filter's impulse response, and vice versa, connected through integration and differentiation. We can use this to design filters to spec.

$$ \class{green}{v_{i→}} = \sin \frac{π}{4} t_i $$

That said, filter design is still an art. IIR filters are feedback systems: once a value enters, it never leaves, bouncing around forever. Controlling it precisely is difficult under real world conditions, with finite arithmetic and noisy measurements to deal with. Much simpler is the finite impulse response (FIR), where each value only affects the output for a limited time. Here I use one lobe of a sine wave over 4 seconds.

$$ \class{blue}{p_{i+1}} = \class{blue}{p_0} + \sum\limits_{k=0}^i\class{green}{v_{i→}} \cdot Δt $$

Even if we don't know how to build the filter, we can still analyze it. We can integrate the impulse response to get the step response. But there's a problem: it overshoots, and not by a little. Ideally the filtered signal should settle at the original height. The problem is that the area under the green curve does not add up to 1.

$$ \class{green}{v_{i→}} = \frac{π}{8} \sin \frac{π}{4} t_i \,\,\,\,\,\,\,\,\,\,\, \class{blue}{p_{i}} = \frac{1}{2} + \frac{1}{2} \cos \frac{π}{4} t_i $$

To fix this, we divide the impulse response by the area it spans, $ \class{green}{\frac{8}{π}} $, or multiply by $ \class{green}{\frac{π}{8}} $, normalizing it to 1. Such filters are said to have unit DC gain, revealing their ancestry in analog electronics. The step response turns out to be a cosine curve, and this filter must therefor act like perpetually interruptible cosine easing.

There's two ways of interpreting the step response. One is that we pushed a step through the filter. Another is that we pushed the filter through a step—an integrator. This symmetry is a property of convolution, which is the integration-like operator we've been secretly using all along.

Convolution is easiest to understand in motion. When you convolve two curves $ \class{purple}{q_i} ⊗ \class{green}{r_i} $, you slide them past each other, after mirroring one of them. As our impulse response is symmetrical, we can ignore that last part for now.

$$ \class{blue}{p_i} = \class{purple}{q_i} ⊗ \class{green}{r_i} = \class{cyan}{\sum\limits_{k=-∞}^{+∞}} \class{purple}{q_k} \cdot \class{green}{r_{i-k}} $$

We multiply both curves with each other, creating a new curve in the overlap: here a growing section of the impulse response. The area under this curve is the output of the filter at that time. The sum goes to infinity in both directions, allowing for infinite tails. We already saw something similar when we used a geometric series to determine the final resting position of an inertial scroll gesture. With a FIR filter, the sum ends.

$$ \class{blue}{p_i} = \class{purple}{q_i} ⊗ \class{green}{r_i} = \class{cyan}{\sum\limits_{k=-∞}^{+∞}} \class{purple}{q_k} \cdot \class{green}{r_{i-k}} $$

But why did we have to mirror one curve? It's simple: from the impulse response's point of view, new values approach from the positive X side, now left, not the negative X side, right. By flipping the impulse response, it faces the other signal, which is what we want.

$$ \class{blue}{p_i} = \class{green}{r_i} ⊗ \class{purple}{q_i} = \class{cyan}{\sum\limits_{k=-∞}^{+∞}} \class{green}{r_k} \cdot \class{purple}{q_{i-k}} $$

If we center the view on the impulse response, it's clear we've swapped the role of the two curves. Now it's the step that's passing backwards through the filter, rather than the other way around.

If we replace the step response with a random burst of signal, the filter can work its magic, smoothing out the input through convolution. It's a weighted average with a sliding window. The filter still lags behind the data, but the tail is now finite.

If we make the window narrower, its amplitude increases due to the normalization. We get a more variable curve, but also a shorter tail. This is like a blur filter in Photoshop, only in 1D instead of 2D. As Photoshop has the entire image at its disposal, rather than processing a real-time signal, it doesn't have to worry about lag: it can compensate directly by shifting the result back a constant distance when it's done.

$$ \class{blue}{ease(f)} = \frac{1}{2} - \frac{1}{2} \cdot \cos πf \,\,\,\,\,\,\,\,\,\,\, \class{green}{slope(f)} = \frac{1}{2}π \cdot \sin πf $$

What about custom filter design? Well, if you're an engineer, that's a topic for advanced study, learning to control the level and phase at exact frequencies. If you're an animator, it's much simpler: you pick a desired easing curve, and use its velocity to make a normalized filter. You end up with the exact same step response, turning the easing curve into a perpetually interruptible animation.

$$ \class{blue}{ease(f)} = (\frac{1}{2} - \frac{1}{2} \cdot \cos πf) \cdot (1 + 20f \cdot (1 - f)^\frac{5}{2}) $$

Which leads to the last trick in this chapter: removing lag on a real-time filtered signal. There's always an inherent delay in any filter, where signals are shifted by roughly half the window length. We can't get rid of it, only reduce it. We have to change the filter to prefer certain frequencies over others, making it resonate to the kind of signal we expect. We use an easing curve that overshoots, and preferably a short one. This is just one I made up.

The velocity—here scaled down—now has a positive and negative part. As neither part is normalized by itself, the filter will first amplify any signal it encounters. The second part then compensates by pulling the level back down.

The result is that the filter actually tries to predict the signal, which you can imagine is a useful thing to do. At certain points, the lag is close to 0, when the resonance frequency matches and slides into phase. When applied to animation, resonant filters can create jelly-like motions. When applied to electronic music at about 220 Hz, you get Acid House.

Let's put it all together, just for fun. Here we have some particles being simulated with Verlet integration. Each particle experiences three forces. Radial containment pushes them to within a certain distance of the target. Friction slows them down, opposing the direction of motion. A constantly rotating force, different for each particle, keeps them from bunching up. The target follows the mouse, with double exponential easing.

Friction links acceleration to velocity. Containment links acceleration to position. And integration links them back the other way. These circular dependencies are not a problem for a good physics engine. Note that the particles do not interact, they just happen to follow similar rules.
Tip: Move the mouse and hold Shift to see variable frame rate physics in action.

If we add up the three forces and trace out curves again, we can watch the particles—and their derivatives—speed through time. Just like you are doing right now, in your chair. As velocity and acceleration only update in steps, their curves will only be smooth if the physics clock and rendering clock are synced.

By manipulating time, we've managed to eliminate frame rate issues altogether, even make it work to our advantage. We've discovered more accurate physics engines, so we don't have to waste time simulating tiny steps. We've also created interruptible animations and turned them into filters. We can choose their easing curves and use feedback systems to remove the need for any manual interruptions altogether.

Here, linear time-invariant systems are very useful building blocks: they are simple to implement, but eminently customizable. Both IIR and FIR filters are simple in their basic form. We can also combine feedback systems with other physical or unphysical forces: we can move the target any way we like, perhaps superimposing variation onto existing curves. If we broaden our horizons a bit, we can find applications outside of animation: data analysis, audio manipulation, image processing, and much, much more.

Of course, there are plenty of non-linear and/or non-time-invariant systems too, too many to cover. When dealing with animation though, we'll prefer systems based on physics. They're just the trick to turn a bunch of artificial data into something that feels slick and natural. That said, physics itself is sometimes non-linear: fluids like water, smoke or fire are perfect examples. Solving those particular boondoggles requires the kind of calculus that frightens most adults and large children, so we won't go into that here. It's the same thing though: you simulate it finitely with a couple of clever tricks and the awesome power of raw number crunching.

Continued in part two.

↧

Animate Your Way to Glory - Part II

September 13, 2013, 12:00 am

≫ Next: On TermKit

≪ Previous: Animate Your Way to Glory

Math and Physics in Motion

Doctor… Who?

All the models we've dealt with until now are programmatic. If we wish to run a sequence of animations, we have to schedule calls appropriately, perhaps using a queue. The proper tool for this job is a timeline. At first glance, it's just a series of keyframes on tracks: a set of coordinates over time, one for each property you're animating, with some easing in between. But it's hard to offer direct controls to a director or animator, without creating uneven or jarring motion, at least in 2D.

We must stop treating space and time as separate things, and chart a course in both at the same time.

This a classic keyframe timeline: a set of frames, with values defined along the way. It could be a vertical or horizontal motion, the opacity of a shape, the volume of a sound, etc. Any one thing we want to animate precisely over a long time.

This is one second of a 60 fps animation and there's a keyframe every 10 frames. We can interpolate between the points with a cosine ease. But there's already a mistake.

By expressing animations as frames, we can only have animations that are multiples of the frame length. In this case, that's 16.6 ms. If we want to space keyframes at 125ms, we can't, because that's 7.5 frames. The closest we can manage is alternating 7 and 8 frame sections.

Just like with variable frame rates, we need to set keyframes in absolute time, not numbered frames. We use a global clock to produce arbitrary in-between frames. If we change our mind and wish to speed up or slow down part of our timeline, there's no snap-to-frame to get in our way. Note that Adobe Flash does not do this: you define your frame rate up front and are stuck with it.

There is a catch though, easy to overlook: by the time we notice the first animation has ended, the second one has already started. We need to account for this slack, and make sure we start partway in, not from the beginning. Otherwise, this error accumulates with every keyframe, leading to noticeable lag.

This is also important for triggered actions like sound. Suppose there is a performance glitch right before it plays, and we lose 7 frames. Rare, but not impossible. If we don't account for slack, we'd have 7.5 frames of permanent lag on the audio, 125ms. More than enough to disrupt lip sync. Instead we should skip ahead to make up for it. To avoid an audible pop when skipping audio, we can apply a tiny fade in: a microramp.

With real-time dependencies like audio, it's better to be safe than sorry though. As the audio subsystem is generally independent, we can avoid this issue by pre-queuing all the sound with a delay. Here we begin playback 100ms earlier, but start each sound with an implied 100ms silence, minus the slack. Now, no audio will be lost in most situations. This too is animation: micromanaging time.

Let's focus our attention back on this easing curve. By treating it as a sequence of individual animations, we've created a smooth path. But it's not a very ideal path: it stops at every keyframe and then starts moving again, creating a curve with stairs. This is more obvious if we plot the velocity.

We need to replace it with a spline, a designable curve. There's too many to name, but we'll stick to a common one: Catmull-Rom splines. It's entirely based on one particular curve. Looks suspiciously like an impulse response, doesn't it?

$$ catmullRom(t) = \frac{1}{2} \cdot \left\{ \begin{array}{rcll} \class{purple}{p_1(t)} & = & t^3 + 5 t^2 + 8 t + 4 & \mbox{if } -2 \leq t \lt -1 \\ \class{royal}{p_2(t)} & = & -3 t^3 - 5 t^2 + 2 & \mbox{if } -1 \leq t \lt 0 \\ \class{purple}{p_3(t)} & = & 3 t^3 - 5 t^2 + 2 & \mbox{if } 0 \leq t \lt 1 \\ \class{royal}{p_4(t)} & = & -t^3 + 5 t^2 - 8 t + 4 & \mbox{if } 1 \leq t \lt 2 \\ \end{array} \right. $$

But actually, it's not one curve, it's 4 separate cubic curves glued together into a symmetric pulse. They're designed so their velocities meet up at the transition, thus creating a single smooth path. But if you look closely, you can see that the velocity (scaled) has two minor kinks in it, one on each side.

There are two other important features. The first is that the curve goes through 0 at all the keyframes except the central one. There, its value is 1. The keyframes are called the knots of the spline.

The other is that its slope is 0 at all the knots except the ones adjacent to the peak. There, it's $ \frac{1}{2} $ and $ -\frac{1}{2} $ respectively. If we trace the slopes out to the center, we go half as high as the peak, to 0.5.

That means if we scale down this curve as a whole, very few things we're interested in actually change. All the horizontal slopes remain horizontal. All the knots at 0 remain at 0. Only the peak shrinks, and the slopes at the adjacent knots go down.

$$ \class{blue}{p_i} \cdot catmullRom(t-i) $$

We can literally treat the curve as the impulse response of a filter, and the knots as a series of impulses. A filter outputs a copy of its impulse response for every impulse it encounters. As this is all theoretical, we don't care about filter lag.

$$ \class{blue}{spline(t)} = \sum\limits_{i=0}^n \class{blue}{p_i} \cdot catmullRom(t-i) $$

If we now add up all the curves, we get the Catmull-Rom spline. Despite the intricate interactions of the curves between the knots, the result is very predictable. The spline goes through every keyframe, because the values at the knots are all 0 except for the peak itself.

What's more, when we move a single value up and down, only two other things change: the two slopes at the adjacent knots. The slope at the knot itself is still constant. This means we can control the initial and final slope of the spline just by adding an extra knot before and after: it won't affect anything else.

See, the slope at a knot is actually just the central difference around that point. This is where the factor of $ \frac{1}{2} $ for the adjacent slopes came from earlier, and why their signs were opposite: it's a difference that spans two keyframes, so we divide by 2. This is the rule that determines how Catmull-Rom splines curve.

There's just one problem: all of this only works if the keyframes are equally spaced. If we change the spacing, our base curve is no longer smooth: there is a kink at the adjacent keyframes. This might not look like much, but it would be noticeable.

There's two ways to solve this. One is to try and come up with a unique curve for every knot. This curve has to be smooth and hit all the right values and slopes. This is the hard way, and can result in odd changes of curvature if done badly, like here.

But actually, you already know the other solution. By distorting the Catmull-Rom spline to fit our keyframes, it's like we've rendered it with a variable speed clock. But one that doesn't change smoothly. This is why the curves have developed kinks out of nowhere. If we can smooth out the passage of time, then we'll stretch the spline smoothly between the keyframes.

We can just create another Catmull-Rom spline to do so. Horizontally, we put equally spaced knots. This dimension has no real meaning: it's just 'spline time' now, independent of real time.

We move the knots vertically to the keyframe time and make a spline. In this case, I tweaked the start and end to be a diagonal rather than a horizontal slope. This curve hits all the keyframes at the right time and transitions smoothly between them. It's a variable clock that goes from constant spline time to variable real time.

To actually calculate the animation, we need to go the other way and invert the relationship: from variable real time to constant spline time. This can be done a few ways, but the easiest is to use a binary search, as the time curve always rises: it's like finding a value in an ordered list. This tells us how fast to scrub through the spline.

With this, we can warp the Catmull-Rom spline to hit all the keyframes at the right times. We'll still need to manually edit the keyframes to get a perfectly ripple-free path, but now we can move them anywhere, anytime we like.

What we just did was to chart a path through 1D space and 1D time, by combining two Catmull-Rom splines. Add time travel, and this is entirely equivalent to charting a random 2D spline through 2D space. To create such an animation, you create two parallel tracks, one for X and one for Y, with identical timings. By scrubbing through spline time, you move in both X and Y, and hence along the curve. However, doing so precisely turns out to be complicated.

In the 1D case, the distance between two keyframes is trivial: going from 0 to 1 means you moved 1 unit. In the 2D case, that's no longer true: the distance travelled depends on both X and Y simultaneously. What's worse is, splines generate uneven paths. If we divide them equally in spline time, we get unequal steps in real time. The apple slows down and then shoots off.

It might seem cool that the spline naturally has a tension in its motion, but it will only get in the way. If we move just a single X coordinate of a single knot, the entire path shifts, and the distance between the steps changes considerably. The easing of the Y coordinate needs to compensate for this. We can't maintain a controlled velocity this way: X and Y are dependent and have to be animated together.

We can resolve this by doing for distance what we did for time: we have to make a map from spline time to real distance. We can step along the spline in small steps and measure the distance one line segment at a time. When we integrate, adding up the pieces, we get a curve that maps spline time onto total distance along the curve.

Again, we can invert the relationship to get a map from distance to spline time.

We can use it to divide the spline into segments of equal length and move an object along the path with a constant speed. This works for any spline, not just Catmull-Rom. We can always turn a curve into a neat set of equally spaced points of our choosing.

The distance map gives us a natural parametrization: a way to move along the curve by the arc length itself. This effectively flattens out the curve into a straight line, and we can treat it like a 1D animation. We can apply straightforward easing again, because distances are once again preserved.

To animate, we just define an easing curve for distance over time. If we want to move along the path at a constant speed, we line the keyframes up along a diagonal.

However, the knots don't have any special meaning anymore. When we move, we pass through them at just the same speed as any other point. That means we can control velocity completely independent of the path itself, using all the tricks from earlier. We can also apply a direct easing curve along the path, for example cubic easing.

To run the animation, we go the other way. The easing curve tells us the desired distance along the path at any moment in time. We have to use the inverse distance map to convert this to spline time for the point in question.

Then we can use the spline time to look up the point on the Catmull-Rom spline. The easing curve makes us scrub smoothly along the distance map. This in turn makes us move smoothly on the spline—albeit with a bit of whiplash.

While that might seem like a lot of work, the good news is, it works in 3D too. We can find a distance map based on 3D distance, and now have three simultaneous Catmull-Rom splines for the X, Y and Z coordinates.

In a way, path-based animation is cheating: it acts like there's an infinitely strong force keeping the object on the track, only we don't need physics to make it happen. If we did add other forces however, we'd get a miniature WipeOut-style racing game. This principle is applied in the demo at the top of this page: the velocity along the track is constant, but the camera and its target are being exponentially eased, creating lag and swings in corners, giving it a natural feel.

Timelines and splines are both sides of the same coin: using piecewise sections to create smoothness. The combination of both gives us path-based animation, pretty close to being the holy grail of controlled animation. We can fit this neatly into any timeline model—provided we don't lose track of all the tiny extra bits of time—with any easing mechanism we want. The track and the motion on it are completely decoupled.

Aside from Catmull-Rom, there's the non-rational splines, the popular Bezier curves as well as other recursive methods. As most of these allow you to control the slope directly, you get direct velocity control on any path in a timeline.

Path-based animations don't have to be restricted to positions either. You can animate RGB colors as XYZ triplets the same way. Or you could animate the parameters of a procedural generator, or a physics simulation. Or animate the volume levels of music in response to gameplay. Or move your robot. Timelines are excellent tools to manage change, but only if you can control the speed precisely at the same time.

Which leaves us only one thing: rotation.

Blowing up the Death Star

How difficult can a few angles be? Very. In 2D, they don't cooperate with our linear models. Even just turning to face a particular direction requires care. In 3D, things get properly messed up. Rotations will turn the wrong way, wobble in place and generally not behave. If you're trying to animate a free-moving camera in 3D, fixing this is pretty important, unless you're making Motion Sickness Tycoon or Cloverfield Returns.

Defeating this particular Goliath will require a careful approach. We'll launch our squadrons of X, Y and Z-wings, use the Force, and attack the weak spot for maximum damage. It better not be a trap.

$$ \phi = 0° $$

What's wrong with angles? Let's ask our trusty friend, the apple. Sorry, I got hungry.

$$ \begin{array}{rcl} \class{blue}{\phi} & = & 2.3 \cdot τ \\ & = & 828° \end{array} $$

Well, they wrap around. Suppose we have an object that's been rotated a couple of times, for example as part of an interactive display. It completed 2.3 turns ($ τ = 2π $) around the circle. For now we'll use degrees, but eventually we'll switch to radians for the heavier stuff.

$$ \class{purple}{\phi_T} = 0° $$

If we animate the apple to a target angle $ \class{purple}{\phi_T} $ at 0°, it will spin all the way back. Our animation system doesn't know that it could stop earlier at 720° or 360°.

$$ \class{purple}{\phi_T} = 315° \\ $$

To fix this, we can't simply reduce all angles to the interval 0…360. If we animate from 0 to 315°, we still go the long way around rather than just 45° in the other direction.

$$ \begin{array}{rcl} \class{blue}{\phi} &=& 315° \\ \class{purple}{\phi_T} &=& 90° \\ \end{array} $$

$$ \begin{array}{rcl} δ &=& \frac{\class{purple}{\phi_T} - \class{blue}{\phi}}{360°} \\[8pt] \class{green}{Δ\phi} &=& 360° \cdot (δ - ├\,δ\,┤) \\ \end{array} $$

We need to reduce the difference in angle $ \class{purple}{\phi_T} - \class{blue}{\phi} $ to less than 180° in either direction. This is a circular difference, easiest when counting whole turns $ δ $, so we can round off to $├\,δ\,┤$. The difference, e.g. $ 3.3 - 3 = 0.3 $ or $ 1.6 - 2 = -0.4 $ is never more than half a turn. If we now set the target to 90°, it tells us to animate by $ \class{green}{+135°} $, that is, the short way around.

$$ \class{purple}{\phi_T} = 90° \\ \class{blue}{\phi} = 450° \\ $$

Our angles are now still continuous, going beyond 360° in either direction, but we never rotate more than 180° at a time unless we actually want to. We can apply this correction whenever we interpolate between two angles, and always end up at an equivalent angle.

Here I use double exponential easing to chase a rapidly changing angle. The once filtered angle jerks whenever it gets lapped, as it suddenly needs to change direction. The twice filtered angle moves smoothly however.

What about 3D? If we're restricting ourselves to a single axis of rotation, nothing really changes. We still control the angle the same way.

But orientation in 3D is a complicated thing. The easiest way to express it is with a 3×3 matrix: this is a set of 3 vectors in 3D. They define a frame of reference in space, a basis: right/left, up/down and forward/back. When we rotate around the vertical axis $ \vec y $, we rotate $ \vec x $ and $ \vec z $ together.

For arbitrary orientations, $ \vec x $, $ \vec y $ and $ \vec z $ can turn in any direction, but always maintain a perfect 90° angle between themselves.

$$ \begin{bmatrix} \class{blue}{a} & \class{green}{d} & \class{orangered}{g} \\ \class{blue}{b} & \class{green}{e} & \class{orangered}{h} \\ \class{blue}{c} & \class{green}{f} & \class{orangered}{i} \end{bmatrix} $$

Each vector is a set of $ (x, y, z) $ coordinates. That means we can write down the matrix as a set of 3 triples of coordinates, one column for each vector. At first it would seem we need 9 numbers to describe a 3D rotation. We can apply this rotation matrix to transform any point $ (x, y, z) $ by adding up proportional amounts of $ \vec x $, $ \vec y $ and $ \vec z $. This is linear algebra.

$$ \begin{bmatrix} \class{blue strike}{a} & \class{green strike}{d} & \class{orangered}{g} \\ \class{blue strike}{b} & \class{green strike}{e} & \class{orangered}{h} \\ \class{blue strike}{c} & \class{green strike}{f} & \class{orangered}{i} \end{bmatrix} $$

But there's tons of redundancy here. Because the 3 vectors are perpendicular, $ \vec z $ can only be in one of two places. The difference between the two is called a left handed or right handed coordinate system: for thumb, index and middle finger, with your hand shaped like a gun and the middle finger sticking out.

$$ \begin{bmatrix} \class{blue}{a} & \class{green}{d} & \class{red gone}{g} \\ \class{blue}{b} & \class{green}{e} & \class{red gone}{h} \\ \class{blue}{c} & \class{green}{f} & \class{red gone}{i} \end{bmatrix} \\[32pt] \class{blue}{\vec x} × \class{green}{\vec y} = \class{orangered}{\vec z} $$

So long as we agree on a common style of coordinate system, for example right-handed, we don't need to track $ \vec z $. We can recover it from $ \vec x $ and $ \vec y $ using something called the vector cross product. The vector that comes out will always be perpendicular to the two we pass in, decided by a left- or right-hand rule. This is by the way how you can aim a camera in 3D: all you need is a target, and an up vector.

We're down to 6 numbers. But there's more. A rotation preserves length, so the basis must always stay the same size. All the vectors must have length 1—be normalized—and hence move on the surface of a sphere.

$$ (\class{purple}{\phi}, \class{slate}{\theta}) $$

Instead of 3 coordinates, we can remember $ \vec x $ as two angles: longitude $ \phi $ and latitude $ \theta $. First we rotate around the Y axis, then around the rotated Z axis. Did we uniquely determine $ \vec y $ as well?

$$ (\class{purple}{\phi}, \class{slate}{\theta}, \class{cyan}{\gamma}) $$

No, there is a third degree of freedom we haven't been using so far. In order to account for all the places where $ \vec y $ can be, we need to allow rotation around $ \vec x $, by another angle $ \gamma $. Now we can describe any orientation in 3D using just three numbers, the so called Euler angles.

This is a YZX gyroscope, after the order of rotations used. We can build one in real life by using concentric rings connected by axles. Make one large enough to put a chair in the middle, and you've got an amusement ride—or something to train pilots with. When we rotate the object inside, we rotate the rings, decomposing the rotation into 3 separate perpendicular ones.

If we animate the individual angles smoothly, like here, we seem to get a smooth orientation. What's the problem? Well, we need to study the gyroscope a bit more.

Let's go back to neutral, setting all angles to 0. You can see the YZX nature of the gyroscope, if you follow the axles from the outside in.

We rotate the first ring by 90° and look at the axles again. Now they go YXZ. We've swapped the last two.

If we rotate the second ring by 90°, the axles change again. They've moved to YXY. This means changing the order or nature of the axles doesn't change the gyroscope, it just rotates all or part of it. That is, unless you make the very useless YYY gyroscope. All functional gyroscopes are identical. Whatever we discover for one applies to all.

This configuration is special however. The axles for the first and third rings are aligned. This is called gimbal lock, though no ring actually locks. If we apply an equal-but-opposite rotation to both, the apple doesn't move. From any of these configurations, we can only rotate two ways, not three. It shows Euler angles do not divide rotations equally.

If we now rotate the inner ring by 90°, all rings have been changed 90° from their initial position. Same for the apple: its final orientation happens to be rotated -90° around the Z axis.

Which means if we rotate the entire gyroscope by 90° around Z, the apple returns to its original orientation. This is what we'd like to see if we simultaneously rotated the three rings of the gyroscope back to zero.

That's not the case however. We try to hold the apple in place, by rotating back the gyroscope as we rotate back all three rings at the same time. The rotations don't cancel out cleanly and the apple wobbles. We'll need to create an angle map, similar to the distance map for splines before. Only now we need to equalize three numbers at the same time.

Another telling sign is when we rotate all rings by 180°: the start and end orientation is the same. Yet the apple performs a complicated pirouette in between. Just like with circular easing, we'll need a way to identify equivalent orientations and rotate to the nearest one.

To see why this is happening, we can rotate the apple around a diagonal axis. You can do this with a real gyroscope just by turning the object in the middle. The three rings—and hence the Euler angles—undergo a complicated dance. The two outer rings wobble back and forth rather than completing a full turn. Charting a straight course through rotation space is not obvious.

In summary: trying to decompose rotations is messy and leads to gimbal lock. We're going to build a different model altogether, using what we just learnt.

First, we make an arbitrary rotation matrix by doing a random X rotation followed by a (local) Y rotation and a (local) Z rotation. This is like using an XYZ gyroscope.

We can apply the same rotations again, acting like a nested XYZXYZ gyroscope. Because the gyroscope is made of two equal parts in series, we've rotated twice as far.

Three points uniquely define a circle. So we can trace an arc for each of the basis vectors. These arcs are not part of the same circle, but they do lie parallel to each other.

$$ \class{cyan}{\vec a} = \class{blue}{\vec x_1} - \class{blue}{\vec x_0} \\ \class{cyan}{\vec b} = \class{blue}{\vec x_2} - \class{blue}{\vec x_1} \\ \class{purple}{\vec c} = \class{cyan}{\vec a} × \class{cyan}{\vec b} \\ \class{slate}{\phi} = \arcsin \frac{|\class{purple}{\vec c}|}{|\class{cyan}{\vec a}| \cdot |\class{cyan}{\vec b}|} $$

Now we can find the common axis of rotation from any of these arcs: we take the cross product of the forward differences. If we divide by the lengths of the differences, the cross product's length tells us about the angle of rotation. We apply an arcsine to get an angle in radians. This is the axis-angle representation of rotations. Note that the axis is oriented to distinguish clockwise from counter-clockwise, here using the right hand rule.

We can do this for any rotation matrix, for any set of Euler angles. It tells us we can rotate from neutral to any orientation by doing a single rotation around a specific axis. Now we have three better numbers to describe orientations: $ \class{purple}{(x, y, z)} $. They don't privilege any particular rotation axis, as both their direction and length can change equally in all 3 dimensions. We can pre-apply the arcsine: we make the vector's length directly equal rotation angle, linearizing it.

We can also identify equivalent angles: if we rotate more than 180° one way, that's equivalent to rotating less than 180° the other way. The axis can flip around when its length reaches $ π $ radians (180°) without any disruption. We can restrict axis-angle to a ball of radius $ π $.

If we interpolate linearly to a different $ \class{purple}{(x, y, z)} $, we get a smooth animation, but there's some wobble. It also goes the long way through the sphere. There's a much shorter way.

We can flip $ \class{purple}{(x, y, z)} $ and then interpolate back, to get a more direct rotation. The wobble remains though: there's a subtle change in direction at the start and end. Hence, axis-angle cannot be used directly to rotate between any two orientations in a single smooth motion.

$$ \begin{bmatrix} \class{blue}{a_1} & \class{green}{d_1} & \class{orangered}{g_1} \\ \class{blue}{b_1} & \class{green}{e_1} & \class{orangered}{h_1} \\ \class{blue}{c_1} & \class{green}{f_1} & \class{orangered}{i_1} \end{bmatrix} $$

$$ \begin{bmatrix} \class{slate}{a_2} & \class{cyan}{d_2} & \class{purple}{g_2} \\ \class{slate}{b_2} & \class{cyan}{e_2} & \class{purple}{h_2} \\ \class{slate}{c_2} & \class{cyan}{f_2} & \class{purple}{i_2} \end{bmatrix} $$

If we have two random rotation matrices $ [\class{blue}{\vec x_1} \,\,\, \class{green}{\vec y_1} \,\,\, \class{orangered}{\vec z_1}] $ and $ [\class{slate}{\vec x_2} \,\,\, \class{cyan}{\vec y_2} \,\,\, \class{purple}{\vec z_2}] $, how can we find the axis-angle rotation that turns one directly onto the other?

We have to invert the first matrix to turn the other way. We could convert it to axis-angle and then reverse the angle. But it turns out that's the same as swapping rows and columns. The latter is obviously a lot less work, but it only works because the three vectors are perpendicular and have length 1. We end up with a matrix that rotates the same amount around the same axis, but in the other direction. For other kinds of matrices, inversion is trickier.

$$ \begin{bmatrix} \class{blue}{a_1} & \class{blue}{b_1} & \class{blue}{c_1} \\ \class{green}{d_1}& \class{green}{e_1} &\class{green}{f_1} \\ \class{orangered}{g_1} & \class{orangered}{h_1} & \class{orangered}{i_1} \end{bmatrix} \cdot \begin{bmatrix} \class{blue}{a_1} & \class{green}{d_1} & \class{orangered}{g_1} \\ \class{blue}{b_1} & \class{green}{e_1} & \class{orangered}{h_1} \\ \class{blue}{c_1} & \class{green}{f_1} & \class{orangered}{i_1} \end{bmatrix} $$

$$ \begin{bmatrix} \class{blue}{a_1} & \class{blue}{b_1} & \class{blue}{c_1} \\ \class{green}{d_1}& \class{green}{e_1} &\class{green}{f_1} \\ \class{orangered}{g_1} & \class{orangered}{h_1} & \class{orangered}{i_1} \end{bmatrix} \cdot \begin{bmatrix} \class{slate}{a_2} & \class{cyan}{d_2} & \class{purple}{g_2} \\ \class{slate}{b_2} & \class{cyan}{e_2} & \class{purple}{h_2} \\ \class{slate}{c_2} & \class{cyan}{f_2} & \class{purple}{i_2} \end{bmatrix} $$

To apply the inverted rotation, we do a matrix-matrix multiplication, which is a fancy way of saying we use it to rotate the other matrix's basis vectors $ [\class{slate}{\vec x_2} \,\,\, \class{cyan}{\vec y_2} \,\,\, \class{purple}{\vec z_2}] $ individually. When applied to the first basis $ [\class{blue}{\vec x_1} \,\,\, \class{green}{\vec y_1} \,\,\, \class{orangered}{\vec z_1}] $, it rotates back to neutral as expected, aligned with the XYZ axes.

We can now convert the relative rotation matrix into axis-angle again. This is the rotation straight from A to B, without any wobble or variable speed. This method is quite involved, and hence is still just a stepping stone towards rotational bliss.

We go back to our axis-angle sphere and apply this rotation, while measuring the total axis-angle every step along the way. We can see the cause of the earlier wobble: when moving straight through rotation space, we need to follow a curved arc rather than a straight line. As both rotations are the same length, this arc follows the surface of the sphere.

To get a better feel for how this works, let's move the other end around. We change it to various rotations of $ \frac{π}{2} $ radians (90°). These are all the rotations on a sphere of radius $ \frac{π}{2} $. Both the arc and axis of rotation change in mysterious ways. The arc snaps back and forth, crossing through the edge of the sphere if that's shorter.

What this really means is that angle space itself is curved. Think of the surface of the earth: if we keep going long enough in any particular direction, we always get back to where we started. As a consequence, you can't flatten an orange peel without tearing it, and you can't make a flat map of the Earth without distorting the shapes and area unequally. Yet we can view such a curved 2D space easily in 3D: it's just the surface of a sphere.

The same applies here, except not just on the surface, but also inside it. Each curved arc is actually straight as far as rotation is concerned, and each straight interpolation is actually curved. The inside of this ball is curved 3D space.

If we want to see curved 3D space without distorting it, we need to view it in four dimensions. This ball is the hypersurface of a 4D hypersphere. So 3D rotation is four dimensional. WTF?

Math is boring. Let's blow up the Death Star.

The Emperor has made a critical error and the time for our attack has come. The data brought to us by the Bothan spies pinpoints the exact location of the Emperor's new battle station. We also know that the weapon systems of this Death Star are not yet operational. Many Bothans died to bring us this information. Admiral Ackbar, please.

Although the weapon systems on this Death Star are not yet operational, the Death Star does have a strong defense mechanism. It is protected by an energy shield, which is generated from the nearby forest Moon of Endor. Once the shield is down, our cruisers will create a perimeter, while the fighters fly into the superstructure and attempt to knock out the main reactor.

Sir, I'm getting a report from Endor! A pack of rabid teddy bears has attacked the generator, tearing the equipment to shreds. The shield is failing…

The Death Star is completely vulnerable! Report to your ships, we launch immediately. We'll relay your orders on the way.

Lieutenant Hamilton, show me the interior of the superstructure.
(That's your cue.)

*Mic screech*
Red Wing, these are your orders. Of the 6 access points to the interior, you will fly your X-wings through the east portal, closest to the superlaser.

There are large passageways leading directly to the central chamber. As these shafts are heavily guarded by fighters, a direct assault is impossible. We will need to avoid patrols by navigating the dense tunnel network that makes up the interior.

The Death Star's inner core is fortified, and all access is restricted. However, one of our operatives has informed us of a large, unsecured ventilation shaft, still under construction. This is our best chance to get into the core and destroy it. You must reach this target at all costs.
Tip: Click and drag to see things from a different angle.

There are large tunnels circling just underneath the surface. You will fly your fighters into: (Choose one)

We will also send a detachment of Y-wings from the north pole. These heavy bombers will rendezvous with the X-wings, taking the long way around, away from the defensive perimeter.

Lieutenant, I have another task for you. Now that the Death Star's shields have conveniently failed, we will launch a probe ahead of our arrival, gathering detailed sensor data of the entire structure.

It will approach directly from the front side. It must pass through each of the large tunnels circling the Death Star to ensure full sensor coverage.

The probe's energy signal is shielded, but it will not escape detection for long. To minimize our chances of detection, we should complete the survey of the entire Death Star without any overlap.

Survey all areas of the Death Star, without entering any tunnel twice.

The probe is approaching the Death Star…

Scan of the interior progressing. Plotting data now.

Hamilton's head hurts. Who would design such a crazy, tangled thing? Yet as he studies the structure, he notices a remarkable symmetry. Grouped by color, the tunnels form a swirling vortex around each central axis. Each vortex is surrounded by a great circle. He labels the three groups $ i $, $ j $ and $ k $, as is the convention in this era.

Yet mysteriously, tunnels always meet at a 90° angle, everywhere: on the central axes, on the circles, even anywhere in between. The colors also maintain their relative orientation at each intersection, including the polarity (positive or negative). Amazed, Hamilton starts scribbling down notes. "Cubic grid, twisted through itself? $ i → j → k $?".

In fact, he's so mesmerized by the display, he's completely lost track of what's going on. As the hustle and bustle of the starship bridge slowly creeps back into focus, he looks up and–…

IT'S A TRAP!

Not to despair. Some lightning gets thrown around, the Emperor is killed, a man finds redemption in death, and the Death Star is destroyed.

That night, after many hours of celebration, the young Lieutenant falls asleep contentedly, and starts dreaming of that maze again. Maybe it was just a flash of inspiration, maybe it was the Force—or maybe the interesting neurochemical effects of fermented Endorian moonberry juice—but a long time ago, in a galaxy far, far away, William Rowan Hamilton figured out quaternions.

More precisely, it was in 1843 in Dublin, Ireland. He was so struck by it, he immediately carved it into the nearest bridge—true story. It shouldn't surprise you that you've been doing quaternion calculations all along: those edges weren't color-coded just to look pretty. They consistently denoted the multiplication by a +X/-X, +Y/-Y or +Z/-Z quaternion, representing a particular rotation around that axis.

A key feature is how the colors wrap around the great circles. They always maintain the same relative orientation at every intersection, but the entire arrangement rotates from one place to the next. For example +Y: it goes up at the core, but circles around the equator horizontally. You also saw what the inside looked like, omitted here for sanity.

But we're missing something: a 6th quaternion at every 'pole'. This space continues outward, we've just been ignoring that part of it.

This suggests there is a second set of 'poles', at twice the radius. We can travel to and from them by multiplying with a quaternion. In fact that's completely true, but with one catch: all the orange points are actually all one and the same point. Huh?

Remember, we're looking at curved space, a hypersphere. To make sense of it, we need to first look at the 2D case.

If the disc represents a curved plane that was projected down to 2D, then in its undistorted form, it's actually a sphere. All the points on the disc's perimeter are actually the same: here they're the north pole, and the disc's center is the south pole.

In curved space, it works similar. We can't visualize this, because this is happening in every direction all at the same time. We experienced the result of it while navigating the Death Star. What we didn't see was that the entire sphere of radius 2π—in axis-angle terms—is all just one and the same point. We never bothered to go beyond radius π before: rotations up to 180° in either direction.

The important thing is to realize that the center of our diagram is not the center of the hypersphere, rather it's just another pole. In order to fit a hypersphere into 3D correctly, we'd somehow have to shrink the entire sphere of radius 2π to a point, to create a new pole, but without passing through the sphere of radius π. This is impossible, you need an extra dimension to make it work.

But why are 3D rotations and quaternions connected? Why does axis angle map so cleanly to half of a hypersphere in quaternion space? And what does a quaternion actually look like? Well. What other kind of mathematical thing likes to turn? When you multiply it by another one of its kind? Where the rotation angle depends on where both inputs are?

$$ \class{blue}{|z| = 1} $$

Complex numbers! Yay! If you're not familiar with them however, not to worry. We won't be needing all the complex numbers: we'll only use those that have length 1. In other words, all points on a circle of radius 1. Much simpler.

$$ z = \class{royal}{\frac{\sqrt{3}}{2}} + \class{blue}{\frac{1}{2} \cdot i} $$

Complex numbers are 2D vectors that lead a double life. Ordinarily, they are written as the sum of two parts. Their horizontal component is a real number, a multiple of $ \class{royal}{1} $. Their vertical component, is a so-called imaginary number, a multiple of $ \class{blue}{i} $, which is a square root of -1. Which supposedly does not exist. Lies.

$$ \class{orange}{z = 1∠30°} $$

It is often better to see them as a length and an angle. $ \class{royal}{1} $ becomes $ \class{royal}{1∠0°} $. The number $ \class{blue}{i} $ becomes $ \class{blue}{1∠90°} $. And $ -1 $ becomes $ 1∠180° $ or $ 1∠-180° $.

$$ \begin{array}{rl} \class{orange}{z} & = & 1∠30° \cdot 1∠90° \\ & = & 1∠120° \\ \end{array} $$

When we multiply two complex numbers, their lengths multiply, and their angles add up. As the lengths are always 1 in our case, we can ignore them. Here, we multiply $ \class{orange}{1∠30°} $ by $ \class{blue}{1∠90°} $ to turn it 90° counter-clockwise. By the same rule, $ \class{blue}{1∠90°} \cdot \class{blue}{1∠90°} = 1∠180° $, better known as $ \class{blue}{i}^2 = -1 $. Complex numbers like to turn, and this gives them interesting properties, explored elsewhere on this site.

Representing 2D rotation with complex numbers is trivial. We can directly map the rotation angle to the complex number's angle, and we can combine rotations by adding up the angles, positive or negative. The angles 0°, 90°, 180°, 270°, 360° become 1, $ \class{blue}{i} $, -1, $ -\class{blue}{i} $, 1. Of course, this adds nothing useful, at least in 2D.

We can expand the model to 3D though, where we have three perpendicular ways of turning. First we'll try to add a second degree of rotation. We add a new imaginary component $ \class{green}{j} $, representing Y rotation, while $ \class{blue}{i} $ is X rotation. Any position in this 3D space is now a quaternion, but we're still limiting them to only length 1, only interested in rotation. We'll be using the surface of what is, for now, a sphere.

But wait, this isn't right. According to this diagram, if we rotate 180° around either the X or Y axis, we end up in the same place—and hence the same orientation. Clearly that's not the case. Yet we based our quaternions on complex numbers, so both $ \class{blue}{i}^2 = -1 $ and $ \class{green}{j}^2 = -1 $.

We can satisfy this condition in a different way though. If we rotate an object by 360° around any axis, we always end up back where we started. So we can make this rule work if we agree that a 360° rotation equals a 180° quaternion.

That means each rotation is represented by a quaternion of half its angle. A rotation by 180° becomes a quaternion of 90°, that is $ \class{blue}{i} $ or $ \class{green}{j} $, and each rotation axis takes us to a unique place. As we still treat $ \class{royal}{1} $ as 0°, the quaternion $ \class{blue}{1∠180°} = \class{green}{1∠180°} = -1 $ now represents a rotation of 360° = 0° around any axis. So $ \class{royal}{1} $ and $ -1 $ are considered equivalent, as far as representing rotation goes.

Furthermore, $ \class{blue}{i} $ and $ \class{slate}{-i} $ are equivalent too, and so are $ \class{green}{j} $ and $ \class{cyan}{-j} $. Each represents rotating either +180° or -180° around the corresponding axis, which is the same thing. In fact, any half of this sphere is now equivalent to the other half, when you reflect it around the central point. This is why we were missing half of the hypersphere earlier: the 'outer half' is a mirror image of the 'interior'.

So what about in-between axes? Well, we could try rotating around $ \class{orange}{(1,1,0)} $ and $ \class{gold}{(1,-1,0)} $, which are the axes that lie ±45° rotated between X and Y. We'd end up tracing circles right between them: this is the only possibility where both rotations are perpendicular, yet maintain an equal distance to both the X and Y situation.

Unfortunately we're missing something important. We've only applied rotations from neutral, from $ 1 $. If we apply a 180° X and Y rotation in series, where do we end up? And what about Y followed by X? The diagram might suggest we'd end up at respectively $ \class{green}{j} $ and $ \class{blue}{i} $.

But this wouldn't make sense: if $ \class{blue}{i} \cdot \class{green}{j} = \class{green}{j} $, and $ \class{green}{j} \cdot \class{blue}{i} = \class{blue}{i} $, then both $ \class{blue}{i} $ and $ \class{green}{j} $ have to be equal to $ 1 $. There'd be no rotation at all. And if we say that $ \class{blue}{i} \cdot \class{blue}{j} = \class{blue}{j} \cdot \class{blue}{i} = -1 $, then the quaternions $ \class{blue}{i} $ and $ \class{green}{j} $ have the exact same effect. We'd only have one imaginary dimension, not two. Even in math, a difference that makes no difference is no difference.

Whether we want to or not, we have to add a third imaginary component, $ \class{orangered}{k} $ to make this click together. So $ \class{orangered}{k}^2 = - 1 $, but it's different from both $ \class{blue}{i} $ and $ \class{green}{j} $. As we've used up our 3 dimensions, we need to project down this new 4th, putting it at an angle between the others. Again, a $ \class{orangered}{k} $ quaternion represents rotation around the Z axis, with the angle divided by two.

We end up with two peculiar relationships: $ \class{blue}{i} \cdot \class{green}{j} = \class{orangered}{k} $ and $ \class{green}{j} \cdot \class{blue}{i} = \class{purple}{-k} $. The quaternion product is not the same when you reverse the factors. Just like an XY gyroscope turns differently than a YX gyroscope. But if we'd started with Z/X or Y/Z, we'd see the exact same thing.

Hence we can rotate and combine these rules to get $ \class{blue}{i} \cdot \class{green}{j} \cdot \class{orangered}{k} = -1 $. This is the $ i^2 = -1 $ of quaternions, the magic rule that links together 3 separate imaginary dimensions and a real one, creating a maze of twisty passages all alike. When you cycle the axes, it still works: $ \class{green}{j} \cdot \class{orangered}{k} \cdot \class{blue}{i} = -1 $ and $ \class{orangered}{k} \cdot \class{blue}{i} \cdot \class{green}{j} = -1 $, demonstrating that quaternions link together three imaginary axes into a cyclic whole.

$$ |\class{blue}{z}| = 1 \\ z = \class{royal}{\cos(\theta)} + \class{blue}{i \cdot \sin(\theta)} $$

So how do we actually use quaternions for rotation? It's quite easy, because they are literally complex numbers whose imaginary component has sprouted two extra dimensions. Compare with an ordinary complex number on the unit circle. Its length (1) is divided non-linearly over the horizontal and vertical component using the cosine and sine: this is trigonometry 101.

$$ |q| = 1 \\ q = \class{royal}{\cos(\frac{\theta}{2})} + (x \cdot \class{blue}{i} + y \cdot \class{green}{j} + z \cdot \class{orangered}{k}) \cdot \sin(\frac{\theta}{2}) $$

For a quaternion on the unit hypersphere, we only make two minor changes. We replace the single $ i $ with a vector $ (x\class{blue}{i}, y\class{green}{j}, z\class{red}{k}) $ where $(x,y,z)$ is the normalized axis of rotation. The cosine and sine stay, though we divide the rotation angle by two. We can visualize the 3 imaginary dimensions directly without projection, after squishing the real dimension to nothing. As the length of the imaginary vector shrinks, the real component grows to compensate, maintaining length 1 for the entire 4D quaternion.

$$ q_1 \cdot q_2 = (w_1 + x_1 \cdot \class{blue}{i} + y_1 \cdot \class{green}{j} + z_1 \cdot \class{orangered}{k}) \cdot (w_2 + x_2 \cdot \class{blue}{i} + y_2 \cdot \class{green}{j} + z_2 \cdot \class{orangered}{k}) \\[10pt] $$

$$ \class{blue}{i}^2 = -1 \,\,\,\,\,\,\, \class{green}{j}^2 = -1 \,\,\,\,\,\,\, \class{orangered}{k}^2 = -1 \\ \class{blue}{i} \cdot \class{green}{j} = \class{orangered}{k} \,\,\,\,\,\,\, \class{green}{j} \cdot \class{orangered}{k} = \class{blue}{i} \,\,\,\,\,\,\, \class{orangered}{k} \cdot \class{blue}{i} = \class{green}{j} \\ \class{green}{j} \cdot \class{blue}{i} = \class{purple}{-k} \,\,\,\,\,\,\, \class{orangered}{k} \cdot \class{green}{j} = \class{slate}{-i} \,\,\,\,\,\,\, \class{blue}{i} \cdot \class{orangered}{k} = \class{cyan}{-j} \\ \class{blue}{i} \cdot \class{green}{j} \cdot \class{orangered}{k} = -1 $$

We can apply the rules of quaternion arithmetic to multiply two quaternions. This is equivalent to performing the rotations they represent in series. Just like complex numbers, two length 1 quaternions make another length 1 quaternion. Of course, all the other quaternions have their uses too, but they're not as common. In a graphics context, you can pretty much forget they exist.

There's only one question left: how to smoothly move between two quaternions, that is, between two arbitrary orientations. With axis-angle, it was a very complicated procedure. With quaternions, it's super easy, because unit quaternions are shaped like a hypersphere. The 'angle map' is the hypersphere itself.

As it turns out though, a hyperspherical interpolation in 4D is exactly the same as a spherical one in 3D. So we really only need to understand the 3D case. We have a linear interpolation between two points on a sphere, and want to replace it with a spherical arc.

The line and the arc share the same plane: the one that contains both points and the center of the sphere. Any such plane cuts the sphere into two equal halves, along an equator-sized great circle. Hence the arc is just an inflated version of the line, with a circular bulge applied in the plane, following the sphere's radius along the way. But we also have to traverse the arc at constant speed: otherwise we'd end up creating an uneven spline-like curve again.

$$ \class{orange}{\theta} = \arccos(\class{green}{x_1} \cdot \class{red}{x_2} + \class{green}{y_1} \cdot \class{red}{y_2} + \class{green}{z_1} \cdot \class{red}{z_2}) $$

$$ \class{cyan}{slerp}(\class{green}{\vec v_1}, \class{red}{\vec v_2}, f) = \frac{\sin((1-f) \cdot \class{orange}{\theta})}{\sin \class{orange}{\theta}} \cdot \class{green}{\vec v_1} + \frac{\sin(f \cdot \class{orange}{\theta})}{\sin \class{orange}{\theta}} \cdot \class{red}{\vec v_2} $$

Luckily, we can apply a little trigonometry again. We can use the 3D (4D) vector dot product to find the angle between two unit vectors (quaternions), after applying an arccosine. Then we weigh the two vectors (quaternions) appropriately so they sum to length 1 and move linearly with the arc length. This is the slerp, the spherical linear interpolation. Working it out yourself can be tricky, but the result is elegant and independent of the number of dimensions.

With all that in place, we can track any orientation with just 4 numbers, and change them linearly and smoothly. Quaternions are like a better axis-angle representation, which simply does the right thing all the time. Of course, you could just look up the formulas and cargo-cult your way through this problem. But it's more fun when you know what's actually going on in there. Even if it's in 4D.

So that's quaternions, the magical rotation vectors. Every serious 3D engine supports them, and if you've played any sort of 3D game within the last 15 years, you were most likely controlling a quaternion camera. Whether it's head-tracking on an Oculus Rift, or the attitude control of a spacecraft (a real one), these problems become much simpler when we give up our silly three dimensional notions and accept 3D rotation as the four dimensional curly beast that it is.

Ultimately though, quaternions can be treated as just vectors with a special difference and interpolation operator. We can apply all our usual linear filtering tricks, and can create sophisticated motions on the hypersphere. Combine that with smooth path-based animation with controllable velocities, and you have everything you need to build carefully tracked shots from any angle.

Meaning from Motion

It's useful to see where we actually are with animation tools, especially on the web. Unfortunately, it doesn't look that great. For most, animation means calling an .animate() method or defining a transition in CSS with a generic easing curve, fire-and-forget style. Keyframe animation is restricted to clunky CSS Animations, where we only get a single cubic bezier curve to control motion. We can't bounce, can't use real friction, can't set velocities or apply forces. We can't blend animations or use feedback. By now you know how ridiculously limiting this is.

In an ideal world, we'd have a perfect animation framework with all the goodies, which runs natively and handles all the math for us while still giving direct control. Until then, consider inserting a little bit of custom animation magic from time to time. Writing a simple animation loop is easy, and offers you fine grained control. Upgrades can be added later when the need presents itself. Your audience might not notice directly, but you can be sure they will remark on how pleasant it is to use, when everything seems alive.

But buyer beware: we need to be thinking as much about what isn't changing, as what is. Just like we use grids and alignment to keep layouts tidy, so should we use animation consistently to bring a rhythm and voice to our interactions.

In what you just saw, little was left to chance. Color, size, proportion, direction, orientation, speed, timing… they're used consistently throughout, there to reinforce the connections that are expressed. I try to hash ideas into memorable qualia, while avoiding optical illusions or accidental resemblances. If it's not the same, it should look, act or speak differently. Even if it's just a slightly different shade of blue, or a 300ms difference in timing.

Though MathBox is a simple script-based animator (for now), it exposes some interesting knobs to play with and can handle arbitrary motion through custom expressions. It also supports slowable time and maintains per-slide clocks. If you map bullet time to a remote control, you can manipulate time mid-sentence: you don't need to follow your slides, your slides follow you. It feels ridiculously empowering when you're doing it. When properly applied, you can build up a huge amount of context and carry it along for extended periods of time at the forefront of people's attention.

Many of these slides are based on live systems that run in the background, advancing at their own pace. The entire diagram reflows, maintaining the mathematical laws and relations that are represented. Often, I use periodic or pseudo-random motion to animate the source data. While it may just seem like a cool trick at first, I think it's actually the main feature. It changes every slide from one example into many different ones. It shows them one after the other, streaming into your brain at 60 frames per second. It maximizes the use of visual bandwidth, yet cause and effect can still be read directly from any freeze frame.

Additionally, looping creates a continuous reinforcement of the depicted mechanisms. In my experience, our intuition can absorb math this way through mere exposure, slowly internalizing models of abstract spaces, as well as the relations and algorithms that operate within. Even if we don't notice right away, it anchors our understanding for later when it's finally expressed formally and verbally.

That's the theory anyway: put the murder weapon on the mantelpiece in the opening scene, and work your way towards revealing it. Only the weapon could be complex exponentials, and the mantelpiece the real number line. I'm not an education specialist or neuroscientist, though I did devour David Marr's seminal work on the human visual system. I just know that whenever I manage to pour a complicated concept into a concise and correct visual representation, it instantly begins to make more sense.

Bret Victor has talked about media as thinking tools, about needing a better way to examine and build dynamical systems, a new medium for thinking the unthinkable. In that context, MathBox is an attempt at viewing the unviewable. To borrow a term, it's qualiscopic. It's about creating graphical representations that obey certain laws, and in doing so, making abstract things tangible. It encourages you to smoothly explore the space of all possible diagrams, and find paths that are meaningful.

By carefully crafting living dioramas of space and time this way, we can honestly say: nothing will appear, change or disappear without explanation.

Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.

↧

On TermKit

May 17, 2011, 12:00 am

≫ Next: Wiki TL;DR – WikiLeaks Reader

≪ Previous: Animate Your Way to Glory - Part II

I've been administering Unix machines for many years now, and frankly, it kinda sucks. It makes me wonder, when sitting in front of a crisp, 2.3 million pixel display (i.e. a laptop) why I'm telling those pixels to draw me a computer terminal from the 80s.

Regular bash terminal

And yet, that's what us tech nerds do every day. The default Unix toolchain, marked in time by the 1970 epoch, operates in a world where data is either binary or text, and text is displayed in monospace chunks. The interaction is strictly limited to a linear flow of keystrokes, always directed at only one process. And that process is capable of communicating only in short little grunts of text, perhaps coalescing into a cutesy little ASCII art imitation of things that grown-ups call "dialogs", "progress bars", "tables" and "graphs".

The Unix philosophy talks about software as a toolset, about tiny programs that can be composed seamlessly. The principles are sound, and have indeed stood the test of time. But they were implemented in a time when computing resources were orders of magnitude smaller, and computer interaction was undiscovered country.

In the meantime, we've gotten a lot better at displaying information. We've also learned a lot of lessons through the web about data interchange, network transparency, API design, and more. We know better how small tweaks in an implementation can make a world of difference in usability.

And yet the world of Unix is rife with jargon, invisible processes, traps and legacy bits. Every new adept has to pass a constant trial by fire, of not destroying their system at every opportunity it gives them.

So while I agree that having a flexible toolbox is great, in my opinion, those pieces could be built a lot better. I don't want the computer equivalent of a screwdriver and a hammer, I want a tricorder and a laser saw. TermKit is my attempt at making these better tools and addresses a couple of major pain points.

I see TermKit as an extension of what Apple did with OS X, in particular the system tools like Disk Utility and Activity Monitor. Tech stuff doesn't have to look like it comes from the Matrix.

Rich Display

It's 2011, and monospace text just doesn't cut it anymore. In the default ANSI color palette, barely any of the possible color combinations are even readable. We can't display graphs, mathematical formulas, tables, etc. We can't use the principles of modern typography to lay out information in a readable, balanced way.

TermKit example

So instead, I opted for a front-end built in WebKit. Programs can display anything that a browser can, including HTML5 media. The output is built out of generic widgets (lists, tables, images, files, progress bars, etc.). The goal is to offer a rich enough set for the common data types of Unix, extensible with plug-ins. The back-end streams display output to the front-end, as a series of objects and commands.

I should stress that despite WebKit it is not my intent to make HTML the lingua franca of Unix. The front-end is merely implemented in it, as it makes it instantly accessible to anyone with HTML/CSS knowledge.

Pipes

Unix pipes are anonymous binary streams, and each process comes with at least three: Standard In, Standard Out and Standard Error. This corresponds to the typical Input > Processing > Output model, with an additional error channel. However, in actual usage, there are two very different scenarios.

One is the case of interactive usage: a human watches the program output (from Std Out) on a display, and types keystrokes to interact with it (into Std In). Another case is the data processing job: a program accepts a data stream in a particular format on Std In, and immediately outputs a related data stream on Std Out. These two can be mixed, in that a chain of piped commands can have a human at either end, though usually this implies non-interactive operation.

These two cases are shoehorned into the same pipes, but happen quite differently. Human input is spontaneous, sporadic and error prone. Data input is strictly formatted and continuous. Human output is ambiguous, spaced out and wordy. Data output is conservative and monolithic.

As a result, many Unix programs have to be careful about data. For example, many tools dynamically detect whether they are running in interactive mode, and adjust their output to be more human-friendly or computer-friendly. Other tools come with flags to request the input/output in specific formats.

This has lead to "somewhat parseable text" being the default interchange format of choice. This seems like an okay choice, until you start to factor in the biggest lesson learned on the web: there is no such thing as plain text. Text is messy. Text-based formats lie at the basis of every SQL injection, XSS exploit and encoding error. And it's in text-parsing code where you'll likely find buffer overflows.

What this means in practice is that in every context, there are some forbidden characters, either by convention or by spec. For example, no Unicode or spaces in filenames. In theory, it's perfectly fine, but in practice, there's at least one shell script on your system that would blow up if you tried. Despite the promise of text as the universal interchange format, we've been forced to impose tons of vague limits.

So how do we fix this? By separating the "data" part from the "human" part. Then we can use messy text for humans, and pure data for the machines. Enter "Data In/Out", "View In/Out".

TermKit data flow diagram

The data pipes correspond to the classical Std pipes, with one difference: the stream is prefixed with MIME-like headers (Content-Type, Content-Length, etc). Of these, only the 'Content-Type' is required. It allows programs to know what kind of input they're receiving, and handle it graciously without sniffing. Aside from that, the data on the pipe is a raw binary stream.

The view pipes carry the display output and interaction to the front-end. Widgets and UI commands are streamed back and forth as JSON messages over the view pipes.

The real magic happens when these two are combined. The last dangling Std Out pipe of any command chain needs to go into the Terminal, to be displayed as output. But the data coming out of Data Out is not necessarily human-friendly.

Thanks to the MIME-types, we can solve this universally. TermKit contains a library of output formatters which each handle a certain type of content (text, code, images, ...). It selects the right formatter based on the Content-Type, which then generates a stream of view updates. These go over the View Out pipe and are added to the command output.

Cat'ing an image

As a result, you can cat a PNG and have it just work. TermKit cat doesn't know how to process PNGs or HTML—it only guesses the MIME type based on the filename and pipes the raw data to the next process. Then the formatter sends the image to the front-end. If you cat a source code file, it gets printed with line numbers and syntax highlighting.

So where does "somewhat parseable text" fit in? It turns out to be mostly unnecessary. Commands like ls output structured data by nature, i.e. a listing of files from one or more locations. It makes sense to pipe around this data in machine-form. Output flags like ls -l become mere hints for the final display, which can toggle on-the-fly between compact and full listing.

In TermKit's case, JSON is the interchange format of choice. The Content-Type for file listings is application/json; schema=termkit.files. The schema acts as a marker to select the right output plug-in. In this case, we want the file formatter rather than the generic raw JSON formatter.

Formatting data in TermKit

Isn't JSON data harder to work with than lines of text? Only in some ways, but parsing JSON is trivial these days in any language. Because of this, I built TermKit grep so it supports grepping JSON data recursively. This happens transparently when the input is application/json instead of text/plain. As a result ls | grep works as you'd expect it to.

To slot in traditional Unix utilities in this model, we can pipe their data as application/octet-stream to start with, and enhance specific applications with type hints and wrapper scripts.

Finally, having type annotations on pipes opens up another opportunity: it allows us to pipe in HTTP GET / POST requests almost transparently. Getting a URL becomes no different from catting a file, and both can have fancy progress bars, even when inside a pipe chain like get | grep.

Synchronous interaction

All interaction in a traditional terminal is synchronous. Only one process is interactive at a time, and each keystroke must be processed by the remote shell before it is displayed. This leads to an obvious daily frustration: SSH keystroke lag.

To fix this, TermKit is built out of a separate front-end and back-end. The front-end can run locally, controlling a back-end on a remote machine. The connection can be tunneled over SSH for security.

TermKit architecture Architecture diagram (TK stands for TermKit)

Additionally, all display updates and queries are asynchronous. The WebKit-based HTML display is split up into component views, and the view pipes of each subprocess are routed to their own view. Vice-versa, any interactive widgets inside a view can send callback messages back to their origin process, as long as it's still running.

This also allows background processes to work without overflowing the command prompt.

String-based command line

A lot of my frustration comes from bash's arcane syntax. It has a particularly nasty variant of C-style escaping. Just go ahead and try to match a regular expression involving both types of quotes.

But at its core, a bash command is a series of tokens. Some tokens are single words, some are flags, some are quoted strings, some are modifiers (like | and >). It makes sense for the input to reflect this.

TermKit command-line

TermKit's input revolves around tokenfield.js, a new snappy widget with plenty of tricks. It can do auto-quoting, inline autocomplete, icon badges, and more. It avoids the escaping issue altogether, by always processing the command as tokens rather than text. Keys that trigger special behaviors (like a quote) can be pressed again to undo the behavior and just type one character.

The behaviors are encoded in a series of objects and regexp-based triggers, which transform and split tokens as they are typed. That means it's extensible too.

Usability

At the end of the day, Unix just has bad usability. It tricks us with unnecessary abbreviations, inconsistent arguments (-r vs -R) and nitpicky syntax. Additionally, Unix has a habit of giving you raw data, but not telling you useful facts, e.g. 'r-xr-xr-x' instead of "You can't touch this" (ba-dum tsshh).

One of the Unix principles is nobly called "Least Surprise", but in practice, from having observed new Unix users, I think it often becomes "Maximum Confusion". We should be more pro-active in nudging our users in the right direction, and our tools should be designed for maximum discoverability.

For example, I want to see the relevant part of a man page in a tooltip when I'm typing argument switches. I'd love for dangerous flags to be highlighted in red. I'd love to see regexp hints of possible patterns inline.

There's tons to be done here, but we can't do anything without modern UI abilities.

Focus and Status

With a project like TermKit, it's easy to look at the shiny exterior and think "meh", or that I'm just doing things differently for difference's sake. But to me, the real action is under the hood. With a couple of tweaks and some uncompromising spring cleaning, we can get Unix to do a lot more for us.

The current version of TermKit is just a rough alpha, and what it does is in many ways just parlour tricks compared to what it could be doing in a few months. The architecture definitely supports it.

I've worked on TermKit off and on for about a year now, so I'd love to hear feedback and ideas. Please go check out the code.

TermKit owes its existence to Node.js, Socket.IO, jQuery and WebKit. Thanks to everyone who has contributed to those projects.

Edit, a couple of quick points:

A Linux port will definitely happen, since it's built out of WebKit and Node.js. Whoever does it first gets a cookie.
TermKit is not tied to JSON except in its own internal communication channels. TermKit Pipes can be in any format, and old-school plain-text still works. JSON just happens to be very handy and very lightweight.
The current output is just a proof of concept and lacks many planned usability enhancements. There are mockups on github.
If you're going to tell me I'm stupid, please read all the other 100 comments doing so first, so we can keep this short for everyone else.

Edit, random fun:

Someone asked for AVS instead of TermKit in the comments... best I could do was JS1K with a PDF surprise:

↧

Wiki TL;DR – WikiLeaks Reader

December 23, 2010, 12:00 am

≫ Next: On Asm.js

≪ Previous: On TermKit

Wiki TL;DR is an extension for Safari and Chrome. It replaces the drab data dumps of WikiLeaks' Cablegate with richly formatted pages optimized for reading.

Wiki TL;DR for Chrome (0.3)
Wiki TL;DR for Safari (0.3)

Abbreviations are expanded, text is reflowed, a map is added and the entire page is laid out with a clean design. Security clearances and message priorities are indicated. A summary is included on top.

It's also got a variety of formatting rules. So far the majority of cables work perfectly, but feedback is welcome. Source code is available on github.

Note: this extension only works if you access wikileaks using the wikileaks.ch domain, instead of wikileaks.org. I'll update it soon.

↧

On Asm.js

November 27, 2013, 12:00 am

≫ Next: Shadow DOM

≪ Previous: Wiki TL;DR – WikiLeaks Reader

Ending The Ice Age of JavaScript

The demo is striking: Unreal Engine, running live in a browser, powered by Mozilla's Asm.js, offering near native performance with hardware accelerated graphics. A gasp here, a wow there, everyone is surely impressed. And look ma', no plug-ins: just JavaScript, Web Audio and WebGL! Just don't mind the 10 second hang on load as it's Preparing JavaScript.

When I heard of it, it sounded great: a highly optimizable subset of JS, that can be dropped seamlessly into any existing code. It reminded me of Winamp's charming evallib compiler, used in AVS and Milkdrop, which only did floating point expressions. It spawned a whole subculture of visual hackery based on little more than a dozen or so math functions and some clever graphics routines. It showed the power of being able to turn scripts into optimal machine code on the fly, and having a multimedia platform at your disposal while doing so.

But that's not what Asm.js is for at all. It's not for people, it's a compiler target, a way of converting non-JS code into a form that browsers handle well. Its design is based on how JavaScript handles numbers: as 64-bit doubles, onto which you can perform select 32-bit integer operations. As such, Asm.js seems like a sweet hack similar to UTF-8, an elegant way of encoding something complicated under strong legacy constraints: typed arithmetic despite a single number type. Yet the part of me that remembers pushing pixels with MMX, that watched this web thing for more than a few years, can't help but ask exactly what it is we're trying to do here.

Asm.js deserves closer inspection for two reasons. First, it's the one "native browser VM" that doesn't massively reinvent wheels. Second, it's the only time a browser vendor's "next-gen JS" attempts have actually gotten everybody else to pay attention. But what are we transitioning into exactly?

LLVM to Asm

To understand Asm.js, you have to understand LLVM. Contrary to its name, it's not really a Virtual Machine but rather a compiler. More precisely, it's the modular core of a compiler, along with assorted tools and libraries. You plug in different front-ends to parse different languages (e.g. C or C++), and different back-ends to target different machines (e.g. Intel or ARM). LLVM can do common tasks like optimization in a platform and language agnostic way, and as such should probably be considered an open-source treasure.

LLVM architecture (source)

Per Atwood's law, it was inevitable that someone decided the back-end should be JavaScript. Thus was born emscripten, turning C into JS—or indeed, anything native into JS. Because the output is tailored to how JS VMs work, this already gets you pretty far. The trick is that native code manages its own memory, creating a stack and heap. Hence you can output JS that just manipulates pre-allocated typed arrays as much as possible, and minimizes use of the garbage collector.

This works particularly well because JavaScript VMs already cheat when it comes to numbers and number types. They receive special treatment compared to other data. You can find a good overview in value representation in javascript implementations. The gist is that JS VMs handle floating point and integer arithmetic separately and efficiently, with lots of low level trickery to speed up computation. Modern VMs will furthermore try to identify code that uses one type only, and emit highly optimized type-specific code with only minimal checks at the boundaries. It's this kind of code that emscripten can emit a lot of, e.g. translating the clang compiler into 48MB of JS.

Which brings us to the bittersweet hack of Asm.js. Once you realize that C can run 'well' as JavaScript even when the VM has to guess and juggle, imagine how much faster it could be when the VM is in on it.

What this means in practice is a directive "use asm" in a block of tailored code, along with implicit type annotations for arguments, casts and return values. Type casts are x|0 for int, +x for double. These annotations are parsed and validated, and optimized code is emitted for the entire code block. This doesn't look bad at all. However, it also looks nothing like real Asm.js code in the wild.

Benchmarks show roughly a 1–2× slowdown compared to native code, significantly faster than normal JS. Hooray, JavaScript wins again, the web is awesome. Because of LLVM, an enormous piece of external, non-web infrastructure. Wait what?

Impostor Syndrome

When something bugs me that I can't put my finger on, there's usually a contradiction that I'm not seeing. After a few talks, articles and conversations, it seems pretty obvious: it puts JavaScript on a pedestal, even as it makes it irrelevant.

It makes JavaScript irrelevant because with LLVM's infrastructure or similar tools, practically anything can or will be compiled into JS.

But it also makes JavaScript more important, focusing the future optimization efforts of browser makers onto it, despite its ill-suited semantics.

It means JavaScript has nothing to do with it, it's just the poison we ended up with, the bitter pill we supposedly have to swallow. So when Brendan Eich says with a smile "Always bet on JavaScript", what he really means is "Always bet on legacy code" or perhaps "Always bet on politics". When you think about it, it's weird to tell JavaScript developers about Asm.js. It's not actually aimed at them.

Looking around, in the browser there's CoffeeScript, TypeScript and Dart. Outside, there's Python, Ruby, Go and Rust. Even CSS now has offspring. The future of the web is definitely multilingual and some people want to jump ship, they're just not sure which one will actually sail yet.

When faced with a legacy mechanism like UTF-8 or indeed Asm.js, we have to ask, is it actually necessary? In the case of UTF-8, it's a resounding yes: we need to assign unique names to things, and this name has to work with both modern and legacy software, passing through unharmed as much as possible. UTF-8 solves a bunch of problems while causing very few.

But with Asm.js, it's just a nice to have. All Asm.js code is new, there is no vault of legacy code that will stop working if we do it wrong. We can already generate functioning JS for legacy browsers, along with something new for alternative VMs. Having one .js file that does both is merely a convenience, and a dubious one at that.

Indeed, the unique appeal of Asm.js is for the browser maker who implements it first: it lets their JS VM race closer to that much desired Native line. It also turns any demo that uses Asm.js into an instant benchmark, which other vendors have to catch up with. It's a rational choice made in self interest, but also a tragedy of the commons.

Maybe that's a bit hyperbolic, but work with me here. There's a serious amount of defeatism and learned helplessness at work here, and again a contradiction. We seek to get ever closer to native performance, yet fall short by design, resigning ourselves to never quite reaching it. I can't be the only one who finds that completely bizarre, when there's laptops and phones running entirely on a web stack now?

If you look at the possible future of Asm.js, there's SIMD extensions, long ints with value objects, math extensions, support for specific JVM/CLR instructions and more. Asm.js is positioned not just as something that works today, but that leads into a bright future to boot. And yet, it all has to be shoehorned into something that is still 100% JavaScript, even as that target itself consists of moving goalposts.

Epic Citadel Code

Part of Unreal Engine, JSified.

History Repeating

So fast forward a year or two. Firefox has completed its wishlist and Asm.js has filled its instruction set gaps. Meanwhile Chrome has continued to optimize V8. Will they have the same new language features that Firefox has? Will they officially support Asm.js? Or push Dart and PNaCl, expanding their influence through ChromeOS and Android? Your guess is as good as mine. As for IE and Safari, I'll just pencil in "behind" for now and leave it at that.

But a certain phrase comes to mind: embrace and extend. From multiple fronts.

It looks like a future where your best bet to get things running fast in a browser is to do decidedly non-web things. You compile something like C to the different flavors of web at your disposal, either papering over their strengths, or tailoring for each individually. That's not something I personally look forward to, as much as it might arouse Epic's executives and shareholders today.

Web developers wouldn't actually be working that differently. They might be using a multi-language framework like Angular, or dropping in a neat C physics library somebody cross compiled for them. I doubt they'll have a nice web-native way to run the same performance-critical code everywhere. You'll just waste some battery life because a computer pretended to be a JavaScript developer. For backwards compatibility with browsers that auto-update every few weeks. Eh?

I admit, I don't know what the post-JS low level future should look like either, but it should probably be closer to LLJS's nicely struct'ed and typed order, than either the featherweight of Asm.js or the monolithic Flash-replacement that is PNaCl.

The big problem with Asm.js isn't that it runs off script rather than bytecode, it's that the code is generated to match how JS engines work rather than how CPUs compute. At best it will be replaced with something more sensible later or just fizz out as an optimization fad. At worst it'll become the IA-32 of the web, still pretending to be an 8086 if asked to.

Looking ahead, there's computation with WebCL, advanced GLSL shaders and more on the horizon. That's a whole set of problems that can become much simpler when "browser" is a language that everyone can speak, to and from, rather than a weird write-only dialect built on a tower of Babel. We don't just need a compilation target, we need a compilation source, as well as a universal intermediate representation.

And this is really the biggest contradiction of them all. Tons of people have invested countless hours to build these VMs, these new languages, these compilers, these optimizations. Yet somehow, they all seem to agree that it is impossible for them to sit down and define the most basic glue that binds their platforms, and implement a shared baseline for their many innovations.

We really should aim higher than a language frozen after 10 days, thawing slowly over 20 years.

↧

Shadow DOM

March 24, 2014, 12:00 am

≫ Next: The Cargo Cult of Game Mechanics

≪ Previous: On Asm.js

SVG, CSS, React and Angular

For a while now I've been working on MathBox 2. I want to have an environment where you take a bunch of mathematical legos, bind them to data models, draw them, and modify them interactively at scale. Preferably in a web browser.

Unfortunately HTML is crufty, CSS is annoying and the DOM's unwieldy. Hence we now have libraries like React. It creates its own virtual DOM just to be able to manipulate the real one—the Agile Bureaucracy design pattern.

The more we can avoid the DOM, the better. But why? And can we fix it?

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
  width="400px" height="400px" viewBox="0 0 400 400" enable-background="new 0 0 400 400" xml:space="preserve">
  <polygon fill="#FDBD10" stroke="#BE1E2D" stroke-width="3" stroke-miterlimit="10" points="357.803,105.593 276.508,202.82 
    343.855,310.18 226.266,262.91 144.973,360.139 153.592,233.697 36.002,186.426 158.918,155.551 167.538,29.109 234.885,136.469 "/>
  <polygon fill="#FDEB10" points="326.982,114.932 259.695,195.408 315.441,284.271 218.109,245.146 150.821,325.623 157.955,220.966 
    60.625,181.838 162.364,156.283 169.499,51.625 225.242,140.488 "/>
</svg>

Dangling Nodes

Take SVG. Each XML tag is a graphical shape or instruction. Like all XML, the data has to be serialized into tags with attributes made of strings. Large data sets turn into long string attributes to be parsed. Large collections of stuff turn into many separate tags to be iterated over. Neither is really desirable.

It only represents basic operations, so all serious prep work has to be done by the user up front. This is what D3 is used for, generating and managing more complex mappings for you.

When you put SVG into HTML, each element becomes a full DOM node. A simple <tag> with attributes is now a colossal binding between HTML, JS, CSS and native. It's a JavaScript object that pretends to be an XML tag, embedded inside a layout model that takes years to understand fully.

Its namespace mixes metadata with page layout, getters and setters with plain properties, native methods with JS, string shorthands with nested objects, and so on. Guess how many properties the DOM Node Object actually has in total? We'll be generous and count style as one.

A hundred is not even close. A plain <div> doesn't fare much better. Just serializing a chunk of DOM back into its constituent XML is a tricky task once you get into fun stuff like namespaces. Nothing in the DOM is as simple as JSON.stringify. Why does my polygon have a base URI?

We have all these awesome dev tools now, yet we're using them to teach a terrible model to people who don't know any better.

DOM Shader

In contrast, there's Angular. I like it because they've pulled off a very neat trick: convincing people to adopt a whole new DOM by disguising it as HTML.

<body ng-controller="PhoneListCtrl">
  <ul>
    <li ng-repeat="phone in phones">
      {{phone.name}}
      <p>{{phone.snippet}}</p>
    </li>
  </ul>
</body>

When you use <input ng-model="foo"> or <my-directive>, you're creating a controller and a scope, entirely separate from the DOM, with their own rules and chain of inheritance. The pseudo-HTML in the source code is merely an initial definition, most of it inert to the browser. Angular parses it out and replaces much of it.

Like React, the browser's live DOM is subsumed and used as a sort of render tree, a generic canvas to be cleverly manipulated to match a given set of views. The real view tree hides in the shadows of JS, where controllers operate on scopes. They only use the DOM to find each other on creation, and then communicate directly. The DOM is mostly there to trigger events, do layout and look pretty. Form controls are the one exception.

It's a bad fit because the DOM was built for text markup and there's tons of baggage in the form of inline spans, floats, alignment, indentation, etc. Most of these are layout systems disguised as typography, of which CSS now has several.

The whole idea of cascading styles is suspect. In reality, most styles don't actually cascade: paddings and backgrounds are set on individual elements. The inherited ones are almost all about typography: font styles, text justification, writing direction, word wrap, etc.

Think of it this way: why should a table have a font size? Only the text inside the table can have a font size, the table is just a box with layout that contains other boxes. Why don't we write table text { size: 16px; } instead of table { font-size: 16px; }? Text nodes exist today.

Well because that's how HTML's <font> tag worked. Instead of just making a selector for text nodes, they gave all the other elements font properties. They didn't get rid of font tags, they made them invisible and put one inside each DOM node.

<html><font>
  <body><font>
    <h1><font>Hello World</font></h1>
    <p><font>Welcome to the future.</font></p>
  </font></body>
</font></html>

Unreasonable Behavior

It was decided the world would be made of block and inline elements—divs and spans—and they saw that it was good, until someone came along and said, hey, so what about my table?

<table>
  <tr>
    <td>Forever</td>
    <td>Alone</td>
  </tr>
</table>

This <table> can't be replicated with CSS 1. Tables require a particular arrangement of children and apply their own box model. It's a directive posing amongst generic markup, just like Angular.

CSS has never been able to deliver on the promise of turning semantic HTML into arbitrary layout. We've always been forced to add extra divs or classes. These are really just attachment points for independent behaviors.

Purists see these as a taint upon otherwise pristine HTML, even though I've never seen someone close a website because the markup was messy. Not all HTML should be semantic. Rather, HTML stripped of its non-semantic parts should remain meaningful to robots.

CSS 2's solution was instead to make <table> invisible too, to go with the invisible <float>, <layer>, <clear> and <frame> tags which we pretended we didn't have. Watch:

17.2.1 Anonymous table objects

[…] Any table element will automatically generate necessary anonymous table objects around itself, consisting of at least three nested objects corresponding to a 'table'/'inline-table' element, a 'table-row' element, and a 'table-cell' element. Missing elements generate anonymous objects (e.g., anonymous boxes in visual table layout) according to the following rules […]

.grid {
  display: table;
}
.grid > ul {
  display: table-row;
}
.grid > ul > li {
  display: table-cell;
}

This is called Not Using Tables.

Without typographical styles, block elements start to look very different. They're styled boxes with implied layout constraints. They stack vertically, expand horizontally and shrink wrap vertically. Floated blocks are boxes that stack horizontally, and shrink wrap both ways. Tables are grids of boxes that are locked together.

Just think how much simpler CSS would be if boxes had box styles and text had text styles, instead of all of them having both. Besides, block margins and paddings don't even work the same on inline elements, there's a whole new layout behavior there.

So we do have two kinds of objects, text and boxes, but several different ways of combining them into layout: inline, stacked, nested, absolute, relative, fixed, floated, flex or table. We have optional behaviors like scrollable, draggable, clipped or overflowing.

They're spread across display, position, float and more, only meaningful in some combinations. And presence is mixed in there too. As a result, you can't unhide an element without knowing its display model. This is a giant red flag.

Thinking with Portals

It should further raise eyebrows that the binary world of inline and block now also includes a hybrid called inline-block.

Medium share thing

You generally don't need to embed a contact form–or all of Gmail—in the middle of mixed English/Hebrew poetry shaped like a bird. You just don't. To attach something to flowing text, you should insert an anchor point instead and add floating constraints. Links are called anchor tags for a reason. Why did we forget this?

Don't shove your entire widget right between the words. You'd inherit styles, match new selectors and bubble all your events up through the text just for the sake of binding a pair of (x, y) coordinates.

Heck, pointer events, cursors, hover states... these are for interactive elements only. Why isn't that optional, so mouse events wouldn't need to bubble up through inert markup? This would completely avoid the mouseover vs mouseenter problem. What is the point of putting a resize cursor on something that is dead without JavaScript? Pointer events shouldn't fire on inert children, and inert parents shouldn't care about interactive children. It's about boundaries, not hierarchy.

Things like SVG are better used as image tags instead of embedded trees, just slotting into place while ignoring their surroundings. They do need their own tree structure, but there is little reason to graft it onto HTML/CSS, inheriting original sin. The nodes have too little in common. At most you can share the models, not the controllers.

We should be able to manipulate them from the outside, like a <canvas>, but define and load them declaratively, like an image tag.

For that matter, MathML should really be a single inline text tag, optimized for math, not a bunch of tags. Regular text spans are not just "plain text". They are trimmed, joined, bidirectionalized, word wrapped and ellipsified before display. It's a separate embedded layout model that makes up the true, invisible <p> tag. A tag that HTML1 actually sort of got right: as an operator.

We create JavaScript with code, not as abstract syntax trees. Why should I build articles and embedded languages out of enormously nested trees, instead of just typing them out and adding some anchor tags around specific interesting parts? The DOM already inserts invisible text nodes everywhere. We didn't need to wrap all our words in <text> tags by hand just to embiggen one of them. The mutant tree on the right could just look like this:

<math>x = (-b &pm; &Sqrt;(b^2 - 4 a c)) / 2a</math>

<math>x = (-b &pm; &Sqrt;(b^2 - 4 a c)) / 2a</math>

Wasn't HTML5 supposed to match how people write it? LaTeX exists.

And which is easier: defining a hairy new category of pseudo-elements like :first-letter and :first-line… or just telling people to wrap their first letter in a span if they really want to make it giant? It was ridiculous to have this feature in a spec that didn't include tables.

The :first-line problem should be solved differently: you define two separate blocks inside a directive, to spread markup across two children with a content binding. It's no different from flowing text across lines and columns.

<mrow>
  <mi>x</mi>
  <mo>=</mo>
  <mfrac>
    <mrow>
      <mrow>
        <mo>-</mo>
        <mi>b</mi>
      </mrow>
      <mo>&#xB1;<!--PLUS-MINUS SIGN--></mo>
      <msqrt>
        <mrow>
          <msup>
            <mi>b</mi>
            <mn>2</mn>
          </msup>
          <mo>-</mo>
          <mrow>
            <mn>4</mn>
            <mo>&#x2062;<!--INVISIBLE TIMES--></mo>
            <mi>a</mi>
            <mo>&#x2062;<!--INVISIBLE TIMES--></mo>
            <mi>c</mi>
          </mrow>
        </mrow>
      </msqrt>
    </mrow>
    <mrow>
      <mn>2</mn>
      <mo>&#x2062;<!--INVISIBLE TIMES--></mo>
      <mi>a</mi>
    </mrow>
  </mfrac>
</mrow>

This is the first example in the MathML spec. Really. "Invisible times".

<join>
  <box class="first-line"></box>
  <box></box>
  <content>Hello New World</content>
</join>

Would this really be insane?

The Boxed Model

CSS got it wrong and we're now suffering the consequences. The HTML feature that was ignored in CSS 1 was the thing they should've focused on: tables, which were directives that generated layout. It set us on a path of trying to fake them by piggybacking on supposedly semantic elements, like lipstick on a div. Really we were pigeonholing non-linear layout as a nested styling problem.

Semantic content was a false spectre on the document level. Making our menus out of <ul> and <li> tags did not help impaired users skip to the main article. Adding roman numerals for lists did not help us number our headers and chapters automatically.

View and render trees are supposed to be simple and transparent data structures, the model for and result of layout. This is why absolute positioning is a performance win for mobile: it avoids creating invisible dynamic constraints between things that rarely change. Styles are orthogonal to that, they merely define the shape, not where it goes.

Flash had its flaws, but it worked 15 years ago. Shoving raw SVG or MathML into the DOM—or god forbid XML3D–is a terrible idea. It's like there's an entire class of developers who've now forgotten how fast computers actually are and how memory is supposed to work. A stringly typed kitchen sink is not it.

So I frown when I see people excited about SVG in the browser in the year 2014, making polygons out of CSS 3D or driving divs with React. Yes I know, it's fun and it does work. And Angular shows the web component approach has merit. But we need a way out.

CSS should be limited to style and typography. We can define a real layout system next to it rather than on top of it. The two can combine in something that still includes semantic HTML fragments, but wraps layout as a first class citizen. We shouldn't be afraid to embrace a modular web page made of isolated sections, connected by reference instead of hierarchy.

Not my problem though, I can make better SVGs with WebGL in the meantime. But one can dream.

↧

The Cargo Cult of Game Mechanics

September 21, 2014, 12:00 am

≫ Next: A DOM for Robots

≪ Previous: Shadow DOM

Form without Function

There's been a lot of fuss about gaming and gaming culture lately, in particular the nature of gaming journalism. Don't worry, I'm so not sticking my face into that particular beehive. However, I do agree the conversation around gaming is crap, so instead I'm posting the kind of opinion piece I wish I'd see on credible gaming sites, as someone who actually knows how the sausage is made.

Dear Esther

But is it Art?

Gamers like to talk—or argue—about graphics, frame rates, physics, hours of play time, item variety, models, textures, downloadable content and microtransactions, and so on. There is a reason the Glorious PC Master Race and the Console Wars are memes. If games are art, if it's a grown up medium, why do we fuss about trivialities so much? You don't debate high literature by critiquing the paper stock or chapter length.

Well because production values are important for immersion. Details and performance really matter. However when we treat games just as mechanical live pictures, we're missing the point entirely. It's confusing form with function. In The Dark Knight, Heath Ledger's Joker should look the part, but he'll be 10x scarier and more interesting once you understand how he operates and thinks. This seems obvious in film, yet not in gaming.

Even "artistic games" like Dear Esther are often criticized for superficial mechanics (or lack thereof), not for what they set out to do. The question isn't whether Dear Esther is just a walking simulator. It's whether it's anywhere near as engaging as walking around a real place, like a park or a museum. If it fails, it's not because there aren't any puzzles. The Anne Frank House in Amsterdam does not require puzzles. It does have a secret passage but the only achievement you get for finding it is sadness.

…

Yup, that awkward pause is where the "gaming as a serious medium" debate usually hangs, and it leaves the conversation severely deadlocked. Trying to add gamified elements for the heck of it, to make a gamier game, rings hollow and does not get us any closer to credibility.

Heavy Rain

The popular alternative is to simply adopt the current forms of Serious Media. To make a game more like a movie or a book, whether blockbuster or arthouse. It generally involves taking away choice, using scripts instead of simulations, with mini-games and quick-time events thrown in to amuse your hindbrain. It's tacitly saying that real storytelling, real human comedy or tragedy, can't happen while a player is in control. It's non-sense of course, plenty of games have done so before.

Somehow though we've forgotten how to do it, and I don't think I'm alone in thinking this. This existential crisis was perfectly embodied in indie gem The Stanley Parable, a post-modern tale of choice. It's a game about playing a game, constantly breaking the fourth wall. There's recursive gags, self-parodying achievements, 'victory' conditions that require you to quit the game, and other surgical strikes at typical gaming habits. It garnered critical praise from gamers and journalists alike, playing like a love-hate letter to its audience: at times cooperative and happy, other times sardonic and sadistic.

The Stanley Parable

I'm pretty sure The Stanley Parable is Art. There's just one thing bothering me. It doesn't actually offer you any choice. The game is an admission of defeat.

Choice is of course a tricky concept, that was the whole point, so let me be more specific. You could feasibly make a 100% Let's Play of Stanley Parable, covering all the branching paths, and turn it into a sort of Dragon's Lair on Laserdisc. It would lose little in translation, most of the gags would still work. It's not a game about your choices, it's still just about watching theirs.

Live in Your World, Play in Ours

If you're looking for someone to blame (you know, in general), it's easy to point to the incestuous industry. Games are big business and cost a ton to produce. The primary purpose of talking about games is to sell things to gamers, in a market that moves very fast, saturated with product. Hence brands and franchises compete over the attention of customers, preferably through lock-in. It goes beyond ordinary sales, and includes pre-orders, season passes, virtual marketplaces and other monetary aids. Be sure to use a condom.

For several years now though, there has been a counterpoint: the wave of DRM-free indies, Humble Bundles and the wild success of Kickstarter. Notably, industry veterans Tim Schafer and Brian Fargo, known for beloved classics like Monkey Island and Wasteland, each held out their hats and promised to bring back the glory days of old. Gamers rewarded them in spades. Budgets ballooned from a few hundred thousand to several million, spawning further spinoffs. Chris Roberts of Wing Commander fame did even better. He kickstarted Star Citizen to the tune of a few million, but continues to raise funds today with virtual goods and perks for the future game. It now exceeds $50 million in backer funding.

Destiny vs Star Citizen

If I were cynical, which I am, I would say a bunch of people have spent hundreds of dollars each on virtual assets with no guarantee they'll ever work as promised. This is the power of nostalgia mixed with in-engine mockups, and it's clearly very good business. Don't get me wrong, I've funded a few games on Kickstarter too, below retail. But what comes out of these projects is raising some eyebrows, with hype, delays and cancellations galore. I think it points to a deeper issue altogether, driven by games but not limited to gaming.

On the surface these developers are giving their fans exactly what they want. Something they already love, modernized and expanded, with early access and feedback. You cannot fault the creators for this. Rather, I think the problem is that gaming fans don't know what they want. It's a know it when I see it kind of affair. So they just ask for more of the same instead, again confusing form with function.

There's an elephant in the room. Everybody does it to some degree, but it's somehow shameful.

Compulsion.

It's even more obvious when you consider that the easy money in gaming isn't actually to bankroll a $200 million console blockbuster, half of which is probably marketing. Rather, it's to put a carefully tuned slot machine under the noses of as many people as possible, like say, a free-to-play smartphone game. With lots of push notifications and time locks, using fictional hooks to create personal investment and a sense of false scarcity. People pull their phones out in elevators and on the toilet, multiple times a day. It's guaranteed brain share if you get in, so much easier than convincing everyone to fork out $50 once, let alone monthly.

The real target audience is a small minority of whales—compulsive users—to buy the virtual currency and goods you mint at will. They subsidize the free users, who in turn provide word of mouth on social media. It's gambling and addiction, by any other name, only now people are betting real money against fake money, so it's legal.

Most gamers are familiar with the "one more turn" itch of strategy or puzzle games, the desire to open every chest and read every log, the zombiefied stares at LAN parties. It's a common trope to be obsessive, but gamers are generally self-aware about it. We don't mind wasting time if it's fun, that's the point, and it gives the Youtubers something to do.

But the Skinner box is still real. Too often we see products that seem to consist mainly of compulsive triggers. Where the developers built a guided theme park ride with only the promise of cake at the end. They set out a generic progression tree and loom a nebulous threat overhead that can only be beaten by a fully armed and operational Level 80 Battlemage. Between you and the end stand a thousand foes and a bunch of fetch/build/shoot/escort quests. Everything will be perfectly scaled to offer the permanent illusion of a challenge you can barely win, and are constantly forced to work for.

I think this kind of game design stems from a fundamental misunderstanding, willful or not, of how games are supposed to work. It's cargo culting the patterns of games and game mechanics, without considering what they're for. Which is the point I'd like to get to.

But first, there's still the elephant.

Star Citizen

Double Fine Adventure

See, the way these shady free-to-play games work... if we're honest, it kinda matches how Kickstarter plays out. Dramatic concept art. A beloved NPC in need. An XP bar to fill. Stretch goals to level up. Massive online multiplayer with social media tie ins, rally your friends. Plus of course, unlimited alpha and beta testing until release, bankrolled by you, with additional paid perks along the way.

With the risk of stating the obvious, but it's more on point than ever: these things are run by game designers, for gamers. No, put away the tin foil hat. I simply want to suggest that what draws people into these projects bears little relation to what comes out at the end, a release which is merely a coda to a multi-year event. That it is no more about game development than Mario is about saving princesses. That maybe Kickstarter is a sequel to Twitter, the world's #1 video game.

It shows in the lack of polish and sophistication in the games that do manage a release, which reviewers and fans consistently gloss over or forgive. Yes I'm getting into taste territory here, but let's look at it objectively. Repetitive shoot em ups that merely consist of dice rolls and numbers going up. RPGs with fenced off wax-museum towns. Meticulously painted backdrops that belie the lack of depth. Or alternatively, pixel art and chiptunes.

Wasteland 2, Shadowrun Returns, Broken Age, Superbrothers: Sword & Sworcery EP

On the surface these games have all the trappings of the classic gaming age, remade in widescreen HD or quirky indie glory, but they lack lasting power once you stop playing. Far from evolving the real classics, of which there are admittedly not actually that many, we've regressed and turned them into caricatures of themselves, mistaking technical limitations for a lack of ambition.

The Carrot and the Stick

If at this point you think I'm wearing rose-tinted glasses so fabulous I'm farting rainbows, allow me to convince you otherwise. I'm not pretending that classic DOS or NES games with giant clunky controls were the height of interaction design, or that early 3D wasn't butt-ugly in retrospect. Features like hint systems and autosaves are nice. Rather, there's a reason people continue to cite the same few classics.

Fallout, Freespace, Outcast, Master of Orion, Rollercoaster Tycoon, System Shock, Thief and Torment are still high points in gaming, and it isn't because they were/weren't Art, or are/aren't crappy by modern standards.

To this day, each of those games presents an understandable, flexible sandbox. They offer you a world with consistent rules, letting you figure out the mechanics to face the game's challenges your way. You explore environments at your own pace, build at your leisure, and you're driven forward because you want to, not because you have to. Compulsion is a side-effect of existing motivations, which naturally result from actively participating in the game world.

Fallout, Outcast, Freespace 2, System Shock 2, Thief 2: The Metal Age

If I go through an airlock in System Shock 2, it's because I need what's on the other side of it, and I hope to return alive from it. The game presents a choice and then dares me to take it.

If I go through an airlock in Mass Effect 3, it closes permanently because everything looks the same and too many players got turned around in testing. There is never a reason to go back. The game presents a mistaken illusion of freedom and has to clamp down to fix it.

Corridor shooters with random chest high barriers, indestructible plot armor, keys hanging next to locks, breadcrumbed objective markers, one-way quick travel or chutes, rock-paper-scissors busywork, teleporting AI... these are all just symptoms of a broken game world, which needs dramatic patch jobs to make basic gameplay not fall apart. If a level designer locks a door with the Red Key, they're just putting a meaningless fetch quest in your path to keep you busy. If they put two elite guards and an alarm there instead, now you have the opportunity for improvisation and consequence. That can only happen when there's options beyond "Use Shotgun on Face" and you've been given space and time to get confident about it.

Instead, many games are explicitly structured in a linear, inflationary manner. What you do at level 50 is mostly the same as level 5, only now the numbers are 10× larger, and you shoot blue instead of green.

The Elder Scrolls IV: Oblivion

The role of game mechanics should not be the oppressive tyrant telling you to fetch and grind and be thankful for your crumbs of XP and DPS as the scenery blazes past. It should be an à-la-carte menu of options which is opened up for your benefit and at your direction. Slow enough that you can get familiar with each element in turn, but fast enough not to frustrate and limit. Unlockables and crafting should be a way to enable new abilities, not just busywork. Level ups should let you specialize in certain tactics, not just keep up with the Joneses who all bought new glass armor and plasma rifles overnight. Compulsion is just a stick, not the carrot.

Ironically I think it's the technical limitations of classic games that often played to their advantage and which modern remakes in particular are screwing up. The spartan graphics served to highlight the mechanics, instead of needing focus rings and prompts. The lack of voices and mocap forced the writing to carry the story. When you can't conjure up massive vistas at will, there's no point in making the player cross giant cities and wastelands. When the entire world is just isometric sprites, it's practical to let the player destroy all of them. For a while there was a really good match between the complexity of the game world and the way it was represented, and I don't think it's a coincidence that this window is where we find many beloved gaming classics.

What might now seem like broken mechanics often had significant effects on gameplay. An amnesiac guard that can't climb ladders has a similar effect as regenerating health: it makes it easier to run away. Except only one of those requires the player to learn their surroundings. Circle strafers had a surprising amount of non-linearity and involved much more acrobatics than FPSes today, and the passive AI of early RTSes acts similar to modern shooter enemies, which don't engage unless you've spotted them.

Crusader: No Regret, Rollercoaster Tycoon, TIE Fighter, Carmageddon

Gaming is ultimately about forgetting the rules of reality and adopting a whole new set. Realism doesn't matter, whacky rules can be fun, as long as they're consistent and interact in interesting ways.

For modern games to evolve to match their now deceiving superstar looks, to move beyond progress bar quest and animated puppets with voice boxes, significant advances have to be made. We need real sandbox simulation, autonomous agents and language-capable AI, and it's not as easy to deliver as another sequel or reboot, mainstream or otherwise. It requires building a game that's meant to be played rather than just reacted to.

I just hope enough people remember what actually made the classics work.

↧

A DOM for Robots

September 26, 2015, 3:00 pm

≫ Next: MathBox²

≪ Previous: The Cargo Cult of Game Mechanics

Modelling Live Data

I want to render live 3D graphics based on a declarative data model. That means a choice of shapes and transforms, as well as data sources and formats. I also want to combine them and make live changes. Which sounds kind of DOMmy.

3D engines don't have Document Object Models though, they have scene graphs and render trees: minimal data structures optimized for rendering output. In Three.js, each tree node is a JS object with properties and children like itself. Composition only exists in a limited form, with a parent's matrix and visibility combining with that of its children. There is no fancy data binding: the renderer loops over the visible tree leaves every frame, passing in values directly to GL calls. Any geometry is uploaded once to GPU memory and cached. If you put in new parameters or data, it will be used to produce the next frame automatically, aside from a needsUpdate bit here and there for performance reasons.

So Three.js is a thin retained mode layer on top of an immediate mode API. It makes it trivial to draw the same thing over and over again in various configurations. That won't do, I want to draw dynamic things with the same ease. I need a richer model, which means wrapping another retained mode layer around it. That could mean observables, data binding, tree diffing, immutable data, and all the other fun stuff nobody can agree on.

However I mostly feed data in and many parameters will end up as shader properties. These are passed to Three as a dictionary of { type: '…', value: x } objects, each holding a single parameter. Any code that holds a reference to the dictionary will see the same value, as such it acts as a register: you can share it, transparently binding one value to N shaders. This way a single .set('color', 'blue') call on the fringes can instantly affect data structures deep inside the WebGLRenderer, without actually cascading through.

I applied this to build a view tree which retains this property, storing all attributes as shareable registers. The Three.js scene graph is reduced to a single layer of THREE.Mesh objects, flattening the hierarchy. Rather than clumsy CSS3D divs which encode matrices as strings, there's binary arrays, GLSL shaders, and highly optimizable JS lambdas.

As long as you don't go overboard with the numbers, it runs fine even on mobile.

<root id="1" scale={600} focus={3}>
  <camera id="2" proxy={true} position={[0, 0, 3]} />
  <shader id="3" code="
uniform float time;
uniform float intensity;

vec4 warpVertex(vec4 xyzw, inout vec4 stpq) {
  xyzw +=   0.2 * intensity * (sin(xyzw.yzwx * 1.91 + time + sin(xyzw.wxyz * 1.74 + time)));
  xyzw +=   0.1 * intensity * (sin(xyzw.yzwx * 4.03 + time + sin(xyzw.wxyz * 2.74 + time)));
  xyzw +=  0.05 * intensity * (sin(xyzw.yzwx * 8.39 + time + sin(xyzw.wxyz * 4.18 + time)));
  xyzw += 0.025 * intensity * (sin(xyzw.yzwx * 15.1 + time + sin(xyzw.wxyz * 9.18 + time)));

  return xyzw;
}"
 time=>{(t) => t / 4} intensity=>{(t) => {
        t = t / 4;
        intensity = .5 + .5 * Math.cos(t / 3);
        intensity = 1.0 - Math.pow(intensity, 4);
        return intensity * 2.5;
      }} />
  <reveal id="4" stagger={[10, 0, 0, 0]} enter=>{(t) => 1.0 - Math.pow(1.0 - Math.min(1,  (1 + pingpong(t))*2), 2)} exit=>{(t) => 1.0 - Math.pow(1.0 - Math.min(1,  (1 - pingpong(t))*2), 2)}>
    <vertex id="5" pass="view">
      <polar id="6" bend={1/4} range={[[-π, π], [0, 1], [-1, 1]]} scale={[2, 1, 1]}>
        <transform id="7" position={[0, 1/2, 0]}>
          <axis id="8" detail={512} />
          <scale id="9" divide={10} unit={π} base={2} />
          <ticks id="10" width={3} classes=["foo", "bar"] />
          <scale id="11" divide={5} unit={π} base={2} />
          <format id="12" expr={(x) => {
        return x ? (x / π).toPrecision(2) + 'π' : 0
      }} />
          <label id="13" depth={1/2} zIndex={1} />
        </transform>
        <axis id="14" axis={2} detail={128} crossed={true} />
        <transform id="15" position={[π/2, 0, 0]}>
          <axis id="16" axis={2} detail={128} crossed={true} />
        </transform>
        <transform id="17" position={[-π/2, 0, 0]}>
          <axis id="18" axis={2} detail={128} crossed={true} />
        </transform>
        <grid id="19" divideX={40} detailX={512} divideY={20} detailY={128} width={1} opacity={1/2} unitX={π} baseX={2} zBias={-5} />
        <interval id="20" width={512} expr={(emit, x, i, t) => {
        emit(x, .5 + .25 * Math.sin(x + t) + .25 * Math.sin(x * 1.91 + t * 1.81));
      }} channels={2} />
        <line id="21" width={5} />
        <play id="22" pace={10} loop={true} to={3} script=[[{color: "rgb(48, 144, 255)"}], [{color: "rgb(100, 180, 60)"}], [{color: "rgb(240, 20, 40)"}], [{color: "rgb(48, 144, 255)"}]] />
      </polar>
    </vertex>
  </reveal>
</root>

Note: The JSX is a lie, you define nodes in pure JS.

Keep it Simple

From afar there's a tree of nodes, similar to SVG tags. This is the MathBox library of vector primitives. The basic shapes are all there: points, lines, faces, vectors, surfaces, etc. These nodes are placed inside a shallow hierarchy of views and transforms.

However none of the shapes draw anything by themselves. They only know how to draw data supplied by a linked source. Data can be an array (static or live), a procedural source, custom JS / GLSL code, etc. This is further augmented by data operators which can be sandwiched between source and shape, forming automatic pipelines between siblings.

The current set of components looks like this:

Base

Group
Inherit
Root
Unit

Camera

Camera

Draw

Axis
Face
Grid
Line
Point
Strip
Surface
Ticks
Vector

Data

Area
Array
Interval
Matrix
Scale
Volume
Voxel

Operator

Grow
Join
Lerp
Memo
Resample
Repeat
Slice
Split
Spread
Swizzle
Transpose

Overlay

DOM
HTML

Present

Move
Play
Present
Reveal
Slide
Step

RTT

Compose
RTT

Shader

Shader

Text

Format
Label
Text
Retext

Time

Clock
Now

Transform

Fragment
Layer
Transform
Transform4
Vertex

View

Cartesian
Cartesian4
Polar
Spherical
Stereographic
Stereographic4
View

To make you feel at home, nodes have an id and classes, and you can use CSS selectors to identify them. Nodes link up with preceding siblings and parents by default, but you can select any node in the tree. This allows for arbitrary graphs, including feedback loops. However all of this is optional: you can also pass in direct node objects or MathBox's own jQuery-like selections. What it doesn't have is a notion of detached document fragments: nodes are immediately inserted on creation.

A node's attributes can be .get() and .set(), though there is also a read-only .props dictionary for fashionable reasons. The values are strongly typed as Three.js colors, vectors, matrices, … but accept e.g. CSS colors and ordinary arrays too. The values are normalized for immediate use, the original values are preserved on the side for printing and serialization.

MathBox Node API

What's unique is the emphasis on time. First, properties can be directly bound to time-dependent expressions, on creation or afterwards. Second, clocks are primitives on their own. This allows for nested timelines, on-demand bullet time, fast forwards and more. It even supports limited time travel, evaluating an expression several frames in the past. This can be used to ensure consistent 60 fps data logging through janky updates, useful for all sorts of things. It's exposed publicly as .bind(key, expr) and .evaluate(key, time) per node. It's also dogfood for declarative animation tracks. The primitives clock/now provide timing, while step and play handle keyframes on tracks.

This is definitely a DOM, but it has only basic features in common with the HTML DOM and does much less. Most of the magic comes from the components themselves. There's no cascade of styles to inherit. Children compose with a parent, they do not inherit from it, only caring about their own attributes. The namespace is clean, with no weird combo styles à la CSS. As much as possible, attributes are unique orthogonal knobs you can turn freely.

Model-View-Projection

On the inside I separate the generic data model from the type-specific View Controller attached to it. The controller's job is to create and manage Three.js objects to display the node (if any). Because a data source and a visible shape have very little in common, the nodes and their controllers are blank slates built and organized around named traits. Each trait is a data mix-in, with associated attributes and helpers for common behavior. Primitives with the same traits can be expected to work the same, as their public facing models are identical.

Controllers can traverse the graph to find each other by matching traits, listening for events and making calls in response. This way only specific events will cascade through cause and effect, often skipping large parts of the hierarchy. The only way to do a "global style recalculation" would be to send a forced change event to every single controller, and there's never a reason to do so.

The controller lifecycle is deliberately kept simple: make(), made(), change(…), unmake(), unmade(). When a model changes, its controller either updates in place, or rebuilds itself, doing an unmake/make cycle. The change handler is invoked on creation as well, to encourage stateless updates. It affords live editing of anything, without having to micro-optimize every possible change scenario. Controllers can also watch bound selectors, retargeting if their matched set changes. This lets primitives link up with elements that have yet to be inserted.

Unlike HTML, the DOM is not forced to contain a render tree as well. Only some of the leaf nodes have styles and create renderables. Siblings and parents are called upon to help, but the effects don't have to be strictly hierarchical. For example, a visual effect can wrap a single leaf but still be applied after all its parents, as transformations are collected and composed in passes.

It'll Do

The result is not so much a document model as it is a computational model inside a presentational model. You can feed it finalized data and draw it directly… or you can build new models within it and reproject them live. Memoization enables feedback and meta-visualization. The line between data viz and demo scene is rarely this blurry.

Here, the notion of a computed style has little meaning. Any value will end up being transformed and processed in arbitrary ways down the pipe. As I've tried to explain before, the kinds of things people do with getComputedStyle() and getClientBoundingRect() are better achieved by having an extensible layout model, one that affords custom constraints and composition on an equal footing. To do otherwise is to admit defeat and embrace a leaky abstraction by design.

The shallow hierarchy with composition between siblings is particularly appealing to me, even if I realize it introduces non-traditional semantics more reminiscent of a command-line. It acts as both a jQuery-style chainable API, and a minimal document model. If it offends your sensibilities, you could always defuse the magic by explicitly wiring up every relationship. In case of confusion, .inspect() will log syntax highlighted JSX, while .debug() will draw the underlying shader graphs.

I've defined a good set of basic primitives and iterated on them a few times. But how to implement it, when WebGL doesn't even fully cover OpenGL ES 2?

MathBox² - PowerPoint Must Die
A DOM for Robots - Modelling Live Data
Yak Shading - Data Driven Geometry
ShaderGraph 2 - Functional GLSL

↧

MathBox²

September 26, 2015, 3:00 pm

≫ Next: ShaderGraph 2

≪ Previous: A DOM for Robots

PowerPoint Must Die

"I think a lot of mathematics is really about how you understand things in your head. It's people that did mathematics, we're not just general purpose machines, we're people. We see things, we feel things, we think of things. A lot of what I have done in my mathematical career has had to do with finding new ways to build models, to see things, do computations. Really get a feel for stuff.

It may seem unimportant, but when I started out people drew pictures of 3-manifolds one way and I started drawing them a different way. People drew pictures of surfaces one way and I started drawing them a different way. There's something significant about how the representation in your head profoundly changes how you think.

It's very hard to do a brain dump. Very hard to do that. But I'm still going to try to do something to give a feel for 3-manifolds. Words are one thing, we can talk about geometric structures. There are many precise mathematical words that could be used, but they don't automatically convey a feeling for it. I probably can't convey a feeling for it either, but I want to try."
– William Thurston, The Mystery of 3-Manifolds (Video)

How do you convince web developers—heck, people in general—to care about math? This was the challenge underlying Making Things With Maths, a talk I gave three years ago. I didn't know either, I just knew why I liked this stuff: demoscene, games, simulation, physics, VR, … It had little to do with what passed for mathematics in my own engineering education. There we were served only eyesore PowerPoints or handwritten overhead transparencies, with simplified graphs, abstract flowcharts and rote formulas, available on black and white photocopies.

Smart people who were supposed to teach us about technology seemed unable to teach us with technology. Fixing this felt like a huge challenge where I'd have to start from scratch. This is why the focus was entirely on showing rather than telling, and why MathBox 1 was born. It's how this stuff looks and feels in my head, and how I got my degree: by translating formulas into mental pictures, which I could replay and reason about on demand.

PowerPoint Syndrome

Initially I used MathBox like an embedded image or video: compact diagrams, each a point or two in a presentation. My style quickly shifted though. I kept on finding ways to transform from one visualization to another. Not for show, but to reveal the similarities and relationships underneath. MathBox encouraged me to animate things correctly, leveraging the actual models themselves, instead of doing a visual morph from A to B. Each animation became a continuous stream of valid examples, a quality both captivating and revealing.

For instance, How to Fold a Julia Fractal is filled with animations of complex exponentials, right from the get go. This way I avoid the scare that ($ e^{i\pi} $) is a meaningful expression; symbology and tau-tology never have a chance to obscure geometrical workings. Instead a web page that casually demonstrates conformal mapping and complex differential equations got 340,000 visits. Despite spotty web browser support and excluding all mobile phones for years.

The next talk, Making WebGL Dance, contained elaborate long takes worthy of an Alfonso Cuarón film, with only 3 separate shots for the bulk of a 30 minute talk. The lesson seemed obvious: the slides shouldn't have graphics in them, rather, the graphics should have slides in them. The diagnosis of PowerPoint syndrome is then the constant trashing of context from one slide to the next. A traditional blackboard doesn't have this problem: you build up diagrams slowly, by hand, across a large surface, erasing selectively and only when you run out of space.

It's not just about permanence and progression though, it's also about leveraging our natural understanding of shape, scale, color and motion. Think of how a toddler learns to interact with the world: poke, grab, chew, spit, smash. Which evolves into run, jump, fall, get back up again. Humans are naturals at taking multiple cases of "If I do this, that will happen" and turning it into a consistent, functional model of how things work. We learn language by bootstrapping random jibberish into situational meaning, converging on a shared protocol.

That said, I find the usual descriptions of how people experience language and thought foreign. Instead, when Temple Grandin speaks about visual thinking, I nod vigorously. Thought to me is analog concepts and sensory memories, remixed with visual and other simulations. It builds off the quantities and qualities present in spatial and temporal notions, which appear built-in to us.

Speech and writing is then a program designed to reconstruct particular thoughts in a compatible brain. There are a multitude of evolving languages, they can be used elegantly, bluntly, incomprehensibly, but the desired output remains the same. In my talks, armed with weapons-grade C2-continuous animations, it is easy to transcode my film reel into words, because the slides run themselves. The string of concepts already hangs in the air, I only add the missing grammar that links them up. This is a puzzle our brains are so good at solving, we usually do it without thinking.

Language is the ability of thoughts to compute their own source code.

(It's not proof, I just supply pudding.)

Manifold Dreams

I don't say all this to up my Rain Man cred, but to lay to rest the recurring question of where my work comes from. I translate the pictures in my head to HD, in order to learn from and refine the view. As I did with quaternions: I struggled to grok the hypersphere, it wouldn't fit together right. So I wrote the code to trace out geodesics in color and fly around in it, and suddenly the twisting made sense. Hence my entire tutorial was built to replicate the same discovery process I went through myself.

There was one big problem: scenes now consisted of diagrams of diagrams, which meant working around MathBox more than with it. Performance issues arose as complexity grew. Above all there was a total lack of composability in the components. None of this could be fixed without ripping out significant pieces, so doing it incrementally seemed futile. I started from scratch and set off to reinvent all the wheels.

$$ \text{MathBox}^2 = \int_1^2 \text{code}(v) dv $$

MathBox 2 was inevitably going to suffer second-system syndrome, parts would be overengineered. Rather than fight it, I embraced it and effectively wrote a strange vector GPU driver in CoffeeScript. (Such is life, this is a blueprint meant to be simplified and made obsolete over time, not expanded upon.) It's a freight train straight to the heart of a graphics card, combining low-level and high-level in a way that feels novel 🐴 when you use it, squeezing 🐴 through a very small opening.

What was tedious before, now falls out naturally. If I format the scene above as XML/JSX, it becomes:

<root>
  <!-- Place the camera -->
  <camera />
  <!-- Change clock speed -->
  <clock>
    <!-- 4D Stereographic projection -->
    <stereographic4>
      <!-- Custom 4D rotation shader -->
      <shader />
      <!-- Move vertices -->
      <vertex>
        <!-- Sample an area -->
        <!-- Draw a set of lines -->
        <area />
        <line />

        <!-- Sample an area -->
        <!-- Draw a set of lines -->
        <area />
        <line />

        <!-- Sample an area -->
        <!-- Draw a set of lines -->
        <area />
        <line />
      </vertex>
    </stereographic4>
  </clock>
</root>

In order to make these pieces behave, a bunch of additional attributes are applied, most of which are strings or values, some of which are functions/code, either JavaScript or GLSL:

<root id="1" scale={300}>
  <camera id="2" proxy={true} position={[0, 0, 3]} />
  <clock id="3" speed={1/4}>
    <stereographic4 id="4" bend={1}>
      <shader id="5" code="
uniform float cos1;
uniform float sin1;
uniform float cos2;
uniform float sin2;
uniform float cos3;
uniform float sin3;
uniform float cos4;
uniform float sin4;

vec4 getRotate4D(vec4 xyzw, inout vec4 stpq) {
  xyzw.xy = xyzw.xy * mat2(cos1, sin1, -sin1, cos1);
  xyzw.zw = xyzw.zw * mat2(cos2, sin2, -sin2, cos2);
  xyzw.xz = xyzw.xz * mat2(cos3, sin3, -sin3, cos3);
  xyzw.yw = xyzw.yw * mat2(cos4, sin4, -sin4, cos4);

  return xyzw;
}"
 cos1=>{(t) => Math.cos(t * .111)} sin1=>{(t) => Math.sin(t * .111)} cos2=>{(t) => Math.cos(t * .151 + 1)} sin2=>{(t) => Math.sin(t * .151 + 1)} cos3=>{(t) => Math.cos(t * .071 + Math.sin(t * .081))} sin3=>{(t) => Math.sin(t * .071 + Math.sin(t * .081))} cos4=>{(t) => Math.cos(t * .053 + Math.sin(t * .066) + 1)} sin4=>{(t) => Math.sin(t * .053 + Math.sin(t * .066) + 1)} />
      <vertex id="6">
        <area id="7" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
        q1.set(0, 0, Math.sin(θ), Math.cos(θ));
        q2.set(0, Math.sin(ϕ), 0, Math.cos(ϕ));
        q1.multiply(q2);
        emit(q1.x, q1.y, q1.z, q1.w);
      }} live={false} channels={4} />
        <line id="8" color="#3090FF" />
        <area id="9" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
        q1.set(0, Math.sin(θ), 0, Math.cos(θ));
        q2.set(Math.sin(ϕ), 0, 0, Math.cos(ϕ));
        q1.multiply(q2);
        emit(q1.x, q1.y, q1.z, q1.w);
      }} live={false} channels={4} />
        <line id="10" color="#20A000" />
        <area id="11" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
        q1.set(Math.sin(θ), 0, 0, Math.cos(θ));
        q2.set(0, 0, Math.sin(ϕ), Math.cos(ϕ));
        q1.multiply(q2);
        emit(q1.x, q1.y, q1.z, q1.w);
      }} live={false} channels={4} />
        <line id="12" color="#DF2000" />
      </vertex>
    </stereographic4>
  </clock>
</root>

Phew. That's how you make a 4D diagram with Hopf fibration as far as the eye can see. Except it's not actually JSX, that's just me and my pretty-printer pretending.

Geometry Streaming

The key is the data itself. It's an array of points mostly, but how that data is laid out and interpreted determines how useful it can be.

Most basic primitives come in fixed size chunks. Particles are single points, lines have two points, triangles have three points. Polygons and polylines have N points. So it made sense to have a tuple of N points be the basic logical unit. You can think in logical pieces of geometry, rather than raw points or individual triangles, unlike GL.

Each primitive maps over data in a standard way. Feed an array of points to a line, you get a polyline. Feed a matrix of points to a surface and you get a grid mesh. Simple. But feed a voxel to a vector, and you get a 3D vector field. The general idea is that drawing 1 of something should be as easy as drawing 100×100×100.

This is particularly useful for custom data expressions, which stream in live or procedural data. They now receive an emit(x, y, z, w) function, for emitting a 4-vector like XYZW or RGBA. This is little more than an inlineable call to fill a floatArray[i++] = x, quite a lot faster than returning an array or object.

mathbox
  .interval({
    expr: function (emit, x, i, t) {
      y = Math.sin(x + t);
      emit(x,  y);
      emit(x, -y);
    },
    width:   64,
    items:    2,
    channels: 2,
  })
  .vector({
    color: 0x3090FF,
    width: 3,
    start: true,
  });

Emitting 64 2D vectors on an interval, 2 points each.

More importantly it lets you emit N points in one iteration, which makes the JS expressions themselves feel like geometry shaders. The result feeds into one or more styled drawing ops. The number of emit calls has to be constant, but you can always knock out or mask the excess geometry.

emit = switch channels
  when 1 then (x) ->
    array[i++] = x
    ++j
    return

  when 2 then (x, y) ->
    array[i++] = x
    array[i++] = y
    ++j
    return

  when 3 then (x, y, z) ->
    array[i++] = x
    array[i++] = y
    array[i++] = z
    ++j
    return

  when 4 then (x, y, z, w) ->
    array[i++] = x
    array[i++] = y
    array[i++] = z
    array[i++] = w
    ++j
    return

Both the expression and emitter will be inlined into the stream's iteration loop.

consume = switch channels
  when 1 then (emit) ->
    emit array[i++]
    ++j
    return

  when 2 then (emit) ->
    emit array[i++], array[i++]
    ++j
    return

  when 3 then (emit) ->
    emit array[i++], array[i++], array[i++]
    ++j
    return

  when 4 then (emit) ->
    emit array[i++], array[i++], array[i++], array[i++]
    ++j
    return

Closures of Hanoi

(4-in-1)²

GPUs can operate on 4×1 vectors and 4×4 matrices, so working with 4D values is natural. Values can also be referenced by 4D indices. With one dimension reserved for the tuples, that leaves us 3 dimensions XYZ. Hence MathBox arrays are 3+1D. This is for width, height, depth, while the tuple dimension is called items. It does what it says on the tin, creating 1D W, 2D W×H and 3D W×H×D arrays of tuples. Each tuple is made of N vectors of up to 4 channels each.

Thanks to cyclic buffers and partial updates, history also comes baked in. You can use a spare dimension as a free time axis, retaining samples on the go. You can .set('history', N) to record a short log of a whole array over time, indefinitely.

All of this is modular: a data source is something that can be sampled by a 4D pointer from GLSL. Underneath, arrays end up packed into a regular 2D float texture, with "items × width" horizontally and "height × depth" vertically. Each 'pixel' holds a 1/2/3/4D point.

Mapping a 4D 'pointer' to the real 2D UV coordinates is just arithmetic, and so are operators like transpose and repeat. You just swap the XY indices and tell everyone downstream that it's now this big instead. They can't tell the difference.

You can create giant procedural arrays this way, including across rectangular texture size limits, as none of them actually exist except as transient values deep inside a GPU core. Until you materialize them by rendering to a texture using the memo primitive. Add in operators like interpolation and convolution and it's a pretty neat real-time finishing kit for data.

Too many WebGL contexts

Continued in Part 2.

↧

ShaderGraph 2

September 26, 2015, 3:00 pm

≫ Next: Yak Shading

≪ Previous: MathBox²

Functional GLSL

For MathBox 1, I already needed to generate GL shaders programmatically. So I built ShaderGraph. You gave it snippets of GLSL code, each with a function inside. It would connect them for you, matching up the inputs and outputs. It supported directed graphs of calls with splits and joins, which were compiled down into a single shader. To help build up the graph progressively, it came with a simple chainable factory API.

It worked despite being several steps short of being a real compiler and having gaps in its functionality. It also committed the cardinal sin of regex code parsing, and hence accepted only a small subset of GLSL. All in all it was a bit of a happy mess, weaving vertex and fragment shaders together in a very ad-hoc fashion. Each snippet could only appear once in a shader, as it was still just a dumb code concatenator. I needed a proper way to compose shaders.

Select a node to view its code

Instanced Data Flow

Enter ShaderGraph 2. It's a total rewrite using Chris Dickinson's bona fide glsl-parser. It still parses snippets and connects them into a directed graph to be compiled. But a snippet is now a full GLSL program whose main() function can have open inputs and outputs. What's more, it now also links code in the proper sense of the word: linking up module entry points as callbacks.

Basically, snippets can now have inputs and outputs that are themselves functions. These connections don't obey the typical data flow of a directed graph and instead are for function calls. A callback connection provides a path along which calls are made and values are returned.

Snippets can be instanced multiple times, including their uniforms, attributes and varyings (if requested). Uniforms are bound to Three.js-style registers as you build the graph incrementally. So it's a module system, sort of, which enables functional shader building. Using callbacks as micro-interfaces feels very natural in practice, especially with bound parameters. You can decorate existing functions, e.g. turning a texture sampler into a convolution filter.

// Build shader graph
var shader = shadergraph.shader();
shader
  .callback()
    .pipe('sampleColor')
    .fan()
      .pipe('sepiaColor')
    .next()
      .pipe('invertColor')
    .join()
    .pipe('combineColors')
  .join()
  .pipe('convolveColor');

GLSL Composer

If you know GLSL, you can write ShaderGraph snippets: there is no extra syntax, you just add inputs and outputs to your main() function. You can use in/out/inout qualifiers or return a value. If there's no main function, the last defined function is exported.

vec3 callback(vec3 arg1, vec3 arg2);

To create a callback input in a snippet, you declare a function prototype in GLSL without a body. The function name and signature is used to create the outlet.

To create a callback output, you use the factory API. You can .require() a snippet directly, or bundle up a subgraph with .callback().….join(). In the latter case, the function signature includes all unconnected inputs and outputs inside. Outlets are auto-matched by name, type and order, with the semantics from v1 cleaned up.

Building basic pipes is easy: .pipe(…).pipe(…).…, passing in a snippet or factory. For forked graphs, you can .fan() (1-to-N) or .split() (N-to-N), use .next() to begin a new branch, and then .join() at the end. There's a few other operations, nothing crazy.

var v = shadergraph.shader();

// Graphs generated elsewhere
v.pipe(vertexColor(color, mask));
v.require(vertexPosition(position, material, map, 2, stpq));

v.pipe('line.position',    uniforms, defs);
v.pipe('project.position', uniforms);

By connecting pairs you create a functional data flow that compiles down to vanilla GLSL. It's not functional programming in GLSL, it just enables useful run-time assembly patterns, letting the snippets do the heavy lifting the old fashioned way.

As GPUs are massively parallel pure function applicators, the resulting mega-shaders are a great fit.

`$ cat *.glsl | magic`

The process still comes down to concatenating the code in a clever way, with global symbols namespaced to be unique. Function bodies are generated to call snippets in the right order, and the callbacks are linked. In the trivial case it links a callback by #defineing the two symbols to be the same. It can also impedance match compatible signatures like void main(in float, out vec2) and vec2 main(float) by inserting an intermediate call.

precision highp float;
precision highp int;
uniform mat4 modelMatrix;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
uniform mat4 viewMatrix;
uniform mat3 normalMatrix;
uniform vec3 cameraPosition;
#define _sn_191_getPosition _pg_103_
#define _sn_190_getPosition _pg_102_
#define _sn_189_getSample _pg_100_
#define _pg_99_ _sn_185_warpVertex
#define _pg_103_ _sn_190_getMeshPosition
#define _pg_100_ _sn_188_getTransitionSDFMask
#define _pg_101_ _sn_189_maskLevel
vec2 _sn_180_truncateVec(vec4 v) { return v.xy; }
uniform vec2 _sn_181_dataResolution;
uniform vec2 _sn_181_dataPointer;

vec2 _sn_181_map2DData(vec2 xy) {
  return fract((xy + _sn_181_dataPointer) * _sn_181_dataResolution);
}

uniform sampler2D _sn_182_dataTexture;

vec4 _sn_182_sample2D(vec2 uv) {
  return texture2D(_sn_182_dataTexture, uv);
}

vec4 _sn_183_swizzle(vec4 xyzw) {
  return vec4(xyzw.x, xyzw.w, 0.0, 0.0);
}
uniform float _sn_184_polarBend;
uniform float _sn_184_polarFocus;
uniform float _sn_184_polarAspect;
uniform float _sn_184_polarHelix;

uniform mat4 _sn_184_viewMatrix;

vec4 _sn_184_getPolarPosition(vec4 position, inout vec4 stpq) {
  if (_sn_184_polarBend > 0.0) {

    if (_sn_184_polarBend < 0.001) {
      
      
      
      
      vec2 pb = position.xy * _sn_184_polarBend;
      float ppbbx = pb.x * pb.x;
      return _sn_184_viewMatrix * vec4(
        position.x * (1.0 - _sn_184_polarBend + (pb.y * _sn_184_polarAspect)),
        position.y * (1.0 - .5 * ppbbx) - (.5 * ppbbx) * _sn_184_polarFocus / _sn_184_polarAspect,
        position.z + position.x * _sn_184_polarHelix * _sn_184_polarBend,
        1.0
      );
    }
    else {
      vec2 xy = position.xy * vec2(_sn_184_polarBend, _sn_184_polarAspect);
      float radius = _sn_184_polarFocus + xy.y;
      return _sn_184_viewMatrix * vec4(
        sin(xy.x) * radius,
        (cos(xy.x) * radius - _sn_184_polarFocus) / _sn_184_polarAspect,
        position.z + position.x * _sn_184_polarHelix * _sn_184_polarBend,
        1.0
      );
    }
  }
  else {
    return _sn_184_viewMatrix * vec4(position.xyz, 1.0);
  }
}
uniform float _sn_185_time;
uniform float _sn_185_intensity;

vec4 _sn_185_warpVertex(vec4 xyzw, inout vec4 stpq) {
  xyzw +=   0.2 * _sn_185_intensity * (sin(xyzw.yzwx * 1.91 + _sn_185_time + sin(xyzw.wxyz * 1.74 + _sn_185_time)));
  xyzw +=   0.1 * _sn_185_intensity * (sin(xyzw.yzwx * 4.03 + _sn_185_time + sin(xyzw.wxyz * 2.74 + _sn_185_time)));
  xyzw +=  0.05 * _sn_185_intensity * (sin(xyzw.yzwx * 8.39 + _sn_185_time + sin(xyzw.wxyz * 4.18 + _sn_185_time)));
  xyzw += 0.025 * _sn_185_intensity * (sin(xyzw.yzwx * 15.1 + _sn_185_time + sin(xyzw.wxyz * 9.18 + _sn_185_time)));

  return xyzw;
}



vec4 _sn_186_getViewPosition(vec4 position, inout vec4 stpq) {
  return (viewMatrix * vec4(position.xyz, 1.0));
}

vec3 _sn_187_getRootPosition(vec4 position, in vec4 stpq) {
  return position.xyz;
}
vec3 _pg_102_(vec4 _io_510_v, in vec4 _io_519_stpq) {
  vec2 _io_509_return;
  vec2 _io_511_return;
  vec4 _io_513_return;
  vec4 _io_515_return;
  vec4 _io_517_return;
  vec4 _io_520_stpq;
  vec4 _io_527_return;
  vec4 _io_528_stpq;
  vec4 _io_529_return;
  vec4 _io_532_stpq;
  vec3 _io_533_return;

  _io_509_return = _sn_180_truncateVec(_io_510_v);
  _io_511_return = _sn_181_map2DData(_io_509_return);
  _io_513_return = _sn_182_sample2D(_io_511_return);
  _io_515_return = _sn_183_swizzle(_io_513_return);
  _io_520_stpq = _io_519_stpq;
  _io_517_return = _sn_184_getPolarPosition(_io_515_return, _io_520_stpq);
  _io_528_stpq = _io_520_stpq;
  _io_527_return = _pg_99_(_io_517_return, _io_528_stpq);
  _io_532_stpq = _io_528_stpq;
  _io_529_return = _sn_186_getViewPosition(_io_527_return, _io_532_stpq);
  _io_533_return = _sn_187_getRootPosition(_io_529_return, _io_532_stpq);
  return _io_533_return;
}
uniform vec4 _sn_190_geometryResolution;

#ifdef POSITION_STPQ
varying vec4 vSTPQ;
#endif
#ifdef POSITION_U
varying float vU;
#endif
#ifdef POSITION_UV
varying vec2 vUV;
#endif
#ifdef POSITION_UVW
varying vec3 vUVW;
#endif
#ifdef POSITION_UVWO
varying vec4 vUVWO;
#endif


vec3 _sn_190_getMeshPosition(vec4 xyzw, float canonical) {
  vec4 stpq = xyzw * _sn_190_geometryResolution;
  vec3 xyz = _sn_190_getPosition(xyzw, stpq);

  #ifdef POSITION_MAP
  if (canonical > 0.5) {
    #ifdef POSITION_STPQ
    vSTPQ = stpq;
    #endif
    #ifdef POSITION_U
    vU = stpq.x;
    #endif
    #ifdef POSITION_UV
    vUV = stpq.xy;
    #endif
    #ifdef POSITION_UVW
    vUVW = stpq.xyz;
    #endif
    #ifdef POSITION_UVWO
    vUVWO = stpq;
    #endif
  }
  #endif
  return xyz;
}

uniform float _sn_188_transitionEnter;
uniform float _sn_188_transitionExit;
uniform vec4  _sn_188_transitionScale;
uniform vec4  _sn_188_transitionBias;
uniform float _sn_188_transitionSkew;
uniform float _sn_188_transitionActive;

float _sn_188_getTransitionSDFMask(vec4 stpq) {
  if (_sn_188_transitionActive < 0.5) return 1.0;

  float enter   = _sn_188_transitionEnter;
  float exit    = _sn_188_transitionExit;
  float skew    = _sn_188_transitionSkew;
  vec4  scale   = _sn_188_transitionScale;
  vec4  bias    = _sn_188_transitionBias;

  float factor  = 1.0 + skew;
  float offset  = dot(vec4(1.0), stpq * scale + bias);

  vec2 d = vec2(enter, exit) * factor + vec2(-offset, offset - skew);
  if (exit  == 1.0) return d.x;
  if (enter == 1.0) return d.y;
  return min(d.x, d.y);
}
uniform float _sn_191_worldUnit;
uniform float _sn_191_lineWidth;
uniform float _sn_191_lineDepth;
uniform float _sn_191_focusDepth;

uniform vec4 _sn_191_geometryClip;
attribute vec2 line;
attribute vec4 position4;

#ifdef LINE_PROXIMITY
uniform float _sn_191_lineProximity;
varying float vClipProximity;
#endif

#ifdef LINE_STROKE
varying float vClipStrokeWidth;
varying float vClipStrokeIndex;
varying vec3  vClipStrokeEven;
varying vec3  vClipStrokeOdd;
varying vec3  vClipStrokePosition;
#endif


#ifdef LINE_CLIP
uniform float _sn_191_clipRange;
uniform vec2  _sn_191_clipStyle;
uniform float _sn_191_clipSpace;

attribute vec2 strip;

varying vec2 vClipEnds;

void _sn_191_clipEnds(vec4 xyzw, vec3 center, vec3 pos) {

  
  vec4 xyzwE = vec4(strip.y, xyzw.yzw);
  vec3 end   = _sn_191_getPosition(xyzwE, 0.0);

  
  vec4 xyzwS = vec4(strip.x, xyzw.yzw);
  vec3 start = _sn_191_getPosition(xyzwS, 0.0);

  
  vec3 diff = end - start;
  float l = length(diff) * _sn_191_clipSpace;

  
  float arrowSize = 1.25 * _sn_191_clipRange * _sn_191_lineWidth * _sn_191_worldUnit;

  vClipEnds = vec2(1.0);

  if (_sn_191_clipStyle.y > 0.0) {
    
    float depth = _sn_191_focusDepth;
    if (_sn_191_lineDepth < 1.0) {
      float z = max(0.00001, -end.z);
      depth = mix(z, _sn_191_focusDepth, _sn_191_lineDepth);
    }
    
    
    float size = arrowSize * depth;

    
    
    float mini = clamp(1.0 - l / size * .333, 0.0, 1.0);
    float scale = 1.0 - mini * mini * mini; 
    float invrange = 1.0 / (size * scale);
  
    
    diff = normalize(end - center);
    float d = dot(end - pos, diff);
    vClipEnds.x = d * invrange - 1.0;
  }

  if (_sn_191_clipStyle.x > 0.0) {
    
    float depth = _sn_191_focusDepth;
    if (_sn_191_lineDepth < 1.0) {
      float z = max(0.00001, -start.z);
      depth = mix(z, _sn_191_focusDepth, _sn_191_lineDepth);
    }
    
    
    float size = arrowSize * depth;

    
    
    float mini = clamp(1.0 - l / size * .333, 0.0, 1.0);
    float scale = 1.0 - mini * mini * mini; 
    float invrange = 1.0 / (size * scale);
  
    
    diff = normalize(center - start);
    float d = dot(pos - start, diff);
    vClipEnds.y = d * invrange - 1.0;
  }


}
#endif

const float _sn_191_epsilon = 1e-5;
void _sn_191_fixCenter(vec3 left, inout vec3 center, vec3 right) {
  if (center.z >= 0.0) {
    if (left.z < 0.0) {
      float d = (center.z - _sn_191_epsilon) / (center.z - left.z);
      center = mix(center, left, d);
    }
    else if (right.z < 0.0) {
      float d = (center.z - _sn_191_epsilon) / (center.z - right.z);
      center = mix(center, right, d);
    }
  }
}


void _sn_191_getLineGeometry(vec4 xyzw, float edge, out vec3 left, out vec3 center, out vec3 right) {
  vec4 delta = vec4(1.0, 0.0, 0.0, 0.0);

  center =                 _sn_191_getPosition(xyzw, 1.0);
  left   = (edge > -0.5) ? _sn_191_getPosition(xyzw - delta, 0.0) : center;
  right  = (edge < 0.5)  ? _sn_191_getPosition(xyzw + delta, 0.0) : center;
}

vec3 _sn_191_getLineJoin(float edge, bool odd, vec3 left, vec3 center, vec3 right, float width) {
  vec2 join = vec2(1.0, 0.0);

  _sn_191_fixCenter(left, center, right);

  vec4 a = vec4(left.xy, right.xy);
  vec4 b = a / vec4(left.zz, right.zz);

  vec2 l = b.xy;
  vec2 r = b.zw;
  vec2 c = center.xy / center.z;

  vec4 d = vec4(l, c) - vec4(c, r);
  float l1 = dot(d.xy, d.xy);
  float l2 = dot(d.zw, d.zw);

  if (l1 + l2 > 0.0) {
    
    if (edge > 0.5 || l2 == 0.0) {
      vec2 nl = normalize(d.xy);
      vec2 tl = vec2(nl.y, -nl.x);

#ifdef LINE_PROXIMITY
      vClipProximity = 1.0;
#endif

#ifdef LINE_STROKE
      vClipStrokeEven = vClipStrokeOdd = normalize(left - center);
#endif
      join = tl;
    }
    else if (edge < -0.5 || l1 == 0.0) {
      vec2 nr = normalize(d.zw);
      vec2 tr = vec2(nr.y, -nr.x);

#ifdef LINE_PROXIMITY
      vClipProximity = 1.0;
#endif

#ifdef LINE_STROKE
      vClipStrokeEven = vClipStrokeOdd = normalize(center - right);
#endif
      join = tr;
    }
    else {
      
      float lmin2 = min(l1, l2) / (width * width);

      
#ifdef LINE_PROXIMITY
      float lr     = l1 / l2;
      float rl     = l2 / l1;
      float ratio  = max(lr, rl);
      float thresh = _sn_191_lineProximity + 1.0;
      vClipProximity = (ratio > thresh * thresh) ? 1.0 : 0.0;
#endif
      
      
      vec2 nl = normalize(d.xy);
      vec2 nr = normalize(d.zw);

      vec2 tl = vec2(nl.y, -nl.x);
      vec2 tr = vec2(nr.y, -nr.x);

#ifdef LINE_PROXIMITY
      
      vec2 tc = normalize(mix(tl, tr, l1/(l1+l2)));
#else
      
      vec2 tc = normalize(tl + tr);
#endif
    
      float cosA   = dot(nl, tc);
      float sinA   = max(0.1, abs(dot(tl, tc)));
      float factor = cosA / sinA;
      float scale  = sqrt(1.0 + min(lmin2, factor * factor));

#ifdef LINE_STROKE
      vec3 stroke1 = normalize(left - center);
      vec3 stroke2 = normalize(center - right);

      if (odd) {
        vClipStrokeEven = stroke1;
        vClipStrokeOdd  = stroke2;
      }
      else {
        vClipStrokeEven = stroke2;
        vClipStrokeOdd  = stroke1;
      }
#endif
      join = tc * scale;
    }
    return vec3(join, 0.0);
  }
  else {
    return vec3(0.0);
  }

}

vec3 _sn_191_getLinePosition() {
  vec3 left, center, right, join;

  float edge = line.x;
  float offset = line.y;

  vec4 p = min(_sn_191_geometryClip, position4);
  edge += max(0.0, position4.x - _sn_191_geometryClip.x);

  
  _sn_191_getLineGeometry(p, edge, left, center, right);

#ifdef LINE_STROKE
  
  vClipStrokePosition = center;
  vClipStrokeIndex = p.x;
  bool odd = mod(p.x, 2.0) >= 1.0;
#else
  bool odd = true;
#endif

  
  float width = _sn_191_lineWidth * 0.5;

  float depth = _sn_191_focusDepth;
  if (_sn_191_lineDepth < 1.0) {
    
    float z = max(0.00001, -center.z);
    depth = mix(z, _sn_191_focusDepth, _sn_191_lineDepth);
  }
  width *= depth;

  
  width *= _sn_191_worldUnit;

  join = _sn_191_getLineJoin(edge, odd, left, center, right, width);

#ifdef LINE_STROKE
  vClipStrokeWidth = width;
#endif
  
  vec3 pos = center + join * offset * width;

#ifdef LINE_CLIP
  _sn_191_clipEnds(p, center, pos);
#endif

  return pos;
}

uniform vec4 _sn_189_geometryResolution;
uniform vec4 _sn_189_geometryClip;
varying float vMask;


void _sn_189_maskLevel() {
  vec4 p = min(_sn_189_geometryClip, position4);
  vMask = _sn_189_getSample(p * _sn_189_geometryResolution);
}

uniform float _sn_192_styleZBias;
uniform float _sn_192_styleZIndex;

void _sn_192_setPosition(vec3 position) {
  vec4 pos = projectionMatrix * vec4(position, 1.0);

  
  float bias  = (1.0 - _sn_192_styleZBias / 32768.0);
  pos.z *= bias;
  
  
  if (_sn_192_styleZIndex > 0.0) {
    float z = pos.z / pos.w;
    pos.z = ((z + 1.0) / (_sn_192_styleZIndex + 1.0) - 1.0) * pos.w;
  }
  
  gl_Position = pos;
}
void main() {
  vec3 _io_546_return;

  _io_546_return = _sn_191_getLinePosition();
  _sn_192_setPosition(_io_546_return);
  _pg_101_();
}

It still does guarded regex manipulation of code too, but those manipulations are now derived from a proper syntax tree. GLSL doesn't have strings and its scope is simple, so this is unusually safe. I'm sure you can still trip it up somehow, but it's worth it for speed. I'm seeing assembly times of ~10-30ms cold, 2-4ms warm, but it depends entirely on the particular shaders.

The assembly process is now properly recursive. Unassembled shaders can be used in factory form, standing in for snippets. Completed graphs form stand-alone programs with no open inputs or outputs. The result can be turned straight into a Three.js ShaderMaterial, but there is no strict Three dependency. It's just a dictionary with code and a list of uniforms, attributes and varyings. Unlike before, building a combined vertex/fragment program is now merely syntactic sugar for a pair of separate graphs.

As it's run-time, you can slot in user-defined or code-generated GLSL just the same. Shaders are fetched by name or passed as inline code, mixed freely as needed. You supply the dictionary or lookup method. You could bundle your GLSL into JS with a build step or include embedded <script> tags.

This is the fragment shader that implements the partial differential equation for this ripple effect (getFramesSample). It samples from a volumetric N×N×2 array, feeding back into itself.

Paging Dr. Hickey

ShaderGraph 2 drives the entirety of MathBox 2. Its shaders are specialized for particular types and dimensions, generating procedural data, clipping geometry, resampling transformed data on the fly, …. The composibility comes out naturally. To do so, I pass a partially built factory by interested parties. This way I build graphs for position, color, normal, mask and more. These are injected as callbacks into a final shader. Shader factories enable ad-hoc contracts, sandwiched between the inner and outer retained layers of Three.js and MathBox, but disappearing entirely in the end result.

Of course, all of this is meta-programming of GLSL, done through a stateful JS lasagna and a ghetto compiler, instead of an idiomatic language. I know this, it's an inner platform effect bathing luxuriously in Turing tar like a rhino in mud. I didn't really see a way around it, given the constraints at play.

While the factory API is designed for making graphs on the spot and then tossing them, you could keep graphs around. There's a full data model underneath. You can always skip the factory entirely.

Plenty of caveats of course. There is no built-in preprocessor, so you can't #define or #ifdef uniforms or attributes and have it make sense. But then the point of ShaderGraph is to formalize exactly that sort of ad-hoc fiddling. Preprocessor directives will just pass through. glsl-parser has gaps too, and it is also exceedingly picky with reserved variable names, so watch out for that.

I did sometimes feel the need for more powerful metaprogramming, but you can work around it. It is easy to dynamically make GLSL one-liner snippets and feed them in. String manipulation of code is always still an option, you just don't need to do it at the macro-level anymore.

ShaderGraph 2 has been in active use now for months, it does the job I need it to very well. In a perfect world, this would be solved at the GPU driver level. Until SPIR-V or WebVulkan gets here, imma stick to my regexes. Don't try this at home, kids.

For docs and more, see the Git repository.

↧

Yak Shading

September 26, 2015, 3:00 pm

≫ Next: Occupy WWW Street (EN)

≪ Previous: ShaderGraph 2

Data-Driven Geometry

MathBox primitives need to take arbitrary data, transform it on the fly, and render it as styled geometry based on their attributes. Done as much as possible on the graphics hardware.

Three.js can render points, lines, triangles, but only with a few predetermined strategies. The alternative is to write your own vertex and fragment shader and do everything from scratch. Each new use case means a new ShaderMaterial with its own properties, so called uniforms. If the stock geometry doesn't suffice, you can make your own triangles by filling a raw BufferGeometry and assign custom per-vertex attributes. Essentially, to leverage GPU computation with Three.js—most engines, really—you have to ignore most of it.

Virtual Geometry

Shader computations are mainly rote transforms. For example, if you want to draw a line between two points, you'll have to make a long rectangle, made out of two triangles. But this simple idea gets complicated quickly once you add corner joins, depth scaling, 3D clipping, and so on. Doing this to an entire data set at once is what GPUs are made for, through vertex shaders which transform points.

Vertex shaders can only do 1-to-1 mappings. This isn't a problem by itself. You can use a gather approach to do N-to-1 mapping, where all the necessary data is pre-arranged into attribute arrays, with the data repeated and interleaved per vertex as necessary.

The proper tool for this is a geometry shader: a program that creates new geometry by N-to-M mapping of data, like making triangles out of points. WebGL doesn't support geometry shaders, won't any time soon, but you can emulate them with texture samplers. A texture image is just a big typed array, and you have random access unlike vertex attributes.

The original geometry acts only as a template, directing the shader's real data lookups. You lose some performance this way, but it's not too bad. Any procedural sampling pattern works, drawing 1 shape or 10,000. As textures can be rendered to, not just from, this also enables transform feedback, using the result of one pass to create new geometry in another.

All geometry rendered this way is 100% static as far as Three.js is concerned. New values are uploaded directly to GPU memory just before the rendering starts. The only gotcha is handling variable size input, because reallocation is costly. Pre-allocating a larger texture is easy, but clipping off the excess geometry in an O(1) fashion on the JS side is hard. In most cases there's the work around of dynamically generating degenerate triangles in a shader, which collapse down to invisible edges or points. This way, MathBox can accept variable sized arrays in multiple dimensions and will do its best to minimize disruption. If attribute instancing was more standard in WebGL, this wouldn't be such an issue, but as it stands, the workarounds are very necessary.

Vertex Party

If you squint very hard it looks a bit like React for live geometry. Except instead of a diffing algorithm, there's a few events, some texture uploads, a handful of draw calls and then an idle CPU. It's ideal for drawing thousands of things that look similar and follow the same rules. It can handle not just basic GL primitives like lines or triangles, but higher level shapes like 3D arrows or sprites.

My first prototype of this was my last christmas demo. It was messy and tedious to make, especially the shaders, but it performed excellently: the final scene renders ~200,000 triangles. Despite being a layer around Three.js … around WebGL … around OpenGL … around a driver … around a GPU … performance has far exceeded my expectations. Even complex scenes run great on my Android phone, easily 10x faster than MathBox 1, in some cases more like 1000x.

Of course compared to cutting edge DirectX or OpenCL (not a typo), this is still very limited. In today's GPUs, the charade of attributes, geometries, vertices and samples has mostly been stripped away. What remains is buffers and massive operations on them, exposed raw in new APIs like AMD's Mantle and iOS's Metal. My vertex trickery acts like a polyfill, virtualizing WebGL's capabilities to bring them closer to the present. It goes a bit beyond what geometry shaders can provide, but still lacks many useful things like atomic append queues or stream compaction.

For large geometries, the set up cost can be noticeable though. Shader compilation time also grows with transform complexity, doubly so on Windows where shaders are recompiled to HLSL / Direct3D. This makes drawing ops the heaviest MathBox primitives to spawn and reallocate. You could call this the MathBox version of the dreaded 'paint' of HTML. Once warmed up though, most other properties can be animated instantly, including the data being displayed: this is the opposite of how HTML works. Hence you can mostly spawn things ahead of time, revealing and hiding objects as needed, with minimal overhead and jank at runtime.

This all relies on carefully constructed shaders which have to be wired up in all their individual permutations. This needed to be solved programmatically, which is where we go last.

↧

Occupy WWW Street (EN)

November 29, 2015, 3:00 pm

≫ Next: Occupy WWW Street (FI)

≪ Previous: Yak Shading

Internet Activism and Media in the Age of Social Justice

English - Français - Nederlands

"I don't have access to any unusual methods of analysis and what special knowledge I have concerning language has no bearing on social and political issues. […] For the analysis of ideology […] a bit of open-mindedness, normal intelligence, and healthy skepticism will generally suffice." – NC

I'm bringing out my serious font again. It's difficult to talk about current affairs without inviting in heaps of context, so I'd like to delve into some recent history.

Reminder, shooting the messenger is poor form, and heresy is medieval.

(🇫🇮 Finnish readers: unarchived links)

The Only Working Class Man In The Village

It's September 2011, the American left has been galvanized. The message is straightforward and aimed at big targets: the 99% of the public is telling the 1% scrooges to stop being greedy and not use corporations as a club. Zuccotti Park in New York fills with activists camping out, discussing and spreading their message.

Cue Jesse LaGreca. With his quintessential American Joe Average look, he takes most folks by surprise, most certainly the Fox News reporter who approaches him. In the impromptu interview, unaired by Fox, LaGreca schools the man and powers through, knocking it out of the park. He calls out the leading questions right away, and notes that if there's one thing the news—"the propaganda machine in the media"—doesn't wish to report on, it's itself. That maybe it's time to have some participatory democracy that "isn't funded by news cameras."

A reporter shows up expecting to report on the world and discovers this story is, at least in part, about themselves. The denial is quick: "We’re here giving you an opportunity on the record […] to put any message you want out there, to give you fair coverage […] So, there is an exception in the case, because you wouldn’t be able to get your message out there without us."

He snaps back: "Certainly. Take for instance when Glenn Beck […] called the president a person who hates white people and white culture. That was a low moment in Americans' history and you guys kind of had a big part in it. So I'm glad you're coming around and kind of paying attention to the [rest of the country] as opposed to the far right fringe who would just prefer to destroy the middle class entirely. […] People would like to see a little bit more economic justice, or social justice, you know, Jesus stuff, as far as feeding the poor, healthcare for the sick. […] I find it really entertaining that people like to hold the bill of rights up while they're screaming at gay soldiers, but they can't wrap their heads around the fact that a for-profit health care system doesn't work. […] If we want the president to do more, let's talk to him on a level that works instead of asking for his birth certificate, […]"

Shots fired, critical hit, no chance of recovery. It's no surprise it was left on the cutting room floor, and no surprise it would leak and shoot to fame as being too extreme for Fox news. Predictable headlines asked if he was the The Smartest Man on Wall Street and explained that Jesse LaGreca Continues to Destroy Media Bias of Occupy Wall Street. All reporters eagerly falling over themselves to demonstrate how they were willing to give anyone a voice.

This was an angle LaGreca himself brought up on air: "The reality is that I’m the only working class person you’re going to see on Sunday news… political news… maybe ever. And I think that is very indicative of the failures of our media to report on the news that matter most importantly…" The host immediately cuts in, "We are trying our best, Jesse."

However when asked if there was a political demand that could sum up Occupy, something to "make this about," he takes a curious turn:

"I think the entire movement is about economic justice. I mean to me – and I’m not speaking on behalf of Occupy Wall Street, I’m just giving my personal opinion – I think it’s a matter of economic rights, and I think it’s a matter of social rights, and social justice. And to the people who would take offense to the word “social” being placed before the word “justice,” I’d invite them to re-read the Constitution."

Ok. I went through the trouble and enacted the labor to educate myself. The United States Constitution makes no references to anything social with regards to justice. Those who wrote it seemed to think that merely "establishing Justice" was plenty enough. It does not rank citizens by privilege, does not define powers in relation to oppression and makes no allusions to punching up. That's what social justice is nowadays, right? Or did he mean the "Jesus stuff"? I'm no constitutional lawyer, feel free to correct me.

More awkwardly, on the subject of taking their cues from other protests, in Greece and Europe and elsewhere, LaGreca claimed "we didn’t take our cue leading off of anybody really."

Occupy may have grown organically from existing groups, but it was jumpstarted by an early blog post and a message blast to Adbusters' subscriber base of "you 90,000 redeemers, rebels and radicals out there," declaring a "worldwide shift in revolutionary tactics is underway," asking if they were ready to have their own "Tahrir moment." The 267 comments show this message was received with enthusiasm and commitment.

LaGreca also wasn't a nobody, he'd been a Daily Kos contributor for three years already. The site's community cheered him on for "not [fitting] into the media’s stereotype of the obtusely disenfranchised ne0-hippies" and helped get the word out while commenting on the spectacle: "My guess is that Hannity is scared. As he should be!"

This is not an indictment of Mr. LaGreca, he mainly talked a lot of sense and said it well. However, four years have passed. Despite the generous airtime afforded by his newfound fame-by-Streisand and the stated desire from reporters to do better, the situation did not improve, not with Fox News and not with the other outlets who define themselves through opposition.

If there's one thing the media loves, it's the exception that confirms the rule: you cannot galvanize the public without access to existing networks and platforms. When you go up against the media, they will repackage you into an entertainment product, and the masses will love it. You might love it too.

Activists Are Dead

So what happened? In June 2012, community hub and non-profit OccupyWallSt.org posts that Reports of Occupy’s Death Have Been Greatly Exaggerated.

"Since the eviction of the protestors at Freedom Plaza last November, it’s become a media cliché to report on the “Death of Occupy.” Articles pop up all over the web, blithely reporting on the failed second wind of Occupy, this lackluster “American Spring,” and the May Day general strike that didn’t quite shut the system down.

It should be no surprise that the mainstream media is eager to report on Occupy’s supposed demise. Even ignoring the fact that the corporate-owned media has a strong desire to never see social movements such as Occupy succeed, the media, as a rule, generally needs to put a dramatic narrative to everything it reports. To them, every story ought to have a captivating story arch with a beginning, middle, and an end."

"In the media’s eyes […] many people heeded the call, [yet] the movement only received its dramatic momentum when cops were photographed attacking and pepper-spraying the nonviolent protestors. It reached its early demise when the police violently cleared out the various encampments. Now, except for a few curmudgeons who can’t seem to understand that Occupy is over, all that remains of Occupy is its populist rhetoric of the 99%—which has been dutifully hawked up by Democratic front-groups such as MoveOn.org to help refuel the Obama election machine."

So much for that then. What's on TV tonight? A presidential debate?

"This popular narrative […] has been so successful that even those who are still active within the Occupy movement can’t help but absorb parts of it. […Meetings] sometimes border on something closely resembling a public support group. On the internet, vaguely self-congratulatory Paul Krugman-y articles, applauding Occupy for “at least shifting the public dialogue,” are posted and reposted […] to remind each other that Occupy at least had a little bit of an effect."

Whelp, that sounds like fun. The post continues to explain at length how Americans see history as "something other people (usually white, upper class men) did long ago" and forget it "took hundreds of thousands of people—especially young people, women, and working class men–to support and spread the ideas of democracy. […] [We] see Occupy as a spectacle […] by people very unlike ourselves. […] Occupy-friendly celebrities and artists, black block style anarchists, and our cities’ despotic mayors are the characters in this drama, [battling it out] for headlines on the stage of our trash-strewn cities."

"The true magic of Occupy was that it rejected all of these things. No one had any more power than anyone else, […] nobody in Occupy really cared that we were ignored by the mainstream media. We don’t need a bunch of hacks at Time Magazine to commend us for our ability to protest. […] The goal of Occupy was to get together as a community of equals, to claim a future different than the ones they gave us, and to reignite a tradition of democratic progress […] It was a way of exploring new ways of interacting with others."

1 2 3 4 5

The author insists we should not "[shrug] off Occupy as a momentary fad or a leftist pipedream." Ok. Aside from a "feeling in the air", the listed effects are that "over 7,200 Occupy protestors have been arrested in the United States" with many "beaten and tortured." Occupy has been demonized because "the media has been strong-armed into reporting […] in an unfavorable light." What else?

"It needs to be remembered that the word “occupy” is […] a call to action, not the action itself. [It] was useful for getting individuals and organizations previously isolated or focused on one-issue grievances out into the streets. […] What is important is that there’s wide community of opposition being formed across many social barriers, and those who hold power are very afraid."

Ignore the posturing at the end, notice something far more telling.

The stated result of Occupy, in a post meant to prove its lasting value, is that 7,200 people made victims and martyrs out of themselves, sustained by passive idealistic resistance from abroad. They have slid into depressed irrelevance, misrepresented by those who should report in the public interest. They were ok with this.

The best lies are half truths. This collective totally-not-mourning ritual was a kernel of rock-hard reality wrapped in narcissistic self-delusion. One might even call it Stockholm Syndrome. These are the evolved Pokémon variants of victim culture, where victimhood is celebrated, because it can offer immense power.

Burgers and Fries with Ketchup

I especially don't buy into the official Occupy story, either the media's or the movement's, because I went and saw for myself in Vancouver. I'm not talking about the 9/11 Truthers or 420 weed signs: they come with the territory, where else would they go? No, I noticed two groups in particular.

First, the starry eyed latté kids manning the human microphone as they called it—just like in Biscotti Park!—with matching friends too fashionably kept to be poor. It struck me as sinister: because of the Occupy-wide suppression practice of banning audio equipment, they were reduced to slowly repeating each other, chanting mantras. This was further sabotaged by a general lack of focus or moderation. Throw in a mix of people who have more feelings than they know what to do with, and it goes nowhere veeeery slowly, like Twitter in bullet time.

This sight was no public forum for enlightened debate, no town hall gathering, it was an obnoxious soapbox no ordinary person would listen to for more than 5 minutes. They had heard of activism, even knew what it looked and sounded like, but they merely went through the motions. Same on many livestreams I tuned into elsewhere.

Second, the actual homeless of the city: unemployed and unemployable, mentally ill, chronically addicted or otherwise rejected by society, mulling around the space they already occupy every day. Put both groups together, quelle surprise, high minded principles about equality, privilege, diversity, ableism, free expression and shared ownership are tested. Youth whose idea of camping is a summer festival with sanitized portapotties and water stations found a public square in winter less accommodating. Furthermore, when the inevitable conflict and crime occurs, those who reach for the words "triggered" and "rape culture" like candy are probably not the best arbiters of actual justice.

Instead of facing reality, they adopted the progressive stack, which "encourages women and traditionally marginalized groups [to] speak before men, especially white men." If you dig into the grapevine, you will find grumblings of "meetings derailed by women and minorities obsessed with identity politics. […] [Plans] were almost always met with objection from feminists, queer activists, blacks, latinos, et al. A lot of the time these people didn't even have specific points of concern. […] They just stood up and started complaining about lack of diversity, the victimization of their specific identity group, etc. […] Smart, able, organized white guys left the protest. They threw up their hands and walked away."

You can find many stories of in-fighting and abuse, it provided ample fodder for the media, social or otherwise. What did or did not occur at specific events was rarely considered relevant, and such concerns became more important than the financial and economic system or the institutions that sustain it. We apparently had to confront the gender-based violence in the Occupy movement: "Exactly what occurred is unclear. Some claim there was an attempted rape. Others shrug off the incident as nothing more than an invasion of a young woman’s personal space by an intoxicated man." Do you suppose that distinction might be important?

Socially-just media pushed the hyperbole, reminding men "don't rape anyone, ever"—thx xojane—and that "women-only spaces are built to expedite advanced conversations about issues that affect women (and, ideally, anyone who does not identify as a man) without these discussions being constantly interrupted by the need to educate other folks in the gathering."

So in other words… Men, stay out, shut up. Women, don't ask any questions. Great job social justice, you brought everything you claim to abhor into your idealistic little utopia, chasing off everyone else.

It's easy to point at the whacky fringe characters with funny hair and glasses as the culprits. If you're Stephen Colbert, you can even mercilessly mock them in front of millions and score lots of points. The real problem is the activist mass itself and the media they buy into. They're too eager to derail themselves over issues they themselves admit are just a microcosm of a human community. The mistake is in assigning universal importance to isolated incidents while ignoring the elephants in the room. That's a disease concentrated in the media, but not only there.

It's difficult to take someone's Important Conversation about Deconstructing Issues seriously, when social justice doesn't know how to socialize.

/Chomsky/ was Right

In Language and Responsibility (1979), Noam Chomsky instead converses eloquently. He mainly explains his theories of linguistics, too niche for most. However, the book's opening chapter "Politics" starts off quite differently, with ideas that would later return in Manufacturing Consent: The Political Economy of the Mass Media (1988). Asked how his methods relate to his more well known political writings:

"I don't have access to any unusual methods of analysis and what special knowledge I have concerning language has no bearing on social and political issues. […] For the analysis of ideology […] a bit of open-mindedness, normal intelligence, and healthy skepticism will generally suffice."

"Take [the] intelligentsia in a society like ours. This social class, which includes historians and other scholars, journalists, political commentators and so on, undertakes to analyze and present some picture of social reality. […] They serve as mediators between the social facts and the mass of the population: they create the ideological justification for social practice. Look at the work of the specialists in contemporary affairs and compare their interpretation with the events, compare what they say with the world of fact. You will often find great and fairly systematic divergences […]."

"With a little industry and application, anyone who is willing to extricate himself from the system […] will readily see through the modes of distortion […] Everybody is capable of doing that. […] Often carried out poorly, […] social and political analysis is produced to defend special interests rather than to account for actual events." (p3-4)

He goes on to talk about the particular situation in the United States. He compares recorded facts with the accepted explanations of events such as the McCarthyist scare, the student revolution, the civil rights movement, the Vietnam war, and more. He covers US interventions at home and abroad, legitimate or illegitimate. The overriding theme is rather obvious:

"Here in the United States there is an astonishing degree of ideological uniformity for such a complex country. Basically, there are two reasons for this. First, there is the remarkable ideological homogeneity of the American intelligentsia in general, who rarely depart from one of the variants of state capitalistic ideology (liberal or conservative), a fact which itself calls for explanation. The second is that the mass media are capitalist institutions. […] The fact that these institutions reflect the ideology of dominant economic interests is hardly surprising." (p9)

An editorial in the New York Times from 1975, a retrospective on the Vietnam War, explains there are those who think "the war […] could have been waged differently" and those who think "a viable non-Communist South Vietnam was always a myth." Chomksy goes all in:

"They don't even mention the logical possibility of a third position: namely, that the United States did not have […] the legal or moral right, to intervene by force in the internal affairs of Vietnam. […] Note that as the Times sets the spectrum of ideological debate, the position of much of the peace movement is simply excluded from consideration. Not that it is wrong, but rather unthinkable, inexpressible. […]

Here we have a marvellous illustration of the functioning of propaganda in a democracy. A totalitarian state simply enunciates official doctrine—clearly, explicitly. […] one can only express opposition at one's peril. In a democratic system of propaganda no one is punished (in theory) for objecting to official dogma. In fact, dissidence is encouraged.

What this system attempts to do is to fix the limits of possible thought: supporters of official doctrine at one end, and the critics—vigorous, courageous, and much admired for their independence of judgement—at the other. The hawks and the doves. But we discover they all share certain tacit assumptions, and that it is these assumptions that are really crucial.

No doubt a propaganda system is more effective when its doctrines are insinuated rather than asserted, […] The more vigorous the debate, the more effectively the basic doctrines of the propaganda system, tacitly assumed on all sides, are instilled. Hence the elaborate pretense that the press is a critical dissenting force—maybe even too critical for the health of democracy—when in fact it is almost entirely subservient to the basic principles of the ideological system." (p36-39)

This was the 70s. What might that look like in the internet age?

Don't Stalk Me Bro

In 2006, Gawker Media launched the Gawker Stalker Maps, a site where visitors were encouraged to share live updates of celebrity sightings in public. Hollywood stars found the usual gaggle of papparazzi joined by even weirder people from the internet, and were not amused.

When called to defend the practice a year later by Jimmy Kimmel, editor Emily Gould goes for the gold in mental gymnastics. Despite the original request for sightings "as quickly as you can send them in," she claims the map is generally slow to update. She laughs away evidence of publishing lies as being "citizen journalism" which "everybody knows […] isn't checked," despite much of the content being editorial. She trips and fails to catch herself, calling it "a stalking…heh…it's not actual stalking", because there is no expectation of not being noticed in public, and it's just acknowledging celebrities as real people. She then immediately asks if celebrities aren't "protected by piles of money from any rocks thrown at them," gets called out and backpedals that "it's not okay to say false things about anyone," invalidating everything she said before.

Kimmel throws a rather fun anachronistic zinger too: "I don't know why anyone would buy advertising on a web site."

It's a rare occasion. Distant celebrities are humanized, old media takes new media to task, and the cheerful talking head is revealed for the blatant opportunist that she is. A chance to show the media does know better, right? Aaron Sorkin saw it too and featured this teachable moment in an episode of HBO's The Newsroom, lifting lines of dialog verbatim:

Except of course, all the sharp edges have been filed off. Instead of a bubbly editor of an internet gossip rag, there's a pudgy Zuckerberg-type pushing his tech startup and app. His protestations seem more genuine, and he's taken aback by the sudden accusations, rather than invited to explicitly address them. The celebrities are not represented, they are off-screen, being stalked and murdered. He's an oblivious technocrat harming society with his self-serving autism, she's the approachable-yet-merciless reporter bringing him to task.

You see how a story that was entirely about the media was turned into something completely different, without even changing most of the words? It's all in the delivery and presentation. The excuse that it can't be entirely realistic or literal does not apply. I find the whole thing rather convenient in pushing certain narratives. It confuses the sociopaths who run certain businesses with the people who maintain the tech, and clear-coats it with some subtle nerd bashing.

It could be a pretty good example of what Chomsky was talking about: when presenting an interpretation of fact, American gatekeepers apply a lens that shines only on a very narrow and predictable window, while insisting otherwise. Like showing you how news is actually made, an elaborate pretense that the press is a critical dissenting force.

Or it could just be a mediocre episode of a faltering TV show. But then I'm sure there are countless opinion pieces and comments already that debate this in excruciating detail.

Either way, in trying to bring reality into their fantasy, they have remade it in their own image. They tend to do that a lot. Not just the ones who write fiction.

Celebrate Diversity

Where might we have seen all of this again recently? Let me lay it all out. What do you do when you're a notable reporter in entertainment, like say, video games, who finds that trade press is mostly obsolete in an age of YouTube, Twitch, Twitter, Reddit and more?

Well, you might have an existential crisis when you realize you were just being a drunk megaphone for corporate marketing interests, and become a mouthpiece for your own interests instead. Get on Twitter and post "Fuck ethics get money," give talks in which you "have no pretense of being unbiased." Feature people and projects you like, promote political and social causes you think are important, and start picking the exclusives yourself instead of waiting for them to drop in your lap. All the while partaking in the global race to the bottom that is modern clickbait.

If enough people in a clique do this, it would create the appearance of an indie renaissance. However it would only feed attention to a very narrow scope of ideas, accessible only to those with specific connections who say the right things. If done in the context of a multi-billion dollar industry, this might create some resentment. Especially if there was years of dubious reporting, conflicts of interest and agenda pushing leading into it, with several careers permanently tarred and feathered through irresponsible rumour and lies. Hypothetically speaking, of course.

In such an environment, imagine evidence pops up that one of the media clique's new darlings is actually a rather disturbing individual. While profiling herself as a paragon of virtue and a talented progressive activist and game developer, she manipulates and lies her way through her career and relationships. This comes in the form of a warning and a detailed description of emotional abuse, proven with, say, lengthy Facebook citations. The details are quite lurid and salacious, namely infidelity with a set of eponymous Five Guys in the industry, including her boss and a Kotaku reporter. On top of that, she is seen to wear the Emperor's New Clothes, in that her "work" mainly consists of a single choose-your-own-adventure Twine book that would barely pass as a first year student's midterm project. It was praised mainly for its subject—depression—rather than its execution.

Imagine that in response to this, other acquaintances come out of the woodwork to describe other things she possibly did. Like framing a forum of depressed people as harassers for attention, leaking personal data on a rival charity, getting friends in media to blacklist them, sabotaging a reality TV game event she was in, raising money for another event that never happened, and generally being resented by past contacts.

More importantly, imagine that this scandal leads to other conflicts of interest being revisited or discovered, previous media lynchings to be brought up and accountability to be demanded, because it involves multiple influential outlets and a lot of people are tired of it.

Rather than admit it's all become a big joke and everyone's been played, reporters might choose to close ranks instead. They might concoct a tale that all the criticism she receives is harassment, that gamers are obtuse shitslingers and wailing hyperconsumers who hate women, declaring the gamer identity to be dead in ten articles published in 24 hours. Though it would alienate their core readership, it would please the newer, more political segment, who would see all their existing biases about video games and gamers confirmed, being ignorant of the medium's established history of complicated themes being explored maturely.

New readers might see it as a license to step in, educate the plebs and defend the fair maidens from the supposed savage horde, in full sight of friends on social media, nodding respectfully. They might rephrase events as a "jilted ex"—even though he dumped her—who has been rallying a hate mob on the evil depths of 4chan—even though he posted on Something Awful and Penny Arcade first and reaction was mixed. They might speak of him leaking "revenge porn" when in actuality it's her own paid erotica circulating. All of these are either outright lies or an admission of not having bothered to read source material.

It would be vital for reporters to never mention the coordinated censorship that might start to take place on Reddit, 4chan and other gaming forums, and the resulting Streisand effect that would only draw in more attention. This could include filing a false DMCA claim on a prominent YouTuber, turning a 25,000 comment thread into a graveyard, banning people for merely mentioning certain names, and declaring the entire topic off-limits on certain allied subreddits. As moderator favors would be traded in behind the scenes to accomplish this, it might create the appearance of even more collusion and even downright conspiracy, leading some to migrate away from 4chan in disgust to a previously minor image board, and later to Voat from Reddit.

Even if someone predicted exactly what would happen, a week in advance, the games press would not, under any circumstance, face facts and admit error. Instead they would begin a long and drawn out media war against consumers who mainly want accountability, and in doing so, become the biggest magnet for trolling, drama, e-celebs and false flags the internet would see since its inception. Especially ones the old internet already knew about.

Exactly like abuse victim Eron Gjoni warned, before being gagged by a contested and possibly unconstitutional legal order, a whole segment of an industry might go insane the subsequent months, over an abuser's self-serving machinations. Gaming and feminist media would have material aplenty with their new scapegoat. As the global news press is really just one big game of telephone, they would all report whatever these experts told them. They'd wield guilt by association like a club without ever bothering to investigate, obliviously committing a big lie so massive it would give Goebbels throbbing priapism. Like say, if one untraceable internet comment nobody can substantiate was momentarily turned into the mushroom kingdom's death of Princess Diana. If random IRC quotes were taken as spy-thriller facts about the opinions and motivations of thousands, without further investigation or right of reply. They might accept the word of a serial liar and self-admitted helldump veteran, starring as both Archduke Ferdinand and Joan of Arc in her very own DongleGate.

All because of vidya, video games, all because a bunch of reporters found themselves kings and queens of a castle nobody really needed after all. They found an audience convinced Twitter and Tumblr was how you did activism, and that campuses and conventions should be places where good faith inquiry goes to die.

But that would be crazy to imagine.

Even crazier would be to see that, unlike every single time it's happened before, this kind of entryism-by-way-of-social-justice met a wall of defiant opposition. That somehow the impossible happened, and people from Reddit and 4chan could put aside their differences long enough to actually get some real work done. That all the oppression olympics, the media shaming, the misrepresentation, the wordplay, the fallacies and moral posturing had finally pushed enough people over the edge to say: no, for once, you move. That they might try turning Twitter into the closest thing it will ever be to a rational discussion space, which is to say, not at all. That people there would be so addicted to digital validation, they would mistake their clumsy ideas being dismantled for harassment. They might even call it cyberviolence at the UN, cherry picking a few idiots out of the mass for effect, and autoblocking the rest in a self-fulfilling prophecy.

You could discover, one year in, that trying to get a Society of Professional Journalists to acknowledge rampant selection bias, copypasting, lazyness and pandering is far trickier than it was in 2011 with Fox News: this time there's bomb threats and few seem to care much who sent them, or even know how to go about reporting on this. That Wikipedia's own infotocracy switches to printing outright propaganda for a while, stuck in citogenesis of its own creation.

Because it's far more lucrative to play dumb after using autistic kids for laughs from a position of unearned privilege.

Brave New World

So let's come full circle on this horseshoe of insanity.

What if it turned out that Occupy's much desired "new way of interacting with others" already existed since 2001, in the form of the Anonymous Image Board? Perhaps not coincidentally an invention from Japan, a culture that turned repression into an art form. A board where identity and status has no value by default and as a result, entirely different rules of engagement apply, building on the practices of early cyberspace.

It does not lead to a utopian forum where intellectuals debate like they're on the set of Star Trek. Quite the opposite, this is humans you're dealing with. It leads to places that mix deep skepticism with casual verbal and graphic abuse, making everyone and everything equally worthless and fake until proven otherwise. Bringing identity as a general point of argument is not just unwelcome, it is shouted down with exactly those insults that deal it the most damage. Relevancy comes from resilience, taking offense is an instant defeat. Ingenuity, hyperbole and humor are the controlling measures to balance it out, acting like constant bullshit tests against the uninspired and intellectually bankrupt. Ideas are forced to compete to remain relevant, just with one caveat: no topic is off limits. Especially taboos. It's not exactly for the faint of heart.

It's the closest thing the internet has to a functional universal "safe space", which is to say, not one at all, not by any offline definition. Once you get used to it though, the porn, the trolling, the shilling, the jew jokes, the "faggotry," it all becomes so banal and transparent as to not be worth getting worked up about. Which is exactly how it should be.

Look, while it's equally vulnerable to all the same pettyness and tribalism that made Twitter a success story, it's far more equipped as both a medium and a culture to deal with it sustainably. That might include razing their own village to the ground and rebuilding it anew in times of great distress. Though when it's all made out of trivially replaceable ones and zeroes that do it for free, is that really something other than an address change? It's not that strange: we used to be nomads, before the promise of a full belly and a permanent roof over their heads made people sacrifice a little bit of freedom for a whole lot of security.

There are many who know all this, none of this is new, and a generation is growing up with this stuff being as normal as Facebook and Snapchat. Yet others look at the chans as some sort of oblivion gate leading to endless bigots, criminals and harassers. Like 9/11 Truth at Occupy, when you push certain ideas into the counter-culture, don't be surprised when speaking openly requires sharing space with shady neighbours, on top of a very long tail of fringe and genuine marginalization. Once again, you may hate this, but you can't ignore it.

For older readers: when you think Dungeons and Dragons turns teenagers into witches and warlocks, or that day-cares murder children in Satanic rituals, expect to be mocked as the ignorant panicky fool that you are. They might have learned to build bridges you had no idea about long ago, and they wouldn't need to declare themselves on the right side of history to do it.

Even more hilarious is when people say this about dear old Reddit, like say, in terrible internet studies. They do their best to ignore their own conclusions and their own camp's troublemakers, with a random "but Women in Tech!" thrown in for good measure. They see what they want to see, not realizing that taking everything there at face value is not just noob mistake #1, it's often a mechanism that explicitly works to keep unmotivated outsiders out. The kind of trolls you can spot right away are the amateurs, and certain people are so useless they are a net drain on any online community they enter, often free from reproach.

The internet stops being uniquely valuable the moment you insist it should work exactly like real life. Our primary instincts and activities do not scale well beyond a single small tribe, that appears to be both empirical fact and lived experience. The default human impulse is to rely on shared identity for social validation. Insecurity leads people to avoid thinking critically, too afraid of going against the grain. They shun ideas rather than tackling them, succumbing to groupthink and echo chambers. The antidote is the same it's always been: good natured humor and reflection, including on oneself, not moral panic.

There's only one question you need to answer to complete the picture. Why are they so eager to believe that all of gaming—or all of society even—hates women and minorities, does not want them around and treats them as inferior, needing special outreach and support, even when there's evidence and testimony to the contrary? I'll give you a hint: social justice warriors are terrible at even just considering opinions that differ from their own. Yet they can't shut up about this one.

The road to hell is not just paved with good intentions, the Devil provides free popcorn at every station.

And when entering rabbit holes, always bring enough fuel to burn back to a higher apoapsis.

The media lied, and I saw many whom I thought knew better fall for it. It was an absolute mindfuck to see educated brains fail this hard and don their favorite brown shirt in the name of progress. It's like they never grew past their high school phase, still trying to become one of the cool kids while resenting them for it. Never realizing that moral posturing is a trick that only works on the weak willed, and it only works so long. Anita pls stop.

Listen and Believe, take your with-us-or-against-us mindset and go think about what you've done. The internet is under total surveillance, global trade agreements threaten to elevate corporations to the level of states, and the willingness to censor the net is at an all time high. Only this time, it's not just the threat of terrorism, pedophiles and child porn used to scare the masses and silence critics. It's "cyberviolence against women" and fringe "white supremacy." You know what "enables a broken system"? Letting a popular female abuser steamroll over her gaslighted male victim and then shamelessly swapping all the labels while applying duct tape.

Look around. One of the most prominent press freedom activists lives in house arrest out of an embassy, accused of the internationally villainous crime of having had consensual sex but without a condom, i.e. "rape." Sexism and racism have been wielded as perverted weapons of reverse victimhood by those with nothing substantial to offer and everything to gain. The newspaper that broke the Snowden leaks has willingly reduced itself to Daily Mail level, shaming scientists instead of focusing on their accomplishments. It's not a conspiracy when the clickbait keeps flowing and people willingly line up to partake, forking over their dough for post-catholic indulgences against patriarchal original sin and white oppressor guilt.

PS: Calling people angry is not a valid retort (we call that "projection"), neither is calling them bigots, harassers or internalized misogynists/homophobes/uncle toms because of things they didn't do. Social justice wanted everyone to educate themselves, so we gave them exactly what they asked for—warts and all—and they and the media continue to misrepresent it. There was no grand deception on the gamers' part, no reefer madness hate mob that chases women out of gaming for being women. Only talentless hacks, plenty of characters with unclear intentions, a lot of third party trolls, and many more people trying to work around them while being called names. I'm not mad as hell, I simply do not wish to take it anymore.

(And they will probably call me right wing, never stopping to think that national socialism created the Volkswagen for a reason. Yes I know, Godwin. Wir haben es nicht gewußt.)

Darmok and Jalad at Tanagra

The tree of liberty must be refreshed from time to time
with the blood of patriots and tyrants,

but they tried to use ketchup.

Note: some people have pointed out the statements above could be seen as quite ominous. I'd just like to clarify, it's meant to be funny and somewhat absurd in a if-we-can't-cry-at-least-we-can-laugh way, not to be taken literally as a call for conflict.

↧

Occupy WWW Street (FI)

November 29, 2015, 3:00 pm

≫ Next: Occupy WWW Street (FR)

≪ Previous: Occupy WWW Street (EN)

Internet Activism and Media in the Age of Social Justice

English - Français - Nederlands

"I don't have access to any unusual methods of analysis and what special knowledge I have concerning language has no bearing on social and political issues. […] For the analysis of ideology […] a bit of open-mindedness, normal intelligence, and healthy skepticism will generally suffice." – NC

I'm bringing out my serious font again. It's difficult to talk about current affairs without inviting in heaps of context, so I'd like to delve into some recent history.

Reminder, shooting the messenger is poor form, and heresy is medieval.

(🇫🇮 Links have been unarchived)

The Only Working Class Man In The Village

However when asked if there was a political demand that could sum up Occupy, something to "make this about," he takes a curious turn:

More awkwardly, on the subject of taking their cues from other protests, in Greece and Europe and elsewhere, LaGreca claimed "we didn’t take our cue leading off of anybody really."

Activists Are Dead

So what happened? In June 2012, community hub and non-profit OccupyWallSt.org posts that Reports of Occupy’s Death Have Been Greatly Exaggerated.

"Since the eviction of the protestors at Freedom Plaza last November, it’s become a media cliché to report on the “Death of Occupy.” Articles pop up all over the web, blithely reporting on the failed second wind of Occupy, this lackluster “American Spring,” and the May Day general strike that didn’t quite shut the system down.

It should be no surprise that the mainstream media is eager to report on Occupy’s supposed demise. Even ignoring the fact that the corporate-owned media has a strong desire to never see social movements such as Occupy succeed, the media, as a rule, generally needs to put a dramatic narrative to everything it reports. To them, every story ought to have a captivating story arch with a beginning, middle, and an end."

"In the media’s eyes […] many people heeded the call, [yet] the movement only received its dramatic momentum when cops were photographed attacking and pepper-spraying the nonviolent protestors. It reached its early demise when the police violently cleared out the various encampments. Now, except for a few curmudgeons who can’t seem to understand that Occupy is over, all that remains of Occupy is its populist rhetoric of the 99%—which has been dutifully hawked up by Democratic front-groups such as MoveOn.org to help refuel the Obama election machine."

So much for that then. What's on TV tonight? A presidential debate?

"This popular narrative […] has been so successful that even those who are still active within the Occupy movement can’t help but absorb parts of it. […Meetings] sometimes border on something closely resembling a public support group. On the internet, vaguely self-congratulatory Paul Krugman-y articles, applauding Occupy for “at least shifting the public dialogue,” are posted and reposted […] to remind each other that Occupy at least had a little bit of an effect."

"The true magic of Occupy was that it rejected all of these things. No one had any more power than anyone else, […] nobody in Occupy really cared that we were ignored by the mainstream media. We don’t need a bunch of hacks at Time Magazine to commend us for our ability to protest. […] The goal of Occupy was to get together as a community of equals, to claim a future different than the ones they gave us, and to reignite a tradition of democratic progress […] It was a way of exploring new ways of interacting with others."

1 2 3 4 5

"It needs to be remembered that the word “occupy” is […] a call to action, not the action itself. [It] was useful for getting individuals and organizations previously isolated or focused on one-issue grievances out into the streets. […] What is important is that there’s wide community of opposition being formed across many social barriers, and those who hold power are very afraid."

Ignore the posturing at the end, notice something far more telling.

Burgers and Fries with Ketchup

It's difficult to take someone's Important Conversation about Deconstructing Issues seriously, when social justice doesn't know how to socialize.

/Chomsky/ was Right

"I don't have access to any unusual methods of analysis and what special knowledge I have concerning language has no bearing on social and political issues. […] For the analysis of ideology […] a bit of open-mindedness, normal intelligence, and healthy skepticism will generally suffice."

"Take [the] intelligentsia in a society like ours. This social class, which includes historians and other scholars, journalists, political commentators and so on, undertakes to analyze and present some picture of social reality. […] They serve as mediators between the social facts and the mass of the population: they create the ideological justification for social practice. Look at the work of the specialists in contemporary affairs and compare their interpretation with the events, compare what they say with the world of fact. You will often find great and fairly systematic divergences […]."

"With a little industry and application, anyone who is willing to extricate himself from the system […] will readily see through the modes of distortion […] Everybody is capable of doing that. […] Often carried out poorly, […] social and political analysis is produced to defend special interests rather than to account for actual events." (p3-4)

"Here in the United States there is an astonishing degree of ideological uniformity for such a complex country. Basically, there are two reasons for this. First, there is the remarkable ideological homogeneity of the American intelligentsia in general, who rarely depart from one of the variants of state capitalistic ideology (liberal or conservative), a fact which itself calls for explanation. The second is that the mass media are capitalist institutions. […] The fact that these institutions reflect the ideology of dominant economic interests is hardly surprising." (p9)

"They don't even mention the logical possibility of a third position: namely, that the United States did not have […] the legal or moral right, to intervene by force in the internal affairs of Vietnam. […] Note that as the Times sets the spectrum of ideological debate, the position of much of the peace movement is simply excluded from consideration. Not that it is wrong, but rather unthinkable, inexpressible. […]

Here we have a marvellous illustration of the functioning of propaganda in a democracy. A totalitarian state simply enunciates official doctrine—clearly, explicitly. […] one can only express opposition at one's peril. In a democratic system of propaganda no one is punished (in theory) for objecting to official dogma. In fact, dissidence is encouraged.

What this system attempts to do is to fix the limits of possible thought: supporters of official doctrine at one end, and the critics—vigorous, courageous, and much admired for their independence of judgement—at the other. The hawks and the doves. But we discover they all share certain tacit assumptions, and that it is these assumptions that are really crucial.

No doubt a propaganda system is more effective when its doctrines are insinuated rather than asserted, […] The more vigorous the debate, the more effectively the basic doctrines of the propaganda system, tacitly assumed on all sides, are instilled. Hence the elaborate pretense that the press is a critical dissenting force—maybe even too critical for the health of democracy—when in fact it is almost entirely subservient to the basic principles of the ideological system." (p36-39)

This was the 70s. What might that look like in the internet age?

Don't Stalk Me Bro

Kimmel throws a rather fun anachronistic zinger too: "I don't know why anyone would buy advertising on a web site."

Or it could just be a mediocre episode of a faltering TV show. But then I'm sure there are countless opinion pieces and comments already that debate this in excruciating detail.

Either way, in trying to bring reality into their fantasy, they have remade it in their own image. They tend to do that a lot. Not just the ones who write fiction.

Celebrate Diversity

But that would be crazy to imagine.

You could discover, one year in, that trying to get a Society of Professional Journalists to acknowledge rampant selection bias, copypasting, lazyness and pandering is far trickier than it was in 2011 with Fox News: this time there's bomb threats and few seem to care much who sent them, or even know how to go about reporting on this. That Wikipedia's own infotocracy switches to (⛔️) printing outright propaganda (⛔️) for a while (⛔️), stuck in citogenesis of its own creation.

Because it's far more lucrative to play dumb after using autistic kids for laughs from a position of unearned privilege.

Brave New World

So let's come full circle on this horseshoe of insanity.

The road to hell is not just paved with good intentions, the Devil provides free popcorn at every station.

And when entering rabbit holes, always bring enough fuel to burn back to a higher apoapsis.

(And they will probably call me right wing, never stopping to think that national socialism created the Volkswagen for a reason. Yes I know, Godwin. Wir haben es nicht gewußt.)

Darmok and Jalad at Tanagra

The tree of liberty must be refreshed from time to time
with the blood of patriots and tyrants,

but they tried to use ketchup.

↧

Occupy WWW Street (FR)

November 29, 2015, 3:00 pm

≫ Next: Occupy WWW Street (NL)

≪ Previous: Occupy WWW Street (FI)

L'Activisme en Ligne et les Médias à l'Époque de la Justice Sociale

English - Français - Nederlands

"Je n'ai pas accès aux méthodes d'analyse inhabituelles, et la connaissance particulière que je possède de langue n'a aucune incidence sur les questions sociales ou politiques. […] Pour l'analyse des idéologies […] un peu d'ouverture d'esprit, de l'intelligence normale, et du scepticisme sain suffisent généralement." – NC

Je m'apporte encore ma police sérieuse. C'est difficile de parler de l'actualité sans inviter des tonnes de contexte, donc je voudrais revenir sur l'histoire récente.

Rappel, personne n'aime le messager porteur de mauvaises nouvelles, et l'hérésie appartient au Moyen Âge.

Le Seul Col Bleu dans le Village

Septembre 2011, la gauche américaine se trouve dynamisée. Le message est direct et vise les grands objectifs: le 99% de la population demande aux 1% avares de tirer sa révérence et d'arrêter de se servir des entreprises comme des matraques. Parc Zuccotti à New York se remplit des activistes qui campent; ils discutent et partagent leur message.

Là aussi Jesse LaGreca. Avec son look caractéristique American Joe Average, il surprend la plupart des gens, certainement le journaliste de Fox News qui l'approche. Dans son interview spontané, jamais diffusée par Fox, LaGreca instruit le pauvre homme et d'y va pleins gaz sans s'arrêter. Il souligne les questions suggestives tout de suite, et remarque que s'il y a une chose sur laquelle les nouvelles—"la machine de propagande dans les médias"—ne veulent pas faire rapport, ce sont eux-mêmes. Que peut-être il est temps pour la démocratie participative qui "n'est pas financée par des caméras de reportage."

Un journaliste se présente à expliquer le monde et découvre que cette histoire s'agit de lui-même, du moins en partie. Le refus suit rapidement: "Nous sommes ici pour vous donner la possibilité […] de diffuser tout ce que vous voulez, pour vous offrir de la couverture équitable […] Alors, il y a une exception dans le cas, parce que vous ne pourriez pas partager votre message sans nous."

Il riposte: "Certainement. Vous vous souvenez quand Glenn Beck […] a décrit le président comme une personne qui déteste les blancs et la culture blanche. C'était un moment faible dans l'histoire des américains et en quelque sorte vous avez joué un grand rôle dans cette affaire. Donc je suis heureux de voir votre change d'avis en faisant attention à [la reste du pays], plutôt que la marge extrême droite qui préfère totalement détruire la classe moyenne. […] On voudrait voir un petit peu plus de la justice économique, ou de la justice sociale, vous savez, des trucs Jésus-Christ, comme nourrir les pauvres, les soins de santé pour les malades. […] Je m'amuse beaucoup que les gens tiennent la constitution quand ils crient aux soldats homosexuels, mais ils ne peuvent pas comprendre qu'un système de santé à la recherche du profit ne marche pas. […] Si nous voulons que le président fait plus, parlons sur un niveau qui fonctionne en lieu de lui demander pour son certificat de naissance, […]"

Shots fired, critical hit, no chance of recovery. Aucune surprise cet entretien a été "oublié," et aucune surprise quelqu'un l'a dévoilé comme trop extrême pour Fox news. Les manchettes prévisibles nous ont demandé s'il était L'Homme Le Plus Intelligent De Wall Street et nous expliquaient que Jesse LaGreca Continue à Détruire La Partialité Des Médias de Occupy Wall Street. Tous des journalistes pressés de montrer qu'ils offriraient une voix à n'importe qui.

LaGreca lui-même l'a évoqué en passant à l'antenne: "La réalité c'est que je suis le seul col blue que vous verrez dans le journal du dimanche… les nouvelles politiques… peut-être jamais. Et je pense que c'est très indicatif des échecs des médias à rapporter les nouvelles les plus importantes…" Il est interrompu immédiatement, "Nous essayons de faire de notre mieux, Jesse."

Lorsqu'on lui demande s'il y a une exigence politique qui peut résumer Occupy, quelque chose "[qu']il s'agit de faire," il prend le virage curieux:

"Je pense que tout le mouvement s'agit de la justice économique. En fait pour moi — et je ne parle pas au nom de Occupy Wall Street, je vous donne mon opinion personelle — je croix que c'est une question des droits économiques, et je croix que c'est une question des droits sociaux, et de la justice sociale. Et pour les gens qui sont fâchés que le mot “sociale” est utilisée avec le mot “justice,” je vous invite de relire la constitution."

Bien sûr. J'ai fait mon devoir et j'ai entrepris le labeur de m'éduquer. La Constitution des États-Unis ne fait en aucun cas référence à quelque chose de sociale concernante la justice. Apparemment, les gens qui l'ont écrit pensaient que simplement "établir de la Justice" suffisait. Elle ne classe pas les citoyens par ordre de privilège, elle ne définit pas les pouvoirs relativement à l'oppression, et fait aucune allusion à punching up. Ça c'est la justice sociale aujourd'hui n'est-ce pas? Ou entendait-il des trucs Jésus-Christ? Je ne suis pas un avocat de droit constitutionnel, n'hésitez pas à me corriger.

Plus gênant, sur le sujet de l'inspiration par d'autres manifestations, en Grèce et en Europe et ailleurs, LaGreca affirme que "nous ne suivions vraiment personne."

Occupy a peut-être connu une croissance organique de groupes existants, mais il a été galvanisée par un blog et un bulletin diffusé aux abonnés de Adbusters, "vous 90,000 rédempteurs, rebelles et radicaux là-bas," qui a déclaré qu'un "changement mondiale des tactiques révolutionnaires est en cours," et leur a demandé s'ils étaient prêts à avoir leur propre "moment Tahrir." Les 267 commentaires ont reçu ce message avec enthousiasme et engagement.

LaGreca n'était pas inconnu non plus, il était contributeur de trois ans chez Daily Kos. Les gens là l'ont encouragé pour "[refuter] le stéreotype des médias de ces ne0-hippies obtuses privés de leurs droits" et répandaient la bonne parole en apprécièrent le spectacle: "Je devine que Hannity a peur. Comme il faut!"

Ceci n'est pas un acte d'accusation au sujet de Mr. LaGreca, il me paraît plutôt plein de bon sens, et assez convaincant. Mais il y a quatre ans. Malgré le ample temps d'antenne que sa nouvelle célébrité-à-Streisand lui a rendu, et le désir des journalistes de faire de mieux, la situation ne s'est pas améliorée, ni chez Fox News ni chez les autres organes de presse qui se définissent par l'opposition.

S'il y a une chose que les médias aiment, elle est l'exception qui confirme la règle: on ne peut pas galvaniser la population sans accès aux réseaux et plateformes existantes. Quand vous vous engagez dans une guerre contre les médias, ils vous transforment en produit de divertissement, et les masses vont l'adorer. Vous peut-être aussi.

Les Activistes sont Morts

Qu'est-ce qui c'est passé? Juin 2012, forum communautaire et association à but non lucratif OccupyWallSt.org annonce que Les Rapports de la Mort de Occupy ont été Grandement Exagérées.

"Depuis l'expulsion des manifestants à Freedom Plaza en novembre dernier, il est devenu un cliché des médias de rendre compte de la “mort de Occupy.” Articles ont été publiés dans l'ensemble du Web, qui racontent allègrement le second souffle de Occupy échoué, ce “Printemps Américain” terne, et la grève générale de mai qui n'a pas exactement arrêté le système.

Ce n'est pas une surprise que les médias générales sont avides de faire rapport sur la fin présumée de Occupy. Même si vous ignorez que les médias détenus par l'entreprise ont bien le désir de ne jamais voir la victoire des mouvements comme Occupy, la règle du pouce est qu'ils ont besoin d'un récit dramatique pour tout ce qu'ils rapportent. Chaque histoire doit avoir un arc captivant avec un début, un milieu et une fin."

"Selon la presse […] beaucoup de gens ont répondu au appel, [mais] le mouvement n'a pas avancé dramatiquement jusqu'à la police a été photographié en s'attaquant aux manifestants non-violents et en appliquant du gaz poivre. Sa disparition précoce était quand la police a démantelé violemment les divers campements. Maintenant, sauf quelques grincheux qui ne peuvent pas comprendre que Occupy est fini, tout ce qu'il reste est sa rhétorique populiste du 99%—qui a été adopté consciencieusement par les organisations Democratiques comme MoveOn.org pour ravitailler la machine d'élection Obama."

Tant pis. Il y a quelque chose à la télé ce soir? Un débat présidentiel quoi?

"Ce récit populaire […] a ainsi réussi a se faire absorber par ceux qui sont encore actives dans le mouvement Occupy. […Les réunions] ressemblent parfois des groupes de soutien publics. Sur Internet, des articles vaguement Paul Krugman-esques sont publiés et republiés en autosatisfaction qui applaudissent Occupy pour “au moins déplacer le dialogue public,” […] pour rappeler à l'autre qu'au moins Occupy a eu un peu d'effet."

Super chouette. L'article raconte en longueur continue que les américains voient leur histoire comme "quelque chose les autres ont fait jadis (surtout les hommes blancs de la classe supérieure)" et oublient "[qu']il a fallu des centaines de milliers de personnes—notamment les jeunes, les femmes, et les hommes ouvriers–pour supporter et diffuser les idées de la démocratie. […] Nous voyons Occupy comme un spectacle […] de gens qui sont très différents de nous. […] Les célébrités et artistes, les anarchistes de style bloc noir, et les maires despotiques sont les personages de ce drame, [qui luttent] pour les manchettes sur la scène de nos villes sales."

"La vraie magie de Occupy était qu'il a rejeté tout ça. Personne n'avait plus de pouvoir que quiconque, […] personne dans Occupy donnait de la merde que nous nous sommes ignorés par les médias traditionnels. Nous n'avons pas besoin des dilettantes au Time Magazine pour nous féliciter de notre capacité à protester. […] Le but de Occupy était de se réunir dans une communauté des égaux, la revendication d'un avenir différent de celui qu'ils nous ont donné, et de raviver une tradition de progrès démocratique […] Il était un effort d'explorer de nouvelles façons d'interagir avec les autres."

1 2 3 4 5

L'auteur insiste sur le fait que "Occupy n'était pas une mode passagère ou une chimère de gauche." D'accord. Sauf le "sentiment dans l'air," les effets énumérés sont que "plus de 7200 manifestants de Occupy sont arrêtés dans les États-Unis" et beaucoup d'eux sont "battus et torturés." Occupy a été diabolisé puisque "les médias ont été poussés à rendre compte […] sous un jour défavorable." Quoi d'autre?

"Il faut se souvenir que “occupy” était […] un appel à l'action, et non pas l'action elle-même. [Il] était utile pour faire sortir dans la rue les individus et les organisations précédemment isolées ou axées sur un seul problème. […] Ce qui est important, c'est la grande communauté d'opposition, étant formé à travers de nombreux obstacles sociaux, et ceux qui détiennent le pouvoir ont beaucoup de peur."

Ne prêtez pas attention aux attitudes à la fin, apercevez-vous de quelque chose beaucoup plus révélatrice.

Le résultat de Occupy, selon un article qui veut prouver sa valeur durable, est que 7200 personnes se sont fait entendre comme des victimes et des martyrs, supportées par la résistance passive et idealiste d'ailleurs. Ils ont glissé vers une non-pertinence déprimée, car ceux qui devraient faire rapport au service de l'interêt public ont donné une impression incorrecte. Ils l'ont trouvé acceptable.

Les meilleurs mensonges sont à moitié vrai. Ce rituel-de-absolument-non-deuil collective était une part de vérité, dûr comme fer, emballé dans l'auto-illusion narcissique. Peut-être il est le syndrome de Stockholm. Ceci sont les espèces Pokémon évolués de la culture de la victime, où la victimisation est célébrée, parce qu'elle peut fournir du pouvoir énorme.

Burgers et Frites Maison au Ketchup

Je ne crois pas l'histoire officielle, ni des médias ni du mouvement, parce que je suis allé voir moi-même en Vancouver. Je n'entends pas la Verité 9/11 ou les activistes de l'herbe: ils appartiennent au territoire, où iraient-ils autrement? Non, je m'apercevais de deux groupes en particulier.

Premièrement, il y avait les redresseurs idealistes au lait qui jouaient au microphone humain—aussi facilement que chez Parc Biscotti!—avec les mecs et nanas trop à la mode pour être pauvre. Cela m'a paru sinistre: comme d'autres endroits de Occupy, le matériel audio était interdit et alors ils étaient contraints de se repeter lentement, comme le chant de mantras. Complémentaire était le manque général d'orientation et modération. Ajoutez des gens qui ont tant de sentiments qu'ils ne savent plus ce qu'ils doivent en faire, et il ne mène à rien très lentement, comme Twitter dans le Matrix.

Je ne voyais pas de discussion ouvert pour le débat éclairé, pas de rassemblement communautaire, c'était une tribune pénible aucune personne ordinaire n'écouterait plus que cinq minutes. Ils avaient entendu parler de l'activisme, ils même savaient comme celui-ci se présente et sonne, mais ils ne le disaient que du bout des lèvres. Exactement comme chez des plusieurs de livestreams d'ailleurs que j'ai vu.

Deuxièmement, les vrais sans-abris de la ville: les chômeurs et les inemployables, les malades mentaux, les toxicomanes chroniques ou des autres rejeté par la société, qui errent dans l'espace qu'ils occupent déjà tous les jours. Combinez les deux, quelle surprise, les grands principes d'égalité, des privileges, de diversité, d'habilisme, d'expression libre et d'appropriation partagée sont testés. Les jeunes qui campent aux festivaux d'été avec des toilettes désinfectées et des robinets en abondance apprennent qu'un endroit public dans l'hiver est moins agréable. Par ailleurs, en cas de conflit ou de crime inévitable, probablement il n'est pas très judicieux de se fonder sur l'arrêt de ceux qui mettent les mots "triggered" et "culture du viol" dans la bouche comme des friandises.

Au lieu d'affronter la réalité, ils ont adopté la liasse progressive, qui "encourage les femmes et les groupes traditionnellement marginalisés [à] prendre la parole devant les hommes, surtout les hommes blancs." Si vous creusez un peu, vous entendez dire qu'il y avait de friction de "réunions déraillés par les femmes et les minorités obsédées par la politique d'identité. […] [Les plans] ont presque toujours rencontré des objections des féministes, des activistes homosexuels, noirs, latinos, etc. Souvent ces personnes n'avaient même pas des préoccupations spécifiques. […] Ils se sont simplement levés pour se plaindre du manque de diversité, de la victimisation de leur groupe d'identité spécifique, etc. […] Des gars blancs intelligents, capables, organisés ont quitté la manifestation. Ils sont partis frustrés."

Vous trouverez beaucoup d'histoires des brouilles et d'abus, ils ont fourni une matière fertile pour les médias, sociaux ou non. Ce qui s'est vraiment passé n'était considéré que rarement pertinent, et de tels problèmes sont devenus plus importants que le système financier et économique, ou les établissements qui le soutiennent. Apparemment on devrait affronter la violence liée au genre dans le mouvement Occupy: "Exactement ce qui s'est passé est incertain. Les uns prétendent qu'il y avait une tentative de viol. Les autres prendent l'incident à la légère, qu'un homme ivre a envahi l'espace vital d'une jeune femme." Pensez-vous que cette distinction peut être importante?

Les médias juste-de-vue-sociale ont dégagé l'hyperbole, que les hommes se souviennent "de violer personne, jamais"—merci xojane—et que "les espaces réservées aux femmes sont construites pour faciliter des conversations avancées concernantes les enjeux qui affectent les femmes (et, idéalement, quiconque se ne considérant pas d'homme) sans être interrompu sans cesse par la nécessité d'éduquer des autres dans le rassemblement."

En d'autres termes… Les gars, restez dehors et taisez-vous. Les filles, pas de questions. Bien fait la justice sociale, t'as apporté tous ce que tu détestes à ta petite utopie, et les autres ont été chassés.

C'est facile de désigner du doigt les personages marginales déjantés, avec leurs cheveux moches et leurs drôles de verres. Si vous êtes Stephen Colbert, vous pouvez vous moquer d'eux devant des millions sans merci en gagnant beaucoup de points. Le vrai problème c'est la masse d'activistes eux-mêmes et les médias auquel ils croient. Ils souhaitent se dérailler par des enjeux qui ils eux-mêmes reconnaîtrent comme un microcosme d'une communauté humaine. L'erreur c'est d'attribuer une signification universelle aux incidents isolés pendant que l'on ignore les éléphants dans le salon. C'est une maladie concentré dans les médias, mais pas seulement là.

C'est difficile de prendre au sérieux une Conversation Importante sur La Déconstruction des Enjeux, si la justice sociale n'a jamais appris à socialiser.

/Chomsky/ Avait Raison

Dans "La Langue et La Responsabilité" (1979), Noam Chomsky converse quelque peu en eloquence. Il explique en grande partie ses théories de niche de la linguistique. Mais le premier chapitre "La Politique" s'ouvre avec des sujets assez différents, et des idées qui retourneraient ensuite dans "La Fabrication du Consentement: L'Économie Politique des Médias de Masse" (1988). Lorsqu'on lui demande quelle est la relation entre son méthodes et son écrits politiques plus célèbres:

"Je n'ai pas accès aux méthodes d'analyse inhabituelles, et la connaissance particulière que je possède de langue n'a aucune incidence sur les questions sociales ou politiques. […] Pour l'analyse des idéologies […] un peu d'ouverture d'esprit, de l'intelligence normale, et du scepticisme sain suffisent généralement."

"Prenons le rôle de l'intelligentsia dans une société comme la nôtre. Cette classe sociale, qui comprend les historiens et les autres universitaires, les journalistes, les commentateurs politiques et ainsi de suite, s'engage pour analyser et présenter une image des réalités sociales. […] Ils servent de médiateurs entre les faits sociaux et la masse de la population: ils créent la justification idéologique pour la pratique sociale. Regardez l'œuvre des spécialistes sur les questions d'actualité et comparez leur interpretation avec se qui s'est passé, comparez ce qu'ils disent avec les faits. Vous trouverez souvent des grandes divergences, assez systématiques […]."

"Avec un peu d'effort et dévouement, chacun qui veut sortir du système […] peut voir à travers les méthodes de distorsion. […] Chacun est capable de le faire. […] Souvent mal exécutées, […] des analyses sociales et politiques sont produites pour défendre des intérêts spécifiques au lieu de rendre compte des événements réels." (p3–4)

Ensuite il parle de la situation précise dans les États-Unis. Il compare les faits enregistrés avec les explications acceptées des affaires comme le maccarthysme, la révolution des étudiants, le mouvement des droits civiles, la guerre de Vietnam, entre autres choses. Il aborde les opérations américaines à la maison et à l'étranger, légitimes ou illégitimes. Le thème dominant est assez évident:

"Ici dans les États-Unis il y a une uniformité idéologique d'une ampleur surprenante pour un pays aussi complexe. Fondamentalement, deux raisons l'expliquent. Premièrement, il y a la homogénéité idéologique notable de l'intelligentsia américaine en général, qui dévient rarement d'une des variantes de l'idéologie état-capitaliste (libérale ou conservatrice), un fait qui lui-même demande une explication. Deuxièmement que les médias de masse sont des institutions capitalistes. […] Il n'est guère étonnant que ces institutions reflètent l'idéologie des intérêts économiques dominants." (p9)

Un article principal du New York Times de 1975, une rétrospective de la guerre de Vietnam, explique qu'il y a ceux qui pensent que "la guerre […] avait pu être menée différemment" et ceux qui pensent "qu'un Vietnam du Sud non-Communiste réalisable était toujours un mythe." Chomsky fait tapis:

"Ils ne mentionnent même pas la possibilité logique d'une troisième position: notamment, que les États-Unis n'avaient pas […] le droit juridique ou moral, de recourir à la force dans les affaires internes du Vietnam. […] À noter que quand le Times fixe le spectre du débat idéologique, la position du plupart du mouvement de paix est simplement exclu de toute considération. Pas qu'elle est fausse, mais plutôt impensable, inexprimable. […]

Voici une excellente illustration de la fonctionnement de la propagande dans une démocratie. Un état totalitaire énonce simplement la doctrine officielle—clairement, explicitement. […] on exprime son opposition à ses propres risques et périls. Dans un système de propagande démocratique, personne n'est punis (en théorie) pour faire objection au dogme officiel. En fait, la dissidence est encouragée.

Ce que le système tente de faire est de fixer les limites des pensées possibles: les supporteurs de la doctrine officielle à une extrémité, et les critiques à l'autre—vigoureux, courageux, et tenue en haute estime pour leur indépendance de jugement. Les faucons et les colombes. Mais nous découvrons qu'ils ont certaines d'hypothèses tacites en commun, et que ce sont ces hypothèses qui sont vraiment cruciales.

Sans doute un système de propagande est plus efficace quand il insinue ses doctrines au lieu de les affirmer, […] Plus le débat est vigoureux, plus les doctrines de base du système de propagande sont instillées, implicitement assumées de tous les côtés. Alors le prétexte élaboré que la presse est une force dissidente critique—peut-être trop critique pour la santé de la démocratie—alors qu'en fait elle est presque complètement subordonnée aux principes de base du système idéologique." (p36–39)

C'était les années 70. À quoi est-ce que ça pourrait ressembler dans un monde connecté?

Ne Me Traque Pas Mec

En 2006, Gawker Media a lancé les Gawker Stalker Maps, un site qui demandait les visiteurs de partager des observations des célébrités en public. Les stars de Hollywood ont vu les paparazzi suivis par des gens encore plus étranges d'Internet, et ils n'en ont pas ri.

Lorsqu'elle est appelé pour défendre cette pratique un an plus tard, rédactrice Emily Gould fonce vers l'or à la gymnastique mentale. Malgré la question initiale des renseignements "si vite que vous les pouvez envoyer," elle prétend que la carte n'est mis à jour que lentement. Elle rejette en riant des preuves d'avoir publié des mensonges, d'être de "journalisme citoyen" que "tout le monde sait qu'il n'est pas vérifié," bien qu'une grande partie du contenu soit rédactionnelle. Elle se prend les pieds et ne récupère pas élégamment, quand elle parle du "stalking… je veux dire… ce n'est pas du vrai stalking" parce qu'il n'y a aucune expectative de se ne pas faire remarquer en public, et il reconnaît que les vedettes ne sont que des gens normaux. Puis elle demande immédiatement si les célébrités "ne sont pas protégés des pierres jetées par des tas d'argent," est sifflée et recule "[qu']il n'est pas convenable de répandre des mensonges sur les gens," invalidant tout ce qu'elle a dit.

Kimmel en jette aussi une remarque drôlement anachronique: "Je ne comprends pas pourquoi quelqu'un voudrait acheter de la pub sur un site Web."

Une occasion rare. Les célébrités distantes sont humanisées, les médias anciens demandent des explications des jeunes, et la jolie tête parlante se montre opportuniste, prise en flagrant délit. Une belle opportunité de prouver les médias savent mieux, non? Aaron Sorkin l'a vu aussi et a inclu ce moment d'enseignement dans un épisode de "The Newsroom," en copiant du dialogue:

Sauf qu'évidemment, tous les arêtes vives ont bien été supprimées. Au lieu d'une rédactrice d'un tabloïd Internet, il y a un type Zuckerberg en rondeur qui nous vend son entreprise en technologie et son app. Ses réfutations semblent plus authentiques, et il est renversé par les accusations soudaines, pas invité expressément pour les aborder. Les célébrités ne sont pas représentées, ils sont ailleurs, en train d'être traquées et assassinées. Il est un technocrate oublieux qui nuit à la société avec son autisme insensible, elle est la journaliste fatale qui lui demande le compte rendu.

Vous voyez comment une histoire qu'il s'agit complètement des médias a été traduite en quelque chose entièrement différente, sans changer la plupart des mots? Tout depend du détaillage et de la présentation. L'excuse qu'ils ne peuvent pas être entièrement réalistes ou littéraux ne s'applique pas. Je trouve tout ça un peu trop opportun pour promouvoir quelques certains récits. Il confond les sociopathes qui dirigent certaines entreprises avec les personnes qui maintiennent la technologie, avec un vernissage subtil de nerd bashing.

Il peut être un bon exemple de ce dont Chomsky a parlé: quand ils présentent une interprétation des faits, les portiers américains appliquent une optique qui éclaire seulement un cadre très étroit et prévisible, tout en précisant au contraire. Comme vous montrer comment on fait vraiment les nouvelles, un prétexte élaboré que la presse est une force dissidente critique.

Ou il peut être un episode médiocre d'un programme TV défaillant. Mais je suis sûr qu'il y a déjà de nombreux d'articles d'opinion et commentaires qui le discutent en détail de profondeur douloureuse.

De toute façon, en essayant de rendre la réalité dans leur fantaisie, ils l'ont refait à leur propre image. Ils le font souvent. Pas seulement ceux qui écrivent de la fiction.

Célébrez la Diversité

Où est-ce qu'on l'aurait pu voir une fois de plus récemment? Regardons. Qu'est-ce que vous faites quand vous êtes journaliste notable de divertissement, disons, les jeux vidéo, qui apprend que les revues professionnelles sont en grande partie obsolète dans un monde de YouTube, Twitch, Twitter, Reddit, etc.?

Peut-être vous auriez une crise existentielle quand vous vous rends compte d'avoir être juste un mégaphone ivre pour les intérêts commerciaux institutionnels, et vous deviendriez un porte-parole pour vos propres intérêts au lieu. Partagez sur Twitter "Fuck ethics get money," et donnez des conférences dans lesquelles vous ne "prétendez pas d'être impartiale." Mettez en vedette les gens et les projets que vous aimez, promouvez des causes politiques et sociales que vous pensez être importantes, et choisissez les exclusivités vous-mêmes plutôt que les attender. Tout ça pendant que vous rejoignez la course mondiale du piège à clics moderne.

Si une clique assez grande le ferait, il pourrait apparaître une renaissance des indies. Mais elle ne nourrirait qu'une sélection d'idées étroite, et resterait seulement accessible aux ceux qui ont les contacts nécessaires et qui disent les bonnes choses. Si on le ferait dans une industrie qui vaut des milliards, il pourrait créer un ressentiment croissant. Surtout s'il y aurait des années de reportage tendancieux, des conflits d'intérêts, et de promotion des agendas politiques, avec plusieurs carrières goudronnées et à plumes, à cause des rumeurs irresponsables et des mensonges. Dans l'hypothèse, bien sûr.

Dans une telle situation, imaginez-vous que des indications émergent qu'une des favoris des médias est en vrai une personne assez déconcertante. En se profilant comme enfant modèle et une activiste progressiste et développeur de jeux, elle se manipule et ment à travers de sa carrière et ses rélations. Particulièrement il s'agit d'un avertissement et une description de maltraitance émotionnelle, prouvée, par exemple, avec de nombreuses citations de Facebook. Les détails sont assez lubriques, notamment l'infidélité avec un ensemble de Cinq Gars dans l'industrie, incluant son chef et un rapporteur chez Kotaku. En outre, elle présente les Habits Neufs de l'Empereur, parce que son œuvre consiste principalement d'un seul livre-à-la-carte fait avec Twine qui passerait à peine comme projet de semestre de la première année. Il a été salué plutôt pour son sujet—la dépression—que pour sa réalisation.

Imaginez qu'en réponse, d'autres connaissances sortent de l'ombre pour décrire d'autres choses qu'elle pourrait avoir fait. Comme faux témoignage qu'un forum de gens déprimés l'a harcelée, dévoiler des renseignements personels d'un organisme de bienfaisance rival, convaincre ses amis de média de les mettre sur la liste noire, saboter un programme de télé-réalité dans laquelle elle figure, collecter de fonds à un événement qui n'a jamais eu lieu, et en général d'être ressentie par ses contacts précédents.

Plus important, imaginez que ce scandale parvient à la révision ou découverte d'autres conflits d'intérêts et lynchages médiatiques, et qu'on exige la responsabilité, parce qu'il s'agit des plusieurs organes de presses influents et beaucoup de gens en ont assez.

Plutôt qu'admettre que c'est devenu une farce monumentale et ils ont été trompés, les rapporteurs pourraient bien serrer les rangs. Ils tenteraient de concocter un conte que tout la critique qu'elle reçoit est de l'harcèlement, que les joueurs sont des frondeurs de merde et des hyperconsommateurs gémissants qui détestent les femmes, en declarant que l'identité des joueurs est mort dans dix articles publiés sur 24 heurs. Pourtant qu'ils aliéneraient leur lectorat ancien, ils attireraient le nouveaux segment, plus politique, qui verrait tous leurs préjugés sur les jeux vidéo et les joueurs confirmés, ignorant que le média avait déjà connu des thèmes sophistiqués explorés avec maturité.

Pour les nouveaux lecteurs ça pourrait apparaître comme une invitation de s'engager, d'instruire la plèbe de Rome et de défendre les belles vierges des sauvages supposés, en pleine vue des amis sur les médias sociaux, hochant la tête respectueuse. Ils pourraient reformuler les événements comme un "amant éconduit"—bien qu'il l'ait abandonné—qui a rassemblé une foule odieuse dans les profondeurs maléfiques de 4chan—bien qu'il l'ait posté premièrement sur Something Awful et Penny Arcade et les sentiments aient été mitigés. Peut-êtres ils diraient qu'il a dévoilé du "porno de vengeance" quand en fait ce sont ses propres photos érotiques à paiement qui circulent. Ce sont tous des mensonges éhontés ou des admissions de ne pas avoir pris la peine de lire des sources.

Il serait essentiel que les rapporteurs ne jamais mentionnent la campagne de censure qui pourrait se produire chez Reddit, 4chan et d'autres fora de joueurs, et l'effet de Streisand résolutoire qui entraînerait encore plus d'attention. Cela pourrait comprendre une fausse réclamation de droits d'auteur addressée à une personnalité Youtube réputée, aménager un fil de 25 000 commentaires en cimetière, interdire l'accès aux personnes qui mentionnent certains noms, et déclarer que le sujet entier est défendu dans certains subreddits alliés. Parce qu'il faudrait un échange de faveurs dans les coulisses chez les modérateurs, cela pourrait laisser penser qu'il y a même plus de la collusion et même un complot, ce qui amène certaines personnes de migrer en dégoût de 4chan vers un forum auparavant mineur, et ensuite vers Voat de Reddit.

Même si quelqu'un savait exactement ce qui se passerait, une semaine à l'avance, la presse des jeux n'affronterait pas en aucune circonstance les faits et n'admettrait pas l'erreur. Au lieu ils commenceraient à faire la guerre médiatique interminable aux consommateurs qui veulent principalement du responsabilité, et ainsi, deviendraient le plus grand aimant pour les trolls, les drames, les e-célébs et fausses bannières que l'internet verrait depuis sa création. Surtout ceux que le vieux internet connaît déjà.

Exactement comme la victime d'abus Eron Gjoni avait averti, avant d'être étouffé par une ordonnance contestée et possiblement inconstitutionnelle, un gros morceau d'une industry pourrait devenir fou les mois suivants, à cause des machinations égocentriques d'une agresseuse. Les médias du jeu et les feministes auraient beaucoup de matière avec leur nouveau souffre-douleur. Vu que la presse mondiale est vraiment guère plus qu'un jeu au téléphone arabe, ils rapporteraient tous ce que ces experts disaient. Ils se fourniraient de la culpabilité par association sans rechercher, en diffusant une grande mensonge aussi énorme que Goebbels en contracterait le priapisme palpitant. Par exemple, si une commentaire introuvable que personne n'en sait rien était exagérée momentanément à la mort de Lady Di dans le Royaume Champignon. Si des citations aléatoire de chat seraient acceptées comme d'espionnage sur les opinions et motivations des milliers, sans autre recherche ou droit de réponse. Peut-être ils croirions sur parole une menteuse-en-série et vétéran des dump d'enfers, qui figure comme l'archiduc Ferdinand ainsi que Jeanne d'Arc dans son propre DongleGate.

Tout ça pour vidya, les jeux vidéo, tout parce que une bande de rapporteurs se sont trouvés les rois et reines d'un palais personne n'en a vraiment besoin après tout. Ils ont découvert un public convaincu que Twitter et Tumblr était comment on fait l'activisme, que les universités et les congrès devraient êtres des endroits où la recherche de bonne foi s'éteind.

Mais ça serait de la folie.

Encore plus hallucinant serait de voir que, contrairement à chaque fois auparavant, ce entrisme-par-voie-justice-sociale rencontrerait de l'opposition provocante. Que d'une façon ou d'une autre, l'impossible avait arrivé, et les gens de Reddit et 4chan pourraient mettre de côté leurs différences assez longtemps pour qu'ils réussissent à obtenir des résultats. Que tous les olympiades d'oppression, la honte médiatique, les fausses représentations, les jeux de mots, les pièges logiques et le ton moralisateur pourraient avoir pressé assez de gens de dire: non, pour une fois, toi tu prends du recul. Qu'il tenteraient d'élever Twitter vers le plus rationel qu'il sera jamais, c'est-à-dire, pas du tout. Que les gens là seraient si dépendant de la validation digitale, qu'il prendraient le démantèlement immédiat de leur idées maladroits comme l'abus malveillant. Ils pourraient le dénommer la violence digitale aux Nations Unies, montrant du picorage de quelques imbéciles, en auto-bloquant le reste comme une prophétie autoréalisatrice.

Vous pourriez découvrir que, un an plus tard, convaincre une Société de Journalistes Professionels de reconnaître du biais de sélection, du copy-paste, de la paresse et de la flatterie est beaucoup plus difficile qu'en 2011 chez Fox News: cette fois, il y a des alertes à la bombe et peu de gens semblent intéressés qui est responsable, ou même savent comment en rendre compte. Que l'infotocratie de Wikipédia décide de disséminer de la propagande totale pendant un certain temps, coincés dans la citogenèse qu'ils créèrent.

Parce qu'il est bien plus lucratif de ne savoir de rien après de se moquer des enfants autistes d'une position de privilège non méritée.

Le Meilleur des Mondes

Enfin, bouclons la boucle sur ce fer à cheval dément.

Qu'arriverait-t-il si les tant attendues "nouvelles façons d'interagir avec les autres" existent déjà depuis 2001, sous la forme du Tableau d'Images Anonyme? Peut-être pas par coïncidence une invention du Japon, une culture qui a transformée la répression en une forme d'art. Un tableau où l'identité et le prestige n'ont aucune valeur par défaut, et par conséquent, des règles de jeu entièrement différentes s'appliquent, faisant fond sur les pratiques du cyberspace ancien.

Il n'entraîne pas de forum utopique où les intellectuels mènent le débat comme les extraterrestres de Star Trek. Au contraire, vous traitez avec les humains. Il produit des endroits qui mélangent du scepticisme profond avec des insultes accessoires et de l'abus graphique, qui rendent chacun et tout également sans valeur et fictif, jusqu'à preuve du contraire. Apporter l'identité comme point générale en discussion n'est pas simplement malvenu, il est conspué avec exactement ces insultes qui infligent le plus de dommages. La pertinence vient de la résilience, qui est offensé perd immédiatement. L'ingéniosité, l'hyberbole et l'humour sont les mesures de contrôle pour balancer tout, comme des garde-fous contre l'homme sans imagination et son faillite intellectuelle. Les idées doivent se rivaliser pour demeurer pertinents, à une seule condition: aucun sujet est défendu. Surtout les tabous. Ce n'est pas exactement pour les cœurs fragiles.

C'est la seule espace d'internet approchante une "safe space" universelle qui fonctionne, c'est-à-dire, pas du tout, pas comme les definitions du monde hors ligne. Néanmoins, une fois habitué, la pornographie, les trolls, les porte-paroles, les blagues sur les juifs, l'enculisme, ça devient tout si banal et transparent qu'il ne vaut pas la peine de s'inquiéter. Comme il vraiment faut.

Regardez, même qu'elle est aussi vulnérable aux plaintes et au tribalisme qui ont remporté Twitter son énorme succès, elle est beaucoup mieux équipée comme une média et une culture d'en traiter durablement. Peut-être ils raseraient totalement leur propre village pour le rebâtir ailleurs dans des moments de grande détresse. Quand il est tout remplaçable et compris de uns et de zéros de concièrge, est-ce vraiment plus qu'un changement de domicile? Il n'est pas si étrange: on était des nomades, avant que la promesse d'un estomac plein et un toit sur nos têtes nous a convaincus de sacrifier un petit peu de liberté pour plein de sécurité.

Il y a bien du monde qui le savent, rien de nouveau à cela, et une génération grandit qui le considère aussi normal que Facebook et Snapchat. Des autres regardent les chans comme une porte de l'enfer qui donne lieu aux bigots, criminels et harceleurs. Comme la Verité 9/11 chez Occupy, quand vous supprimez certaines idées vers la contre-culture, ne vous étonnez pas quand le dialogue ouvert doit partager l'espace avec des voisins louches, sur une longue queue de frange et de vraie marginalisation. Le détestez à votre guise, mais ne l'ignorez pas.

Pour les vieux lecteurs: quand vous pensez que Donjons et Dragons endoctrine les jeunes dans des sorcières et mages, ou que les crèches tuent les enfants dans des rituels sataniques, vous vous attendez d'être ridiculisé comme bouffon de panique. Peut-être ils ont appris de créer des ponts que vous n'en saviez rien, il y a longtemps, et ils n'avaient pas besoin de se déclarer d'être du bon côté de l'histoire.

Encore plus hilarant sont les gens qui craignent notre cher Reddit, comme par example, des études d'internet terribles. Ils font de leur mieux pour ignorer leurs propres observations et les trouble-fêtes de sa propre coalition, avec un "mais les Femmes de Numerique" spontané pour ajouter de la couleur. Ils voient ce qu'ils veulent voir, sans réaliser que prendre au pied tout ce qu'on y peut lire n'est pas simplement la faute numéro un des débutants, c'est souvent un mechanisme qui fonctionne précisément pour décourager les étrangers peu motivés. Les trolls qu'on repère tout de suite sont les amateurs, et certaines personnes sont si inutiles qu'ils sont une perte sèche pour chaque communauté en ligne où ils s'amarrent, souvent sans reproche.

L'internet pert son valeur unique dès que vous exigez qu'il fonctionne exactement comme la vie réelle. Nos intuitions et activités primaires ne s'adaptent pas bien en dehors d'un seul petit tribu, il paraît un fait empirique aussi que l'expérience vécu. Le reflexe humain par défaut est de compter sur l'identité partagée pour la validation sociale. Leur insecurité amène les gens à éviter la pensée approfondie, ayant trop de peur d'aller à contre-courant. Ils méconnaissent les idées au lieu de les assaillir, et succombent à la pensée de groupe et la chambre d'échos. L'antidote est toujours la même: l'humour et la réflexion affable, y compris la connaissance de soi-même, pas de panique morale.

Il y a une seule question nécessaire pour compléter ce tableau. Pourquoi sont-ils si avides de craindre que le monde des jeux—ou la société entière—déteste les femmes et les minorités, ne les veut pas, les considère inférieures, nécessitant de l'aide et de la sensibilisation unique, même quand il y a des preuves et des témoins qui nous disent autrement? Voici un indice: les guerriers de la justice sociale ont toujours énormes difficultés de simplement considérer des opinions différentes des leurs. Mais celui-ci, il ne le peuvent pas laisser béton.

L'enfer n'est pas simplement pavé de bonnes intentions, le Diable vous sert du pop-corn gratuit.

Et quand vous descendez dans le terrier du lapin, apportez assez de carburant pour brûler vers un apoapse plus élevé.

Les médias ont menti, et j'ai vu beaucoup de gens se tromper que je pensait savoir mieux. C'était absolument baise-cerveau de voir des gens éduqués faire défaut si durement, et mettre ses favoris chemises brunes au nom du progrès. C'est comme ils ne sont jamais sortis du lycée, encore tentant de rejoindre la groupe populaire alors que s'en irriter. Jamais réalisant que le ton moralisateur ne marche que sur les velléitaires, et seulement pour un certain temps. Anita t'arrête svp.

Écoutez et Croyez, prends ta avec-nous-ou-contre-nous mentalité et va réfléchir longtemps à ce que tu as fait. L'Internet est sous surveillance complète, les ententes commerciales mondiales sont sur le point d'élever les entreprises à la niveau des états, et on n'a jamais été si prêt de censurer le net. Mais cette fois, ce n'est pas juste la menace de terrorisme, des pédophiles ou de la pornographie juvénile que l'on utilise pour effrayer les masses et faire taire les critiques. C'est "la violence digitale contre les femmes" et la "suprématie blanc" de la marge. Vous savez ce qui "maintient un système rompue"? Quand une agresseuse populaire jette aux oubliettes un homme victime et on échange les étiquettes sans honte en appliquant du ruban adhésif.

Regardez autour de vous. L'un des plus éminents activistes pour la liberté de presse vit en détention à domicile dans une ambassade, accusé du crime internationalement infâme des rapports sexuels consensuel mais sans préservatif, c'est-à-dire, "un viol." Le sexisme et le racisme ont été utilisés comme armes pervertis de la victimisation inversée par ceux qui n'ont rien de substantiel à offrir mais qui peuvent tirer le grand profit. Le quotidien qui a publié les fuites Snowden s'a réduit au niveau de tabloïd. Ils déshonorent les scientifiques plutôt que les admirer pour leurs réussites. Ce n'est pas de complot, pas dans un forêt des pièges à clics qui continue d'attracter des lecteurs consentants. Ils continuent de mettre la main au portefeuille pour acheter des indulgences post-catholiques pour la faute originelle patriarcale et la culpabilité oppresseur blanche.

PS: Rejeter les autres comme en colère n'est pas une riposte valable (ça s'appelle la projection), ni les décrire comme des bigots, des harceleurs ou des misogynistes/homophobes/racistes intériorisés à cause de choses qu'ils n'ont pas fait. La justice sociale désirait que tout le monde s'éduque, alors, nous leur avons donné exactement ce qu'ils voulaient—avec ses défauts et ses qualités—et ils de même que les médias continuent à le déformer. Il n'y avait pas de grande tromperie au nom de joueurs, pas de foule odieuse type reefer madness qui chasse les femmes du secteur pour être femmes. Juste des dilletantes sans talent, de nombreux de personages des intentions peu claires, beaucoup de trolls de tierce partie, et de nombreux autres qui essayent de travailler autour d'eux pendant qu'on les insulte. Je ne suis pas folle de rage, je simplement n'en veux plus.

(Et ils me diront probablement de droite, sans réfléchir que le socialisme national créa le Volkswagen pour une raison. Oui je sais, Godwin. Wir haben es nicht gewußt.)

Darmok and Jalad at Tanagra

L'arbre de la liberté doit être ravivé de temps en temps
avec le sang des patriotes et des tyrans,

mais ils ont essayé du ketchup.

↧

Occupy WWW Street (NL)

November 29, 2015, 3:00 pm

≫ Next: How to Fold a Julia Fractal

≪ Previous: Occupy WWW Street (FR)

Internetactivisme en Media in het Socialerechtvaardigheidstijdperk

English - Français - Nederlands

"Ik heb geen toegang tot ongewone onderzoeksmethodes en de speciale kennis die ik heb van taal is niet van toepassing op sociale en politieke kwesties. […] Om ideologie te analyseren […] heb je algemeen gezien enkel een beetje open wereldsopvatting, gewone intelligentie en gezond skepticisme nodig." - NC

't Is weer tijd voor mijn serieus lettertype. Helaas is het moeilijk om over courante zaken te spreken zonder meteen een kilo context erbij te kappen. In plaats daarvan ga ik weer in onze recente geschiedenis duiken.

Denk er aan: de boodschapper neerschieten is onbeleefd, en ketterij is middeleeuws.

The Only Working Class Man In The Village

Het is september 2011, Amerikaans links staat in vlam. De boodschap is kordaat en mikt op grote doelen: het 99% publiek wilt dat die 1% gierige vrekken er nu eens mee ophouden met hun giga-ondernemingen als knuppel te gebruiken. Zuccotti Park in New York vult zich met kamperende activisten, daar om hun boodschap mondig te verspreiden.

Zo ook Jesse LaGreca. Met zijn typisch Amerikaanse "Joe Average" look verrast hij velen, vooral de Fox News verslaggever die hem aanspreekt. In het impromptu interview, nooit op Fox uitgezonden, deelt LaGreca forse kletsen uit, met vonken erbij. De suggestieve vragen worden meteen aangeduid, en hij merkt op dat als er nu iets is waarover het nieuws—"de propagandamachine in de media"—niet wilt spreken, dan is het wel zichzelf. Dat het misschien tijd is voor wat participatieve democratie die "niet met nieuwscamera's gefinancierd wordt."

Een journalist duikt op om de wereld te verklaren maar ontdekt dat dit verhaal deels over hemzelf gaat. Hij ontkent het meteen: "We zijn hier om je een kans te geven […] om eender wat je wilt uit te zenden, met eerlijke berichtgeving. […] Dus is er een uitzondering in dit geval, want je zou je boodschap niet zonder ons kunnen verspreiden."

Hij kaatst terug: "Zeker. Neem bijvoorbeeld toen Glenn Beck […] de president beschreef als iemand die blanke mensen en blanke cultuur haat. Dat was een laag moment in de Amerikaanse geschiedenis en jullie hebben daar eigelijk wel een grote rol in gespeeld. Dus 't is tof te zien dat jullie mening aan't veranderen is en dat je wat meer aandacht aan de [rest van het land] besteedt, in plaats van een extreem rechtse marge die de middenstand liever compleet zou vernielen. […] De mensen zouden liever wat economische rechtvaardigheid of sociale rechtvaardigheid zien, je weet wel, Jesusdingen, zoals de armen voeden of voor de zieken zorgen, maar ze kunnen maar niet vatten dat een systeem van winstjagende gezondheidsdiensten niet werkt. […] Ik vind het echt grappig dat de mensen met de grondwet zitten wapperen terwijl ze naar homo-soldaten schreeuwen. […] Als we willen dat de president meer doet, laat ons op een functioneel niveau spreken in plaats van om zijn geboortecertificaat te vragen, […]"

Shots fired, critical hit, no chance of recovery. 't Is begrijpelijk dat de clip "vergeten" werd, en begrijpelijk dat het tot groot faam zou uitlekken als zijnde too extreme for Fox news. Voorspelbare koppen vroegen ons of hij de Slimste Mens op Wall Street was, en legden uit hoe Jesse LaGreca de Vooroordelen van de Media Over Occupy Wall Street Blijft Vernielen. Allemaal verslaggevers die gretig wilden tonen hoe graag zij wel niet iedereen een stem wilden geven.

LaGreca raakte hier zelf aan op TV: "De realiteit is dat ik de enige mens van de arbeidersklasse ben die je op het journaal op zondag gaat zien… of politiek nieuws… misschien ooit. En ik denk dat dat zeer indicatief is voor de mislukkingen van onze media om over de belangrijkste zaken bericht te geven…" De gastheer onderbreekt meteen: "We doen ons best toch, Jesse."

Komt de vraag echter of er een politieke eis is die Occupy zou kunnen samenvatten, iets om "het over te doen gaan," neemt hij een vreemde bocht:

"Ik denk dat de hele beweging over economische rechtvaardigheid gaat. Ik bedoel, voor mij—en ik spreek niet voor Occupy Wall Street, ik geef enkel mijn mening—ik denk dat het een kwestie is van economische rechten en een kwestie van sociale rechten, en sociale rechtvaardigheid. En voor de mensen die het kwalijk nemen dat het woord “sociale” voor “rechtvaardigheid” staat, ik nodig u uit om de grondwet te herlezen."

Ok. Ik heb effe het laborieuze werk volbracht om mijzelf te onderwijzen. De grondwet van de Verenigde Staten verwijst nergens naar sociale zaken als het op rechtvaardigheid aankomt. Het is alsof de schrijvers dachten dat het "oprichten van rechtvaardigheid" genoeg was. Burgers worden niet ingedeeld volgens privilege, macht wordt niet bepaald volgens benadeling en onderdrukking, en het maakt geen allusie naar punching up. Dat is hedendaagse sociale rechtvaardigheid, niet? Of bedoelde hij die "Jesusdingen"? Ik ben geen grondwettelijk advocaat, corrigeer mij gerust.

Een beetje genanter, als het op het inspiratie van andere protestbewegingen komt, in Griekenland en Europa en elders, beweert LaGreca dat "we eigelijk niemand echt volgden."

Occupy is misschien wel spontaan van bestaande groeperingen gegroeid, maar het startschot was een vroege blogpost en een nieuwsblast naar Adbusters' 90 000 abbonnees van "jullie verlossers, rebellen en radicalen," die beweerde dat er een "wereldwijde shift in revolutionaire tactieken is," en vroeg of ze klaar stonden voor hun eigen "Tahrirmoment." De 267 reacties hebben dit bericht met enthousiasme en inzet ontvangen.

LaGreca was ook geen onbekende, hij was al drie jaar lang bijdrager bij Daily Kos. De leden van de site juichten hem toe dat hij "niet in het mediastereotype van die stomme rechteloze ne0-hippies viel" en verspreidden het goede woord terwijl ze op het spektakel toekeken: "Ik vermoed dat Hannity schrik heeft. Zoals het moet!"

Dit is geen aanklacht aan het hof van Mr. LaGreca, hij heeft veel zinvols gezegd en met verve. Maar we zijn nu vier jaar later. Ondanks de gulle zendtijd die zijn nieuwe fame-by-Streisand uitkeerde, en de belofte van verslaggevers om het beter te doen, is de situatie niet verbeterd, niet bij Fox News en niet bij de andere pers die zichzelf definiëert via oppositie.

Als er nu iets is waar de media dol op is, dan is het wel de uitzondering die de regel bevestigt: je kan het publiek niet galvanizeren zonder toegang tot bestaande netwerken en platformen. Als je het opneemt tegen de media, word je omgetoverd tot vermaaksproduct, en de massa zal het geniaal vinden. Jij ook misschien.

Activists Are Dead

Wat is er gebeurd? In juni 2012 verklaart gemeenschappelijk forum en vzw OccupyWallSt.org dat Berichten over de Dood van Occupy Mateloos Overdreven zijn.

"Sinds de buitenzetting van de manifestanten op Freedom Plaza vorige november, is het een media-cliché geworden om over de "Dood van Occupy" bericht te geven. Artikels zijn over heel het web opgesprongen die vrolijk over de mislukte tweede wind van Occupy spreken, deze tamme "Amerikaanse Lente," en de algemene staking in mei die het systeem niet bepaald platlegde.

Het mag geen verrassing zijn dat de algemene pers graag vertelt dat Occupy vermoedelijk gesneuveld is. Zelfs als we negeren dat media onder zakelijk bestuur een sterke wens heeft om sociale bewegingen als Occupy nooit te zien slagen, heeft de media als algemene regel voor ieder verslag een dramatische vertelling nodig. Voor hen moet ieder verhaal een boeiende boog hebben met een begin, een midden en een einde."

"Volgens de media […] weerklonk de oproep bij velen, [maar] kreeg de beweging haar dramatische impuls pas toen de politie op foto stond terwijl ze geweldloze manifestanten aanvielen en pepper-sprayden. Het bereikte haar vroege einde toen de politie de verschillende kampen met geweld opruimde. Het enige dat er nog van over blijft, buiten enkele norse paters die maar niet kunnen begrijpen dat Occupy gedaan is, is de eigen populistische retoriek van de 99%—die moedwillig opgepikt is door Democratische front-groeperingen zoals MoveOn.org om de Obama-verkiezingsmachine te voeden."

Allez soit. Is er iets op TV vanavond? Een presidentieel debat zegt ge?

"Deze populaire vertelling […] is zo succesvol geweest dat zelfs zij die nog steeds in de Occupybeweging actief zijn het deels absorberen. […Vergaderingen] zijn vaak borderline publiekstherapie. Op het internet worden ietwat zelfophemelende Paul Krugman-y artikels gepost en gerepost dat Occupy “tenminste de publieke dialoog verschoven heeft” […] om elkaar te herinneren dat Occupy tenminste een klein effect had."

Plezant zeg. De post legt verder uit hoe Amerikanen geschiedenis zien als "iets dat andere mensen (vaak blanke, welstandige mannen) lang geleden deden" en vergeten dat er "honderdduizenden mensen nodig waren—vooral jonge mensen, vrouwen, en laagstandige mannen–om de ideeën van democratie te verspreiden. […] [We] zien Occupy als een spektakel […] van mensen die anders zijn dan ons. […] Occupy-vriendelijke celebrities en artiesten, zwarte-blok-stijl anarchisten en de despotische burgemeesters van onze steden zijn de personages in dit drama, [die het uitstrijden] voor nieuwskoppen op het podium van onze verloederde steden."

"De echte magie van Occupy was dat het dit allemaal afwees. Niemand had meer macht dan iemand anders, […] het kon niemand in Occupy echt schelen dat we door de algemene pers genegeerd werden. We hebben geen groep idioten van Time Magazine nodig om ons te prijzen voor onze protestvaardigheid. […] Het doel van Occupy was om samen te komen als een gemeenschap van gelijken, om een andere toekomst te vorderen dan we gegeven werden, en om een traditie van democratische vooruitgang te hernieuwen […] Het was een manier om met elkaar nieuwe omgangsvormen te onderzoeken."

1 2 3 4 5

De auteur benadrukt dat we "Occupy niet als tijdelijke rage of linkse koortsdroom mogen afwijzen." Ok. Buiten een "feeling in the air", zijn de beschreven effecten dat "meer dan 7200 Occupyprotesteerders gearresteerd zijn in de Verenigde Staten" met vele "geslagen en gemarteld." Occupy is gedemoniseerd omdat "de media onder druk gezet werd om […] in een ongunstig licht bericht te geven." Wat nog?

"We moeten herinneren dat het woord “occupy” een […] oproep tot actie is, niet de actie zelf. [Het] was nuttig om individuen en organisaties buiten te brengen die vroeger geïsoleerd waren of zich slechts op één kwestie richtten. […] Het belangrijkste is dat er een brede gemeenschap van oppositie aan het vormen is die vele sociale barrières brugt, en zij die de macht hebben zijn nu heel bang."

Negeer de dikkenekkerij op 't einde, er is iets veel markanter en verhelderend.

Het resultaat van Occupy, zo zegt een bewijs van zijn blijvende waarde, is dat 7200 mensen zichzelf tot slachtoffers en martelaars gemaakt hebben, ondersteund door passief idealistisch verzet van buitenaf. Ze zijn langzaam depressief en irrelevant geworden, nadat de pers die in de publieke interesse moest spreken hen verkeerd afgeschilderd heeft. Ze vonden dit ok.

De beste leugens zijn half-waar. Dit collectieve absoluut-niet-rouwritueel was een kern van steenharde realiteit in een schil van narcistisch zelfbedrog. Men zou het zelfs Stockholm Syndroom kunnen noemen. Dit zijn de geëvolueerde Pokémongenera van de cultuur van het slachtoffer, waar slachtofferschap gevierd wordt, omdat het je immense macht kan bieden.

Burgers and Fries with Ketchup

Ik geloof het officiële Occupyverhaal niet, noch dat van de media noch van de beweging zelf, omdat ik zelf ben gaan kijken in Vancouver. Ik bedoel niet de 9/11 Truthers of de wietactivisten: die horen er nu eenmaal bij, waar moeten zij anders naartoe? Nee, ik merkte twee specifieke groepen op.

Eerst waren er de glanzende latté kids die human microphone speelden—net zoals in Biscotti Park!—met vriendjes assorti en te modieus om arm te zijn. Het leek mij creepy: omdat geluidsversterking verboden was zoals bij vele Occupy's, moest iedereen elkaar naroepen, alsof ze mantras zongen. Verdere sabotage was een algemeen gebrek aan focus of moderatie. Voeg een mix mensen toe met meer gevoelens dan ze mee weg kunnen, en het gaat oertraag nergens naar toe, net als Twitter in The Matrix.

Dit zicht was geen publiek forum voor verlicht debat, geen stadsvergadering, het was een irritante zeepkist waar geen ordinaire mens meer dan 5 minuten naar zou luisteren. Ze hadden van activisme gehoord, ze wisten zelfs hoe het eruit zag en hoe het klonk, maar ze dansten enkel de pasjes. Net zoals op vele livestreams die ik elders checkte.

Ten tweede, de eigelijke daklozen van de stad, werkloos en onwerkbaar, mentaal gestoord, chronisch verslaafd of op andere wijze verstoten door de maatschappij, die net in die ruimte rondhingen die ze iedere dag al occuppeerden. Gooi deze twee groepen samen, quelle surprise, de verlichtte idealen over gelijkwaardigheid, privilege, diversiteit, discriminatie, vrije meningsuiting en gemeenschappelijk bezit worden getest. Jeugd wier idee van kamperen niveau Werchter is, met proper sanitair en waterkraantjes, vonden een openbare plaats in putje winter minder aangenaam. Meer nog, als het onvermijdelijke conflict en de misdaad opduiken, is het waarschijnlijk geen goed idee om rechtvaardigheid over te laten aan zij die met woorden als "triggered" en "rape culture" omgaan als ware het snoep.

In plaats van deze werkelijkheid te confronteren, adopteerden ze de progressive stack, die "vrouwen en traditioneel gemarginaliseerde groepen aanspoort om vóór mannen te spreken, vooral blanke mannen." Als je wat rondgraaft vind je klachten over "vergaderingen ontspoort door vrouwen en minderheden die geobsedeerd waren met identiteitspolitiek. […] [Plannen] werden praktisch altijd beantwoord met tegenstand van feministen, queeractivisten, zwarten, latino's, enz. Vaak hadden deze mensen zelfs geen specifieke zorgen. […] Ze stonden gewoon recht en begonnen te klagen over een gebrek aan diversiteit, over de victimisatie van hun specifieke identiteitsgroep, etc. […] Slimme, vaardige, georganiseerde blanke kerels vertrokken. Ze trapten het af in frustratie."

Er zijn veel verhalen van ruzie en misbruik, het was vruchtige grond voor de media, sociaal of niet. Wat er nu juist wel of niet gebeurd is bij specifieke incidenten was zelden relevant, en zulke zorgen werden belangrijker dan het financieel en economisch systeem of de instituten die het onderleggen. We moesten blijkbaar het gendergericht geweld in de Occupybeweging confronteren: "Wat er nu juist gebeurd is, is onduidelijk. Sommigen beweren dat er een poging tot verkrachting was. Anderen schilderen heel de affaire af als niets meer dan een jonge vrouw wier persoonlijke ruimte werd binnengevallen door een dronken man." Denk je dat dat onderscheid misschien belangrijk is?

Sociaalvaardige media overdreef gretig, mannen werden herinnerd om "niemand te verkrachten, ooit"—thx xojane—en dat "ruimtes voor vrouwen gebouwd worden om geavanceerde gesprekken te expediteren over kwesties waar vrouwen mee omgaan (en, idealiter, iedereen die zich niet als man identificeert) zonder dat die discussies regelmatig onderbroken worden omdat andere mensen in de groep onderwezen moeten worden."

Met andere woorden… Mannen, buiten en zwijgen. Vrouwen, geen vragen stellen. Schitterend gedaan social justice, je hebt alles wat je zogezegd haat naar je idealistische utopie gebracht, en al de anderen weggejaagd.

Het is gemakkelijk om de schuld af te schuiven op whacky marginalen met vreemd haar en een brilleke. Als je Stephen Colbert heet, dan kan je ze zelfs genadeloos uitlachen voor miljoenen en grote punten scoren. Het echte probleem is de activistenmassa zelf en de media waar ze in geloven. Ze geraken te graag vast in hun eigen problemen, dingen waarvan ze zelf toegeven dat het eigelijk puur een microcosmos is van een menselijke gemeenschap. De fout is om geïsoleerde incidenten als universeel voor te stellen, terwijl de olifant in de kamer genegeerd wordt. Dat is een ziekte geconcentreerd in de media, maar niet alleen bij hen.

Het is moeilijk om een Belangrijk Gesprek over het Deconstrueren van Zaken serieus te nemen, als social justice niet weet hoe te socializen.

/Chomsky/ was Right

In Language and Responsibility (1979) legt Noam Chomsky echter welsprekend uit. Hij beschrijft vooral zijn theorieën van linguïstiek, te niche voor de meesten. Maar het eerste hoofdstuk "Politiek" begint heel anders, met ideeën die later zouden terugkomen in Manufacturing Consent: The Political Economy of the Mass Media (1988). Gevraagd wat de relatie is tussen zijn methodes en zijn bekendere politieke schrijfsels:

"Ik heb geen toegang tot ongewone onderzoeksmethodes en de speciale kennis die ik heb van taal is niet van toepassing op sociale en politieke kwesties. […] Om ideologie te analyseren […] heb je algemeen gezien enkel een beetje open wereldsopvatting, gewone intelligentie en gezond skepticisme nodig."

"Neem [de] intelligentsia in een maatschappij als de onze. Deze sociale klasse, met historici en andere geleerden, journalisten, politieke commentators en zo voort, neemt het op zich om een beeld van sociale werkelijkheid te brengen via analyse. […] Zij dienen als bemiddelaars tussen de sociale feiten en de bevolkingsmassa: zij scheppen de ideologische onderbouw voor de sociale praktijk. Lees het werk van de specialisten in hedendaagse zaken en vergelijk hun interpretatie met wat er gebeurd is, vergelijk wat ze zeggen met de feiten. Je vindt vaak grote en redelijk systematische afwijkingen […]."

"Met een beetje moeite en toepassing [ziet] eender wie van het systeem wilt vrijkomen […] dwars door de methodes van vervorming. […] Iedereen kan dat. […] Vaak slecht uitgewerkt, […] produceert men sociale en politieke analyse om speciale interesses te verdedigen in plaats van een feitelijke verklaring te brengen." (p3-4)

De specifieke situatie in de Verenigde Staten wordt toegelicht. Hij vergelijkt de opgenomen feiten met de aanvaardde verklaring van bijvoorbeeld de McCarthy-paniek, de studentenrevolutie, de burgerrechtenbeweging, de Vietnamoorlog, en meer. Hij praat over interventies van de VS in het thuisland en elders, legitiem of illegitiem. Het overheersende thema is duidelijk:

"Hier in de VS is er een verbazingwekkende graad van ideologische uniformiteit voor zulk een complex land. Hier zijn algemeen twee redenen voor. Eerst is er de verbazende ideologische gelijkheid van de Amerikaanse intelligentsia in groep, die zelden afwijken van één van de varianten van staat-kapitalistische ideologie (liberaal of conservatief), een vaststelling die voor verklaring vraagt. Tweede is dat de massa-media kapitalistische instituten zijn. […] Het feit dat deze instituten de ideologie van overheersende economische interesses weerspiegelen kan geen verrassing zijn." (p9)

Een hoofdartikel in de New York Times van 1975, een terugblik op de Vietnamoorlog, legt uit dat er mensen zijn die denken dat "de oorlog […] anders gevoerd had kunnen worden" en zij die denken dat "een haalbaar niet-Communistisch Zuid-Vietnam altijd een mythe was." Chomsky speelt vlot kaart:

"Ze zeggen niets over de logische mogelijkheid van een derde positie: namelijk, dat de VS […] het legaal of moreel recht niet had, om zich met macht in de interne zaken van Vietnam te moeien. […] Merk op dat wanneer de Times het spectrum van ideologisch debat opzet, de positie van een groot deel van de vredesbeweging eenvoudigweg uitgesloten wordt. Niet dat deze fout is, maar eerder ondenkbaar, onuitdrukbaar. […]

Hier hebben we een fantastische illustratie van hoe propaganda werkt in een democratie. Een totalitaire staat legt officiële doctrine gewoon op—duidelijk, expliciet. […] oppositie spreekt op eigen risico. In een democratisch systeem van propaganda wordt niemand gestraft (in theorie) voor bezwaren tegen officieel dogma. Meer nog, dissidentie wordt aangemoedigd.

Wat dit systeem probeert te doen is om de limieten van mogelijke denkgang vast te zetten: supporters van officiële doctrine aan de ene kant, en de critici—levendig, moedig, en veel bewonderd voor hun onafhankelijkheid van oordeel—aan de andere. De haviken en de duiven. Maar we ontdekken dat ze allemaal bepaalde stille veronderstellingen hebben, en dat het net deze zijn die cruciaal zijn.

Ongetwijfeld is een propagandasysteem effectiever wanneer de doctrines geïnsinueerd worden in plaats van geasserteerd. […] Hoe levendiger het debat, hoe beter de algemene doctrines van het systeem ingeprent worden, zwijgend verondersteld aan alle kanten. Vandaar de uitgebreide pretentie dat de pers een kritische oppositionele kracht is—misschien zelfs té kritisch voor een gezonde democratie—terwijl zij praktisch compleet onderdanig is aan de algemene principes van het ideologisch systeem." (p36-39)

Dit was in de jaren 70. Hoe zou dat er in het internettijdperk uitzien?

Don't Stalk Me Bro

In 2006 lanceerde Gawker Media de Gawker Stalker Maps, een site waar iedereen gevraagd werd om publieke meldingen van bekende gezichten in te sturen. Hollywoodsterren zagen de typische bende papparazzi gevolgd door nu nog vreemdere mensen van op het internet, en konden er niet bepaald mee lachen.

Wanneer Jimmy Kimmel hen een jaar later op het matje roept, gaat redactrice Emily Gould voor het goud in de mentale gymnastiek. Ondanks de oorspronkelijke vraag voor info "zo snel als je het kan insturen," beweert ze dat de kaart meestal traag updatet. Ze lacht bewijs van gepubliceerde leugens weg als zijnde "burgerjournalistiek" waarvan "iedereen weet dat dat niet nagekeken wordt," hoewel veel van de inhoud redactioneel is. Ze struikelt en mist haar recovery, met "een stalking…heh…het is geen echte stalking," omdat er nu eenmaal geen verwachting is om niet in het openbaar opgemerkt te worden, en het bevestigt alleen maar dat bekende gezichten gewone mensen zijn. Ze vraagt dan meteen of celebrities niet "door bergen geld beschermd worden als men stenen naar hen gooit," wordt afgefloten en moonwalkt in respons dat "het niet ok is om leugens over de mensen te vertellen," in een complete invalidering van al het vorige.

Kimmel gooit er een leuke anachronistische zinger bij: "Ik snap nu eenmaal niet waarom iemand advertenties op een website zou willen kopen."

Het is zeldzaam. Verre celebrities worden gehumaniseerd, nieuwe media krijgt lik op stuk van de oude, en de vrolijke poppenkast stelt zich bloot als de schaamteloze opportuniste die ze is. Een kans om te tonen dat de media beter weet, niet? Aaron Sorkin zag het ook en nam dit teachable moment op in een aflevering van The Newsroom, met dialoog deels letterlijk overgenomen:

Helaas natuurlijk, al de scherpe kantjes zijn weggevijld. In plaats van een bubbly redactrice van een internetroddelblad, is er een mollig Zuckerberg-type die zijn tech startup en app zit te verkopen. Zijn bezwaar lijkt oprechter, en de plotse beschuldigingen zijn onverwacht, niet het geplande onderwerp. De celebrities worden niet voorgesteld, zij zijn elders, druk bezig achtervolgd en vermoord te worden. Hij is een onwetende technocraat die de maatschappij schaadt met zijn egoïstisch autisme, zij is de charmant-maar-genadeloze journaliste die hem op de feiten drukt.

Zie je hoe een compleet media-on-media verhaal herschreven werd tot iets helemaal anders, zonder zelfs de meeste woorden aan te raken? Het zit 'em volledig in de afwerking en presentatie. Het excuus dat het niet helemaal realistisch of letterlijk mag zijn is niet van toepassing. Ik vind de hele zaak nogal handig om bepaalde visies in een goed licht te stellen. Het schept verwarring tussen de sociopaten die bepaalde zaken draaien, en de mensen die de technologie onderhouden, met een subtiel laagje nerd bashing er bovenop.

Het zou best wel eens een goed voorbeeld kunnen zijn van wat Chomsky zei: wanneer ze een interpretatie van de feiten brengen, schuiven Amerikaanse retorici een lens in die enkel op een heel smal en voorspelbaar kader schijnt, terwijl ze net het omgekeerde beweren. Zoals tonen hoe het nieuws nu echt gemaakt wordt, een uitgebreide pretentie dat de pers een kritische oppositionele kracht is.

Of misschien is het gewoon een middelmatige aflevering van een uitsputterend TV-programma. Alleen, ik ben zeker dat er al ontelbare artikels en reacties zijn die dit in pijnlijk diep detail nagaan.

In ieder geval, terwijl ze net realiteit in hun fantasie wilden brengen, hebben ze het in hun eigen beeld herschapen. Ze doen dat wel vaker. Niet alleen degene die fictie schrijven.

Celebrate Diversity

Waar zouden we dit onlangs weer allemaal gezien hebben? Laat ik het effe allemaal uitleggen. Wat doe je als je een bekende verslaggever bent in entertainment, bijvoorbeeld video games, en ontdekt dat een handelspers grotendeels overbodig is wanneer YouTube, Twitch, Twitter, Reddit en anderen bestaan?

Wel, misschien heb je dan een existentiële crisis, wanneer je beseft dat je eigelijk gewoon een zatte megafoon was voor zakelijke marketinginteresses, en beslis je daarom een antenne voor je eigen interesses op te zetten. Spring op Twitter en post "Fuck ethics get money," geef talks waar je "geen pretentie hebt van onpartijdig te zijn." Geef aandacht aan mensen en projecten die je leuk vindt, verspreid politieke en sociale ideeën die je belangrijk vindt, en kies de exclusieve hits zelf in plaats van te wachten tot ze in je schoot landen. Terwijl je meedoet aan de globale neerwaartse race zijnde modern clickbait.

Als genoeg mensen in een clique dit doen, dan zou dat best wel eens de schijn van een indie renaissance kunnen opwekken. Maar het zou enkel aandacht voeden aan een heel smalle context, enkel toegankelijk voor hen met de juiste connecties en die de juiste dingen zeggen. Als dit allemaal gebeurt in een multi-miljarden-dollars-industrie, dan zou dat best wel eens op wat tenen kunnen trappen. Vooral als er bijvoorbeeld jarenlange problem waren met tendentieuze journalistiek, belangenverstrengeling, en politieke agenda's, met meerdere carrières permanent besmeurd met pek en veren, via onverantwoordelijke roddel en leugens. Puur hypothetisch, natuurlijk.

In zulke omgeving, stel je voor dat bewijsmateriaal opduikt dat één van de nieuwe medialievelingen eigelijk een redelijk gestoord individu is. Terwijl ze zich profileert als Moeder Theresa en een getalenteerde progressieve activiste en game-ontwikkelaar, manipuleert en liegt ze zich dwars door haar carrière en relaties. Dit alles is in de vorm van een waarschuwing en gedetailleerde beschrijving van emotioneel misbruik, bewezen met, bijvoorbeeld, uitgebreide Facebookcitaten. De details zijn vrij pikant, namelijk ontrouw met een handvol zogenaamde Five Guys in de game industrie, inclusief haar baas en een Kotaku-verslaggever. Bovendien heeft deze keizerin geen kleren aan, want haar "werk" bestaat grotendeels uit één kies-je-eigen-verhaal Twine boek dat amper zou voldoen als een eerstejaarsstudente's half-jaarproject. Het werd vooral geloofd voor het onderwerp—depressie—niet de uitwerking.

Stel je bovendien voor dat in respons, andere kennissen uit de kast komen om andere verhalen te vertellen. Zoals valse beschuldigingen dat een forum van depressieve mensen haar lastig viel, persoonlijke gegevens lekken van een goed doel gezien als rivaal, hen zwartschilderen tegen vrienden in media, een reality TV game show waar ze in meespeelt saboteren, geld vragen voor een ander evenement dat nooit plaats had, dit alles door vorige contacten redelijk kwalijk genomen.

Maar belangrijker, stel je voor dat dit schandaal een druppel is, en dat vorige belangensverstrengeling herbezocht of ontdekt wordt, vorige lynchpartijen in de media besproken worden, en verantwoording gevraagd wordt, omdat het over meerdere invloedrijke sites gaat en een hoop mensen het beu zijn.

In plaats van toe te geven dat het allemaal één grote klucht geworden is, en iedereen misleid werd, zouden verslaggevers best wel eens de rangen kunnen sluiten. Ze zouden een verhaal kunnen bedenken dat al de kritiek die zij ontving onaanvaardbare intimidatie was, dat gamers een stelletje stompe strontgooiers en tierende hyperconsumenten zijn die vrouwen haten, in een declaratie dat de gameridentiteit dood is, in tien artikels gepubliceerd in 24 uur. Hoewel dit hun bestaande publiek zou wegjagen, zou dit met plezier ontvangen worden door het nieuwere, meer politieke segment, die al hun voorbestaande foute ideeën over video games en gamers bevestigd zouden zien, compleet onwetend dat het medium al een lange geschiedenis had van ingewikkelde thema's op volwassen wijze gebracht.

Nieuwe lezers zouden dit misschien kunnen zien als startschot om het plebs van Rome te onderwijzen en de vrome maagden te verdedigen van de zogezegde barbaarse horde, terwijl iedereen op social media kan meekijken, nodding respectfully. Ze zouden heel de affaire best wel eens kunnen herschrijven als een "afgewezen ex"—hoewel hij haar dumpte—die een haatbende bijeen riep op 4chan—hoewel hij eerst op Something Awful en Penny Arcade postte en de reactie gemengd was. Ze zouden misschien spreken over "wraakporno" hoewel het eigelijk haar eigen betaalde erotica is die circuleert. Dit zijn allemaal ofwel directe leugens ofwel een bekentenis dat het te veel moeite was om bronmateriaal te lezen.

Het zou cruciaal zijn dat berichten nooit spreken over de gecoördineerde censuur die op Reddit, 4chan en andere gamingfora zou kunnen plaatsnemen, en het resulterende Streisand-effect dat er enkel meer aandacht naar zou brengen. Bijvoorbeeld een valse copyrightclaim op een prominente YouTuber, een kerkhof van 25 000 gewiste reacties, mensen bannen gewoon omdat ze bepaalde namen durven vermelden, en heel het onderwerp interdit maken op bepaalde geallieerde subreddits. Hiervoor zouden gunsten moeten ingeruild worden achter de schermen bij de moderators, en dit zou best wel eens de schijn van nog meer collusie en zelfs regelrechte samenzwering kunnen opwekken, zodat sommigen in degout zouden verhuizen van 4chan naar een voordien piepklein forum, en daarna naar Voat van Reddit.

Zelfs als iemand volledig voorspeld had wat er ging gebeuren, een week voordien, dan zou de gamingpers niet, onder welke omstandigheden ook, de realiteit confronteren en fouten toegeven. In plaats daarvan zouden ze een lange en uitgerekte mediaoorlog beginnen tegen consumenten die vooral verantwoording willen, en zo de grootste magneet voor trolls, drama, e-celebrities en valse vlaggen worden die het internet tot nu toe gezien had. Vooral diegene die het oude internet al lang kende.

Net zoals het slachtoffer van misbruik Eron Gjoni gewaarschuwd had, voor hij monddood gemaakt werd door omstreden en mogelijk ongrondwettelijke juristiek, zou een heel segment van een industrie wel eens compleet geschift kunnen gaan de maanden nadien, wegens de egoïstische machinaties van een misbruikster. Games en feministische media zouden eindeloos materiaal hebben met hun nieuwe zondebok. Omdat de globale pers eigelijk vooral telefoontje speelt, zouden ze herhalen wat deze experts hen gezegd hadden. Ze zouden met de knuppel van collectieve schuld zwieren zonder verder te kijken dan hun neus lang is, en onwetend zulk een massaal grote leugen vertellen dat Goebbels er kloppend priapisme van zou krijgen. Bijvoorbeeld als één anonieme internet comment waar niemand iets van weet ogenblikkelijk de dood van Princess Diana in the Mushroom Kingdom werd. Als willekeurige IRC quotes als spion-thriller feiten aanzien werden over de meningen en motivaties van duizenden, zonder onderzoek of recht op antwoord. Misschien zouden ze het woord nemen van een seriële leugenaar en zelf-beschreven helldump-veteraan, die zowel als aartshertog Ferdinand en Jeanne d'Arc in haar eigen DongleGate figureert.

Dit allemaal om vidya, video games, allemaal omdat een hoop verslaggevers koningen en koninginnen werden in een kasteel waar amper iemand nog naar omkeek. Ze vonden een publiek dat overtuigd was dat je activisme via Twitter en Tumblr deed en dat een campus of een congres een plaats was waar onderzoek ter goeder trouw sterft.

Maar dat zou zot zijn.

Nog hallucinanter zou zijn om te zien dat, in tegenstelling tot al de vorige keren, dit soort entryisme-via-social-justice tegen een muur van moedwillige oppositie zou knallen. Dat op één of andere manier het onmogelijke gebeurd was, en dat mensen van Reddit en 4chan hun verschillen lang genoeg konden overkomen om effectief werk gedaan te krijgen. Dat al de oppression olympics, de lynchpartijen, de karikaturen, het woordenspel, de denkfouten en morele aanstellerij eindelijk genoeg mensen over de schreef had geduwd om te zeggen: nee deze keer, neem jij eens een stap terug. Dat ze zouden proberen om Twitter om te vormen naar het rationeelste discussieforum dat het ooit zou zijn, namelijk, helemaal niet. Dat de mensen daar zo verslaafd zouden zijn aan digitale bevestiging en goedkeuring, dat ze de kordate afbraak van hun slordige ideeën zouden verwarren voor misbruik. Ze zouden het misschien zelfs cybergeweld kunnen noemen bij de VN, met enkele idioten geselecteerd als toonbeeld voor effect, en de rest geautoblokkeerd in een zelfverwekkende profetie.

Het zou best kunnen dat je, een jaar in, ontdekt dat het nogal moeilijk is om de Society of Professional Journalists te doen erkennen dat er probleemniveaus van partijdigheid, copypasting, luiheid en strooplikkerij zijn. Veel moeilijker dan in 2011 met Fox News: deze keer zijn er bommeldingen en niemand lijkt bepaald geïnteresseerd wie de daders zijn of zelfs hoe hierover bericht te geven. Dat Wikipedia's infotocratie gewoonweg omschakelt naar voluit propaganda voor een tijd, vastgerot in de citogenesis die ze zelf schepten.

Omdat het nu eenmaal veel lucratiever is om dom te doen nadat je autistische kinderen voor de lol gebruikt, vanuit een positie van onverdiend voordeel.

Brave New World

Laten we de toer afmaken op dit hoefijzer van waanzin.

Wat als het nu bleek dat Occupy's langgezochte "nieuwe manier om met elkaar om te gaan" al bestond sinds 2001, in de vorm van het Anonieme Image Board? Misschien niet toevallig een uitvinding van Japan, een cultuur die repressie tot een kunstvorm bracht. Een forum waar identiteit en status geen standaardwaarde hebben, en bijgevolg compleet andere spelregels gehanteerd worden, als evolutie van de praktijk van vroege cyberspace.

Het leidt niet tot een utopisch forum waar intellectuelen pontificeren alsof ze op de set van Star Trek staan. Integendeel, dit gaat over homo sapiens. Het leidt tot plaatsen met een mix van diep skepticisme, willekeurige beledigingen en grafische schendingen, waar iedereen en alles even waardeloos en nep is tot het tegendeel bewezen is. Het is niet alleen onwelkom om identiteit als algemeen argument op te brengen, het wordt afgeschreeuwd met net die beledigingen die de meeste schade doen. Relevantie is het resultaat van weerstand, wie zich beledigd voelt verliest. Creativiteit, overdrijving en humor zijn de controlemechanismen om het allemaal uit te balanceren, als non-stop bullshit test tegen de ongeïnspireerde mens en zijn intellectueel bankroet. Ideeën moeten concurreren om relevant te blijven, maar wel met een belangrijke nota: geen enkel onderwerp is verboden. Vooral taboe. 't Is niet bepaald voor boterbloemekes.

Het is de functioneelste, meest universele safe space die het internet heeft, namelijk, helemaal niet, niet volgens enige offline definitie. Eens je het gewoon bent… de porno, de trolls, de marchanten, de jodengrappen, de jeanetterij, het wordt allemaal zo banaal en transparant dat het de moeite niet is om je er druk in te maken. En dat is nu nét hoe het zou moeten.

Kijk, hoewel het even zwak is voor al het gevit en al het tribalisme dat Twitter zo successvol maakte, is het veel capabeler zowel als medium en cultuur om er duurzaam mee overweg te gaan. Misschien zouden ze zelfs hun eigen dorp platbranden en elders opnieuw opbouwen in grote nood. Als het allemaal bestaat uit triviale eentjes en nullekes van de conciërge, is dat veel meer dan een adreswijziging? Dit is niet zo vreemd: onze voorouders waren nomaden, totdat de belofte van een volle buik en een permanent dak boven ons hoofd ons overtuigde om een beetje vrijheid op te geven voor een hele hoop veiligheid.

Er zijn velen die dit weten, er is niks nieuw aan, en een generatie is er nu mee aan't opgroeien, even normaal als Facebook en Snapchat. Sommige anderen echter kijken toe op de chans als ware het een hellepoort naar eindeloze kwezels, criminelen en lastposten. Net zoals 9/11 Truth bij Occupy, als je bepaalde ideeën in de tegencultuur drukt, verbaas je niet dat zij die openlijk willen spreken ruimte moeten delen met twijfelachtige buren, met een heel lange staart van fringe en echte marginalizatie. Je vindt dit misschien niet fijn, maar je kan er niet naast kijken.

Voor oudere lezers: als je denkt dat Dungeons and Dragons tieners tot heksen en tovenaars omdoctrineert, of dat creches kinderen doodden in satanische rituelen, verwacht dat mensen je uitlachen als onwetende paniekerige lomperik. Misschien hebben ze wel jarenlang geleden bruggen leren bouwen zonder dat je het wist, en was het niet nodig om zich aan de "juiste kant" van de geschiedenis te attacheren.

Het is nog hilarischer wanneer ze dit zeggen over dear old Reddit, bijvoorbeeld in belachelijk slecht internetonderzoek. Ze doen hun best om hun eigen conclusies en de troepschoppers aan hun kant te negeren, met een willekeurige "but Women in Tech!" ertussen. Ze zien wat ze willen zien, zonder te beseffen dat alles letterlijk nemen niet alleen de voorspelbaarste fout is, het is vaak een mechanisme dat dient om ongemotiveerde buitenstaanders buiten te houden. De trolls die je meteen opmerkt zijn de amateurs, en bepaalde mensen zijn zo nutteloos dat ze een netto kost zijn voor iedere onlinegemeenschap waar ze aanmeren, vaak zonder wedergevolg.

Het internet verliest zijn unieke waarde van zodra je wilt dat het net als het echte leven werkt. Onze primaire instincten en activiteiten zijn niet zo geschikt voor meer dan één kleine stam, dat lijkt zowel empirisch feit als persoonlijke ervaring. Onze standaardimpuls is om op gedeelde identiteit terug te vallen voor sociale validatie. Onzekerheid weerhoudt mensen van kritisch te denken, te bang om tegen de stroom in te zwemmen. Ze schrikken weg van ideeën in plaats van ze te confronteren, en gaan onder aan groupthink en echokamers. Het tegengif is nog steeds hetzelfde: goedwillige humor en reflectie, ook van zichzelf, niet morele paniek.

Er is maar één vraag die je moet beantwoorden om dit plaatje te vervolledigen. Waarom geloven ze zó graag dat iedereen in gaming—of in heel de maatschappij—vrouwen en minderheden haat, er niet wilt mee omgaan, ze als inferieur behandelt, zodat speciale programma's en ondersteuning nodig zijn, zelfs als er tegenbewijs en getuigen zijn die anders beweren? Hier is een hint: Social Justice Warriors hebben altijd enorm veel moeite om een verschillende mening te beschouwen. Maar deze, daar kunnen ze niet over zwijgen.

The road to hell is not just paved with good intentions, the Devil provides free popcorn at every station.

En als ge een konijnenpijp binnenrijdt, zorgt dat ge genoeg brandstof meepakt om terug te vallen naar nen hogeren apoapsis.

De media heeft gelogen, en veel mensen die beter zouden moeten weten trapten er in. Het was complete hersenneukerij om opgeleide breinen zo hard te zien falen en hun favoriete bruine hemd voor de vooruitgang uit de kast te halen. 't Is alsof ze nooit het middelbaar ontgroeiden, nog steeds bij de toffe groep proberen te horen terwijl ze het hen kwalijk nemen. Nooit beseffend dat moreel theater een truk is die enkel op de zwakwilligen werkt, en slechts voor een tijd. Anita pls stop.

Listen en Believe, neem u met-ons-of-tegen-ons ideologie en ga eens goed nadenken over wat ge nu juist gedaan hebt. Het internet wordt volledig onderschept, globale handelsverdragen staan op punt om multinationals tot het niveau van staten te eleveren, en men is nog nooit zo bereid geweest om het net te censureren. Deze keer is het niet alleen de bedreiging van terrorisme, pedofilie of kinderporno die gebruikt worden om de massa af te schrikken en critici de mond te snoeren. Het is "cybergeweld tegen vrouwen" en marginale "white supremacy." Weet ge wat een "kapot systeem in stand houdt"? Als ge een populaire misbruikster over haar ondergeknoet mannelijk slachtoffer laat bulldozeren, en dan schaamteloos de etiketjes omwisselt terwijl ge mondtape opplakt.

Kijk eens rond. Één van de prominentste activisten voor de persvrijheid zit in huisarrest op een ambassade, beschuldigd van de internationaal-monsterlijke misdaad van consensuele seks maar zonder condoom, namelijk "verkrachting." Seksisme en racisme worden gebruikt als perverse wapens voor omgekeerde slachtoffers door mensen die niks substantieel te bieden hebben maar enorme winst kunnen jagen. De krant die de Snowden leaks bracht heeft zichzelf vrijwillig tot Daily Mail roddel gereduceerd, met wetenschappers beschaamd in plaats van bewonderd voor wat ze bereikt hebben. Het is geen samenzwering wanneer de clickbait blijft vloeien en iedereen gretig in rij staat om mee te doen; ze halen moeiteloos hun portemonnee boven om post-katholieke aflaten te kopen voor de patriarchische erfzonde en de blanke koloniale schuld.

PS: Mensen afwimpelen als kwaad is geen geldig antwoord (dat noemen we "projectie"), noch ze kwasten, lastposten of geïnternaliseerde vrouwenhaters/homofoben/racisten noemen voor dingen die ze helemaal niet gedaan hebben. Social Justice wou dat iedereen zich zou educaten, wel, we gaven ze net wat ze gevraagd hebben—ook het minder positieve—en zij en de media blijven het verkeerd voorstellen. Er was geen groot bedrog vanwege gamers, geen reefer madness haatbende die vrouwen uit gaming jaagt omdat ze vrouwen zijn. Enkel ongetalenteerde sjoemelaars, een hoop personages met onduidelijke bedoelingen, een hoop trolls in derde persoon, en veel meer volk dat rond hen probeert werk gedaan te krijgen terwijl ze uitgescholden worden. Ik ben niet mad as hell, ik heb er eenvoudigweg genoeg van.

(En ze zullen mij waarschijnlijk rechts noemen, zonder er bij stil te staan dat nationaal socialisme de Volkswagen voor een bepaalde reden gemaakt heeft. Ja ik weet het, Godwin. Wir haben es nicht gewußt.)

Darmok and Jalad at Tanagra

De boom der vrijheid moet van tijd tot tijd gevoed worden
met het bloed van vaderlanders en tirannen,

maar ze hebben het met ketchup geprobeerd.

↧

How to Fold a Julia Fractal

January 4, 2013, 3:00 pm

≫ Next: To Infinity… And Beyond!

≪ Previous: Occupy WWW Street (NL)

A tale of numbers that like to turn

"Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy. And yet," Death waved a hand, "And yet you act as if there is some ideal order in the world, as if there is some… some rightness in the universe by which it may be judged."
– The Hogfather, Discworld, Terry Pratchett

Mathematics has a dirty little secret. Okay, so maybe it's not so dirty. But neither is it little. It goes as follows:

Everything in mathematics is a choice.

You'd think otherwise, going through the modern day mathematics curriculum. Each theorem and proof is provided, each formula bundled with convenient exercises to apply it to. A long ladder of subjects is set out before you, and you're told to climb, climb, climb, with the promise of a payoff at the end. "You'll need this stuff in real life!", they say, oblivious to the enormity of this lie, to the fact that most of the educated population walks around with "vague memories of math class and clear memories of hating it."

Rarely is it made obvious that all of these things are entirely optional—that mathematics is the art of making choices so you can discover what the consequences are. That algebra, calculus, geometry are just words we invented to group the most interesting choices together, to identify the most useful tools that came out of them. The act of mathematics is to play around, to put together ideas and see whether they go well together. Unfortunately that exploration is mostly absent from math class and we are fed pre-packaged, pre-digested math pulp instead.

And so it also goes with the numbers. We learn about the natural numbers, the integers, the fractions and eventually the real numbers. At each step, we feel hoodwinked: we were only shown a part of the puzzle! As it turned out, there was a 'better' set of numbers waiting to be discovered, more comprehensive than the last.

Along the way, we feel like our intuition is mostly preserved. Negative numbers help us settle debts, fractions help us divide pies fairly, and real numbers help us measure diagonals and draw circles. But then there's a break. If you manage to get far enough, you'll learn about something called the imaginary numbers, where it seems sanity is thrown out the window in a variety of ways. Negative numbers can have square roots, you can no longer say whether one number is bigger than the other, and the whole thing starts to look like a pointless exercise for people with far too much time on their hands.

I blame it on the name. It's misleading for one very simple reason: all numbers are imaginary. You cannot point to anything in the world and say, "This is a 3, and that is a 5." You can point to three apples, five trees, or chalk symbols that represent 3 and 5, but the concepts of 3 and 5, the numbers themselves, exist only in our heads. It's only because we are taught them at such a young age that we rarely notice.

So when mathematicians finally encountered numbers that acted just a little bit different, they couldn't help but call them fictitious and imaginary, setting the wrong tone for generations to follow. Expectations got in the way of seeing what was truly there, and it took decades before the results were properly understood.

Now, this is not some esoteric point about a mathematical curiosity. These imaginary numbers—called complex numbers when combined with our ordinary real numbers—are essential to quantum physics, electromagnetism, and many more fields. They are naturally suited to describe anything that turns, waves, ripples, combines or interferes, with itself or with others. But it was also their unique structure that allowed Benoit Mandelbrot to create his stunning fractals in the late 70s, dazzling every math enthusiast that saw them.

Yet for the most part, complex numbers are treated as an inconvenience. Because they are inherently multi-dimensional, they defy our attempts to visualize them easily. Graphs describing complex math are usually simplified schematics that only hint at what's going on underneath. Because our brains don't do more than 3D natively, we can glimpse only slices of the hyperspaces necessary to put them on full display. But it's not impossible to peek behind the curtain, and we can gain some unique insights in doing so. All it takes is a willingness to imagine something different.

So that's what this is about. And a lesson to be remembered: complex numbers are typically the first kind of numbers we see that are undeniably strange. Rather than seeing a sign that says Here Be Dragons, Abandon All Hope, we should explore and enjoy the fascinating result that comes from one very simple choice: letting our numbers turn. That said, there are dragons. Very pretty ones in fact.

Like Hands on a Clock

What does it mean to let numbers turn? Well, when making mathematical choices, we have to be careful. You could declare that $ 1 + 1 $ should equal $ 3 $, but that only opens up more questions. Does $ 1 + 1 + 1 $ equal $ 4 $ or $ 5 $ or $ 6 $? Can you even do meaningful arithmetic this way? If not, what good are these modified numbers? The most important thing is that our rules need to be consistent for them to work. But if all we do is swap out the symbols for $ 2 $ and $ 3 $, we didn't actually change anything in the underlying mathematics at all.

So we're looking for choices that don't interfere with what already works, but add something new. Just like the negative numbers complemented the positives, and the fractions snugly filled the space between them—and the reals somehow fit in between that—we need to go look for new numbers where there currently aren't any.

We'll start with the classic real number line, marked at the integer positions, and poke around.
We imagine the line continues to the left and right indefinitely.

$$ \class{blue}{2} + \class{green}{3} = \class{red}{5} $$

But there's a problem with this visualization: by picturing numbers as points,
it's not clear how they act upon each other.
For example, the two adjacent numbers $ \class{blue}{2} + \class{green}{3} $ sum to $ \class{red}{5} $ …

$$ \class{blue}{-2} + \class{green}{-1} = \class{red}{-3} $$

… but the similarly adjacent pair $ \class{blue}{-2} + \class{green}{-1} = \class{red}{-3} $.
We can't easily spot where the red point is going to be based on the blue and green.

A better solution is to represent our numbers using arrows instead, or vectors.
Each arrow represents a number through its length, pointing right/left for positive/negative.

The nice thing about arrows is that you can move them around without changing them.
To add two arrows, just lay them end to end. You can easily spot why $ \class{blue}{-2} + \class{green}{-1} = \class{red}{-3} $ …

… and why $ \class{blue}{2} + \class{green}{3} = \class{red}{5} $, similarly.
As long as we apply positives and negatives correctly, everything still works.

$$ \times \class{green}{1.5} ... $$

Now let's examine multiplication. We're going to start with $ \class{blue}{1} $ and then we'll multiply it by $ \class{green}{1.5} $ repeatedly.

With every multiplication, the vector gets longer by 50 percent.
These vectors represent the numbers $ \class{red}{1}, \class{red}{1.5}, \class{red}{2.25}, \class{red}{3.375} $, $ \class{red}{5.0625} $, a nice exponential sequence.

$$ \times (\class{green}{-1.5}) ... $$

Now we're going to do the same, but multiplying by the negative, $ \class{green}{-1.5} $, repeatedly.

The vectors still grow by 50%, but they also flip around, alternating between positive and negative.
These vectors represent the sequence $ \class{red}{1}, \class{red}{-1.5}, \class{red}{2.25}, \class{red}{-3.375}, \class{red}{5.0625} $.

But there's another way of looking at this. What if instead of flipping from positive to negative, passing through zero, we went around instead, by rotating the vector as we're growing it?

We'd get the same numbers, but we've discovered something remarkable: a way to enter and pass through the netherworld around the number line. The question is, is this mathematically sound, or plain non-sense?

$$ +180^\circ $$

$$ 0^\circ $$

The challenge is to come up with a consistent rule for applying these rotations. We start with normal arithmetic. Multiplying by a positive didn't flip the sign, so we say we rotated by $ 0^\circ $. Multiplying by a negative flips the sign, so we rotated by $ \class{green}{180^\circ} $. The lengths are multiplied normally in both cases.

$$ \times \class{green}{1.5 \angle 90^\circ} ... $$

$$ +90^\circ $$

$$ +270^\circ $$

Now suppose we pick one of the in-between nether-numbers, say the vector of length $ 1.5 $, at a $ 90^\circ $ angle. What does that mean? That's what we're trying to find out! We'll write that as $ \class{green}{1.5 \angle 90^\circ} $ (1.5 at 90 degrees). It could make sense to say that multiplying by this number should rotate by $ \class{green}{90^\circ} $ while again growing the length by 50%.

This creates the spiral of points: $ \class{red}{1 \angle 0^\circ} $, $ \class{red}{1.5 \angle 90^\circ} $, $ \class{red}{2.25 \angle 180^\circ} $, $ \class{red}{3.375 \angle 270^\circ} $, $ \class{red}{5.0625 \angle 360^\circ} $. Three of those are normal numbers: $ +1 $, $ -2.25 $ and $ +5.0625 $, lying neatly on the real number line. The other two are new numbers conjured up from the void.

$$ +135^\circ $$

$$ +45^\circ $$

$$ +225^\circ $$

$$ +315^\circ $$

$$ \times \class{green}{1 \angle 45^\circ} ... $$

Let's examine this rotation more. We can pick $ 1 $ at a $ \class{green}{45^\circ} $ angle. Multiplying by a $ 1 $ probably shouldn't change a vector's length, which means we'd get a pure rotation effect.

By multiplying by $ \class{green}{1 \angle 45^\circ} $, we can rotate in increments of $ 45^\circ $.
It takes 4 multiplications to go from $ +1 $, around the circle of ones, and back to the real number $ -1 $.

And that's actually a remarkable thing, because it means our invented rule has created a square root of $ -1 $.
It's the number $ \class{green}{1 \angle 90^\circ} $.

$ (\class{green}{1 \angle 90^\circ})^2 = \class{blue}{-1} $

If we multiply it by itself, we end up at angle $ \class{green}{90} + \class{green}{90} = \class{blue}{180^\circ} $, which is $ \class{blue}{-1} $ on the real line.

But actually, the same goes for $ \class{green}{1 \angle 270^\circ} $.

$ (\class{green}{1 \angle 270^\circ})^2 = \class{blue}{-1} $

When we multiply it by itself, we end up at angle $ \class{green}{270} + \class{green}{270} = \class{blue}{540^\circ} $. But because we went around the circle once, that's the same as rotating by $ \class{blue}{180^\circ} $. So that's also equal to $ \class{blue}{-1} $.

$$ \pm180^\circ $$

$$ 0^\circ $$

$$ -90^\circ $$

$$ +90^\circ $$

$$ -135^\circ $$

$$ -45^\circ $$

$$ +135^\circ $$

$$ +45^\circ $$

$ (\class{green}{1 \angle -90^\circ})^2 = \class{blue}{-1} $

Or we could think of $ +270^\circ $ as $ -90^\circ $, and rotate the other way. It works out just the same. This is quite remarkable: our rule is consistent no matter how many times we've looped around the circle.

$ (\class{green}{1 \angle 90^\circ})^2 = \class{blue}{-1} $

$ (\class{green}{1 \angle 270^\circ})^2 = \class{blue}{-1} $

Either way, $ \class{blue}{-1} $ has two square roots, separated by $ 180^\circ $, namely $ \class{green}{1 \angle 90^\circ} $ and $ \class{green}{1 \angle 270^\circ} $.
This is analogous to how both $ 2 $ and $ -2 $ are square roots of $ 4 $.

$$ \class{blue}{a} \cdot \class{green}{b} = \class{red}{c}$$

Complex multiplication can then be summarized as: angles add up, lengths multiply, taking care to preserve clockwise and counterwise angles. Above, we multiply two random complex numbers a and b to get c.

$$ \class{blue}{a} \cdot \class{green}{b} = \class{red}{c}$$

When we start changing the vectors, c turns along, being tugged by both a and b's angles. It wraps around the circle, while its length changes. Hence, complex numbers like to turn, and it's this rule that separates them from ordinary vectors.

$$ \hspace{35 pt} + $$

$$ - \hspace{35 pt} $$

We can then picture the complex plane as a grid of concentric circles. There's a circle of ones, a circle of twos, a circle of one-and-a-halfs, etc. Each number comes in many different versions or flavors, one positive, one negative, and infinitely many others in between, at arbitrary angles on both sides of the circle.

$$ \pm180^\circ $$

$$ 0^\circ $$

$$ +90^\circ $$

$$ \hspace{15pt} \class{blue}{i} $$

Which brings us to our reluctant and elusive friend, $ \class{blue}{i} $. This is the proper name for $ \class{blue}{1 \angle 90^\circ} $, and the way complex numbers are normally introduced: $ i^2 = -1 $. The magic is that we can put a complex number anywhere a real number goes, and the math still works out, oddly enough. We get complex answers about complex inputs.

Complex numbers are then usually written as the sum of their (real) X coordinate, and their (imaginary) Y coordinate, much like ordinary 2D vectors. But this is misleading: the ugly number $ \class{red}{\frac{\sqrt{3}}{2} + \frac{1}{2}i } $ is actually just $ \class{green}{1 \angle 30^\circ} $ in disguise, and it acts more like a $ 1 $ than a $ \frac{1}{2} $ or $ \frac{\sqrt{3}}{2} $. While knowing how to convert between the two is required for any real calculations, you can cheat by doing it visually.

$$ \pm180^\circ $$

$$ 0^\circ $$

$$ -90^\circ $$

$$ +90^\circ $$

$$ -135^\circ $$

$$ -45^\circ $$

$$ +135^\circ $$

$$ +45^\circ $$

$$ \class{blue}{+1} $$

$$ \hspace{55pt}\class{green}{+i} $$

$$ \class{blue}{-1} $$

$$ \class{green}{-i}\hspace{55pt} $$

But looking at individual vectors only gets us so far. We study functions of real numbers by looking at a graph that shows us every output for every input. To do the same for complex numbers, we need to understand how these numbers-that-like-to-turn, this field of vectors, change as a whole.
Note: from now on, I'll put $ +1 $, i.e. $ 0^\circ $ at the 12 o'clock position for simplicity.

When we apply a square root, each vector shifts. But really, it's the entire fabric of the complex plane that's warping. Each circle has been squeezed into a half-circle, because all the angles have been halved—the opposite of squaring, i.e. doubling the angle. The lengths have had a normal square root applied to them, compressing the grid at the edges and bulging it in the middle.

But remember how every number had two opposite square roots? This comes from the circular nature of complex math. If we take a vector and rotate it $ 360 ^\circ $, we end up in the same place, and the two vectors are equal. But after dividing the angles in half, those two vectors are now separated by only $ 180 ^\circ $ and lie on opposite ends of the circle. In complex math, they can both emerge.

Complex operations are then like folding or unfolding a piece of paper, only it's weird and stretchy and circular. This can be hard to grasp, but is easier to see in motion. To help see what's going on, I've cut the disc and separated the positive from the negative angles in 3D.

When we square our numbers to undo the square root, the angles double, folding the plane in on itself. The lengths are also squared, restoring the grid spacing to normal.

After squaring, each square root has now ended up on top of its identical twin, and we can merge everything back down to a flat plane. Everything matches up perfectly.

Thus the square root actually looks like this. New numbers flow in from the 'far side' as we try and shear the disc apart. The complex plane is stubborn and wants to stay connected, and will fold and unfold to ensure this is always the case. This is one of its most remarkable properties.

There's no limit to this folding or unfolding. If we take every number to the fourth power, angles are multiplied by four, while lengths are taken to the fourth power. This results in 4 copies of the plane being folded into one.

However, things are not always so neat. What happens if we were to take everything to an irrational power, say $ \frac{1}{\sqrt{2}} $? Angles get multiplied by $ 0.707106... $, which means a rotation of $ 360^\circ $ now becomes $ \sim 254.56^\circ $.

Because no multiple of $ 360 $ is divisible by $ \frac{1}{\sqrt{2}} $, the circular grid never matches up with itself again no matter how far we extend it. Hence, this operation splits a single unique complex number into an infinite amount of distinct copies.

For any irrational power $ p $, there are an infinite number of solutions to $ z^p = c $, all lying on a circle. For a hint as to why this is so, we can look at Taylor series: an arbitrary function $ f(z) $ can be written as an infinite sum $ a + bz + cz^2 + dz^3 + ... \,$ When z is complex, such a sum doesn't just represent a finite amount of folds, but a mindboggling infinite origami of complex space.

We've seen how complex numbers are arrows that like to turn, which can be made to behave like numbers: we can add and multiply them, because we can come up with a consistent rule for doing so. We've also seen what powers of complex numbers look like: we fold or unfold the entire plane by multiplying or dividing angles, while simultaneously applying a power to the lengths.

Pulling a Dragon out of a Hat

With a basic grasp of what complex numbers are and how they move, we can start making Julia fractals.

At their heart lies the following function:

$$ f(z) = z^2 + c $$

This says: map the complex number $ z $ onto its square, and then add a constant number to it. To generate a Julia fractal, we have to apply this formula repeatedly, feeding the result back into $ f $ every time.

$$ z_{n+1} = (z_n)^2 + c $$

We want to examine how $ z_n $ changes when we plug in different starting values for $ z_1 $ and iterate $ n $ times. So let's try that and see what happens.

Our region of interest is the disc of complex numbers less than $ 2 $ in length. I've marked the circle of ones as a reference.

We take an arbitrary set of numbers, like this grid, and start applying the formula $ f(z) = z^2 + c $ to each. Rather than use vectors, I'll just draw points, to avoid cluttering the diagram.

First we square each number. That is, their lengths are squared, their angles are doubled.
The squaring has a dual effect: numbers larger than $ 1 $ grow bigger and are pushed outwards, numbers less than $ 1 $ grow smaller and are pulled inwards.

Next, we reset the grid back to neutral, keeping the numbers in their new place.
We also pick a random value for the constant $ \class{green}{c} $, e.g. $ \class{green}{0.57 \angle -59^\circ} $.

Now we add $ \class{green}{c} $ to each point, completing one round of Julia iteration, $ f(z) = z^2 + c $. As a result, some numbers have ended up closer towards the origin (i.e. $ 0 $), others further away from it. The combination of folding + shifting has had a non-obvious effect on the numbers.

We begin the second iteration and square each number again. Any number not inside the critical circle of $ 1 $ in the middle will get pushed out again. The other numbers continue to linger in the middle.

If we zoom out, we can see the larger numbers are spiralling outwards and are permanently lost. The minor nudge by $ \class{green}{c} $ won't be enough to bring them back.

Others remain in the middle, being drawn in, but are also at risk of being pushed out of the circle by $ \class{green}{c} $.

Resetting the grid again, we add the same value $ \class{green}{c} $ to our vectors again to finish. At this point, our original grid of numbers has been completely jumbled up.

If we continued this process would any numbers remain in the middle? Or would they eventually all get flung out? Unfortunately it's very hard to see what's going on while iterating forwards, because we lose track of where each point came from.

So we're going to go backwards instead. We'll establish a safe-zone of all numbers less than $ 2 $, forming a solid disc of all those which aren't irretrievably lost. We want to know where all these numbers can possibly come from. To help track these points, I've coloured one area in a different shade.

First we have to shift the numbers again, this time in the opposite direction to subtract $ c $.

Now we apply the square root to find $ z_{n-1} = \pm \sqrt{z_n - c} $, which is a Julia iteration in reverse.

After one backwards iteration, the disc has been squished down into an oval at an angle.
These are all the points that will definitely stay in the middle after one iteration.

When we apply the second iteration, a pattern starts to develop. Because of the repeated unfolding, we create two bulges wherever there was previously only one.

At the same time, the square root alters the length of each number as well. As a result, we squeeze in the radial direction, scaling down earlier features as they combine with newly created ones.

After 4 iterations, we start to see the first hints of self-similarity. The shape's lobes are sprouting into spirals.

But all we've really done is narrow down our blue safe-zone to include only those points that 'survive' up to 5 Julia iterations.

Remarkably this seems to distort the fractal evenly: our highlighted circles don't stretch into ovals. This is not a coincidence. Complex operations are indeed stubborn, in that they all preserve right angles everywhere. To do so, the mapping must act like a pure scaling and rotation at every point, without shearing off in any particular direction. This is what allows the fractal to look like itself at different scales.

Skipping ahead to iteration 12, we've definitely abandoned the realm of neat, traditional geometry.
Despite curving wildly, the total mapping $ z_{12} $ still has this property of evenness, which is properly referred to as a conformal mapping.

After 128 iterations, we end up with this intricate dragon-like shape, approximating the safe zone for the true fractal map $ z_\infty $. The numbers that make up the blue area are the hardiest points that will survive the next 128 attempts on their life. All the others will definitely get flung out.

Yet this complicated shape is merely the result of folding over and over again, adding a simple constant in between. If we perform a forwards Julia iteration, i.e. squaring and shifting, we see this shape matches up with itself, and looks identical before and after.

For different values of $ \class{green}{c} $, the fractal morphs into other shapes. There's literally an infinite variety to discover. Some sets are made up of disconnected parts. In this case, $ |c| $ is large enough to push the solid disc away from the center in a single iteration, but not so far that some points can't fold back in. If $ |c| $ gets much larger, the set vanishes.

For a smaller $ c $, Julia sets are solid. Even a small shift in the value of $ c $ can accumulate into a large difference. Here we zone in on some fluffy clouds right outside the 'solid zone'. Oddly enough, it seems when $ c $ is not inside of its own Julia set, the set is not solid. Note that in this case, 128 iterations is not sufficient: large solid patches remain, which would be divided further with more iterations.

This area of fractal space is dubbed Seahorse Valley, for rather obvious reasons.

Nearby, we find these jewel-like spirals.

Buried deep inside, there are remarkable combinations of shapes, like this pearl necklace covered in something resembling palm trees.

And we can even make snowflakes. The dramatic changes due to $ c $ reveal the chaotic nature of fractals. Mathematically, chaos occurs when even the tiniest change can accumulate and blow up to an arbitrarily large effect.

If we change our iteration formula, for example to a fourth power $ f(z) = z^4 + c $, the entire shape changes. Because each iteration now turns one bulge into four, the resulting shape has four-fold rotational symmetry.

Again, different values of $ \class{green}{c} $ make different shapes, precipitating dramatic changes.

To understand the effect of $ c $ we need to make a Mandelbrot set. This is similar to a Julia set, but the formula is applied differently. We'll use $ z^2 + c $ again. Instead of different starting values $ z_1 $, we choose different values of $ c $ and start with $ z_1 = 0 $ every time. Because $ c $ is no longer constant, the mapping stops being a simple folding operation. Each iteration is now unique and not so easy to visualize.

Because the Mandelbrot set traverses all possible values of $ c $ across its surface, it has a part of every associated Julia set in it. Around any number $ \class{green}{c} $ it looks like the Julia set which has that value as its constant. Here, we move towards the three-way cross at the bottom of the Mandelbrot set. The Julia set develops similar features.

Where the Mandelbrot set is round and bulbous, the Julia set is too.

The spirals and seahorses from earlier are located here. You can literally see the shapes on both sides of the valley evolving towards horseheads and spirals respectively. But the Mandelbrot set acts like a map to Julia sets in a much more direct way: anywhere the Mandelbrot set is filled in (blue), the corresponding Julia set is solid too. The white areas are values of $ c $ which create disconnected Julia sets.

That the Mandelbrot set is a 'pixel-perfect' map of Julia sets is a big clue. It reflects that they're actually both slices of a single higher dimensional object. By viewing these slices as we travel through, we can get a vague idea of its shape and complexity. In this object, every point in the Mandelbrot set is connected to the center of the corresponding Julia set. Actually picturing this 4D object is a challenge.

But like any fractal, the Mandelbrot set also contains copies of itself, buried inside its edge. This is just one of the many varied copies. As a result, deep Mandelbrot zooms can reach astonishing levels of beauty in complexity. This is best done with specialized software that can calculate with hundreds of digits of precision.

Making fractals is probably the least useful application of complex math, but it's an undeniably fascinating one. It also reveals the unique properties of complex operations, like conformal mapping, which provide a certain rigidity to the result.

However, in order to make complex math practical, we have to figure out how to tie it back to the real world.

Travelling without Moving

It's a good thing we don't have to look far to do so. Whenever we're describing wavelike phenomena, whether it's sound, electricity or subatomic particles, we're also interested in how the wave evolves and changes. Complex operations are eminently suited for this, because they naturally take place on circles. Numbers that oppose can cancel out, numbers in the same direction will amplify each other, just like two waves do when they meet. And by folding or unfolding, we can alter the frequency of a pattern, doubling it, halving it, or anything in between.

More complicated operations are used for example to model electromagnetic waves, whether they are FM radio, wifi packets or ADSL streams. This requires precise control of the frequencies you're generating and receiving. Doing it without complex numbers, well, it just sucks. So why use boring real numbers, when complex numbers can do the work for you?

$$ w(x) = \sin(x) $$

Take for example a sine wave $ w(x) $.

$$ w(x, t) = \sin(x - t) $$ $$ \class{blue}{\frac{\partial w(x, t)}{\partial t}} $$

For the wave to propagate across a distance, its values have to ripple up and down over time.
The rate of change over time is drawn on top. This is the vertical velocity at every point. Both the wave and its rates of change undergo a complicated numerical dance.

$$ w(x, t) = \sin(x - t) $$ $$ \class{blue}{\frac{\partial w(x, t)}{\partial t}} \,\, \class{green}{\frac{\partial^2 w(x, t)}{\partial t^2}} $$

But to properly describe this motion, we have to go one level deeper. We have to examine the rate of change of the vertical velocity of the wave. This is its vertical acceleration. We see that green vectors tug on blue vectors as blue vectors tug on the wave.

$$ w(x, t) = \sin(x - t) $$ $$ \class{green}{\frac{\partial^2 w(x, t)}{\partial t^2}} = \,? $$

It's easier to see what's going on if we center the vectors vertically. The acceleration appears to be equal but opposite to the wave itself.

$$ w(x, t) = \sin(x - t) + 1 $$ $$ \class{green}{\frac{\partial^2 w(x, t)}{\partial t^2}} = \,? $$

But that's just a lucky coincidence. If we shift the wave up by one unit, its opposite shifts down by a unit. Yet its velocity and acceleration are unaltered. So acceleration is not simply the opposite of the wave.

What's actually going on is that the green vectors match the curvature of the wave, positive inside valleys, negative on top of crests. Intuitively, this can be explained by saying that waves tend to bounce towards an average level: this is going to pull the value up out of valleys and down from peaks.

$$ w(x, t) = \sin(x - t) + 1 $$ $$ \class{green}{\frac{\partial^2 w(x, t)}{\partial t^2}} = \class{red}{\frac{\partial^2 w(x, t)}{\partial x^2}} $$

But curvature is the rate of change of the slope, and slope is the rate of change over a distance. So to describe real waves, we need to relate 'second level' change over time and change over distance, each deriving twice. This is Complicated with a capital C.

Let's try this with complex numbers instead. Until now, we had a 2D graph, showing the real value of the wave over real distance. We're going to make the wave's value complex. Mapping a 1D number (distance) to a 2D number (the wave function), means we need a 3D diagram.

The complex plane is mapped into the old Y direction (real) and the new Z direction (imaginary).

$$ w(x) = (1 \angle x) $$

To make a complex wave, we do the thing complex numbers are best at: we make them turn, and make a helix. In this case, our wave function is simply the variable number $ 1 \angle x $ , a constant length with a smoothly changing rotation over distance.

$$ w(x, t) = (1 \angle x) \cdot (1 \angle t) = 1 \angle (x + t) $$

$$ \class{blue}{\frac{\partial w(x, t)}{\partial t}} = \,? $$

To make the wave move, we can simply twist it in-place. Which we now know is the same as multiplying by an increasing angle $ 1 \angle t $. If we plot the complex velocity of each point, at first sight this might not look any simpler than the real wave. But in fact, these vectors are not changing in length at all, unlike the real version. As the wave is pulled by the velocity vectors, both undergo a pure rotation.

$$ \class{blue}{\frac{\partial w(x, t)}{\partial t}} = i \cdot w(x, t) $$

At all times, the velocity is offset by $ 90^\circ $ from the wave itself. And that means that described in complex numbers, wave equations are super easy. Instead of involving two derivatives, i.e. the rate of rate of change, we only need one. There is a direct relationship between a value and its rate of change. The necessary rotation by $ 90^\circ $ can then be written simply as multiplying by $ i $.

To recover a real wave from a complex wave, we can simply flatten it back to 2D, discarding the imaginary part. By using complex numbers to describe waves, we give them the power to rotate in place without changing their amplitude, which turns out to be much simpler.

$$ \frac{1}{2} (\class{blue}{ 1 \angle (x + t) } + \class{green}{ 1 \angle -(x + t) }) = \cos(x + t) $$

In fact, flattening the wave has a perfectly reasonable complex interpretation: it's what happens when we average out a counter-clockwise wave (positive frequency) with a clockwise wave (negative frequency). By twisting each in opposite directions, the combined wave travels along, locked to the real number line.

$$ \frac{1}{2} (\class{blue}{ 1 \angle (x + t) } + \class{green}{ 1 \angle -(\frac{3}{2}x + t) }) = \,? $$

But if we add up two arbitrary complex frequencies, their sum immediately turns into a spirograph pattern that manages to evolve and propagate, even as it just rotates in place. Though the original waves both had a constant amplitude of $ 1 $, the relative differences in angles (i.e. the phase) allows them to cancel out in surprising ways.

Neither curve is actually moving forward: they're just spinning in place, creating motion anyway. This is actually what quantum superposition looks like, where two or more complex probability waves combine and interfere. Where the result cancels out to zero, that's where two separate possible states are cancelling out each other, creating interference. That the underlying numbers are complex doesn't prevent them from describing real physics, indeed, it seems that's how nature actually works.

This serene display hides a whirlwind of phase. We can plot the velocity of the two frequencies, and their combination, scaled down for clarity. Once again you can see the power of describing waves with complex numbers, letting you split up a complicated motion into simple, repetitive rotations… into numbers that like to turn.

The End Is Just The Beginning

In visualizing complex waves, we've seen functions that map real numbers to complex numbers, and back again. These can be graphed easily in 3D diagrams, from $ \mathbb{R} $ to $ \mathbb{C} $ or vice-versa. You cross 1 real dimension with the 2 dimensions of the complex plane.

But complex operations in general work from $ \mathbb{C} $ to $ \mathbb{C} $. To view these, unfortunately you need four-dimensional eyes, which nature has yet to provide. There are ways to project these graphs down to 3D that still somewhat make sense, but it never stops being a challenge to interpret them.

For every mathematical concept that we have a built-in intuition for, there are countless more we can't picture easily. That's the curse of mathematics, yet at the same time, also its charm.

Hence, I tried to stick to the stuff that is (somewhat!) easy to picture. If there's interest, a future post could cover topics like: the nature of $ e^{ix} $, Fourier transforms, some actual quantum mechanics, etc.

For now, this story is over. I hope I managed to spark some light bulbs here and there, and that you enjoyed reading it as much as I did making it.

Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.

More like this: To Infinity… And Beyond!.

For extra credit: check out these great stirring visualizations of Julia and Mandelbrot sets. I incorporated a similar graphic above. Hat tip to Tim Hutton for pointing these out. And for some actual paper mathematical origami, check out Vihart's latest video on Snowflakes, Starflakes and Swirlflakes.

↧

To Infinity… And Beyond!

January 27, 2013, 3:00 pm

≫ Next: On WebGL

≪ Previous: How to Fold a Julia Fractal

Exploring the outer limits

“It is known that there are an infinite number of worlds, simply because there is an infinite amount of space for them to be in. However, not every one of them is inhabited. Therefore, there must be a finite number of inhabited worlds.

Any finite number divided by infinity is as near to nothing as makes no odds, so the average population of all the planets in the universe can be said to be zero. From this it follows that the population of the whole universe is also zero, and that any people you may meet from time to time are merely the products of a deranged imagination.”
– The Restaurant at the End of the Universe, Douglas Adams

If there's one thing mathematicians have a love-hate relationship with, it has to be infinity. It's the ultimate tease: it beckons us to come closer, but never allows us anywhere near it. No matter how far we travel to impress it, infinity remains disinterested, equally distant from everything: infinitely far!

$$ 0 < 1 < 2 < 3 < … < \infty $$

Yet infinity is not just desirable, it is absolutely necessary. All over mathematics, we find problems for which no finite amount of steps will help resolve them. Without infinity, we wouldn't have real numbers, for starters. That's a problem: our circles aren't round anymore (no $ π $ and $ \tau $) and our exponentials stop growing right (no $ e $). We can throw out all of our triangles too: most of their sides have exploded.

A steel railroad bridge with a 1200 ton counter-weight.
Completed in 1910. Source: Library of Congress.

We like infinity because it helps avoid all that. In fact even when things are not infinite, we often prefer to pretend they are—we do geometry in infinitely big planes, because then we don't have to care about where the edges are.

Now, suppose we want to analyze a steel beam, because we're trying to figure out if our proposed bridge will stay up. If we want to model reality accurately, that means simulating each individual particle, every atom in the beam. Each has its own place and pushes and pulls on others nearby.

But even just $ 40 $ grams of pure iron contains $ 4.31 \cdot 10^{23} $ atoms. That's an inordinate amount of things to keep track of for just 1 teaspoon of iron.

Instead, we pretend the steel is solid throughout. Rather than being composed of atoms with gaps in between, it's made of some unknown, filled in material with a certain density, expressed e.g. as grams per cubic centimetre. Given any shape, we can determine its volume, and hence its total mass, and go from there. That's much simpler than counting and keeping track of individual atoms, right?

Unfortunately, that's not quite true.

The Shortest Disappearing Trick Ever

Like all choices in mathematics, this one has consequences we cannot avoid. Our beam's density is mass per volume. Individual points in space have zero volume. That would mean that at any given point inside the beam, the amount of mass there is $ 0 $. How can a beam that is entirely composed of nothing be solid and have a non-zero mass?

Bam! No more iron anywhere.

While Douglas Adams was being deliberately obtuse, there's a kernel of truth there, which is a genuine paradox: what exactly is the mass of every atom in our situation?

To make our beam solid and continuous, we had to shrink every atom down to an infinitely small point. To compensate, we had to create infinitely many of them. Dividing the finite mass of the beam between an infinite amount of atoms should result in $ 0 $ mass per atom. Yet all these masses still have to add up to the total mass of the beam. This suggests $ 0 + 0 + 0 + … > 0 $, which seems impossible.

If the mass of every atom were not $ 0 $, and we have infinitely many points inside the beam, then the total mass is infinity times the atomic mass $ m $. Yet the total mass is finite. This suggests $ m + m + m + … < \infty $, which also doesn't seem right.

It seems whatever this number $ m $ is, it can't be $ 0 $ and can't be non-zero. It's definitely not infinite, we only had a finite mass to begin with. It's starting to sound like we'll have to invent a whole new set of numbers again to even find it.

That's effectively what Isaac Newton and Gottfried Leibniz set in motion at the end of the 17th century, when they both discovered calculus independently. It was without a doubt the most important discovery in mathematics and resulted in formal solutions to many problems that were previously unsolvable— our entire understanding of physics has relied on it since. Yet it took until the late 19th century for the works of Augustin Cauchy and Karl Weierstrass to pop up, which formalized the required theory of convergence. This allows us to describe exactly how differences can shrink down to nothing as you approach infinity. Even that wasn't enough: it was only in the 1960s when the idea of infinitesimals as fully functioning numbers—the hyperreal numbers—was finally proven to be consistent enough by Abraham Robinson.

But it goes back much further. Ancient mathematicians were aware of problems of infinity, and used many ingenious ways to approach it. For example, $ π $ was found by considering circles to be infinite-sided polygons. Archimedes' work is likely the earliest use of indivisibles, using them to imagine tiny mechanical levers and find a shape's center of mass. He's better known for running naked through the streets shouting Eureka! though.

That it took so long shows that this is not an easy problem. The proofs involved are elaborate and meticulous, all the way back. They have to be, in order to nail down something as tricky as infinity. As a result, students generally learn calculus through the simplified methods of Newton and Leibniz, rather than the most mathematically correct interpretation. We're taught to mix notations from 4 different centuries together, and everyone's just supposed to connect the dots on their own. Except the trail of important questions along the way is now overgrown with jungle.

Still, it shows that even if we don't understand the whole picture, we can get a lot done. This article is in no way a formal introduction to infinitesimals. Rather, it's a demonstration of why we might need them.

What is happening when we shrink atoms down to points? Why does it make shapes solid yet seemingly hollow? Is it ever meaningful to write $ x = \infty $? Is there only one infinity, or are there many different kinds?

To answer that, we first have to go back to even simpler times, to Ancient Greece, and start with the works of Zeno.

Achilles and the Tortoise

Zeno of Elea was one of the first mathematicians to pose these sorts of questions, effectively trolling mathematics for the next two millennia. He lived in the 5th century BC in southern Italy, although only second-hand references survive. In his series of paradoxes, he examines the nature of equality, distance, continuity, of time itself.

Because it's the ancient times, our mathematical knowledge is limited. We know about zero, but we're still struggling with the idea of nothing. We've run into negative numbers, but they're clearly absurd and imaginary, unlike the positive numbers we find in geometry. We also know about fractions and ratios, but square roots still confuse us, even though our temples stay up.

So the story goes: the tortoise challenges Achilles to a footrace.

"If you give me a head start," it says, "any start at all, you can never win.".
Achilles laughs and decides to be a good sport: he'll only run twice as fast as the tortoise.

The tortoise explains: "If you want to pass me, first you have to move to where I am. By the time you get there, I'll have walked ahead a little bit."

"While you cross the next distance, I will move yet again. No matter how many times you try to catch up, I'll always be some small distance ahead. Therefor, you cannot beat me."

Achilles realizes that talking tortoises are not a sign of positive mental health, so he decides to find a wall to run into instead. It will either confirm the theory, or end the pain.

See, the race is actually unnecessary, because the problem remains the same.
In order to reach the wall, Achilles first has to cross half the way there.

Then he has to go half that distance again, and again. No matter how many times he repeats this, there will always be some distance left. So if Achilles can't cross this distance in a finite amount of steps, why is he wearing that stupid helmet?

$$ … $$

The distance travelled forms a never ending sequence of expanding sums.
We have to examine the entire sequence, rather than individual numbers in it.

By definition, the distance travelled and distance to the wall always add up to $ 1 $. So one simple way to resolve this conundrum is to say: Well yes, it's going to take you infinitely long to glue all those pieces together, but only because you already spent an infinite amount of time chopping them up!
But that's not a very mathematically satisfying answer. Let's try something else.

The distance to the wall is always equal to the last step taken. We know that each step is half as long as the previous one, starting with $ \frac{1}{2} $. Therefor, the distance to the wall must decrease exponentially: $ \frac{1}{2}, \frac{1}{4}, \frac{1}{8}, \frac{1}{16}, … $, getting closer to zero with every step.

But why can we say that this gap effectively closes to zero after 'infinity steps'? The number that we're building up is $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … \,$

We know our sum will never exceed $ 1 $, as there is only $ 1 $ unit of distance being divided. This means $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … \leq 1 $, which eliminates every number past the surface of the wall—but not the surface itself.

Suppose we presume $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … < 1 $ and hence that this number lies some tiny distance in front of the wall.

Well in that case, all we need to do is zoom in far enough, and we'll see our sequence jump past it after a certain finite number of steps.

If we try to move it closer to the wall, the same thing happens. This number simply cannot be less than $ 1 $. Therefor $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … \geq 1 $

The only place $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … \, $ can be is exactly $ 0 $ units away from $ 1 $. If two numbers have zero distance between them, then they are equal.

$$ … $$

What we've actually done here is applied the principle of limits: we've defined a procedure of steps that lets us narrow down the interval where the infinite sum might be. The lower bound is the sequence of sums itself: it only increases towards $ 1 $, never decreases. For the upper bound, we established no sum could exceed $ 1 $. Therefor the interval must shrink to nothing, and the sequence converges.

$$ \lim_{n \to +\infty} x_n = \mathop{\class{no-outline}{►\hspace{-2pt}►}}_{\infty\hspace{2pt}} x_n $$

The purpose of a limit is then to act as a supercharged fast-forward button. It lets us avoid the infinite amount of work required to complete sums like $ \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \frac{1}{16} + … $ and simply skip to the end. To do so, we have to step back, spot the pattern, and pin down where it ends. So limits allow us to literally reach the unreachable. But in fact, you already knew that.

$$ \frac{2}{3} = 0.66666… $$

$$ 0.6 + 0.06 + 0.006 + …\hspace{2pt} $$

As soon as you learned to divide, you found $ 2 \div 3 = 0.666… = 0.6 + 0.06 + 0.006 + …\hspace{2pt} $
Even in primary school the opportunity to examine infinity is there. Rather than tackle it head on, it's simply noted and filed. Eight years later it's regurgitated in the form of cryptic epsilon-delta definitions.

$$ 1 - 1 + 1 - 1 + 1 … $$

But then there's those pesky consequences again. By allowing the idea of infinity, we can invent an entire zoo of paradoxical things. For example, imagine a lamp that's switched on ($1$) and off ($0$) at intervals that decrease by a factor of two: on for $ \frac{1}{2} $ second, off for $ \frac{1}{4} s $, on for $ \frac{1}{8} s $, off for $ \frac{1}{16} s $, …
After $ 1\,s $, when the switch has been flipped an infinite amount of times, is the lamp on or off?

$$ (1 - 1) + (1 - 1) + (1 - 1) + … = 0 \,? $$

$$ 1 + (-1 + 1) + (-1 + 1) + … = 1 \,? $$

Another way to put this is that the lamp's state at $ 1\,s $ is the result of the infinite sum $ 1 - 1 + 1 - 1 + … $
Intuitively we might say each pair of $ +1 $ and $ -1 $ should cancel out and make the entire sum equal to $ 0 $.
But we can pair them the other way, leading to $ 1 $ instead. It can't be both.

If we zoom in, it's obvious that no matter how close we get to $ 1\,s $, the lamp's state keeps switching. Therefor it's meaningless to attempt to 'fast forward' to the end, and the limit does not exist. At $ 1\,s $ the lamp is neither on nor off: it's undefined. This infinite sum does not converge.

But actually, we overcomplicated things. Thanks to the power of limits, we can ask a simpler, equivalent question. Given a lamp that switches on and off every second, what is its state at infinity? The answer's the same: it never settles.

Limits are the first tool in our belt for tackling infinity. Given a sequence described by countable steps, we can attempt to extend it not just to the end of the world, but literally forever. If this works we end up with a finite value. If not, the limit is undefined. A limit can be equal to $ \infty $, but that's just shorthand for the sequence has no upper bound. Negative infinity means no lower bound.

Breaking Away From Rationality

Until now we've only encountered fractions, that is, rational numbers. Each of our sums was made of fractions. The limit, if it existed, was also a rational number. We don't know whether this was just a coincidence.

It might seem implausible that a sequence of numbers that is 100% rational and converges, can approach a limit that isn't rational at all. Yet we've already seen similar discrepancies. In our first sequence, every partial sum was less than $ 1 $. Meanwhile the limit of the sum was equal to $ 1 $. Clearly, the limit does not have to share all the properties of its originating sequence.

We also haven't solved our original problem: we've only chopped things up into infinitely many finite pieces. How do we get to infinitely small pieces? To answer that, we need to go looking for continuity.

Generally, continuity is defined by what it is and what its properties are: a noticeable lack of holes, and no paradoxical values. But that's putting the cart before the horse. First, we have to show which holes we're trying to plug.

Let's imagine the rational numbers.

Actually, hold on. Is this really a line? The integers certainly weren't connected.

Rather than assume anything, we're going to attempt to visualize all the rational numbers. We'll start with the numbers between $ 0 $ and $ 1 $.

$$ \class{blue}{\frac{0 + 1}{2}} $$

Between any two numbers, we can find a new number in between: their average. This leads to $ \frac{1}{2} $.

$$ \frac{a + b}{2} $$

By repeatedly taking averages, we keep finding new numbers, filling up the interval.

If we separate out every step, we get a binary tree.

You can think of this as a map of all the fractions of $ 2^n $. Given any such fraction, say $ \frac{13}{32} = \frac{13}{2^5} $, there is a unique path of lefts and rights that leads directly to it. At least, as long as it lies between $ 0 $ and $ 1 $.

Note that the graph resembles a fractal and that the distance to the top edge is divided in half with every step. But we only ever explore a finite amount of steps. Therefor, we are not taking a limit and we'll never actually touch the edge.

$$ \frac{2 \cdot a + b}{3} $$

$$ \frac{a + 2 \cdot b}{3} $$

But we can take thirds as well, leading to fractions with a power of $ 3^n $ in their denominator.

As some numbers can be reached in multiple ways, we can eliminate some lines, and end up with this graph, where every number sprouts into a three-way, ternary tree. Again, we have a map that gives us a unique path to any fraction of $ 3^n $ in this range, like $ \frac{11}{27} = \frac{11}{3^3} $.

$$ \frac{21}{60} = \frac{21}{2^2 \cdot 3 \cdot 5} $$

Because we can do this for any denominator, we can define a way to get to any rational number in a finite amount of steps. Take for example $ \frac{21}{60} $. We decompose its denominator into prime numbers and begin with $ 0 $ and $ 1 $ again.

$$ \frac{21}{60} = \frac{21}{2^2 \cdot 3 \cdot 5} $$

There is a division of $ 2^2 $, so we do two binary splits. This time, I'm repeating the previously found numbers so you can see the regular divisions more clearly. We get quarters.

The next factor is $ 3 $ so we divide into thirds once. We now have twelfths.

For the last division we chop into fifths and get sixtieths.

$ \frac{21}{60} $ is now the 21st number from the left.

But this means we've found a clear way to visualize all the rational numbers between $ 0 $ and $ 1 $: it's all the numbers we can reach by applying a finite number of binary (2), ternary (3), quinary (5) etc. divisions, for any denominator. So there's always a finite gap between any two rational numbers, even though there are infinitely many of them.

The rational numbers are not continuous. Therefor, it is more accurate to picture them as a set of tick marks than a connected number line.

To find continuity then, we need to revisit one of our earlier trees. We'll pick the binary one.
While every fork goes two ways, we actually have a third choice at every step: we can choose to stop. That's how we get a finite path to a whole fraction of $ 2^n $.

But what if we never stop? We have to apply a limit: we try to spot a pattern and try to fast-forward it. Note that by halving each step vertically on the graph, we've actually linearized each approach into a straight line which ends. Now we can take limits visually just by intersecting lines with the top edge.

Right away we can spot two convergent limits: by always choosing either the left or the right branch, we end up at respectively $ 0 $ and $ 1 $.

These two sequences both converge to $ \frac{1}{2} $. It seems that 'at infinity steps', the graph meets up with itself in the middle.

But the graph is now a true fractal. So the same convergence can be found here. In fact, the graph meets up with itself anywhere there is a multiple of $ \frac{1}{2^n} $.

That's pretty neat: now we can eliminate the option of stopping altogether. Instead of ending at $ \frac{5}{16} $, we can simply take one additional step in either direction, followed by infinitely many opposite steps. Now we're only considering paths that are infinitely long.

But if this graph only leads to fractions of $ 2^n $, then there must be gaps between them. In the limit, the distance between any two adjacent numbers in the graph shrinks down to exactly $ 0 $, which suggests there are no gaps. This infinite version of the binary tree must lead to a lot more numbers than we might think.
Suppose we take a path of alternating left and right steps, and extend it forever. Where do we end up?

We can apply the same principle of an upper and lower bound, but now we're approaching from both sides at once. Thanks to our linearization trick, the entire sequence fits snugly inside a triangle.

If we zoom into the convergence at infinity, we actually end up at $ \class{orangered}{\frac{2}{3}} $.
Somehow we've managed to coax a fraction of $ 3 $ out of a perfectly regular binary tree.

If we alternate two lefts with one right, we can end up at $ \class{orangered}{\frac{4}{7}} $. This is remarkable: when we tried to visualize all the rational numbers by combining all kinds of divisions, we were overthinking it. We only needed to take binary divisions and repeat them infinitely with a limit.

Every single rational number can then be found by taking a finite amount of steps to get to a certain point, and then settling into a repeating pattern of lefts and/or rights all the way to infinity.

If we can find numbers between $ 0 $ and $ 1 $ this way, we can apply the exact same principle to the range $ 1 $ to $ 2 $. So we can connect two of these graphs into a single graph with its tip at $ 1 $.

But we can repeat it as much as we like. The full graph is not just infinitely divided, but infinitely big, in that no finite box can contain it. That means it leads to every single positive rational number. We can start anywhere we like. Is your mind blown yet?

No? Ok. But if this works for positives, we can build a similar graph for the negatives just by mirroring it. So we now have a map of the entire rational number set. All we need to do is take infinite paths that settle into a repeating pattern from either a positive or a negative starting point. When we do, we find every such path leads to a rational number.
So any rational number can be found by taking an infinite stroll on one of two infinite binary trees.

Wait, did I say two infinite trees? Sorry, I meant one infinitely big tree.
See, if we repeatedly scale up a fractal binary tree and apply a limit to that, we end up with almost exactly the same thing. Only this time, the two downward diagonals always eventually fold back towards $ 0 $. This creates a path of infinity + 1 steps downward. While that might not be very practical, it suggests you can ride out to the restaurant at the end of the universe, have dinner, and take a single step to get back home.

Is it math, or visual poetry? It's time to bring this fellatio of the mind to its inevitable climax.

$ \class{blue}{0} $

$ \class{green}{1} $

$ \class{blue}{0} $

$ \class{green}{1} $

$ \class{blue}{0} $

$ \class{green}{1} $

You may wonder, if this map is so amazing, how did we ever do without?
Let's label our branches. If we go left, we call it $ 0 $. If we go right, we call it $ 1 $.

$$ \frac{5}{3} = \class{green}{11}\class{blue}{0}\hspace{2pt}\class{green}{1}\class{blue}{0}\hspace{2pt}\class{green}{1}\class{blue}{0}… $$

We can then identify any number by writing out the infinite path that leads there as a sequence of ones and zeroes—bits.

But you already knew that.

$$ \frac{5}{3} = \class{green}{1}.\class{green}{1}\class{blue}{0}\hspace{2pt}\class{green}{1}\class{blue}{0}\hspace{2pt}\class{green}{1}\class{blue}{0}…_2 $$

See we've just rediscovered the binary number system. We're so used to numbers in decimal, base 10, we didn't notice. Yet we all learned that rational numbers consist of digits that settle into a repeating sequence, a repeating pattern of turns. Disallowing finite paths works the same, even in decimal: the number $ 0.95 $ can be written as $\, 0.94999…\, $, i.e. take one final step in one direction, followed by infinitely many steps the other way.

$$ \frac{4}{5} = \class{blue}{0}.\class{green}{11}\class{blue}{00}\hspace{2pt}\class{green}{11}\class{blue}{00}…_2 $$

When we write down a number digit by digit, we're really following the path to it in a graph like this, dialing the number's … er … number. The rationals aren't shaped like a binary tree, rather, they look like a binary tree when viewed through the lens of binary division. Every infinite binary, ternary, quinary, etc. tree is then a different but complete perspective of the same underlying thing. We don't have the map, we have one of infinitely many maps.

$$ π = \class{green}{11}.\class{blue}{00}\class{green}{1}\class{blue}{00}\class{green}{1}\class{blue}{0000}\class{green}{1}…_2 $$

Which means we can show this graph is actually an interdimensional number portal.
See, we already know where the missing numbers are. Irrational numbers like $ π $ form a never-repeating sequence of digits. If we want to reach $ π $, we find it's at the end of an infinite path whose turns do not repeat. By allowing such paths, our map leads us straight to them. Even though it's made out of only one kind of rational number: division by two.

$$ π = \mathop{\class{no-outline}{►\hspace{-2pt}►}}_{\infty\hspace{2pt}} x_n \,? $$

So now we've invented real numbers. How do we visualize this invention? And where does continuity come in? What we need is a procedure that generates such a non-repeating path when taken to the limit. Then we can figure out where the behavior at infinity comes from.

Because the path never settles into a pattern, we can't pin it down with a single neat triangle like before. We try something else. At every step, we can see that the smallest number we can still reach is found by always going left. Similarly, the largest available number is found by always going right. Wherever we go from here, it will be somewhere in this range.

We can set up shrinking intervals by placing such triangles along the path, forming a nested sequence.

$$ \begin{align} 3 \leq & π \leq 4 \\ 3.1 \leq & π \leq 3.2 \\ 3.14 \leq & π \leq 3.15 \\ 3.141 \leq & π \leq 3.142 \\ 3.1415 \leq & π \leq 3.1416 \\ 3.14159 \leq & π \leq 3.14160 \\ \end{align} $$

$$ \begin{align} 11_2 \leq & π \leq 100_2 \\ 11.0_2 \leq & π \leq 11.1_2 \\ 11.00_2 \leq & π \leq 11.01_2 \\ 11.001_2 \leq & π \leq 11.010_2 \\ 11.0010_2 \leq & π \leq 11.0011_2 \\ 11.00100_2 \leq & π \leq 11.00101_2 \\ \end{align} $$

What we've actually done is rounded up and down at every step, to find an upper and lower bound with a certain amount of digits. This works in any number base.

Let's examine these intervals by themselves. We can see that due to the binary nature, each interval covers either the left or right side of its ancestor. Because our graph goes on forever, there are infinitely many nested intervals. This tower of $ π $ never ends and never repeats itself, we just squeezed it into a finite space so we could see it better.

If we instead approach a rational number like $ \frac{10}{3} = 3.333…\, $ then the tower starts repeating itself at some point. Note that the intervals don't slide smoothly. Each can only be in one of two places relative to its ancestor.

In order to reach a different rational number, like $ 3.999… = 4 $, we have to establish a different repeating pattern. So we have to rearrange infinitely many levels of the tower all at once, from one configuration to another. This reinforces the notion that rational numbers are not continuous.

If the tower converges to a number, then the top must be infinitely thin, i.e. $ 0 $ units wide. That would suggest it's meaningless to say what the interval at infinity looks like, because it stops existing. Let's try it anyway.

There is only one question to answer: does the interval cover the left side, or the right?

Oddly enough, in this specific case of $ 3.999…\, $ there is an answer. The tower leans to the right. Therefor, the state of the interval is the same all the way up. If we take the limit, it converges and the final interval goes right.

But we can immediately see that we can build a second tower that leans left, which converges on the same number. We could distinguish between the two by writing it as $ 4.000…\, $ In this case the final interval goes left.

If we approach $ 10/3 $, we take a path of alternating left and right steps. The state of the interval at infinity becomes like our paradoxical lamp from before: it has to be both left and right, and therefor it is neither, it's simply undefined.

The same applies to irrational numbers like $ π $. Because the sequence of turns never repeats itself, the interval flips arbitrarily between left and right forever, therefor it is in an undefined state at the end.

But there's another way to look at this.
If the interval converges to the number $ π $, then the two sequences of respectively lower and upper bounds also converge to $ π $ individually.

Remember how we derived our bounds: we rounded down by always taking lefts and rounded up by always taking rights. The shape of the tower depends on the specific path you're taking, not just the number you reach at the end.

That means we're approaching the lower bounds so they all end in $ 0000… \, $ Their towers always lean left.

If we then take the limit of their final intervals as we approach $ π $, that goes left too. Note that this is a double limit: first we find the limit of the intervals of each tower individually, then we take the limit over all the towers as we approach $ π $.

For the same reason, we can think of all the upper bounds as ending in $ 1111 …\, $ Their towers always lean right. When we take the limit of their final intervals and approach $ π $, we find it points right.

But, we could actually just reverse the rounding for the upper and lower bounds, and end up with the exact opposite situation. Therefor it doesn't mean that we've invented a red $ π $ to the left and green $ π $ to the right which are somehow different. $ π $ is $ π $. This only says something about our procedure of building towers. It matters because the towers is how we're trying to reach a real number in the first place.

See, our tower still represents a binary number of infinitely many bits. Every interval can still only be in one of two places. To run along the real number line, we'd have to rearrange infinitely many levels of the tower all at once to create motion. That still does not seem continuous.

We can resolve this if we picture the final interval of each tower as a bit at infinity. If we flip the bit at infinity, we swap between two equivalent ways of reaching a number, so this has no effect on the resulting number.

In doing so, we're actually imagining that every real number is a rational number whose non-repeating head has grown infinitely big. Its repeating tail has been pushed out all the way past infinity. That means we can flip the repeating part of our tower between different configurations without creating any changes in the number it leads to.

That helps a little bit with the intuition: if the tower keeps working all the way up there, it must be continuous at its actual tip, wherever that really is. A continuum is then what happens when the smallest possible step you can take isn't just as small as you want. It's so small that it no longer makes any noticeable difference. While that's not a very mathematical definition, I find it very helpful in trying to imagine how this might work.

$ 1, 2, 3, 4, 5, 6, … $

Finally, we might wonder how many of each type of number there are.
The natural numbers are countably infinite: there is a procedure of steps which, in the limit, counts all of them. Just start at the beginning, and fast-forward.

$$ 1, 2, 3, 4, 5, 6, … $$

$$ \class{orangered}{2, 4, 6, 8, 10, 12, …} $$

$$ \class{green}{0, 1, -1, 2, -2, 3, …} $$

We can find a similar sequence for the even natural numbers by multiplying each number by two. We can also alternate between a positive and negative sequence to count the integers. We can match up the elements one-to-one, which means all three sequences are equally long. They're all countably infinite.
There are as many even positives as positives. Which is exactly as many as all the integers combined. As counter-intuitive as it is, it is the only consistent answer.

$$ \begin{array}{cccccccc} 1 \hspace{2pt}&\hspace{2pt} 2 \hspace{2pt}&\hspace{2pt} 3 \hspace{2pt}&\hspace{2pt} 4 \hspace{2pt}&\hspace{2pt} 5 \hspace{2pt}&\hspace{2pt} 6 \hspace{2pt}&\hspace{2pt} … \\[6pt] \frac{1}{2} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{2}{2}} \hspace{2pt}&\hspace{2pt} \frac{3}{2} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{4}{2}} \hspace{2pt}&\hspace{2pt} \frac{5}{2} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{6}{2}} \hspace{2pt}&\hspace{2pt} \\[3pt] \frac{1}{3} \hspace{2pt}&\hspace{2pt} \frac{2}{3} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{3}{3}} \hspace{2pt}&\hspace{2pt} \frac{4}{3} \hspace{2pt}&\hspace{2pt} \frac{5}{3} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{6}{3}} \hspace{2pt}&\hspace{2pt} \cdots \\[3pt] \frac{1}{4} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{2}{4}} \hspace{2pt}&\hspace{2pt} \frac{3}{4} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{4}{4}} \hspace{2pt}&\hspace{2pt} \frac{5}{4} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{6}{4}} \hspace{2pt}&\hspace{2pt} \\[3pt] \frac{1}{5} \hspace{2pt}&\hspace{2pt} \frac{2}{5} \hspace{2pt}&\hspace{2pt} \frac{3}{5} \hspace{2pt}&\hspace{2pt} \frac{4}{5} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{5}{5}} \hspace{2pt}&\hspace{2pt} \frac{6}{5} \hspace{2pt}&\hspace{2pt} \\[3pt] \frac{1}{6} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{2}{6}} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{3}{6}} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{4}{6}} \hspace{2pt}&\hspace{2pt} \frac{5}{6} \hspace{2pt}&\hspace{2pt} \class{grey}{\frac{6}{6}} \hspace{2pt}&\hspace{2pt} \\[3pt] \hspace{2pt}&\hspace{2pt} \vdots \hspace{2pt}&\hspace{2pt} \hspace{2pt}&\hspace{2pt} \vdots \hspace{2pt}&\hspace{2pt} \hspace{2pt}&\hspace{2pt} \hspace{2pt}&\hspace{2pt} \hspace{2pt}&\hspace{2pt} \class{white}{\ddots} \end{array} $$

But we can take it one step further: we can find such a sequence for the rational numbers too, by laying out all the fractions on a grid. We can follow diagonals up and down and pass through every single one. If we eliminate duplicates like $ 1 = 2/2 = 3/3 $ and alternate positives and negatives, we can 'count them all'. So there are as many fractions as there are natural numbers. "Deal with it", says Infinity, donning its sunglasses.

$$ \begin{array}{c} 0.\hspace{1pt}\class{green}{1}\hspace{1pt}0\hspace{1pt}0\hspace{1pt}1\hspace{1pt}1\hspace{1pt}1\hspace{1pt}0\hspace{1pt}…_2 \\ 0.\hspace{1pt}1\hspace{1pt}\class{blue}{0}\hspace{1pt}0\hspace{1pt}1\hspace{1pt}0\hspace{1pt}0\hspace{1pt}1\hspace{1pt}…_2 \\ 0.\hspace{1pt}1\hspace{1pt}0\hspace{1pt}\class{green}{1}\hspace{1pt}0\hspace{1pt}0\hspace{1pt}1\hspace{1pt}0\hspace{1pt}…_2 \\ 0.\hspace{1pt}0\hspace{1pt}1\hspace{1pt}1\hspace{1pt}\class{green}{1}\hspace{1pt}0\hspace{1pt}1\hspace{1pt}1\hspace{1pt}…_2 \\ 0.\hspace{1pt}1\hspace{1pt}0\hspace{1pt}1\hspace{1pt}1\hspace{1pt}\class{blue}{0}\hspace{1pt}0\hspace{1pt}1\hspace{1pt}…_2 \\ 0.\hspace{1pt}0\hspace{1pt}1\hspace{1pt}0\hspace{1pt}1\hspace{1pt}0\hspace{1pt}\class{blue}{0}\hspace{1pt}0\hspace{1pt}…_2 \\ 0.\hspace{1pt}0\hspace{1pt}1\hspace{1pt}1\hspace{1pt}1\hspace{1pt}1\hspace{1pt}0\hspace{1pt}\class{green}{1}\hspace{1pt}…_2 \\ … \\ \\ 0.\hspace{1pt}\class{blue}{0}\hspace{1pt}\class{green}{1}\hspace{1pt}\class{blue}{0\hspace{1pt}0}\hspace{1pt}\class{green}{1\hspace{1pt}1}\hspace{1pt}\class{blue}{0}\hspace{1pt}…_2 \end{array} $$

The real numbers on the other hand are uncountably infinite: no process can list them all in the limit. The basic proof is short: suppose we did have a sequence of all the real numbers between $ 0 $ and $ 1 $ in some order. We could then build a new number by taking all the bits on the diagonal, and flipping zeroes and ones.
That means this number is different from every listed number in at least one digit, so it's not on the list. But it's also between $ 0 $ and $ 1 $, so it should be on the list. Therefor, the list can't exist.

This even matches our intuitive explanation from earlier. There are so many real numbers, that we had to invent a bit at infinity to try and count them, and find something that would tick at least once for every real number. Even then we couldn't say whether it was $ 0 $ or $ 1 $ anywhere in particular, because it literally depends on how you approach it.

What we just did was a careful exercise in hiding the obvious, namely the digit-based number systems we are all familiar with. By viewing them not as digits, but as paths on a directed graph, we get a new perspective on just what it means to use them. We've also seen how this means we can construct the rationals and reals using the least possible ingredients required: division by two, and limits.

Drowning By Numbers

In school, we generally work with the decimal representation of numbers. As a result, the popular image of mathematics is that it's the science of digits, not the underlying structures they represent. This permanently skews our perception of what numbers really are, and is easy to demonstrate. You can google to find countless arguments of why $ 0.999… $ is or isn't equal to $ 1 $. Yet nobody's wondering why $ 0.000… = 0 $, though it's practically the same problem: $ 0.1, 0.01, 0.001, 0.0001, … $

Furthermore, in decimal notation, rational numbers and real numbers look incredibly alike: $ 3.3333… $ vs $ 3.1415…\, $ The question of what it actually means to have infinitely many non-repeating digits, and why this results in continuous numbers, is hidden away in those 3 dots at the end. By imagining $ π $ as $ 3.1415…0000… $ or $ 3.1415…1111… $ we can intuitively bridge the gap to the infinitely small. We see how the distance between two neighbouring real numbers must be so small, that it really is equivalent to $ 0 $.

That's not as crazy as it sounds. In the field of hyperreal numbers, every number actually has additional digits 'past infinity': that's its infinitesimal part. You can imagine this to be a multiple of $ \frac{1}{\infty} $, an infinitely small unit greater than $ 0 $, which I'll call $ ε $. You can add $ ε $ to a real number to take an infinitely small step. It represents a difference that can only be revealed with an infinitely strong microscope. Equality is replaced with adequality: being equal aside from an infinitely small difference.

You can explore this hyperreal number line below.

As $ ε $ is a fully functioning hyperreal number, $ ε^2 $ is also infinitesimal. In fact, it's even infinitely smaller than $ ε $, and we can keep doing this for $ ε^3, ε^4, …\,$ To make matters worse, if $ ε $ is infinitesimal, then $ \frac{1}{ε} $ must be infinitely big, and $ \frac{1}{ε^2} $ infinitely bigger than that. So hyperreal numbers don't just have inwardly nested infinitesimal levels, but outward levels of increasing infinity too. They have infinitely many dimensions of infinity both ways.

So it's perfectly possible to say that $ 0.999… $ does not equal $ 1 $, if you mean they differ by an infinitely small amount. The only problem is that in doing so, you get much, much more than you bargained for.

A Tug of War Between the Gods

That means we can finally answer the question we started out with: why did our continuous atoms seemingly all have $ 0 $ mass, when the total mass was not $ 0 $? The answer is that the mass per atom was infinitesimal. So was each atom's volume. The density, mass per volume, was the result of dividing one infinitesimal amount by another, to get a normal sized number again. To create a finite mass in a finite volume, we have to add up infinitely many of these atoms.

These are the underlying principles of calculus, and the final puzzle piece to cover. The funny thing about calculus is, it's conceptually easy, especially if you start with a good example. What is hard is actually working with the formulas, because they can get hairy very quickly. Luckily, your computer will do them for you:

We're going to go for a drive.

We'll graph speed versus time. We have kilometers per hour vertically, and hours horizontally. We've also got a speedometer—how fast—and an odometer—how far.

Suppose we drive for half an hour at 50 km/h.

$ \class{orangered}{25} $

We end up driving for 25 km. This is the area of spanned by the two lengths: $ 50 \cdot \frac{1}{2} $, a rectangle.

$ \class{orangered}{60} $

Now we hit the highway and maintain 120 km/h for the rest of the hour. We go an additional 60 km, the area of the second rectangle, $ 120 \cdot \frac{1}{2} $.
Whenever we multiply two units like speed and time, we can always visualize the result as an area.

$ \class{slate}{85} $

Because we crossed 85 km in one hour, this is equivalent to driving at a constant speed of 85 km/h for the duration. The total area is the same.

If this were a race between two different cars, we would see a photo finish. The distance travelled in kilometers is identical at the 1 hour mark. Where they differ is in their speed along the way, with the red car falling behind and then catching up.

The difference is visible in the slope of both paths. The faster the car, the more quickly it accumulates kilometers. If it drove 25 km in half an hour, then its speed was 50 km/h, $ \frac{25}{0.5} $. This is the distance travelled divided by the time it took, vertical divided by horizontal.

Slope is a relative thing. If we shrink the considered time, the distance shrinks along with it, and the resulting speed is the same. What we're really doing is formalizing the concept of a rate of change, of distance over time.

Constant speed means a constant increase in distance. We can directly relate the area being swept out left to right with the accumulated distance by each car. This is clue number 1.

Now suppose the red car starts ahead by 10 km and drives the same speeds.
It will also end up 10 km ahead after 1 hour, its path has simply been shifted by 10 units. The slope is unchanged: it doesn't matter where you are and where you've been, only how fast you're going right now. It's what's called an instantaneous quantity, it describes a situation only in the moment. This is clue number 2.

In order to get ahead, the red car had to drive there. So we can imagine it started earlier, $ \frac{1}{5} $ of an hour, driving for 10 km at the same speed. Again, the equality holds: area swept out equals accumulated distance, we add another $ 50 \cdot \frac{1}{5} $. Constant slope still equals constant speed.

One curve describes how the other changes in the moment, therefor the two quantities are linked somehow. We add up area to go from speed to distance; we find slope to go from distance to speed. We're going to examine this two-way relationship more.

Real cars don't start or stop on a dime, they accelerate and decelerate. So we're going to try more realistic behavior.

Suppose the speed follows a curve. In one hour, the car starts from 0 km/h, accelerates to over 100 km/h and then smoothly decelerates back to standstill. The distance travelled also curves smoothly, from 0 to 60 km, so we've driven 60 km in total.

We can immediately see that at the point where the car was going fastest, the distance was increasing the most. Its slope is steepest at that point. The relationship between the two curves holds.

But actually measuring it is a problem. First, there are no more straight sections to measure the slope on. If we take two points on a curve, the line that connects them doesn't touch the curve, it crosses it at an angle.

Second, we can no longer measure the area by dividing it into rectangles, or any other simple geometric shape. There will always be gaps. We can solve both of these problems with a dash of infinity.

We'll start with area. We have to find an upper and a lower bound again.
We're going to divide the curve into 4 sections.

First, the upper bound. We find the highest value in each section and make a rectangle of that height. This approach is too greedy and overestimates.

The lower bound is similar. We find the smallest value in each interval and make rectangles of that height.
This underestimates and leaves areas uncovered.

If we do 7 divisions instead. We can see that the upper bound has decreased: there is less excess area. The lower bound has increased: the gaps are smaller and more area is covered.

With 10 divisions, it's even better. It seems the upper and lower bounds are approaching each other.

And the same at 13 divisions. If we keep doing this, our slices will get thinner and thinner, and we'll be adding more of them together. If we take a limit, each slice becomes infinitely thin, and there are infinitely many of them. Let's step back and see what that means.

Take for example the sequence of lower bounds.

Because every slice is equally wide, we can glue them together into a single rectangle per step.
Its width $ w $ is the thickness of a single slice, and its height $ h $ is the sum of the heights of the slices.

In the limit, this rectangle becomes both infinitely thin and infinitely tall. This is a tug of war between Zero and Infinity where at first sight, they both seem to win. That's a problem. Luckily, we're not interested in the rectangle itself, but rather its area.

We can change a rectangle's sides without changing its area. We multiply its width by one factor (e.g. $ 2 $), and divide the height by the same amount. The area $ 2w \cdot \frac{h}{2} $ is unchanged. Hence, we can normalize our rectangles to all have the same width, for example $ 1 $.

We can do the same for the upper bounds. We can see that both areas are converging on the same value. This is the true area under the curve, which is neither zero nor infinite. In this tug of war, both parties are equally matched.

Now our sequence looks very different: it's approaching a definite area, sandwiched between red and blue.

$ \class{slate}{60} $

If we take the limit, we get the area under our curve.

$ \class{orangered}{60} $

This way we can find the area under any smooth curve. This process is called integration. The symbol for integration is $ \int_a^b $ where $ a $ and $ b $ are the start and end points you're integrating between. The S-shape stands for our sum, adding up infinitely many pieces.

$$ \int_0^T \! f(t) \mathrm{d} t $$

We can then integrate one curve to make another, by sweeping out area horizontally from a fixed starting point. We move the end point to a time $ T $ and plot the accumulated value along the way. Using limits, we can do this continuously. This takes us from speed to distance travelled. The quantity $ \,\mathrm{d}t\, $ is the infinitesimal width of each slice, an infinitely small amount of time.

Now we just need to figure out the reverse and find slopes. We'll go back to our failed attempt from earlier.

If we shrink the distance we're considering, our slope estimate gets closer to the true value. But if we try to take a limit, we end up dividing $ 0 $ by $ 0 $.

Instead we need to normalize our sequence again so it doesn't vanish.

We only care about slope: the ratio of the two right sides. Which means, if we scale up each triangle, the ratio is unchanged. That just comes down to multiplying both sides by the same number. Again we can scale them all to the exact same width.

Now we've created a limit that does converge to something rather than nothing.

This finite value is the slope at the point we were homing in on. Because we can apply this process at any point on the curve, we can find the exact slope anywhere. This is called finding the derivative or differentiation.

$$ \frac{ \mathrm{d} f(t) }{\mathrm{d} t} $$

We can also apply this process over an entire curve to generate a new one. So now we know how to go the other way: distance to speed. Mathematically, we are dividing an infinitesimal piece of the distance, $ \,\mathrm{d} \class{slate}{f(t)}\, $, by an infinitesimal slice of time $ \,\mathrm{d} t\, $. Working with infinitesimal formulas is tricky however. There's always an implied limit being taken in order to reach them in the first place. Indeed, it took centuries to formalize this fuzzy explanation into what we call differential forms today.

We can note that if we shift the distance curve up or down, the speed is unchanged. When you take a derivative, any constant value you've added to your function simply disappears. This shows again that speed is always in the moment, it only describes what's going on in an infinitely short piece of curve.

Differentiation is then like x-ray specs for curves and quantities, and it's turtles all the way down. For example, if we differentiate speed, we get acceleration. This is another rate of change, of speed over time. We see the car's acceleration is initially positive, speeding up, and then goes negative, to slow down, i.e. accelerate in the opposite direction.
Note: The acceleration has been divided by 4 to fit.

If we integrate acceleration to get speed, we have to count the second part as negative area: it is causing the speed to decrease.

We can see that the point of maximum speed is the point where the acceleration passes through $ 0 $. One of the most useful applications of derivatives is indeed to find a maximum or minimum of a curve more easily. No matter where it is, the slope at such a point must always be horizontal—provided the curve is smooth.

Let's end this with a more exciting example. What's tall, fast and makes kids scream?

A roller coaster! We'll construct a little track by welding together pieces of circles and lines.

Alas, we shouldn't be too proud of our creation. Even though it looks smooth, there's something very wrong. This is how you build roller coasters when you don't want people to have fun. To see the problem, we need to use our x-ray specs.

$$ \class{orangered}{f^{\prime}(x)} = \frac{\mathrm{d}\class{slate}{f(x)}}{\mathrm{d}x} $$

We differentiate the height into its slope. It has sharp corners all over the place. Even though the track itself looks smooth, it doesn't change smoothly. The slope is constant in the straight sections and changes rapidly in the curved sections.

$$ \class{green}{f^{\prime\prime}(x)} = \frac{\mathrm{d^2}\class{slate}{f(x)}}{\mathrm{d}x^2} $$

If we take the derivative of the slope, i.e. find the slope's slope, we get a measure of curvature. It's positive inside valleys, negative on top of crests. This graph is even worse: there are sharp peaks and cliffs. Note that in the formula, we are now dividing by the square of the infinitesimal distance $ \mathrm{d}x $. This is like going two levels down on the hyperreal number line and back up again.

$$ \class{teal}{κ(x)} = \frac{1}{ρ} = \frac{ \class{green}{f^{\prime\prime}(x)} } { (1 + \class{orangered}{f^{\prime}(x)}^2)^{3/2} } $$

We can see better if we replace the second derivative with the 2D curvature.
This is the radius of the circle that touches the curve at a given point. As this radius gets infinitely big on straight sections, we use its inverse, $ \class{teal}{κ} $. Because of how we built the track, $ κ $ switches between $ 0 $ and a constant positive or negative value.
At every switch, there will be a corresponding change in force, a jerk.

Let's simulate a ride. As riders go through our curved sections, their inertia will push them to the outside of the curve. From their point of view, this is a centrifugal force up or down. We'll plot the (subjective) vertical G force including gravity. It starts at a comfy 1 G, but then swings wildly between 0.5 G and 1.25 G.

Even though the track seems smooth, we can see that the vertical G's are not. Every time we enter a curve, we experience a sudden jerk up or down. This is due to the jumps in the curvature. The G's are themselves curved, because the rider's sense of gravity decreases as the cart goes vertical. The sharp dips below 0.5 G are not simulation errors: this is actually what it would feel like.

To really highlight the problem, we need to x-ray the G's and derive again. G forces are a form of acceleration. The derivative of acceleration is a change in force, called jerk. Whenever it's non-zero, you feel jerked in a particular direction.

To fix this, we need to alter the curve of the track and smooth it out at all the different levels of differentiation. Here I've applied a relaxation procedure. It's like a blur filter in photoshop: we replace every point on the track with the average of its neighbours. We get a subtly different curve. Its height hasn't changed much at all, it's just a little bit less tense.

But this minor change has a huge effect on both slope and radius of curvature. They are completely smoothed out, with all corners and jumps removed.

If we do another simulation, the G force graph looks completely different. There are no more jumps.

But the real difference is in jerk. There are no more actual jerks, only smooth oscillations. Instead of bruises, riders will get butterflies. Thanks to calculus, we avoided that painful lesson without ever having to ride it ourselves.

Please check your pockets for loose items. Lost property will not be returned.

Let's start with the original, unrelaxed track. Thanks to calculus, we can simulate head-bobbing so you can get a feel for how jerky this is. Even virtually, this isn't very pleasant.

This is the improved track. Notice the smooth transitions in and out of curves.

And that's how you make sweet roller coasters: by building them out of infinitely small, smooth pieces, so you don't get jerked around too much.

That was differential and integral calculus in a nutshell. We saw how many people actually spend hours every day sitting in front of an integrator: the odometers in their cars, which integrate speed into distance. And the derivative of speed is acceleration—i.e. how hard you're pushing on the gas pedal or brake, combined with forces like drag and friction.

By using these tools in equations, we can describe laws that relate quantities to their rates of change. Drag, also known as air resistance, is a force which gets stronger the faster you go. This is a relationship between the first and second derivatives of position.

In fact, the relaxation procedure we applied to our track is equivalent to another physical phenomenon. If the curve of the coaster represented the temperature along a thin metal rod, then the heat would start to equalize itself in exactly that fashion. Temperature wants to be smooth, eventually averaging out completely into a flat curve.

Whether it's heat distribution, fluid dynamics, wave propagation or a head bobbing in a roller coaster, all of these problems can be naturally expressed as so called differential equations. Solving them is a skill learned over many years, and some solutions come in the form of infinite series. Again, infinity shows up, ever the uninvited guest at the dinner table.

Closing Thoughts

Infinity is a many splendored thing but it does not lift us up where we belong. It boggles our mind with its implications, yet is absolutely essential in math, engineering and science. It grants us the ability to see the impossible and build new ideas within it. That way, we can solve intractable problems and understand the world better.

What a shame then that in pop culture, it only lives as a caricature. Conversations about infinity occupy a certain sphere of it—Pink Floyd has been playing on repeat, and there's usually someone peddling crystals and incense nearby.
"Man, have you ever, like, tried to imagine infinity…?" they mumble, staring off into the distance.

"Funny story, actually. We just came from there…"

Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.

More like this: How to Fold a Julia Fractal.

↧

On WebGL

March 10, 2013, 4:00 pm

≫ Next: Storms and Teacups

≪ Previous: To Infinity… And Beyond!

More than pretty pictures

Like a dragon, WebGL slumbers. But you've seen them, right? Those seemingly magical demos that transform your ordinary browser into a lush 3D world with one click?

While available in Chrome and Firefox on the desktop, WebGL is still not widely supported. So far it's mostly used for demo projects and flashy one-off brochures. On the few mobile devices that support it, you need developer access to enable it. It's certainly nowhere near to being ready for prime time. So why should you care?

AlteredQualia

The Black Sheep

The goal of WebGL is to bring the graphics capabilities of traditional apps and games into the browser, with performance as the main benefit. The graphics hardware does the work directly, leaving the CPU to just coordinate. Yet those developers look on with skepticism: "You mean we have to code in JavaScript?" There's grumbling about the limited capabilities too, which lag a few years behind the latest OpenGL and Direct3D APIs, and there's worries about copyright and modding.

First, we have to be honest: there's no question that native apps and 3D engines will continue to excel, bringing cutting edge graphics and performance. The layers of indirection in both HTML5 and WebGL cannot be circumvented.

But they do serve a purpose: to provide a safe sandbox for untrusted code from the web at large. Even triple-A games still occasionally crash, a result of their complexity, with thread synchronization, memory management and manual context-switching the price to pay. Random phishers shouldn't have that level of access to your system, nor should it be required.

AlteredQualia

WebGL represents a different way of using high-performance graphics: not as a bare metal API with caveats, but as a safe service to be exposed, to be flicked on or off without a second thought. It may not sound like much, but the security implications are big and will only be solved carefully, over time. It's undoubtedly a big reason behind Apple and Microsoft's reluctance to embrace it.

We should also note that this isn't a one-way cross-over. HTML has already snuck into the real-time graphics scene. First we saw in-game web views and browsers, then UIs such as Steam's overlay. In fact, all of Steam is WebKit. The main benefit is familiarity: designers can use the well-known techniques of the web both inside and outside the game. This mirrors the way Adobe Flash entered the gaming space before, being used to drive menus and overlays in many games.

It's been said that the skills required for front-end web development and game development eventually converge on the same thing. The technologies certainly have.

Felix Woitzel

The Procedural Canvas

The web is the world's only universal procedural medium. Content is downloaded in partially assembled form, and you and your browser decide how it should be displayed. The procedural aspect has always been there, and today's practice of responsive design is just another evolution in procedural page layout. It all started with resizable windows and tables.

But when we decide to put a graphic into a page, we still bake it into a grid of pixels and send that down the pipe. This has worked great as a delivery mechanism, but is starting to show its age, due to high DPI displays and adaptive streaming.

It's also pushed the web further towards consumption: YouTube and Tumblr are obvious results. Both sites have a huge asymmetry between content creator and consumer, encouraging sharing rather than creating.

Turing pattern gradient attractor feedback

Felix Woitzel

Real-time graphics level the playing field: once built, both creator and consumer have the same degree of control—at least in theory. All the work necessary to produce the end result is ideally being done 60 times per second. The experience of e.g. playing a game is like a sort of benign DRM, which requires you to access the content in a certain way. All native apps implement such 'DRM' by accident: their formats are binary and often proprietary, the code is compiled. Usually modding is supported in theory—that's what Downloadable Content is, an official mod—but the tools simply aren't included.

The web is different. No matter how obfuscated, all code eventually has to talk to an interface that is both completely open and introspective. You can hook into any aspect of it and watch the data. There isn't a serious web developer around who would argue that this is a bad thing, who hasn't spent time deconstructing a site through a web inspector on a whim.

Florian Bösch

This is where WebGL gets interesting. It takes the tools normally reserved for well, the hardcore geeks, and makes them much more open and understandable. I can certainly say from experience that coding with an engine like Three.js is an order of magnitude more productive than e.g. Ogre3D in C++. For most of the things I want to do with it, the performance difference is negligible, but there is much less code. Once you get your dev environment going, creating a new 3D scene is as simple as opening a text file. You can interact with your code live through the console for free.

More so, it integrates with the publishing tools we already know. I wonder for example how many hours of dev time the game industry has spent reinventing the wheel for fonts, menus, option screens, etc. To be fair, they often do so with amazing production value. But guess what: you now have CSS 3D, and soon you'll have CSS shaders. You don't need custom in-house tools when your designers can just use Chrome's Inspector and get the exact same result. Content delivery is easy: you have cloud storage, CDNs and memory caches at your disposal.

There is a missing link however: WebGL is a canvas inside the page, isolated from what's outside. But you could imagine APIs to help bring HTML content into a WebGL texture, taking over responsibility for drawing it. After all, most web browsers already use hardware acceleration to compose 2D web pages on screen. The convergence has already started.

Florian Bösch

The web has a history of transformative changes. CSS gave us real web design, Flash gave us ubiquitous video, Firebug gave us Web Inspectors, jQuery gave us non-painful DOM manipulation, and so on. None of these ideas were new in computing when they debuted, the web merely adapted to fill a need. WebGL is an idea in a similar vein, a base platform for an ecosystem of specialized frameworks on top.

It can help lead to a WolframAlpha-ized LCARS future, where graphics can be interactive and introspective by default. Why shouldn't you be able to click on a news graphic to filter the view, or download the dataset? For sure, this is not something that uniquely requires WebGL, and tools like d3.js are already showing the way with CSS and SVG. As a result, the last mile of interactivity becomes a mere afterthought: everything is live anyway. What WebGL does is raise the limit significantly on what sort of content can be displayed in a browser. It's not until those caps are lifted that we can say with a straight face that web apps can rival native apps.

Still, we shouldn't be aiming to recreate Unreal Engine in HTML / JS / GL, though someone will probably try, and eventually succeed. Rather we should explore what happens when you put a 3D engine inside a web page. Is it web publishing, or demoscene? Does it matter?

Ro.me team

Mr.doob

Chrome Workshop

A Useful Baseline

In this light, WebGL's often lamented limitation becomes its strength. WebGL is not modelled after 'grown-up' OpenGL, but mirrors OpenGL ES (Embedded Systems). It's a suite of functionality supported by most mobile devices, but eclipsed by even the crummiest integrated laptop graphics from 3 years ago.

This needn't be a worry for two reasons. First, WebGL supports extensions, which add to the functionality and continue to be specced out. A WebGL developer can inspect the capabilities of the system and determine an appropriate strategy to use. Many extensions are widely supported, and even without extensions, all GL code is already subject to the platform's size limits on resources. WebGL is no different from other APIs, it just puts the bar a bit lower than usual.

Second of all, it means WebGL is the only 3D API that has a shot at being universal, from desks to laps to pockets to living rooms, and everything in between. Your game console could be an Android computer, handheld or appliance. Your TV might run Linux or iOS. So might your fridge. WebGL fits with where hardware and software is going, and adapting to various devices is nothing new for the web. I imagine we might see a standardized benchmark library pop up, and developer tools to make e.g. desktop Chrome mimic a cellphone's limited capabilities.

For the Christmas demo above, I included a simple benchmark that pre-selects the demo resolution based on the time needed to generate assets up front. Additionally, it was built on a 4 year old laptop GPU, so it should run well for the majority of viewers on first viewing. The same can't be said for cutting-edge demoscene demos, which often only run smoothly on top of the line hardware. I know I'm usually resigned to watching them on YouTube instead. As neat as tomorrow's tech is, for most people it only matters what they have today.

This is the biggest philosophical difference between WebGL and OpenGL. WebGL aims to be a good enough baseline that you can carry in your pocket as well as put on a big screen, and make accessible with a simple link. I don't expect graphics legends like John Carmack to take anything but a cursory glance at it, but then, it's not encroaching on his territory. It is a bit surprising though that the demoscene hasn't taken to the web more quickly. It has never been about having top of the line hardware, only what you use it for. Contests like JS1K continue to demonstrate JavaScript's expressiveness, but we haven't really seen the bigger guns come out yet.

And it really is good enough. Here's 150,000 cubes, made out of 1.8 million triangles:

AlteredQualia

Next up is a fractal raytracer. At 30 frames per second, 512x512 pixels, 40 iterations per pixel, each folding 3D space 18 levels deep… that's 5.6 billion folds per second. This intricate visualization is little more than raw number crunching power. That's just the core loop and excludes set up and lighting. It's all driven by a couple kilobytes of JavaScript wrapped in some HTML, delivered over HTTP.

$Distance estimation with fractals$

Why wouldn't you want to play with that? Come try WebGL, the water's fine.

Storms and Teacups

March 23, 2013, 4:00 pm

≫ Next: Why Android Hates You

≪ Previous: On WebGL

If you've been paying attention, you'll have seen a lot more discussions about gender, feminism and harrassment lately. The conversation mostly revolves around the latest incident of the day. I'd like to reflect on the bigger picture instead, and talk about some uncomfortable truths.

This is about how we act, online and offline, and why we do it.
Please read it top to bottom, or not at all.

Special thanks go to the folks who took time to provide feedback on drafts.

The examples used in this article, whether positive or negative, are chosen for their representative nature. They are not unique exceptions that deserve special sympathy, scrutiny or scorn.

Juliana Coutinho

The Shametweet

Atlassian, provider of software development infrastructure, sends out a tweet to advertise one of their services:

If you're ready for a build server so pretty you could take it to the prom, you're ready for @Atlassian Bamboo.

The response is immediate and harsh:

Sexist ads won't win you fans!
Grow up and don't use gendered terms to promote your tech products

A reply is made:

Sorry you don't like the wording!
We weren't being gender specific though. Men are pretty too!

Finally, cue the condescending follow ups:

For fuck's sake, way to exhibit absolutely no understanding whatsoever of the subtleties of patriarchy. Get educated.

Look closely and you'll see this pattern pop up more and more, in various forms. The key word is always educate, or more accurately, re-educate. The tone varies from feigned concern to outright hostility. If only you weren't so ignorant, you wouldn't have made such horribly offensive statements. Apologies are dismissed as insincere, a refusal to admit one's true sins.

But let's step back for a bit and look at what was said. First, Atlassian's reply is right, they weren't being gender specific, they merely compare a piece of software to prom. That's not what the indignant reader saw. They read between the lines, and substitute it with something like this:

Women are expected by society to always be pretty. We think this is great.
Prom is a celebration of this institutional sexism. Let's trivialize it by comparing it to server technology.
We think you'll enjoy our use of sexism and buy our products.

For sure, everyone has their own interpretation and (I hope) I'm exaggerating. But the tweet's supposed sexism is not actually there. The speaker's intent is completely ignored, the hurt feelings of the offended take priority. The reinterpretation itself is sexist: only women can be pretty.

The worst form of this behavior is what I call the Shametweet. This is when someone retweets a statement—usually a perceived insult directed at themselves—without any further comment. The tweeter seemingly considers it beneath themselves to address the insolence directly. Instead, they choose to demonstrate their superior sensibilities to their followers. Those will then jump to his or her's defense, making the problem go away with a single click of a button, while they maintain an aura of innocent plausible deniability.

To my lack of surprise, it's mostly women who I see doing this, voluntarily turning themselves into objects, letting others claim their agency, and usually men who are all too eager to jump to the rescue, even when it's not requested. Some celebrities do it too, sicking a million followers on a target who failed to stroke their ego that morning. More than a few of these fragile celebs are men.

Objectification

Anita Sarkeesian dislikes sexist tropes and objectification of women in video games and wants to bring this problem to light. As one might expect with anyone who does anything on the internet, trolls show up, and insults and accusations of sexism start flying around. Things get ugly, and valid criticism is lost in a sea of crud. Anita cleverly uses the Streisand effect to her advantage, gets publicity in both feminist and general media, ending in a successful $158K Kickstarter campaign to produce a web video series.

Jezebel, billing itself as "Celebrity, Sex, Fashion for Women", is one of the sites eagerly siding with Anita. It appeals to their readership: a young audience of mostly women who enjoy seeing another woman doing her own thing, more so when it irritates men and advances the status of the sisterhood—if the comments are anything to go by.

Fast forward. Jezebel asks "Why Is Michelle Williams in Redface?", "You should know better".

Her transgression was to appear on a fashion magazine cover "dressed in a braided wig, dull beads, and turkey feathers [...] in a flannel shirt, jeans, and [...] some sort of academic or legal robe. [...] An attempt to portray reservation nobility [...] like she's the member of another race."

But they don't stop there. This tasteless display is in fact "akin to putting a picture of a Gentile in a stereotypical Jewish getup on the cover of Adolf Hitler's Mein Kampf". Godwin triumphs once again.

The writer may indeed have a point in there somewhere, that is, about stereotypes of First Nations cultures. But the irony is so thick you can spread it like Nutella.

Jezebel eagerly celebrates the advances of women over male-dominated society at every turn, decries Patriarchy and rings the alarm bell whenever supposed standards of equality and self-determination are violated. Now they complain that an industry they focus on, which treats people like objects to be dressed and painted, didn't objectify a woman in a tasteful enough fashion.

They should do an exposé on the Emperor's wardrobe next.

Who is it really, that is pressuring women to be passive, immaculate and above all, politically correct dolls? Is it really all men's fault? Or is it fueled by media and advertising that bills itself "For Women" in giant pink letters, but really seems to be just about "Judging Women" instead, telling them they need to look better, be likeable supermoms as well as executives, but deserve to have it all, honest?

On the other side, gaming sister-site Kotaku asks "She's Sexy. Now kill her?", questioning the "humiliation of sexualized females" in God of War: Ascension. In this game's bloody quest of revenge, after a couple hours of brutally murdering several armies of mythological creatures one by one, you stab the Medusa-like Gorgon in the chest. On top of its giant snake body, right where its breasts are. Gasp.

This scene summarizes "all [the] issues with violence against sexualized female characters in one nutshell." But after describing it in the context of the game, only one real objection remains: "Breasts code some enemies as female, [...] violence against [these] body parts is disturbing," and is not the usual "norm in games".

The game is presenting "a form of feminine beauty that associates exposed, large breasts as beautiful." The author seems to be confusing "sexualized" and "sexy", as if sexualization is only what turns him on—I think it's breasts—and something must be sexualized before it can be arousing. Apparently if the Gorgon had been obese and flat-chested, there'd be no issue in putting it down. Which is exactly what Euryale looked like, the repulsive Gorgon the author must've killed in the previous game.

This attempted pro-woman analysis of sexualized portrayal seems to suggest that a feminized body is automatically sexual, but only if she's hot enough, like say, the "final, sexy boss."

The Social Justice Warriors

Skeptic blogger and retired medical doctor Harriet Hall writes a post, titled I Am Not Your Enemy: An Open Letter to My Feminist Critics. She clarifies exactly what she said and meant on a previous occasion. The comments then continue to argue back and forth about what it all means.

It goes back to a t-shirt she wore at a conference, stating she "felt safe and welcome" and was "just a skeptic, not a 'skepchick', not a 'woman skeptic', just a skeptic". This shirt was apparently so offensive and dehumanizing it reduced one of its victims to tears.

All of this is fallout from the scandal known as ElevatorGate. A man at a conference asked Rebecca Watson up for coffee in an elevator, after a late night in the hotel bar, and accepted no for an answer. Cue the public shaming based on her one-sided account, using her position as a conference speaker, and the inevitable backlash. The man himself however has wisely chosen to stay out of it and remains unidentified. It prompted Richard Dawkins to point to more serious women's issues to possibly worry about, who was then chastized for speaking from white male privilege. This scandal, entirely based on hearsay, is still going on a year later.

In fact, Harriet's thread features an appearance from Rebecca herself. She takes "ten precious minutes" out of her busy schedule to explain she "doesn't really think of [her] at all", after clarifying why she feels the post talks about her directly. Despite admitting to writing and deleting both a blog post and a private email on the subject, Rebecca says Harriet "doesn’t actually deserve an explanation, [or] real estate in my head" which is why she "let others argue over it". Which she says right after arguing over it.

Does this sound at all familiar? She includes that she would be "concerned for [her] personal hygiene" for wearing one shirt several days in a row. I'm not making this up.

Like Dawkins, I wonder: Don't these people have more important things to get angry at? Are they just self-absorbed, seeking publicity through controversy? Some undoubtedly are, but for the majority I think it's far more simple.

It's fair to ask: why are they so bothered and offended, spending their free leisure time organizing miniature online protests, thread after thread? Was the t-shirt (or the tweet) a direct, personal insult? Did it insult a class of people they belong to? Is it specific enough that someone could reasonably argue it applies to them, but not the next person? No.

So why take it personally? It's because it reminds us of an uncomfortable truth about ourselves or the world. In Atlassian's case, it's that beauty has a dark side, and it gives some people an unfair advantage or disadvantage. Did I get this job because of my talents or my looks? Do I present myself badly? Do people judge me by things beyond my control? Do I have a weird face? It reminds us of all the times we've experienced this ourselves, and if you have children, of all the times they will too. The internet becomes a mirror for our own insecurities, and we read our worries into everything.

In Harriet Hall's case, it's the acknowledgement that life is what we make of it, that people disagree with us more than we like to admit, and that often the best thing to do is shrug and not let it bother you, and focus on results rather than labels. Though again, everyone's interpretation is different.

But we don't want to admit that, our pride does not allow it. We'd much rather explain our unease by assuming it was inflicted deliberately, and we make up convenient reasons why that is so, why we were targeted. See, Atlassian is just another sexist tech company, they can't even tweet without insulting every woman on the planet! Harriet Hall, born in 1945, the second ever female intern in the US Air Force, must be an ignorant ditz when it comes to matters of feminism, because of one smelly t-shirt. If you don't see it the same way, well, you're just not educated enough to read between the lines.

It's both men and women who do it. We can argue who is more at fault until the cows come home, but when it comes to sexism it's fair to say men take the brunt of the blame, and are the ones expected to make amends. It's completely one sided, and it's another one of those convenient excuses that we substitute for the real thing. We don't want to talk about the full complexity at play here. Indeed, the closest feminism gets to acknowledging this is, Patriarchy hurts men too! So it's not my fault, just the result of every single choice I've ever made?

When someone points out that viewing everything through a uniquely feminist and female-oriented lens gives a skewed perspective, a rapid fire meme is returned: "But what about the mennzzz?" Attempting to show that inequality applies to both genders, quite often in women's favor, is considered derailing. Showing that the feminist interpretation of history as unbridled Patriarchy is unrealistic, and that feminism has long ago developed its own oppressive and hateful character, is dismissed as misogyny, even when it's women saying it.

There's more handy tropes to end attempts at nuance and shut down discussion: Check Your Privilege, Stop JAQing off (Just Asking Questions), Mansplaining, Victim Blaming, Nice Guy, Schrödinger's Rapist. The list goes on, and all of a sudden, concerns about gendered slurs no longer apply.

The so-called "safe space" that these online social justice groups claim to seek, is just another word for a censored space, and a hypocritical one at that. It's one where certain ideas and thoughts are not to be uttered, and must be replaced by less realistic and less worrisome ones. But no true safe space exists, as offense is always in the eye of the beholder.

Listening involves an interpretation of what people thought it meant they heard.

Women in Open Source

Statistics show that women observe sexism online to a higher degree than men, particularly in tech and open source. Recommendations are made on how to make the community more friendly to women, and most suggestions involve re-educating men to reduce their blindness. More so, it's implied that once the atmosphere is respectful enough, women will join and equality will be achieved.

Sorry, but I don't buy it, because as late as 2006, 28% of participants in proprietary software were women, but only 1.5% in open source. Most open source projects start out as hobbies, created by one person in their spare time. If the community was such a sexist hell for women, wouldn't you expect the web to be littered with the abandoned works of that 1/4th of professionals who are women, who were turned off by how it was received once published? Instead, I find that female-founded projects are far and few, and calls for women to participate consist mainly of inviting them into existing projects, and speaking at established conferences about existing technologies.

Is the increasing role of women in open source a consequence of empowerment and self-direction? Or does it stem from the fact that open source is becoming more important in commercial use, and now more women are tagging along? It's both, naturally, but the huge gap between the two gender ratios can't be reduced to abuse and sexism. For a multitude of reasons, women simply aren't as interested as a group.

A big part of the problem is confidence, and starts much earlier: you must be this smart to be in open source, or so people think. Angela Byron, winner of the 2008 Google-O'Reilly Best Contributor award, called to "Fight the Einstein Perception" in Women in Open Source. It took Google's Summer of Code to convince her to take the plunge and make the career change. Programs like that are great to bring fresh talent into a community, but they won't cause the seismic shift in gender balance that feminism requires. If we want more women in open source, shouldn't we encourage them to just do their own thing, as those 98.5% of contributors who were male seemed to be doing?

Open source is claimed to be a meritocracy, but it really isn't. Once two people start modifying the same code, politics get involved, and I can certainly speak from experience that decisions at the top of an open source project are more about people and their interests than code. It isn't enough to create a good solution, it must be advocated and accepted, and apply to a wide variety of existing scenarios. If the work isn't good enough and fails, reputations take a hit. Like this:

Linus Torvalds can act like a complete asshole, self-admittedly so, chew out his (male) contributors, and nobody in particular seems to mind. Linux is successful either despite or because of it.

Linus builds and directs software millions rely on. His abrasive tone reflects the importance of the issues he deals with on a daily basis. So far, his peers have deemed it socially acceptable. You may hate this, but you can't ignore it.

Can we really say with a straight face that he could talk the exact same way to a female contributor, and nothing would be different? In a culture where "never hit a woman" is considered a valid rule by many, men are the default assumed aggressor in domestic violence, and expected to chase the burglar—another man no doubt—out of the house to protect their wife and children? Or would it spawn thread after thread of discussions of just how bad the transgression was, and how to make sure it never happens again?

Open source culture is quite competitive, but the biggest problem an open source contributor has isn't criticism, it's getting people to pay attention in the first place. Ironically, this is something women are innately privileged in: studies show women have automatic in-group bias—women like women more than men like men—that people prefer their mothers to their fathers, and men are universally associated with negative behavior such as violence. It's propagated in the popular stereotypes of the bumbling husband, the insensitive jock, the aggressive bully, and so on.

That perspective is dismissed by feminists as lashing out from male privilege, and the fear of losing it. But how privileged are men over women, when their life expectancy recedes further from that of women the lower the standard of living? Is there a Kickstarter I can donate to for that? No, instead National Geographic states matter of factly that it's a "troubling trend" and a "wake up call" that men's life expectancy is getting closer to that of women in the US, because it means women are gaining less. They use the margin by which women outlive men as if it's some sort of index of prosperity.

Hey, remember that time when Hillary Clinton said "Women have always been the primary victims of war"? Because they "lose their husbands, their fathers, their sons in combat." A woman who survives is more of a victim than a man who dies for her, please be sure to educate yourself on this.

Could it be that the sexism women say they are constantly subjected to online, is merely the flipside of a coin? One that allows them to cultivate attention with nothing more than a well-chosen avatar, and which men eagerly give to them? How many women forego the make-up in their profiles and videos before lamenting the unsolicited date proposals, awkward as they may be?

I'm not ignoring cases like Kathy Sierra and the persistent, real harassment she received, but let's not forget that it was inflicted by individuals upon individuals, not on womankind.

When the overwhelming majority of open source contributors are men fighting for recognition, do you suppose some of them might feel some resentment that a woman can walk into a room, real or virtual, and make everyone's head turn? If so, do women's concerns deserve automatic precedence over men's? The country I live in has a Minister for the Status of Women after all. Not for Equality.

The Anti-Harrassment Policy

To attend or speak at JSConf, you must agree to a code of conduct. Its goal is to create a positive, harassment free environment, something which I am all for. The policy is starting to be adopted verbatim by other conferences, like PyCon.

But the wording explicitly defines harrassment as including "offensive verbal comments", specifically "related to gender, sexual orientation, disability, physical appearance, body size, race, religion, sexual images in public spaces, deliberate intimidation, stalking, following, harassing photography or recording, sustained disruption of talks or other events, inappropriate physical contact, and unwelcome sexual attention."

How many of the storms in teacups above would fall under this wide umbrella? If the yardstick to be applied is offense, then this basically forces everyone to walk on egg shells and admit guilt ahead of time. "Participants asked to stop any harassing behavior are expected to comply immediately." There is no room here to discuss the merit of a particular case, to measure the validity of a claim.

Keeping it on-topic: the problem with discussing sex at technical conferences

Indeed, the latest is that we cancel the talk first, ask questions later, based on the concerns of a single complaint over a title without a summary. The threat of going public was possibly made, but accounts differ. I find the Ada Initiative's first response to the situation revealing.

While stressing the real issue is staying on topic and not devolving into unnecessary sexual talk, every negative point raised appears to concern only women. "Sexual topics [...] can be perceived as encouragement to humiliate, objectify, and assault women, regardless of the intent of the speaker." And, "Many people are unable to separate 'talking about sex' and 'saying derogatory things about women'." Their response shows they assumed the talk would not be "done in a woman-positive way". That is, a talk featuring a female speaker who blogs about harm reduction.

At no point do they express regret at having silenced a voice. "Be considerate and thoughtful," it ends.

Let me borrow a quote from Stephen Fry: "The only people who are obsessed with food are anorexics and the morbidly obese, and that in erotic terms is the Catholic church in a nutshell." You'll never see more talk of sexism and rape than on feminist websites.

Trigger warnings, humiliation, objectification, assault, rape culture: feminism's opinion of neither men nor women's abilities to act mature around each other seems particularly high.

As an aside, have you ever noticed how Tumblr isn't just a hub for bold feminism, but also erotic fanfics? And by 'erotic' I mean gay sex of dubious consent set in the Twilight universe. You know. Rape. That fangirls write and fantasize about. And joke about in hushed tones at Comic-Con. Is that woman-positive enough, or are the lines blurring a bit?

More recently, someone lost their job after public shaming involving an overheard and misinterpreted comment about "forking" and "dongles", and the guy still felt the need to apologize profusely to the female offendee. Her media presence exceeds his by far and includes tweeting about "[putting] something into your pants [...] like a bunch of socks". Meanwhile followers thanked her for her bravery, that is, snapping a picture with a smile and throwing it to the lions. Who was abusing who here?

Of course it blew up into its own internet storm, but can you blame people for responding in kind to an example that's been so clearly set?

People read Woman fired for getting upset at man's joke and fill in the rest of the story themselves, like this animated GIF equivalent of a temper tantrum. More dignified publications instead carefully explain "Why asking what [she] could have done differently is the wrong question", that is, the one question in this entire fiasco the rest of us could actually learn something from.

Judging a book by its cover is the new tolerance. We throw people into the stocks based on feelings while ignoring intent and assuming victimhood. This is why I fundamentally disagree with equating offense with harassment: it provides unlimited ammo and shuts down discussion rather than giving people the benefit of doubt. It elevates the exception to the norm, by presuming the worst.

Here's a clause I'd like to see instead: if you choose to air minor incidents in public one-sidely—or threaten to do so—rather than resolving the matter in private, you lose by default. Leave the soapbox for the people who actually need it. Also, if a speaker has been invited and has spent time preparing a talk, it's the most basic courtesy to honor that invitation, no matter what. Let people judge it on its own merits. We attend conferences to hear other points of view, not to be sheltered from them.

As for the creeper move cards, please toss them out, because that's not how adults resolve differences. How gender-neutral is the word creep anyway, and how would you respond to being dismissed with a generic scrap of paper printed from the internet?

If you reduce communication to such a passive aggressive and childish statement, color me unsurprised when you receive an equally childish response, especially in a community that thrives on subversion and creative re-use of things they're not supposed to toy with. It's the exact same attitude that protects us from DRM, eagerly tests claims of privacy and security, and liberates closed technology for those without access. You cannot have one without the other.

Conferences are social gatherings, and sexuality is a normal part of that. I know several happy couples who met at a tech conference, coming from different cities or even countries. Are we to assume that none of them used this opportunity to hook up, and that relationships never happen without ambiguity and misunderstanding? It's not a binary choice between tweeting #ITappedThat and turning conferences into convents.

But why does it seem like there are so many socially maladjusted men roaming these conferences? Does anyone care about the reasons at all, like say, the high rate of autism-spectrum disorders among geeks? Could it be due to the emphasis schools and universities place on non-intellectual pursuits like sports and popularity, and the bullying that results from it? Because it seems to me what some socially awkward hackers have done is exactly what the social justice warriors want: they've created a safe space for themselves, where only their own rules apply.

I never hear much about the effect "Jock culture" has on men, but quite a lot about "Rape culture" and women. We stereotype geeky men as neckbearded basement dwellers whom women are to be protected from unilaterally, rather than working towards real resolution. I don't mind the word neckbeard personally, it can be a humorous badge of pride, but if it's offensive to anyone, surely that's men, not women?

Beating Which Odds?

In a post titled, Beating the Odds, the JSConf organizers explain how they got 25% of their speakers to be women. The choice quote is: "Our industry systematically biases against 50% of great speakers and misses out on a significant amount of talks, topics, discussion and thus progress." The argument is that, despite only 10% of proposals coming from women, an anonymized selection process disproportionately favored female speakers.

Under a more traditional selection process, these women's valuable and apparently superior contributions would have been ignored. Note how they ignore the ratio of men and women in the industry, and assume this would not affect the gender ratio of good candidates: 50% of them are assumed to be women. That's not how statistics work.

The results: "Our highest ranked talk is from a woman and we know we wouldn’t have gotten that talk without the outreach we did." And: "We invited 35 women to submit to the [Call For Proposals], of these 13 ended up submitting one or more proposals, 5 women submitted on their own."

So basically, there is a significant amount of pre-selection going on here. In their outreach to female candidates, organizers naturally prefer women who they already think will make good speakers. These candidates then further self-selected based on their own confidence and skill. Less than half of female speakers submitted on their own. Meanwhile, the 162 proposals from men came from the usual pool, requiring no unique outreach. Despite extolling the virtues of anonymized selection, the process was biased to favor talented women from the get go, and it's no surprise women sent in better proposals as a group.

Given the rates of commercial and open source tech participation for women, getting 25% female speakers is a high number, assuming fair random sampling, beating the odds. But it's not random at all. The cure for sexism is apparently... more special treatment for women?

It also bothers me on a personal level: I'm gay, and feel equally excluded when someone puts a picture of Natalie Portman in their JavaScript talk. But even if I wasn't, who's to assume my opinions on the matter would fall in line with the cliché? When people do diversity spot checks of speaker panels and rally the horde, I get counted as just another dude propagating patriarchy and heteronormativeness. What does it tell you when the first thought upon seeing a lone woman in a line-up is token female rather than trailblazer?

Now, I'm not against setting a good example, and I realize the perception of a boy's club can be a barrier to entry. However that shouldn't distract us from what equality of opportunity actually looks like. In tech, it's nowhere near a 50 / 50 gender split, because the imbalance starts much earlier, with more men than women going into STEM fields, despite the fact that 3 women now graduate for every 2 men.

Can we at least give women the benefit of the doubt and assume that they go after what interests them, rather than being unable to choose differently? Even in the most gender-equal country in the world, Norway, STEM fields are still male dominated and the social sector remains female dominated, despite decades of fervent pro-equality policy and education.

How solid and gender-neutral is the research that traces this all back to social pressure? The 2010 documentary Hjernevask (Brainwash) provided a very revealing answer to this question and others, causing a stir in the Norwegian academic community. I highly recommend watching it, there are English subtitles. I found the resemblance to creationism and intelligent design striking: supposed scientists were dismissing observations out of hand because of perceived implications, questioning the author's motives instead. But sexual dimorphism doesn't imply patriarchy, any more than evolution implies social darwinism.

Some choice facts from honest nature vs. nurture research: even day-old babies show a measurable difference in interest between boys and girls, when presented with both a mechanical toy and a human face. Genetically identical twins have similar IQs and depression rates and research with adopted children shows a similar relation to their biological parents, much more than their adoptive ones. This is no reason to treat individuals any different, but some averages differ innately across gender lines, and I don't see that as something we can or should fix by overcompensating.

Breaking Out of The Filter Bubble

Above all, there's a common thread I can't ignore. The women I admire and respect in tech did so primarily on their own merit, letting nobody speak for them but themselves. Like the men I look up to, they point people to their accomplishments, not their likeability. Their Twitter bios don't consist of one ism after another, showing their adherence to a pre-approved set of beliefs. They don't let random trolls derail them, and they don't find themselves at the center of fires of their own making, expecting others to put them out.

It's also the ideal I aim for. When a couple thousand people on YouTube told me I had no life, I laughed my ass off at the absurdity. I'd just created an accidental experiment in viral media, and learned tons in the process. Meanwhile they just watched a video they apparently didn't like, and then wasted more of their time to point this out. They weren't talking about me, they were talking about themselves.

When people told me I killed Unix, that I should be shot, and that I was just some idiot designer who didn't understand code, I didn't have the privilege to retweet the offense and let my posse roll in. I could only ignore it, taking the reputation hit, or refute the misconceptions with arguments and insight, changing people's minds one post at a time. The arrogant Unix greybeards who bugged me in private? Simple: you bait them into telling you everything they know, pan for gold amongst the mud, and move on. One person against the might of Twitter, HackerNews and Reddit: it's really not so bad, just don't take it too seriously. Once the novelty wears off, the bystander effect kicks in, unless you keep stoking the fires yourself.

Of course, I did let it inform my choices: I stopped working on that project in public, realizing I wasn't going to get much useful participation until much later, and I could do without the distraction. But it no longer bothers me, it's just one in a long line of useful experiments. The lingering frustration I feel is about people's short sightedness, not bruised ego. Ever since then, I treat the internet like I would a lovable-but-backwards grandparent, who makes racist comments over Christmas dinner. Yes Grandma, it's all the damn commie jews and faggots' fault, now, who wants dessert?

No, I don't feel bad for dropping in those sacrilegious words in there just now. I like to think you are mature enough to let those letters pass under your eyes, without burning me at the stake because it reminds you of something unpleasant. I trust you to focus on the couple thousand words I started with, rather than just two at the end. See, the reason people say the n-word instead of nigger when talking about racism, is that they don't yet realize they too would have owned slaves back then.

When the internet gets its panties in a bunch for the umpteenth time, it's worth asking: where are people getting their information from? The plural of anecdote is not data, after all. Every incident I've heard of lately was massively blown out of proportion. Kony 2012 anyone? Look, finally a cause we can all be equally offended by.

Women are adamant about not being pigeonholed by their gender. I see no reason why we should encourage and celebrate doing it to men. Whether male or female, or any of the shades in between and around, people can have wildly different points of view, and reducing everything to a gender battle is as myopic as pretending no issues exist at all.

The most reasonable people are now afraid to speak their mind. They rightly fear being shamed and harassed by those who scream the loudest of abuse. I've debated writing about this for a while, because I know what a certain part of the response will be. But I'm not the only one saying it, so I'm doing it here, once, in full length, with honest citations, after discussion with people of experience. Women and men, in case you're wondering. "Good luck" was a common theme.

Remember, I'm not the one trying to make hay out of gender issues, turning them into ad revenue, TV appearances or book sales. In my line of work, we're expected to fix things, not just tell people they're broken in increasingly hyperbolic words.

Don't man the cannons or summon the horde. Instead, go check out the ton of links I just dropped into your lap, listen to what's already been said, and see if you can't hear the sound of a record skipping somewhere in the distance. It's not the one you think it is.

For the future then, something to think about. If I step outside, I can walk a couple blocks in any direction to encounter these.

I've taken the liberty of making them more honest:

Dead Rocks

Audi

This is what we allow advertisers to paste onto our streets, our newspapers, our TV shows. Our brains. And then the media turns around to tell us how everyone's being selfish and insecure, but sexism is to blame.

As a smarter person put it, it's narcissism repackaged as a gender battle.

Don't say it doesn't affect you, not when a picture of dollar bills makes you more reluctant to help someone pick up pencils.

↧