What does typography actually sound like?

You can try out HALMA’s interactive start screen right here in the illustration!

Curtain up: Big cinema on the screen

The HALMA start screen is based on an HTML canvas element, on which the animation is projected. To achieve the desired effect, we decided to use the physics engine Matter.js.

Physics Engine Example

In the first step, the text’s underline needs to be animated. For this, individual spheres were created with Matter.js, held in place by several “springs.” A function ensures their correct positioning along the descenders of the text – where an underline will later appear – by automatically placing these bodies at equal intervals between two coordinates.

May the force be with you – and your mouse pointer

With a kind of “force field” surrounding the mouse pointer’s position, the bodies can now be set into oscillation. Since they are connected, they begin to oscillate, creating the desired “wobble” effect. To prevent chaotic behavior, we limit the maximum distance from the origin point with a simple vector calculation.

Tuning, or: “Fiddling around”

By combining various parameters such as mass, gravity, tensile force, and damping, we can specifically influence and fine-tune the simulation.

Here you can see the first setup of the individual vector points.

On the screen: Now rendering

The animations calculated in this way are not yet visible; they only exist virtually in the form of coordinates and vectors. For the whole thing to concretely manifest on the screen as a nice animation, the next step is to render concrete pixels from it. For this, we don’t use the renderer integrated into Matter.js but rather the P5.js library, as it allows drawing splines.

With the coordinates of the individual bodies, a line can be output on the canvas element, which, without further calculations, would initially be very angular. Initial attempts to smooth the edges with Bézier curves showed that Bézier curves are better suited for fewer anchor points. They require the position of the three subsequent coordinates, which makes the calculations very complex and time-consuming.

To generate a curved line, we therefore used the formula of the cubic Hermite spline instead, which is easier to apply to the individual vector points of the line. Since this has to be recalculated with each frame and should run on every device with a framerate of at least 30 FPS, it is very important that this function remains as performant as possible. We let the physics engine run in its own animation loop, thus being independent of the framerate and able to avoid dubious simulation behavior.

The content dynamically generated by Javascript now needs to be scaled according to screen size and resolution, correctly placed on the canvas, and recalculated every time the browser window is resized.

Finished application in the frontend

No sound is good etiquette

Finally, as soon as the mouse’s force field collides with one of the bodies, “all that’s left” is to play a sound. Since conventional <audio> elements buffer before playing the sound, a different solution was needed. To make this work cross-browser, all sounds are played silently once when the page loads, effectively achieving a preload. This essentially front-loads the loading and buffering of the sound files.

Now, it’s well-known good practice not to play sound without the user’s explicit request. In some browsers and on almost all mobile devices, this is also enforced, and sound can only be played after a user interaction. So, we’ll add a button that allows you to toggle sound output on and off.

An annoying bug: If you want “do not disturb,” you have to feel it

The start screen was finally finished, and we began the testing phase. It quickly became clear that the visual implementation worked as desired on almost all devices. However, we received reports from individual testers who couldn’t hear any sound on their iOS devices. What a setback, as sound was a fundamental part of the whole idea! Weeks of puzzling and debugging followed: the problem couldn’t be reproduced or narrowed down, it seemed to occur completely randomly. The only commonality was its appearance on iOS devices, but neither the iOS nor Safari version, nor the device itself, seemed to have any influence.

Numerous frustrated lines of code were written. In search of answers or at least small clues as to how the bug could be reproduced, documentation was scoured and internet forums were searched. All HALMAs and eventually even their family and friends were recruited as testers, and extensive test series were carried out. All without success. We almost scrapped the whole idea. But then, rather by chance, we were able to pinpoint the error: True to the motto “It’s not a bug, it’s a feature”, the playback of dynamic sounds on iOS does not run through the media channel like on other devices, but through the ringtone channel! This means that no sound can be played as long as the phone is muted or “do not disturb” is active. Therefore, for all iPhone and iPad users, the rule is now: Turn off “do not disturb” and turn on sound!

The finished start screen on an iPad with a corresponding note to disable silent mode in the device settings