With 2022 in sight, it is high time to finish those write-ups about some of the projects we worked on this year. This past spring, we developed a custom NAP application that animates, controls and pre-visualizes about 9000 LEDs embedded in a massive ring at 1000Hz for the special performance Within Without lead by Nick Verstand in Royal Theater Carré. The performance was such a success, that the ring continued to appear at the VMA's and Awakenings ADE. Here's a peek behind the scenes.
Let's start with some technical details. The ring has a diameter of 10 meters and consists of 48 HD107S led strips (+/- 180 leds each, so roughly 8700 in total), embedded on the in- and outside, controlled by 12 custom-made microcontrollers designed by JDT Engineering. The led controllers receive UDP packets which are pushed to the SPI ports of the led strips. Our application produces several visual effects for the ring that can be controlled from a remote machine by means of OSC commands at an arbitrary frequency, and are updated and sent as UDP packets to the microcontrollers at 1000Hz. It also renders a real-time pre-visualization of the ring, making the application a useful show-programming tool when the physical light ring is not connected. The software runs on a compact <€300,- Intel NUC (barebones, 8th Gen i5) running Ubuntu 20.04 and a low-latency kernel to ensure high-precision timing guarantees to minimize jitter.
We choose to build all of our applications with NAP, as it provides a framework to quickly build custom software solutions that offer the flexibility of versatile instruments. A notable example of how we leverage NAP this way is in Within-Without's effect- and parameter mapping system.
The GIF below shows the application window when there are two software-controllers (which can be viewed as 'rings') configured; one for the inner and outer led-ring, each of which manages their own CPU-thread on which effects are computed. In the case of Within-Without, each software-controller addresses 6 of the 12 physical microcontrollers that make up a single ring and automatically generates UDP packets that are subsequently pushed to their appropriate destinations at the desired frequency. The dropdown window on the left shows an automated control-interface based on the effects that are bound to a particular ring.
Effects must be bound to a controller for them to affect the appearance of the ring. This is possible by configuring the application data in our editor napkin. Here, I navigate to the outer controller under Resources and add a Glitch effect (nap::LedEffectGlitch) to its Effects property in the Inspector.
After pressing ctrl s to save the new configuration, the application will automatically hot-reload, and show the newly added glitch effect in the control window. All effect parameter presets can be saved to and loaded from disk at any time.
The application also automatically generates OSC addresses which can be used to control specific effect parameters from another application running on a remote machine like Max for Live.
Incoming OSC messages are received on a dedicated thread, which subsequently buffers handling events for execution on the main thread. This is also responsible for rendering the pre-visualization and therefore not in sync with the rings' effect computation threads. As the update rate of the OSC messages and rendering is expected to be much lower than the high update frequency of the ring controllers, all effect parameter values are smoothed on the controller threads. This way, the effect animations on the led rings will always appear smooth, no matter the update frequency of the incoming OSC control messages.
All effects in napkin are computed in order and perform a modification on the software-controller's associated led buffer. This means that users can build their own DSP-chain of effects in their data. For instance, clever use of a Multiply effect at the bottom of the effect pipeline can create a low-pass filter for the composition of smooth effect trails. For the purpose of added flexibility, it was also decided that multiplications should not affect the full led buffer. Therefore, each software-controller manages additional led buffers for the most important effects to which a separate multiplication can be performed before the result is blended into the main buffer. An example of this process is illustrated in the effect pipeline below.
Taking into consideration the potential computation workload of an arbitrary number of effects (though within reasonable bounds) applied to 9k-element led buffers with a budget of 1 millisecond (1000Hz) per controller thread, it was important to implement the effects with high performance in mind. Moreover, we had to be wary of thermal conditions for when the app would be running over extended periods of time. As the projected effects could easily be parallelized, it made a lot of sense to compute them on the GPU. However, compute shaders and shader storage buffers were not supported by NAP at the time. Therefore, all effect computations had to be performed on the CPU, and the number of evaluations of the led buffer's pixels had to be minimized.
Our approach to shaping the ring's animations leverages the effect-chain system's capability of producing feedback loops. This way, for 'tails' that travel along the circle, we are granted the advantage of using the previous frame's led buffer without having to reiterate over previously drawn pixels. Internally, we only have to keep track of a tail's location on the ring and its velocity. This information is used to determine which subset of pixels must be evaluated for the current frame; typically only a small fraction of its perceivable length. The length of tails can therefore be controlled by applying a subsequent effect that performs an arithmetic operation on the buffer e.g. Multiply.
Arithmetic operations and buffer blending (which are fundamentally applied over the entire buffer) have also been optimized to utilize SIMD lanes, minimizing their computational overhead. The result looks excellent and easily maintains stable frametimes on the target machine.
That concludes this short overview of our contribution to Within-Without. The physical installation is absolutely a sight to behold so we definitely recommend checking it out if you get the chance!
NAP Framework is open source software. Suggestions and contributions are welcome.
Lesley van Hoek | Software Engineer Naivi
