When I first started playing around with filters, there was a type of effect that interested me a lot, but I could not figure out how to reproduce it. It was the one that transformed you into several small elements, like shapes. For example, there is this one who looks like you were made of legos or this one who puts you in an excel spreadsheet. However, the one I was most curious about was Craig Lewis’s ASCII filter. While probably not doing the same technique as him, this is the one we’re going to tackle today.

At the time, I knew it involved doing UV transformations, but neither I knew what exactly was a “UV” or how to do the transformations. This article intends to be the one I wish I had when I was starting going deeper in Spark AR. It will show how to make ASCII shader in 2 ways: non-procedurally, with an external texture, and procedurally, with SparkSL. Before implementing both ways, we’ll take a look at the theory behind them. You will then be able to make plenty of cool effects by modifying it only a little. You can follow along by downloading the files for free on my Github 🤖 !

Behind the matrix

A key part of the effects shown above is pixelization and it is also the first step of both of our ASCII shaders. This technique reduces the resolution of an image or a section of it.

It is achieved by downsampling the pixels, taking only one out of two pixels for example. In practice, we can achieve this by manipulating UV. We have to multiply each coordinate by a certain X value, round it up, and then divide by X. I take this opportunity to present a tool that I intend to use in the next tutorials, the GLSL editor! It allows you to create and display shaders with GLSL (OpenGL Shading Language) directly on a web page. If you are familiar with programming, there are some things you should recognize, but you don’t have to understand everything to be able to use it. You only have to uncomment certain lines to have a visual of the operations explained. If you feel like it, don’t hesitate to experiment with different operations to deepen your understanding.

Note that there are already two patches in the AR library that do this. However, I have always had difficulty understanding how shaders work when looking at them in a nodal editor. I find it quickly becomes intimidating, even for simple effects. So I decided to do it again by myself to be sure I understood correctly and so that they were better suited to the project.

Taking the blue pill

The process to achieve the non-procedural effect looks like this:

First, we have to separate our pixelated image into multiple binary masks according to the pixel’s gray level. Then, we must create character patterns according to the pixelization level, the number of cells, and the characters we want. In Figure 1, I used a single texture that already has all the characters I needed. It is good practice to operate this way performance-wise, but it would be possible to do it with different textures for each character too. Then, we have to mix the binary masks with the corresponding character patterns and finally, mix each of the masks together. You can use the final mask the way you want! In the example I did, I used it with colors that change over time. Also, I separated the image into 5 gray levels, but doing it with any level is possible.

Implementation

1. Gray layers

To divide the image according to the gray level of each pixel, we can use the step function. If n is the gray level, and we want our function to return 1 if it’s between the interval [a, b], we can use :

2. Character patterns

When creating the character patterns, the number of cells that result from pixelization is needed. We will tile the characters that many times. If a sprite sheet is used, we must also be able to select only a part of it. Thus, the size of the icons and their position in the sheet is needed. The operations to be performed on UVs are as follows.

3. Result

It only remains to mix the patterns and the binary masks. You can use the final mask with the colors you want to achieve different effects. I went with rainbowy color and took advantage of the UI slider to adjust the pixelization level.

Swallowing the red pills

The procedural method, strongly inspired by this shader, is a bit more complex. In this case, there is no external texture so we are starting from scratch. Operating this way allows us to create a very light filter – 5 Kb instead of approximately 500 Kb. It also gives us a different type of flexibility in the characters used. However, this technique only works for an ASCII-type effect, while the previous one could be used with a wide variety of textures. The process is as follows :

First, as with the character patterns shows above, every cell needs to have its “own” UV. We need to remap and discretized them according to the number of bits we want our character to have. In Figure 2, we have 3×3 bits per character. Each pixel would then have an index and this index will be used with the character patterns in the getValue function to determine the pixel’s value :

Once done, each of the pixels of the image has a value of 1 and 0 and we have our final mask!

Implementation

Before I start, I want to point out that the figures in this article use 3-bit characters for clarity. Obviously, this does not allow much latitude in design choices. An 8-bit design is used in my effect instead.

UV Manipulation

The purpose of this step is to create “grids” in each cell of the same size as the symbols. Inside, each sub-cell corresponds to a pixel of the character. Two approaches are possible to achieve this: one borderless and one with border. The first method allows creating continuous patterns, while the second allows obtaining effects similar to the non-procedural technique. The details of each method are presented in the editor that follows. I advise you to experiment with both of them to get a good feel for how they work.

getValue()

To determine the value of a pixel, we need to get his index in the grid cell we made in the UV Manipulation step and take the value of the desired character pattern at this index. As shown in Figure 3, the method used requires two binary operations: a right shift (>>) and a bitwise AND (&). Unfortunately, SparkSL does not support these operations, but there are methods to emulate them :

Characters patterns

To create your own characters, you can use the tool below. It will give you directly the integer value to use in your .sca file.

Notice here that it gives four integers instead of one. Following my tests, I found that ~24 bits symbol is the maximum before getting numerical instabilities. This precision limit doesn’t give much flexibility in creating patterns, not even a 5×5 grid. To address this problem, I decided to use an approach that uses 4×16 bits instead. Making slight changes in the index management is needed, but it allows to have larger characters. Using this technique allows you to create patterns of any size without worrying about the precision limit of SparkSL!

Result

To demonstrate the result, I decided to make a maze effect. This is achieved by using a different part of a cross for each character pattern. In this case, I used the full-size approach in the UV manipulation to get a continuous design :

Conclusion

Here it is! I hope you enjoyed this tutorial and will use it to create awesome effects. Make sure you follow me on Instagram to be informed of my next articles. 🙂