Image Processing

Image processing transforms an image by examining individual pixels or the values around them. A blur averages nearby colors; an edge detector measures differences; a threshold separates values above and below a chosen level.

In Utomata, an image is just another field of values. Once an image is stored in a field, we can process its pixels with ordinary arithmetic and neighborhood lookups.

In this guide, we will capture a rotating 3D scene, apply a few filters, and introduce feedback so that the processed image retains traces of its past. The Image Filters example uses the same operations to display several results side by side.

Setup

We begin with a small rotating mesh and a viewport looking at it:

~mesh {
   i_geo = (-0.35, -0.35, 0);
   i_res = (4, 16, -0.5);
   i_scl = (0.15);
   rot = (mlt(step, 0.004), mlt(step, 0.01), 0);
}

^vp {
   look = (0, 0, 0);
   pos = (0, 0, 0.5);
   light = (1, 0.5, 0);
   fill = (0.1);
   bg = (0);
}

The mesh rotates as step increases. ^vp specifies the camera position, lighting, and background for the image we will process.

Add two fields and a formation to display the result:

&dim = (256, 256);

#img {
   dim = &dim;
   run = ^vp;
}

#filtered {
   dim = &dim;
   run = #img;
}

~filtered {
   col = #filtered;
   pos = (-1, 1, 0);
   scl = (0.75);
}

run = ^vp; reads the rendered view into #img. The field's dimensions determine the image resolution: here, 256 × 256 pixels. Each cell stores red, green, and blue values in its x, y, and z components.

For now, #filtered simply copies the source image. Both fields share &dim, so each output cell corresponds to one source pixel. The formation displays the result beside the mesh, away from the close view captured by ^vp.

Working with color

Let's replace the run expression in #filtered with a threshold:

run = lrg(#img, 0.5);

lrg returns 1 where a value is greater than 0.5, and 0 otherwise. The comparison applies independently to all three components. Each color channel becomes either fully on or fully off, producing a small set of saturated colors, along with black and white.

Another way to simplify the image is to reduce the number of available color levels:

run = div(flr(mlt(#img, 4)), 4);

Multiplying by 4, rounding down with flr, and dividing by 4 reduces each channel to multiples of 0.25. Smooth shading becomes a series of distinct bands. This effect is called posterization. Try replacing both occurrences of 4 with a larger number to preserve more detail.

We can also lower the output resolution for a pixelated appearance and fewer cells to process. Change the dim property of #filtered to:

dim = (32, 32);

The smaller field still samples across the full source image, so the scene and display formation need no changes. #img continues to capture at its original resolution. Restore dim = &dim; in #filtered before continuing.

Neighborhood filters

The color operations above read one source pixel at a time. A blur also reads nearby pixels and averages their colors:

run = #img.AVG((-1, -1), (1, 1));

AVG takes two vectors that define the corners of a rectangular sampling region. Here, offsets from -1 to 1 on both axes select a 3 × 3 region centered on the corresponding cell in #img. The result is the average of all nine values, including the center cell, calculated separately for each color channel. This softens differences between adjacent pixels, producing a simple box blur.

To control the blur size, add these macros outside the field declarations and use &blur in #filtered:

&len = 6;
&blur = #img.AVG((-&len, -&len), (&len, &len));

#filtered {
   dim = &dim;
   run = &blur;
}

Try smaller or larger values, or use different extents on each axis for a directional blur. Here, &len = 6 extends six cells in each direction, giving a 13 × 13 region. That means averaging 169 samples for every output cell, so larger regions require more work.

Finding edges

An edge is a place where nearby values differ. We can highlight these differences by subtracting a small neighborhood average from the original pixel:

&avg1 = #img.AVG((-1, -1), (1, 1));
&edges = mlt(4, abs(sub(#img, &avg1)));

run = &edges;

In uniform areas, the pixel and its neighborhood average are similar, so subtraction produces a value close to zero. Around boundaries, the difference is larger. abs makes differences in either direction positive, and multiplying by 4 makes them more visible.

We can also compare individual neighbors to emphasize a particular direction:

&emboss = add(sub(#img(-1, -1), #img(1, 1)), 0.5);

run = &emboss;

The two lookups read pixels on opposite sides of the current position. Adding 0.5 places unchanged areas at mid-gray, while differences produce lighter and darker relief.

The weights applied to neighboring pixels can be arranged in a small matrix called a convolution kernel. The Image Filters example includes a Sobel filter that uses weighted sums to measure changes along both axes. Its longer expression is built from the same relative lookups, multiplication, and addition used here.

Adding feedback

So far, each filter has read only #img. We can also read the previous state of #filtered itself.

Add two more macros:

&glow = add(mlt(#img, 0.5), mlt(&blur, 0.5));

&feedback = add(
   mlt(#, 0.01),
   mlt(#.AVG((-1, -1), (1, 1)), 0.9)
);

Then replace the output rule:

run = add(&glow, mlt(&feedback, 0.5));

&glow mixes the source image with its blur. In &feedback, the unnamed # refers to the field evaluating the expression: here, #filtered. It reads that field's previous values, while #.AVG reads their neighborhood average.

Each step therefore adds a softened trace of the previous output to the incoming image. As the mesh rotates, earlier shapes linger and spread. Try reducing the feedback multiplier from 0.5 toward 0 to shorten the traces, or increasing it slightly to retain more of the past.

Try combining other filters with feedback, or reading from several image fields. Each output can respond both to incoming imagery and to its own previous state.

Real-time diagnostics: histogram

Fields can also help us inspect an image as it changes. A histogram groups pixel values into ranges, called bins, and measures how often values fall into each one:

&bins = 32;

#histo {
   dim = (&bins, 1);
   set = (0);
   val_dom = REAL;
   coord_dom = SIGNED;
   run = #filtered.AVG[(-1, -1), (1, 1)] {
      eql(
         clp(flr(mlt(add(V, 1), div(&bins, 2))), 0, sub(&bins, 1)),
         clp(flr(mlt(add(C.x, 1), div(&bins, 2))), 0, sub(&bins, 1))
      )
   };
}

Each of the 32 cells in #histo represents a bin spanning part of the signed value range, from -1 to 1. For each bin, the reducer scans #filtered and returns 1 for channel values that belong to it, or 0 otherwise. AVG turns these matches into a fraction of the image, giving separate distributions for red, green, and blue.

Because the calculation runs every step, the histogram updates alongside the filtered image. Displaying these values as bars makes it easier to see how thresholding concentrates colors into a few levels, or how feedback pushes values toward the upper end of the range.