This is a writeup for a concept I had been working on – an image processing application. It did not come to pass but I’d like to share some of the ideas behind it (previous posts touches upon some ideas too). It might be of interest to those of you working on developing full-scale applications integrated in web browsers.
The early days
An early sketchup for the application took inspiration from audio-processing software. In particular SynC Modular (later known as Reaktor). It had several levels of building instruments and effects for audio processing. At the high level, non-expert users could wire up high-level modules that had well-defined input/output sockets. In addition, UI for parameters could be designed that allowed users to configure the internals of modules. At the low level, expert users could program up logic for signal manipulation using predefined low-level primitives. The expert users could then package up the internals in a high-level module that exposed inputs/outputs for wiring.
Putting the ideas into action
Here is a small picture showing the general layout of the application. It shows the result of mandelbrot fractal applied to an image. The module generating the fractal has several input parameters and an input image to apply it to, then an output image to store the resulting image to.
When editing the Fractal node we can see the actual Scala code that generates the image. In this code, the input image is transformed by mapping each pixel over a lambda function which yields the transformed pixel. Behind the scenes, a new image is produced, then stored as the output image.
Pixels in images are not manipulated directly with xy-coordinates. This allows the application to subdivide the image into segments that can be transformed concurrently. In addition, an initial pass on a miniaturized image is performed, allowing the user to view a quick preview. The final image is then progressively refined in the last pass.