Jan 19, 2020       tags: | art | visualisation | js |

Real time abstract art generation using a neural net (CPPN)

I came across https://www.deeplearning.ai/ai-notes/optimization/ some time ago and was very intrigued by the banner visualisation at the top of the page. I initially thought it was some clever CSS trick, but poking around revealed it was the result of a “compositional pattern-producing network” (CPPN) running in real time. I started looking for more information and found a few very interesting resources on the subject (see References at the bottom). One of the most helpful ones was https://tehnokv.com/posts/visualizing-audio-with-cppns/. That’s where I learned it was just a simple feed forward neural net that took pixel positions as inputs and produced RGB values for each pixel using random weights. With some trigonometry thrown in as activation functions, captivating patterns could be produced. Of course, to make the patterns dynamic, there also needs to be a temporal input.

It wasn’t difficult to reproduce the method in Python with Keras/TF. However, it wasn’t very spectacular in a jupyter notebook, so I decided to make my own implementation in JavaScript. Experiments with vanilla js showed it was achievable but CPU-costly. Fortunately, thanks to Tensorflow.js, I had the option of getting GPU acceleration through WebGL. Even without a GPU, it should be more efficient than vanilla js thanks to incorporation of WebAssembly code. To add interactivity, I used mouse/touch coordinates as extra inputs to the neural net (scroll down for the architecture diagram).


The architecture of this neural net is very simple (diagram generated by http://alexlenail.me/NN-SVG/index.html):


The sliders control the magnitude of all activations in a layer. You can select various combinations of activation functions. For example, lin-lin-cos-sin, lin-sin-lin-sin make some interesting patterns given the right weights/magnitudes. Try moving the mouse/finger on the canvas to change patterns quickly for extra “wow”.

Click "Generate wallpaper" or press "spacebar" to generate a 1920x1080 wallpaper (may take a while!). The image will appear below. It will look like a thumbnail, but it's actually full-res and can be saved. If you would like to make patters with a different set of random weights, click "New weights".

Of course, actual CPPNs are used for much more than generation of random abstract patterns. I recommend the resources linked below for additional information.



Suggested post:

Popularity of car colours in the Greater Toronto Area
According to a survey conducted in 2012 by PPG Industries, white (21%) and black (19%) were the two most popular colours in North America followed closely by silver and grey (16% each). Red and blue accounted for 10 and 8% respectively. I decided to test this data by taking photographs of an intersection in Mississauga, Ontario and analyzing them with the help of YOLOv3 as well as OpenCV and scikit-learn libraries.