Jan 19, 2020 tags: | art | visualisation | js |
Real time abstract art generation using a neural net (CPPN)
I came across https://www.deeplearning.ai/ai-notes/optimization/ some time ago and was very intrigued by the banner visualisation at the top of the page. I initially thought it was some clever CSS trick, but poking around revealed it was the result of a “compositional pattern-producing network” (CPPN) running in real time. I started looking for more information and found a few very interesting resources on the subject (see References at the bottom). One of the most helpful ones was https://tehnokv.com/posts/visualizing-audio-with-cppns/. That’s where I learned it was just a simple feed forward neural net that took pixel positions as inputs and produced RGB values for each pixel using random weights. With some trigonometry thrown in as activation functions, captivating patterns could be produced. Of course, to make the patterns dynamic, there also needs to be a temporal input.
It wasn’t difficult to reproduce the method in Python with Keras/TF. However, it wasn’t very spectacular in a jupyter notebook, so I decided to make my own implementation in JavaScript. Experiments with vanilla js showed it was achievable but CPU-costly. Fortunately, thanks to Tensorflow.js, I had the option of getting GPU acceleration through WebGL. Even without a GPU, it should be more efficient than vanilla js thanks to incorporation of WebAssembly code. To add interactivity, I used mouse/touch coordinates as extra inputs to the neural net (scroll down for the architecture diagram).
The architecture of this neural net is very simple (diagram generated by http://alexlenail.me/NN-SVG/index.html):
The sliders control the magnitude of all activations in a layer. You can select various combinations of activation functions. For example, lin-lin-cos-sin, lin-sin-lin-sin make some interesting patterns given the right weights/magnitudes. Try moving the mouse/finger on the canvas to change patterns quickly for extra “wow”.
Click "Generate wallpaper" or press "spacebar" to generate a 1920x1080 wallpaper (may take a while!). The image will appear below. It will look like a thumbnail, but it's actually full-res and can be saved. If you would like to make patters with a different set of random weights, click "New weights".
Of course, actual CPPNs are used for much more than generation of random abstract patterns. I recommend the resources linked below for additional information.
References
- Using CPPNs to generate abstract visualizations from audio data (tehnokv.com)
- Generative Art with Compositional Pattern Producing Networks and GANs by Kevin Jiang (personal blog)
- CPPN mini-project by Troels A. Rasmussen (medium.com)
- Neurogram by Studio Otoro (otoro.net)
- Generating Large Images from Latent Vectors by Studio Otoro (otoro.net)
- cppn-tensorflow repository on github by hardmaru (Studio Otoro)
- An implementation of realtime CCPPNs using WebGL by pngupngu (pngupngu.com)
- Understanding Compositional Pattern Producing Networks (Part One) by Cameron Wolfe (towardsdatascience.com)
- Making deep neural networks paint to understand how they work by Paras Chopra (towardsdatascience.com)
Source
Suggested post:
Popularity of car colours in the Greater Toronto Area
