source code for https://derivative.works
Derivative-Works is an experiment in using machine learning to create image collages.
The algorithm cuts out shapes from images and rearranges them to create a face.
All of the reference images are in the public domain, created in Artbreeder using BigGAN and StyleGAN.
- A patch generator (DCGAN) trained on Perlin noise was taken from a previous project. It creates a high diversity of shapes and is fully-differentiable.
- There are a fixed number of patches that each has a corresponding latent vector and transformation matrices. These transformations control where in the reference image the patch is cut from and where in the canvas it is placed.
- These variables are then optimized (using Adam) to do feature inversion over a face classifier (DLIB’s CNN model).
The primary difference between this method and vanilla inversion is the input medium: instead of optimizing pixels directly, we optimize parameters. This simple technique lead to a variety of textures and compositions and the videos show the actual optimizations.
The website listed above. Uses svelte + typescript + sapper + tailwind
See make_collage.ipynb to make your own
Exporting the data for the site is a little buggy 😇
- @jacobgil for his dlib port + feature inversion repo which made things work :)
- @reiinakano for Neural Painters, which was helpful during our research