Héctor Rodríguez
Deep Archetypes: Meditation on Violence, (2021)
This video uses deep learning archetypal analysis methods to analyze and visualize the rhythmic flow of Maya Deren’s 1948 film Meditation on Violence, made in collaboration with martial artist Chao-Li Chi (Ji Chaoli).
Archetypal analysis belongs to the family of representation learning methods. It encodes every image in a dataset as a convex combination of a set of basic or archetypal images. The archetypes themselves are constrained to be convex combinations of images in the dataset.
In this visualization, the archetypes discovered in Deren's movie are arranged as thumbnails in the foreground. As the original movie unfolds in the background, those archetypes that play a stronger role in generating the current frame become sharper. This effect is accomplished by making the alpha value of each archetypal image a function of its activation coefficient.
The activation of each archetype is also represented by the height of a curve. The foreground curve represents the activation responses of the archetypes to the current frame. As time goes on, the current curve gradually recedes into the background, visualizing the temporal flow of the movie as a memory machine.
Archetypal analysis was chosen because it is readily interpretable. The algorithm identifies images that are part of the dataset and highly representative of it. One can think of these archetypes as a visual lexicon extracted from this particular movie. The decomposition of each movie image is then rendered as a combination of this lexicon. This result produces an immediately intelligible visualization of how the algorithm “interprets” the dataset.
Credits
This project uses the Archetypal Analysis Network (AAnet), an autoencoder proposed in:
D. van Djik, D. Burkhardt, M. Amodio, A. Tong, G. Wolf and S. Krishnaswamy. Finding Archetypal Spaces Using Neural Networks. In: arxiv.org/pdf/1901.09078.pdf
A linear method, the Simplex Volume Maximization, is then applied to the images in archetypal space to select the final set of archetypes:
C. Thurau, K. Kersting, and C. Bauckhage. Yes We Can - Simplex Volume Maximization for Descriptive Web-Scale Matrix Factorization. In Proc. Int. Conf. on Information and Knowledge Management. ACM. 2010. Code in: github.com/cthurau/pymf
The machine learning element of the work was programmed in python using tensorflow. The visualization was made with Processing.
The artistic and technical director of the project was Héctor Rodríguez. The project was coded by Sam Chan with Héctor Rodríguez.
