Gallery inside out. Videomapping as Performative Mediation of the Vasulkas’ Videoart Archive

Jana Horáková (Brno)

Introduction

A large-scale interactive projection on the facade of the Brno House of Arts took place on 20 December 2021, on the second anniversary of the death of Woody Vasulka (1937–2019). It was part of the exhibition triptych Vasulka Live Archive / Interfaces (2021–2022). The exhibition has a dispersed character. Its components are situated in public space, virtual reality, and online. The goal of the exhibition project is to promote the work of the Vasulkas collected in the archive managed by the Vasulka Kitchen Brno; the project concept is also based on the idea of building on the creative poetics of this innovative artistic duo. Because they were known for experiments with the implementation of up-to-date technology in the creative process, we emphasize the properties of the technology used in our applied research project, which is artificial intelligence, particularly machine learning carried out by artificial neural networks.[1]

Vasulka Live Archive / Interfaces is not a traditional exhibition project because there are neither new images nor remakes of the Vasulkas’ work exhibited. Instead, the exhibits are new types of interfaces to the contents of the Vasulkas’ archive, which belong to a family of new cultural forms that provide access to and manipulation of data and information. Their techniques include hypermedia, databases, search engines, data mining, image processing, visualization, and simulation. [Manovich 2003: 23] The exhibition triptych is one of the main outputs of a three-year research and application project focusing on the iconographic analysis of the Vasulkas’ audiovisual works using artificial intelligence, particularly machine learning, and on the mediation of archive content using innovative, interactive formats.[2]

Vasulka Live Archive

The focus of the Vasulka Live Archive (VLA) project was on the development and application of intelligent software used for image and sound recognition processing of the archive’s content with a focus on audio and visual objects. The archive we had at our disposal contains audiovisual, image, and text files of various types, formats, and quality with a volume of 536 GB. In our project, we focused only on the videos, which consisted of 1252 items with a volume of 137 GB and a total length of 6 days, 20 hours and 27:30 minutes. The Vasulka Live Archive website traces 105 works by the Vasulkas, represented by 124 videos.

In terms of technology, the project’s main outputs were two software tools and a specially designed website. The software tools were developed using machine learning for the iconographic analysis of the Vasulkas’ works, focusing on the visual and audio components, respectively. The third piece of software is the interactive web interface Vasulka Live Archive [VLA 2021]. It is equipped with specially designed functionalities that make it a unique epistemological tool for the iconographic analysis of the Vasulkas’ works. Machine learning is a method of training intelligent software capable of analyzing a dataset, in our case, a set of the Vasulkas’ videos, frame by frame, with pinpoint accuracy, based on predefined recurring visual and acoustic motifs. This process is non-linear and transmedia. The outputs take the form of statistical data relating to the frequency and probability of occurrence of these motifs across the Vasulkas’ work. Therefore, the Vasulka Live Archive project should be understood in the context of an emerging discipline, the iconography of the age of artificial intelligence [Spratt 2017: 12], and as part of a broader trend in the use of digital tools in the humanities (digital humanities).

Vasulka Live Archive / videomapping

A non-linear approach based on the search for similar motifs across the Vasulkas’ work was also used by creative director Jiří Mucha and his team in the creation of audiovisual work for the video mapping production.[3] The selection of the videos combined theoretical knowledge of the Vasulkas’ work and automatic classification of similar motifs performed by artificial neural networks. The shortlisted videos included works by the Vasulkas that are considered fundamental or typical of their work (such as Art of Memory, Artifacts, and Violin Power). In searching for certain visual and audio motifs, the artificial neural networks specially trained to recognize selected motifs across the Vasulkas’ videos were used. In this way, the motifs such as Woody, Steina, machine vision, interior, camera, special effect of Rutt/Etra synthesizer, electronic music or singing, were detected[4] . However, the visual and audio motifs were not only decomposed and put together in a new order, but they were remixed and animated too. For example, from the spinning circle in Grazing, Mucha created columns by stacking them on top of each other.

Fig. 1 Grazing: From the spinning circle in Grazing (Woody Vasulka, 1976),
Mucha created columns by stacking them on top of each other.

Another example, one of the Vasulkas' favorite motifs, the orb, was taken from Vocabulary, removed from the video frame, and animated to circle the entire front wall of the gallery.

Fig. 2 Vocabulary: The orb, was taken from Vocabulary (Woody Vasulka, 1973), removed from the video frame, and animated to circle the entire front wall of the gallery

The sound from all the videos was provided to the creators of the mapping as a separate audio track for further processing as well.

From 124 unique video works in the Vasulkas’ archive, Mucha selected 18[5], from which only fragments were used in the final work.

List of videoworks used for mapping:

These video works were subjected to sampling, remixing, and various forms of manipulation (using Adobe After Effect and Cinema 4D software tools). In addition, some of them were processed by Pavel Sikora with the help of the Deep Dream software [Mordvintsev u.a. 2015] and Lucid Library so that the viewers of the video mapping could also observe the pseudo-cognitive processes of artificial neural networks work in the analysis of these videos. The result is a new audiovisual work referring to the remix aesthetics of music videos, based on the associative ordering and layering of audio and video motifs, and on dynamic editing that rhythms the viewer’s experience in a way which is reminiscent of experimental avant-garde film.

The projection lasted about 10 minutes and contained several parts in which clusters of motifs were taken from the Vasulkas’ videos and reconnected in new ways. In the beginning, the façade of the Brno House of Arts was transformed into a projection screen by basic symbols, numbers and letters (1-2-3-4, Theatre of Hybrid Automata), and then was replaced by motifs of a midi violin used as an interface and a score (Violin Power, Voice Windows, Vocalizations, Vocaquartet); these were followed by surreal multiplications of Steina's mouth (Let it Be); then Woody Vasulka let the audience peek at his artistic laboratory (Artifacts), whose workshop-like informality was interspersed with associative images of Paganini from The Commission; finally, Mucha projected photographs of Steina and Woody Vasulka while playing audio from The Art of Memory, referencing the occasion of the projection release, which was the second anniversary of Vasulka's death.

Mucha used remix aesthetics based on multiplications, superpositions, transformations, samplings, and deconstructions of the videos. But also a Situationists strategy of deflection can be identified in the form of combining sounds and images from different videos, or videos manipulated to the point that they become raw, abstract material for further use. The remix wos a creative strategy based on the transformation (sampling) of the Vasulkas’ videos into a new work. It is a gesture of both life-giving appropriation, and homage to the creators. The resulting audiovisual work must also be seen as an authorial way of presenting the archive’s content, i.e., as a curatorial achievement with a significant authorial contribution.

Remix and machine learning

In the chosen procedure, a motivic, thematic arrangement prevailed in which we can recognize analogies to the functionalities of artificial neural networks. Matthew Fuller says that translation from one medium to another is an effective way of making the overly complex or hidden processes beneath the surface of computer screens visible and therefore understandable. [Fuller 2008] However, Eduardo Navas understands the relationship between remix and machine learning (the work of intelligent software) as analogous, a more or less complex example of remix, and in terms of human vs. algorithmic activity, as a relationship between creativity and metacreativity, or in the case of remix, as products of meta-creativity and meta-metacreativity respectively (in the sense of a remix-based model of human creativity). [Navas 2021]

Video mapping as a performative exhibition

Video mapping, projection mapping or spatial augmented reality, is a term for “projection ... onto any surface, turning common objects of any 3D shape into interactive displays. … [Video mapping is] the display of an image on a non-flat or non-white surface.” [Projection mapping n.d.] The definition of video mapping emphasizes the interaction between the film and the surface on which it is projected. In this case, it takes the form of mutual informing and conditioning – as the projection transforms the façade, the facade structures the projected video.

Fig. 3 Let It Be: Example of image multiplication

The originally decorative Art Nouveau facade of the front wall of the Brno House of Arts was redesigned in a functionalistic style by architect Bohuslav Fuchs after the Second World War and reconstructed in the same style in 2007 by architects from the Hrůša a spol. atelier. (BAM 2022) It consists of a rectangular façade, which is symmetrically surrounded by the building’s L-shaped walls in the rear plan.

The facade of the House of Arts has become a vital structuring factor for video mapping. Its clean geometric shapes and numerous, symmetrical articulations and layering make it an ideal setting for video mapping. The architectural layouts thus essentially defined the language of the audiovisual work – a rhythmic work with superimpositions, juxtapositions, and layering that resembles keying or interactions between background and figure used in many video works.

Fig. 4 Architectural drawing: The facade of the Brno House of Arts

Conclusion – Gallery upside down

Video mapping Vasulka Live Archive has presented a spectacular alternative to conventional gallery practices. At the same time, it was an optimal solution for the exhibition as its implementation was not subject to anti-pandemic measures. When exhibition institutions, like galleries or museums, looked for ways to engage with their audiences during the COVID-19 pandemic, many projects were created online or in gallery windows and walls used for the display of artifacts. In the Vasulka Live Archive /video mapping, the facade was used as a gallery space – it literally turned the gallery inside out. Rather than the gallery’s interior, the exterior walls become the backdrop and stage for the video projection. Video mapping on the facade of an exhibition institution should be understood as a gesture extending the Vasulkas’ intention to free the medium of video from constraints, whether it was the frames of television boxes or perhaps the white cube of a gallery or the black cube of a cinema theatre. The large-scale projection onto a textured background also refers to the environments the Vasulkas created in their peak period. The form of augmented reality can be found, for example, in Steina Vasulka’s installations (Cascades, 2000; Mynd, 2000), and Woody Vasulka’s staged videos in the space of the New Mexico desert in The Art of Memory (1987).

Video mapping has provided both a tribute to the Vasulkas’ artistic legacy as well as a model of the way artificial intelligence (artificial neural networks) sort and cluster audio-visual motifs across the Vasulkas’ work during its pseudo-cognitive operations of learning the content of the Vasulkas’ videos database.

https://vasulkalivearchive.net/Pictures/videomaping.mp4

Fig. 5: Video-documentary of the Vasulka Live Archive /
video mapping. (MUNI ARTS, 2022)