Art Teleported

I went to Turku, Finland to be part of a show in New York City organised by the Czong Institute for Contemporary Art from South Korea. That’s right! I participated in Art Teleported, the art show and conference for nomadic artists, by teaming up with the Videokaffe collective. That’s a whole lot of name dropping, so let’s back up for a second.

During Supermarket Art Fair I met the founder of Videokaffe: Sebastian Ziegler. Immediately there was a (goose bumps inducing) connection because of the way we both deal with the intersection of the virtual and physical world. It fuelled my desire to collaborate, so I didn’t need to think twice when they invited me to work together. Their way of working is new (exciting and a bit scary) terrain for me. So I wrote down a plan for my personal artistic development, which I can execute thanks to the Reisma Van den Burg Grant. Main goal of the plan: to develop the ‘collaborativity’ of myself and my work.

Since Supermarket we’ve been in regular contact through video conferencing using Discord, had a show in DesignLab Gallery in California and have been working together in a cross-location live performance format called Screen Breach. These are different every time, from focusing on collaborate drawing, to connecting kinetic sculptures, or even live performance involving the human body. What remains constant in each Screen Breach is that cameras and projectors are used to connect the studios and/or gallery spaces together through the web.

Screen Breach, short example

Until now, all of our collaboration happened without me having to leave Groningen (thanks internet)! So I found it about time to pack my bags and work together in a physical space for a change. Destination: Turku, Finland. But it wasn’t just me on the move. With the Art Teleported show coming up, three Videokaffe members gathered in NYC (Mark Andreas, Tom Burtonwood and Holly Holmes), and Sebastian Ziegler, Jack Balance, Jenny Mild, Olli Suorlahti, Erno Pystynen + me gathered in Turku to prepare and perform another Screen Breach!

Setup on the NYC side: tabletop projection + kinetic sculpture
Mark at the table, the kinetic sculpture in the top left.

In the photos above you see the setup we had at the Art Teleported event, which consisted of two ‘portals’: the table and the kinetic sculpture on the wall.

Kinetic sculpture: wings + body

Let’s look at the sculpture first, which consists of wings and a body. The rhythm of the body drives the wings. But the two parts aren’t directly connected. In fact, they’re not even on the same continent. The body was mounted to the wall in our Finland location, live-streamed and projected on top of the body in New York. There, wall mounted light sensors pick op the movement of the body, and actuate the motors that move the wings. It’s a single sculpture, which exists in two places at once, with the two halves connected through light, reminding me of quantum entanglement

Interactive tabletop projection, New York side, with attributes

Next let’s dive into the table portal, the part I was most looking forward to 🙂 A camera above the table in New York live-streams what happens on the one table, which is then projected onto the second table in Turku. The same happens the other way around, merging both tabletops into a single surface open for play.

Spontaneous drawing, responding to the other side

This portal is ‘hands on’! The audience engaged with us through spontaneous drawing, playing games like Tic Tac Toe or Rock Paper Scissors (with an actual rock, paper and scissors!) and improvising with the various attributes on the table. Doing this we aim to promote curiosity, the joy of invention, and embed values we’re in danger of leaving behind (like handcraft) in the digital infrastructure for the future.

With my Web Spaces projects I’ve been breaking out of the confined space of the screen. What initially only existed virtually, began to manifest itself in the material world. This still fascinates me. Since meeting Videokaffe, collaborating (in real time) has become another point of interest, so I began developing a tool which would allow me, and others, to playfully connect and make things happen in the physical world triggered by what happens on the screen.

The tool is a computer program which allows you to measure the average colour of a screen area, by resizing and dragging around a rectangle. The rectangle acts as a sensor, and based on the average colour within it, it turns a smart power socket on or off via Bluetooth.

For this occasion we connected the smart socket to an air compressor with a bouquet of balloons. Then we dragged the digital sensor rectangle on top of the video feed from the table in New York. This made the rectangle appear on our table surface. But the people at the Art Teleported venue could see it too, since everything on our table is projected on their table too (and vice versa).

Now we could inflate the balloon bouquet by covering the projected rectangle. The people in New York could do the same, and blow up our balloons too. But triggering the sensor for too long would over-inflate the balloons and make them pop!

Blowing up the balloons with my sensor app

Popping the balloons would hurt our ears, not theirs. So the players on the other side of the portal were less careful to prevent over inflating the balloons… Combined with a bit of lag over the network, it turned out to be a suspenseful game.

Balloons about to pop
The Turku side with Olli Suorlahti, Jack Balance, Jenny Mild, Jip de Beer and Sebastian Ziegler

Traveling to Turku was worth it. Besides the performance itself, I spent the week developing my cross platform sensor application for the performance, either in the library or at Sebastian and Heini’s studio at Logomo (where they have delicious all you can eat lunch buffets, with plenty of vegan options!), and had a fun intermezzo making clay sausages in preparation for Heini’s exhibition at Gallery Anhava, Helsinki (6 Feb – 1 Mar 2020).

Making clay sausages

Heini and Sebastian were so kind to invite me over for dinner with their family. Since Kayleigh and I hope to have kids in the future, I enjoyed seeing how they combine their work as artists with raising two adorable kids. I suppose it’s a challenge, but their lightheartedness makes it flow 🙂

With Videokaffe assembling, I also found a moment to resurface my first big project: Rapping.Reviews (2016), which deals with big data in a humorous way. The result is a website with rap battle music video’s, compiled from a beat, a movie trailer, and quotes from movie reviews to form positive and negative rhyme sequences. It used to automatically update daily, but has stopped ever since IMDb blocked my method of easily accessing millions of user reviews… The project is still online (tested in Google Chrome) and hosts 13.802 computer generated rap battles! In return Jack Balance showed me how he’s currently experimenting with A.I. to generate text. Given a sentence, the computer model will continue it from ‘its own imagination’. Here’s one we made:

Model prompt >>> There is this guy from The Netherlands in Finland. He doesn't drink beer. They'll have a performance soon.
=================== SAMPLE 1 ====================
 He bet us $30,000 that we won't get hammered, but what he was after was access to celebrities. He got into a video game, and says movies and golf clubs will be uploaded live in October. But what he wants to see with his very handy Oculus Citro Fractal camera is not an actual video game, but a work of art. He of course does his own build of Infinity Engine for the game. It's committing piracy, and that's fine." We're intrigued.

We documented the Screen Breach on both sides (NYC and Turku) and generated almost 100gb of audiovisual material in the process. Now we need an AI to process the footage… 

Published by Jip de Beer

Jip de Beer, initiator of MADtech.me, is an artist who operates on the intersection of art and computer science. He's fascinated by automation and attempts to optimize his artistic work(flow) with programming. His work usually involves a Web Browser and tends to be generative, interactive and strongly visual. In his daily life, Jip tries to have a positive influence on the world by living a vegan lifestyle.

Leave a Reply