The installation Our Harmony is a result of a collaboration between me and two master students from ‘New Audiences and Innovative Practices’ (NAIP) from the conservatory: Kim Bosch and Stefan Kirschbaum. For several months NAIP students and MADtech students had been collaborating to create a one day experimental music festival that we in the end called Hotch Potch. So during this festival on May 13, 2017, many visitors were walking in and out of Our Harmony. By moving their shared position people could trigger sounds and music. Only in collaboration they could really move the music. During performances, the audience was also able to interact with live musicians, and create an atmosphere together.
1. How does it work?
Our Harmony is an interactive music installation. By walking through a space you can move the music together as a group of people. There is a ‘play field’ with crosses on the floor. The Kinect sensor on the ceiling measures the joint position of people in the space. The moving cursor on the floor represents this joint position. When the group of people moves the cursor over the crosses in the play field, they will trigger different sounds and musical elements.
The joint position works as follows: for example, if two people face each other at two opposite ends of the play field, the cursor will be projected in the center. If two people are close to each other, the cursor will be projected just in front of their feet. As an individual, you do not directly influence the cursor. You have to focus on each other, because only together you can move the music.
2. What is it about?
Our Harmony is about connection. In this modern age, concepts such as self-deployment and personal authenticity have become very popular. But because of this, we are getting more and more turned into ourselves, isolated in our own worlds, staring at our mobile phones. The music, sounds, and texts in the installation play with these concepts: on the one hand the love for the beautiful ideal of personal authenticity, and on the other hand the importance of being together. The experience of Our Harmony encourages people to open themselves to the people around them. Only interaction with each other affects the installation. Only together you can move the music. Together, you create a harmony.
The installation is also a way to make music and concerts more accessible. The audience can influence the music in an intuitive way. We have also combined Our Harmony with live music performers. The audience was invited to walk through the installation and move the music. The musicians responded musically to this. Thus, there was a shift from passive attendance to actively involved in a music performance, while musical knowledge is of no importance. A deeper connection existed between the audience and the musicians. Suddenly, the musicians didn’t only create a harmony for the public, but collaborated with the public.
During the experience of Our Harmony at the Hotch Potch event, I saw that the installation encourages visitors to move, socialise, discover, and have fun. I have heard from several people that they found it an immersive experience, felt a sea of calm, that made them curious and as if they could walk for hours. Also, it was not a technical experience, but almost magical.
3. Technical development process
Before the installation Our Harmony was festival-ready, there were months of brainstorming, experimentation, and trial-and-error. An in depth summary can be found in this previous blog post. Back then I wrote: “I am really enthusiastic about the concepts of involving the audience in a music performance and allowing people to create music together in an intuitive way (without needing music knowledge).” I wanted to use the Kinect to track how people were moving in a space and translate that to music.
Generally I experimented with two approaches. The first was to map one person to one musical instrument. So if a person represents a piano, and makes changes in movement, the piano music will change too. But this was complicated, messy, and difficult for the audience to understand what was happening. The second approach was to track general movement, which is all about the shared position. So if person A is located all the way to the right side, and person B all the way to the left, the location of general movement would be in the center. If person A and B are both at the left side, the location of general movement would be on the left. I mapped the location of the general movement to sounds and music. In the next figure you can see a visual representation of the floor (the Kinect records from above). The circles represent areas that contain musical elements and sounds. The closer you get to the center, the louder that musical element becomes.
This approach worked much smoother, but it still had some issues when we tried it out.
I mapped single musical instruments to the areas. But when trying out at the conservatory, we realised that it would be nicer to have different atmospheres, with more musical elements in one area. In this way the space would have more distinct sounds. With this version we also tried to play together with live musicians, as can be seen in the videos.
I saw another issue when I analysed someone interacting with the installation, trying to figure out what his influence was. He was moving fast, like he wanted to play the drums with his full body movement. This was very different from how I interacted with it. Because I knew the intention of the installation, I walked slowly. So that made us think.. I really didn’t want it to become a body drum kit, no, I wanted people to walk slowly through it and explore the sounds. So how to make sure people walk slowly? First of all, it would help if the space would be bigger. Like in the video, the floor that the Kinect recorded was not much bigger than 2 meters by 2 meters. So you will be able to move very quickly from one area to the next. If the space is a lot bigger, it takes mores steps to get to other areas, and it will naturally take longer. We were also thinking about hanging books down from the ceiling on a wire at the center of the areas, on which people could read or write. In this way people would definitely have a reason to slow down and stop at a center. It could also give it more conceptual depth. But in the end this didn’t seem necessary anymore, which I will get back on.
The final issue was that people needed to look at the visual representation (like in the image before) to understand where their shared position was and to understand what was happening. It worked in a way that people understood what was happening, but it took the magic away. You were in this immersive space, in which your movements control the music, but then suddenly you have to look at something very abstract, and it takes you out of the immersiveness. So finally we ended up projecting a cursor on the floor that represented the shared location of the people in the installation. I made an icon with four abstract images of people with their heads pointed to each other. Then we also taped crosses on the floor representing mysterious sonic elements. We had the feeling that in this way, you are much more focused on the music, the space, and each other. And since there was enough to focus on, and it was more natural now to move slowly, we decided not to use the books as mentioned before.
4. Conceptual development process
But tracking general movement didn’t only work better in a technical way, it also seemed to suit better to my concept. Because at the same time as I was making this system, I was also reading Everyone is an Artist by Ruben Jacobs and The Ethics of Authenticity by Charles Taylor. They both think that in our modern society, the value of authenticity and the pursuit of self-fulfillment has become important to such an extent that we tend to forget causes beyond the self, community, living together, etc. They still believe that authenticity is a beautiful ideal, it’s just that nowadays we have taken it too far.
“True authenticity also involves a recognition of and an openness to what he calls “horizons of significance” — certain larger contexts within which humans move. These contexts might include respect for and benevolence toward others and toward the natural world. They provide a sense of personal connection with a larger political, social, or religious source of meaning.” – source
It’s funny that whenever someone started to interact with the installation-in-progress, he/she was figuring out how his/her own movements were influencing the installation. But if there are more people in the installation, it won’t make sense like that. I had to explain them that you have to focus on each other and your shared location. Only together you can move the music. I have realised (even with my co-creators) that this concept is uneasy at first, and not usual. Most interactive installations are kind of one-to-one: I do something, the installation responds. But with this installation, people are almost forced to look beyond the self, otherwise it won’t work. At first I thought that the use of general movement would be a limitation, but eventually it turned out to be the very core of the concept. With this installation I was really hoping to get my visitors to focus on each other, to look beyond the self, to feel the togetherness.
I still liked the idea with the books hanging down, because of the possibility to provide conceptual depth. I didn’t want my visitors to only focus on each other because then “the installation works”. I also wanted to give them a clue why. So instead of using books, I hid audio quotes in the space. Just like people can trigger music by moving the cursor over the crosses together, they can also trigger quotes in the same way. With these quotes I explore on one side the beauty of the value of authenticity and self-fullfilment, and on the other side questioning whether we haven’t taken this too far and lost our “horizon of significance”. I decided to use the following quotes in the installation:
- The privilege of a lifetime is to become who you truly are
- Before authenticity we felt part of a larger order, the great chain of being
- Do we still care?
- We need each other in order to be anybody
- Where are the hero’s? What is our purpose? Something worth dying for?
- Where and with who am I?
- We lived among angels, heavenly bodies, and our fellow earthly creatures
- Being starts with being with
- Look inside
I was hoping that these quotes, together with the act of working together would spark thought in visitors about the subject of authenticity and community.
5. The Festival Experience
But the festival experience had to tell me whether this turned out the way I intended. And it told me it did. Sometimes I explained to people beforehand that they have to move the cursor together. And in that way they focused on each other. But when I didn’t explain I saw the process in people while they were trying out Our Harmony: first they were stepping on the crosses, but soon realising that their individual actions didn’t influence anything. I saw how their focus slowly shifted from themselves to the others who were also present in the installation. At first moving very slowly, and eventually playing and experimenting with different locations and movements with the other people. It really invoked a sense of connection between people in the installation.
During the festival day there were three performance moments with Our Harmony. We didn’t only like to see how visitors were connecting with each other. We also wanted to try out how Our Harmony could be used to create a more direct connection between live music performers and the audience. So the audience could walk through the installation together, create atmospheres, on which the live musicians react. In this way it’s not only “musicians imposing their music on the audience” but more like creating a musical performance together. The performances were a unique experience. During the whole performance, the audience was walking in and out of the installation. I also saw one guy walking through the installation and constantly smiling at the musicians, almost as if he was challenging them (as you can see in the following picture).
Several people told me that Our Harmony feels very immersive, relaxing, and that they could do it for hours, because there was so much to explore. I also heard from two different people that it made them think of another project called Tovertafel: “Playful interactive light projects tempt children, adults and the elderly to move and have fun together.” I really liked to see this link, because I am still very much interested in human factors and health care. But sometimes I feel like I’m not doing enough with this interest. But apparently, my projects, like Our Harmony, still have a close relationship to those interests of mine.