Making the Code

Digital Backdrops in Theater, and How to Create Them

By Warren D Riley

This article is part of the Stage Raw/Unusual Suspects Youth Journalism Fellowship

In early November, audience members filed into the Radford Studio Backlot to witness the opening of Live From LA. An immersive hip-hop theatre experience, the play was performed and devised by local Los Angeles high school students in a collaboration between the Unusual Suspects, Street Poets, and other youth-focused arts education organizations. 

 The verdant creatives danced, rapped, and soliloquized while a series of collages projected onto the buildings behind them – acting as pulsating backdrops that underlined themes of immigration, Mexican identity, and resistance. Perhaps unbeknownst to the enthralled audience, these digital works of art were made possible by a group of PhD students, who taught the high school cohort how to code and produce their own projection designs. 

 As fellows of Processing Foundation, an organization that promotes software learning within the arts, Payton Croskey, Ana C, and Jiwon Ham sat down with Stage Raw to discuss their experiences with Live From LA: what brought them to the project, and their thoughts on ensuring computer literacy in underserved communities. 

 Jiwon Ham is a media artist who is particularly interested in challenging the relationship between humans and physical space through the use of data. With a great deal of experience creating art for large-scale digital screens back in Seoul, Ham now contributes her technical and artistic expertise to the students of Live From LA.

 Payton Croskey is pursuing her PhD in Media Arts & Technology at UC Santa Barbara. With a focus on empowering marginalized communities with emerging tech, Croskey’s research is at the intersection of engineering and human-computer interaction.

 Also interested in human-computer interaction, Ana C  is a fellow PhD candidate at UCSB alongside Croskey. Ana studies how to use site-specific augmented reality as a means for social justice causes, a background that makes her uniquely equipped for the social and historical themes of Live From LA.

Stage Raw:  Can you tell me a little about this project/fellowship? What made you get involved?

Jiwon Ham: The theme of the fellowship this year was Data Storytelling. And, the story of the play was based on the [experiences of the] high school students.

Payton Croskey: What spoke to us about the initial concept was the fact that the students would be in control of the narrative. [They’d] be in control of various aspects of the theatrical process, so we wanted to make sure that they also had control over what the projections would look like.

Ana C: We engaged with this tool called p5.js, which is designed to make code more accessible for visual artists and creatives. So I think we had the unique opportunity, in the sense of a collaboration, of leveraging this tool and making it even more accessible to the creative process.

PC: [We are] working with the Latinx community here in LA for this production, a community that is traditionally excluded from these spaces and doesn’t actually have the opportunities, oftentimes, to even learn about tools like projection and tools like p5.js.

SR: So you took on a teaching role while working with the cohort?

PC: “From some initial rehearsals through to the end, we would [explain] what projections are, what projection mapping is and looks like, and what is possible. We thought it was important [to impart] concepts like coding, or these kinds of technical ideas: What does it look like to set up the projections? What does the hardware look like? This is a technological age, but very few people are typically able to participate in these technologies. How can these communities be part of not just the creation of these tools, but also be knowledgeable users of these tools?”

SR: Besides on a technical level, were you involved in other aspects of the production process?

AC: Each of the organizations [including Processing Foundation] worked together to develop the script and storyline. We could chime in during the production meetings with the students.

PC: [The students] were trying to figure out how to cut the script down. So, we might say, ‘Oh, I really like this moment between these characters.’ And also we would offer things that we thought might be useful for projections. Sometimes they might have suggestions for what the projections might look like, and then we could have a dialogue with that person saying: ‘That would work well.’ Or: ‘Maybe this part won’t read as well to the audience at this scale.’

SR: Did you come in with initial plans for how you would include projections in the performance?

AC: The way we conceptualized them was as a narrative device. The idea was that they would help contextualize part of what was happening in the play. So the projection shows visual [examples] of what the students are talking about. Like when Natalia talks about growing and, you know, breaking through.

Here, Ana is referencing a moment from the performance when the character of Natalia, played by young actress Stephanie Nieves, delivered a poem. As the character talked about herself and her fellow community members as “a forest born of saplings” her words were underscored by animated flowers twisting and blooming on the backdrop behind her.

SR: Expanding on that: Ana, for those unable to see these projections live, could you describe them? 

AC: So, for example, in the sequence where Andres explains the parallels between the raids in Venezuela and the [ICE] raids in Los Angeles, the projection mapping was aimed at representing and communicating to the audience what was happening in Venezuela, for the people who were not familiar. The projection designs build off a lot of the historical social movements that inspired the students.

SR: What would you say was the greatest obstacle you overcame during the process?

AC:. We couldn’t go to the site very often. So, we had to conceptualize all of these on-site projections virtually. Maybe Jiwon can speak a little bit to another approach that we took, which was how to conceptualize all of these on-site projections virtually. That’s something that she spearheaded.

SR:. Could you tell us about that experience, Jiwon?

JH: For the projection test, we needed to go [to the Radford] early in the morning, like 5 am or before sunrise, so that we could have a dark environment. Also, we didn’t have the full, total setup equipment and the projectors, so we could only test with, like, one or two projectors. So, instead [of working onsite], I used Blender, a free 3D modeling tool, so that I could set up the scene we digitally scanned after our first visit. We used photogrammetry and also 3D scanning, so we could recreate the real site virtually. We sent rendering images to communicate with the students, make them visualize how the final product should look.

Amidst the cohort of Fellows at “Live from L.A.” Jiwon Ham (far left) and Payton Croskey (front right) (Photo by Angie Lee)

SR You guys are incredibly adept engineers. But, out of curiosity, did you have any experience in theater as a kid? Perhaps as a performer?

JH:  [Laughs] I don’t really remember anything except, I did one theatre play in kindergarten, I don’t even remember what I did. I just remembered that it was embarrassing.

AC:  [Laughs] Painting backgrounds once, so I could get permission to get out of class and then get my friends out of class.

PC: In high school, I was in a theater magnet program. So we learned the craft of, you know, being on stage and your emotions and blocking. And then we also took classes in technical theater, meaning all of the backstage elements. I learned set design, lighting design, sound design, costume design, but what I primarily focused on was stage management.

SR: That can be a rather thankless role, no?

PC:  Yeah, but I secretly liked it because I had all the control. I tell the actors to get in place. I’m telling the sound people to stand by. You know, I’m in charge. [Laughs] I did enjoy that.

SR:  For you, Payton, you’ve talked about your work in technology as securing marginalized groups a place in the ‘historical record.’ Can you expand on what that means?

PC:  Well, you ask [current artificial intelligence] a question like, ‘What protest movements existed in the ’60s?’ Sometimes it has a very specific lens, a white capitalist lens. The stories of marginalized communities are often pushed to the side, and they’re not included in the initial data sets. We’re watching whole archives being erased and destroyed.

SR: . Why was it important to you that these students shared their lived experiences – as part of  L.A. LIVE or otherwise? How does this project reflect your mission?

PC: People who do research like this and work like this are being defunded. We’re witnessing it live. This is an opportunity to counteract that narrative, where these students are able to share their story, contribute data that is consent-full, instead of consent-less, and it is directed by them; they chose what they wanted to share. The student whose [indigenous culture] has historically been ignored, wanted to make sure that the exact specifics of her culture were included in this production, and that’s something that would be lost if we weren’t able to do a project like this. It’s a different type of agency that’s happening here, and I think that’s really important in the creation of new technologies.