On the Set of Fox’s Trippy New Reality TV Show, Which Turns Singers Into… Aliens?

Human pop stars? That’s so pre-pandemic.

I was skeptical, a few weeks ago, as I approached the doors to a taping of Alter Ego, Fox’s new singing competition with a high-tech twist, but it felt the same as entering any old sound stage — at least for the first minute or so. On Alter Ego, which premieres this Wednesday night (September 22nd), the contestants don’t perform on stage, but rather behind a curtain whilst donning motion-capturing suits that control their own highly fantastical, augmented-reality avatars.

In typical music-TV fashion, the winner will get a cash prize ($100,000), as well as mentorship opportunities from celebrity judges: willi.i.am, Grimes, Alanis Morissette, and Nick Lachey. But the judges, along with host Rocsi Diaz and in-person audiences, watch the performances on eye-level monitors placed strategically around the room — so as to appear as if they’re looking centerstage. Moreover, the judges won’t learn the true identity of the winner until they’ve already won.

To film, Alter Ego relies on 14 cameras, eight of which use advanced camera-tracking technologies. “It’s not something that’s done in post,” creative producer Michael Zinman — who previously partnered with Fox on The Masked Singer — tells Rolling Stone on the still-empty studio floor. Above our heads, thousands of Infrared Reflective (IR) markers — one-inch-by-one-inch silver squares that, essentially, create a map for these cameras — twinkle like a mini galaxy.

“You have to go back to your five-year-old self, play pretend, and talk with your imaginary friend”

The smart cameras then communicate with Unreal Engine, a video-game design software, to render the avatars in real time. (A company called Silver Spoon, which was responsible for creating virtual crowds for Major League Baseball in 2020, designed the 3D models in advance with creative input from the contestants. An augmented reality company called Lulu helped with the stage-plotting.)

Avatar data — things like eye color, height, and special effects — motion-capture data, lighting data, and camera data all meet in a hub of servers next to the mini stage behind the actual stage. “Then, if it all goes right, it spits it out and you see the composite,” Zinman explains. As the host, Diaz is the only one who really has to act, as she’s tasked with standing “next to” the avatars when they’re critiqued at the end of their performances: “You have to go back to your five-year-old self, play pretend, and talk with your imaginary friend,” she says. “It’s actually a lot of fun.”

Such a mixed-reality spectacle isn’t something production companies could’ve pulled off five years ago. “You probably could’ve done it last year, but it wouldn’t look as good,” Zinman says, adding that it could take up to six months to get through the filming process — as opposed to the three weeks required for this show. “We, collectively on the Fox side, spent at least a year developing the technology. To say that it’s even supposed to happen right now… We’re all blown away.”

Not only have graphics evolved significantly, but these latest versions of different tech pieces were not designed to interact with each other — and that’s what Fox really had to figure out fast to be ahead of the curve. The result: Avatars that can cry if and when the person cries, avatars with shadows and reflections that follow them appropriately, avatars that can appear brighter or darker depending on the lighting they’re in at the moment, avatars that can brush their hair out of the way and blush.

While the limited elasticity of the avatars’ faces can sometimes make it look like everyone just got the same bad Botox job, the show is still miles ahead of anything that’s ever successfully been done before. In watching the first episode, it’s easy to forget that the avatars aren’t actually on stage.

One of director Sam Wrench’s biggest priorities was making sure the competition still had “the visual language of a traditional show,” Wrench tells Rolling Stone in the cavernous control room, which smells faintly of pizza. He wanted to shoot it “like a normal entertainment show — just with this added technology — and not let the technology prohibit that.”

“There are a bunch of nuances as to why that’s harder to do, mainly because the character must always exist on the front of any image,” he says. “You can never have them stood behind anyone or anything. So, just before they move behind a piano, we make sure a certain camera cuts at exactly that frame at that time to make sure it aligns accurately. Because, if he stepped back, he’d actually appear in front of that piano.” The same live blending issue presents itself with low-lying fog, he points out.

Alter Ego‘s capabilities have attracted singers with a variety of reasons for being closed off — and believing they couldn’t make it as a performer until now. Some have stage fright, some don’t look like models, and some have already been sent packing by record executives. “Imagine changing the life of someone who just cannot perform on stage because of a disability or mental block,” Diaz urges, as a beauty team hovers around her, checking to make sure all sequins and such remain intact. “This is going to change the music game.”

One contestant, Matthew Lord, was once a baby left in a dumpster in Compton, California. These days, he’s a big, balding 60-year-old who sings opera. (On a dare and a bet, he auditioned for Juilliard Opera Center: “And the bastards let me in,” he says.)

Lord was on tour with his band, The Three Redneck Tenors when Covid hit and he was forced to find a new line of work. Before talent scouts for Alter Ego convinced him to come out to L.A., he was driving an 18-wheeler cross-country. When they called, he was sleeping in the back of his truck at a Campbell’s Soup factory in Alabama. “I wanted to be the next Johnny Vegas when I was younger,” he tells Rolling Stone in the green room on set.

Instead, he became a blue, chihuahua-sized werewolf named Wolfgang Champagne. “I didn’t know what an avatar was,” he says. “I’ve never seen the movie. When I did the first audition, I kept calling it an emoji. But, yea, Wolfie very much has my personality; he’s just young and thin with a full head of hair.” He goes on: “I’m old. There’s not a place in this world for people like me. And that’s not being maudlin; the TV and music industry is an industry for young people. This show gave me a chance to be relevant.”

As a plus-sized woman, Yasmin Shawamreh says she “got overlooked and rejected” while trying to make it in the music industry on her own. Shawamreh, who’s half Palestinian and half Jewish, is from the West Bank but was raised primarily in Chicago. Growing up, musical theater was always her anchor and guiding light no matter how hard she struggled with feeling misunderstood. “I’m a child of war,” she says of naming her avatar Siren. “I’ve survived that, I’ve survived Chicago, I’ve survived the opioid crisis. I write music about heavy things. I go there in the music that I write. My avatar encompasses being a strong warrior.”

Dasharra Bridges — another one of the four contestants Rolling Stone met this summer — always loved singing R&B, gospel, and pop, but she put her dreams of a career on pause to become a mother at 18. “I didn’t know if I could focus on music because I had them, but this show helped me get to a place where I could do both,” she says, adding that her daughter helped picked her name, Queen Dynamite. (Her kids are now 12 and seven.)

Anthony Flammia was also an R&B singer, but he’s realizing now that he might not want to be in that genre after all. “You could guess what I was going to sing by looking at me,” he says. Flammia, who goes by Lover Boy on Alter Ego, started playing around with rock more on the show, and something shifted in him. “Actually, my voice sounds even better with rock. You don’t expect the grunginess.”

But recording contracts often center around putting aspiring stars into a marketable boxes: Flammia used to have a deal with Republic Records, who connected with him as he was making beats for 6ix9ine — even though he’d already been trying to “make it” with his own music for more than a decade. “I got signed in 2018 and dropped in 2019,” he says. “They asked for 10 songs, I gave them 20. I got my skydiving license so I could shoot a video in the sky. I produced all my own music too. But I realized that, if the record label doesn’t have that green button that makes you famous anymore, then they don’t want [you].” Once again, he was on his own.

Really, the crux of Alter Ego’s premise is quite simple and almost overdone at this point: Find America’s next musical star without prioritizing image. Most obviously, The Voice does this — but as soon as a judge turns his or her chair around, after they’ve already chosen their vocally equipped fighter, the mystery is lost. And when the winner of The Voice enters the music biz’s ecosystem of record-label rollouts and marketing plans, it’s harder to push off the urge to blend in.

But if sheltering at home during the Covid-19 pandemic taught us anything it’s that little must be done “IRL” these days. Like it or not, we now live in a world that breeds digital superstars.

CGI-based influencer Lil Miquela, for example, boasts three million followers on Instagram and a quarter of a million monthly listeners on Spotify. Arguably, it was she who clawed open the floodgates when she first went online in 2016. But following the offline world’s shutdown, Warner Music Group invested in Spirit Bomb, a virtual artist label, in 2020 — as well as Wave, a virtual entertainment company, and Roblox, a video game that hosts virtual concerts, in 2021.) Of course, livestreaming was a huge trend during the pandemic: Even the behemoth that is Live Nation acquired a company in this realm, the Benji and Joel Madden-run Veeps. And one-off shows inside games like Fortnite are no longer uncommon. In theory, the winner of Alter Ego could go on to fill their calendar with shows without ever having to be seen in the flesh.

But tech developers are also looking at ways to outfit an in-person concert team with all the necessary tools to handle a proper tour for a digital artist. While this isn’t on the immediate horizon, it’s not as distant as one might think. Zinman says, as a result of pandemic-related downtime, the hurdles Fox cleared in one year should’ve taken three. And in a piece for Rolling Stone about the future of concerts, which ran this spring, concert designer and mixed-reality specialist Cory Fitzgerald said that the creative developments he expected to go mainstream in his field in two or three years instead happened in months.

“I think in the future, we’ll be using something like a Google glass — wherein the audience wears the tech, and they see through it while it renders the actual avatar,” Zinman says, unknowingly echoing similar statements made by Fitzgerald and another concert tech expert, Silent House CEO Baz Halpin, for the earlier piece: “Augmented reality in that way is going to be our future,” Halpin said, urging readers to imagine wearing glasses while Lady Gaga performed “Rain on Me” with virtual, colorful rain falling above and around the crowd. (A lot of this will depend on the speed at which 5G becomes commonplace, and that doesn’t feel too far off.)

When Alter Ego is over, executive producer Matilda Zoltowski says that “the four judges will mentor the artist at different levels to help them put out their first record.” When asked for more details about what that entails, Zoltowski says Fox is “not exactly” sure yet, explaining that the idea came from the judges: “There are endless opportunities and options, and it hasn’t really been done before, so we’re still talking about how we’re going to do that.”

Grimes, one of most tech-savvy judges, declined to comment for this story, but will.i.am tells Rolling Stone that, after watching the Alter Egos, he sparked up about the possibilities of developing Metaverse-level talent and how to keep working with the avatars post-show, regardless of who wins. He says he’s “musing” about how he could map out the specifics during the next few months as the episodes unfold.

So, while experts continue to plot the future of touring, and the savviest of the savvy attempt to navigate the still-unfolding grey area, will things like social media, new streaming options, virtual-concert platforms, and in-game opportunities provide enough juice to turn a TV show-winner into a career artist? Therein lies the real experiment.