Over the weekend, the Smashing Pumpkin’s Billy Corgan joined Marilyn Manson onstage for “Antichrist Superstar,” which morphed into a perplexing cover of Cyndi Lauper’s “Girls Just Wanna Have Fun.” As if that isn’t interesting enough, Corgan was dressed as a nun. Only with Manson, right? You can check out a fan-shot video of the performance below.

Read more: Marilyn Manson on Columbine massacre: “It really shut down my career”

Corgan and Manson have spent much of the summer together on their co-headlining tour, which wrapped up last week.