Viggle Creates Customizable AI Characters for Memes and Bringing Ideas to Life

Even if you’re unfamiliar with Viggle AI, there’s a good chance their viral creations have caught your eye. This AI startup from Canada has become famous for generating various renditions of rapper Lil Yachty jumping around on stage during a music festival. In a notable edit, Lil Yachty is substituted with Joaquin Phoenix’s portrayal of the Joker, and in another, it appears Jesus is energizing the audience. These variations are endless, all thanks to one AI firm pushing the boundaries of meme culture, with Viggle’s chief executive revealing that YouTube clips are a significant resource for their AI innovation.

Propelling forward in AI technology, Viggle has developed the JST-1, a foundational 3D video model that boasts an authentic grasp of physics, according to official statements from the organization. Hang Chu, Viggle’s CEO, emphasizes the distinction Viggle has from other AI endeavors in video creation. Specifically, Viggle offers an avenue for users to determine precise movements for characters, a stark contrast to the often unrealistic animations produced by other models that disregard physical laws.

“What we’re essentially forging is an entirely new species of graphics engine that operates solely on neural networks,” Chu articulated in a discussion. “Ours is fundamentally different from the conventional video generators which focus on pixel manipulation without a true comprehension of physics’ structure and behavior. Our approach provides deep insights into these aspects, which greatly elevates our platform in terms of manipulation ease and generation capability.”

For example, to replicate the Joker performing Lil Yachty’s dance moves, one simply needs to upload the original footage alongside an image of the desired character. Viggle also accords the flexibility of combining character images with textual cues for animation or even crafting animation-ready characters purely from text instructions.

While these memes have garnered substantial attention, they represent just a fraction of Viggle users. Chu points out that their technology has found broader applicability as a visualization instrument among the creative community. Despite the videos being somewhat raw—with unstable imagery and lack of facial expressions—Chu acknowledges their utility in assisting filmmakers, animators, and game designers to materialize their visions.

Viggle now provides a no-cost, restricted version of its AI system via Discord and a web application, along with a $9.99 subscription option for expanded features. The company also extends special privileges to certain creators through a dedicated program. Viggle continues to explore potential collaborations with movie and game producers, noting a growing interest from independent artists and content developers as well.

Recently, Viggle declared a $19 million Series A investment round led by Andreessen Horowitz, with Two Small Fish also participating. This funding is expected to enable Viggle to scale its operations, hasten product evolution, and bolster its team. The startup acknowledges its partnership with Google Cloud, among other providers, to refine and deploy its AI algorithms, though it’s mentioned that YouTube clips specifically do not form the basis of AI training through these collaborations.

Training data

In a dialogue with TechCrunch, inquiries about the specific data sources for Viggle’s AI models brought forth a comparison to OpenAI’s training process for Sora, as per Chu.

Chu confirmed the inclusion of YouTube videos in their dataset when questioned, a stance that could stir controversy. YouTube’s chief, Neal Mohan, had previously outlined in a Bloomberg interview that utilizing YouTube content for AI generation without permission would breach the platform’s user agreement, hinting at the legal complexities surrounding such practices.

Following the interview, a Viggle spokesperson reached out to clarify the CEO’s remarks, suggesting a reticence to divulge complete details about their data sourcing strategy while confirming that YouTube videos do play a role in their training material.

In its commitment to compliance, Viggle synergizes various publicly accessible sources, including YouTube, assuring adherence to all applicable service terms. The firm underscores its dedication to maintaining amicable ties with platforms like YouTube, steering clear of extensive downloads or any activities that might entail unauthorized use of video content.

This stance on using YouTube as a data reservoir aligns Viggle with other industry players who navigate the murky waters of content usage. Reports have surfaced about major tech entities, including OpenAI and Nvidia, leveraging YouTube materials for AI training, highlighting a prevalent yet discreet practice within the tech sphere. Admitting such use openly, however, remains uncommon.