Connect with us

Hi, what are you looking for?

Technology

Nvidia’s ‘Audio2Face’ tech uses AI to generate lip-synced facial animations for audio files


AI innovation: Game development is an incredibly time-consuming and expensive process, with art and animation budgets often eating up a hefty chunk of a team’s cash reserves. Believable facial animation, in particular, is critical for cutscene-heavy titles. That’s why Nvidia is working on an AI-based tool that can read audio files and create matching facial animations in real-time; no mocap is required.

This tech, called “Audio2Face,” has been in beta for several months now. It didn’t seem to get much attention until more recently, despite its potentially-revolutionary implications for game developers (or simply animators as a whole).

As you’d probably expect from any technology that is both powered by AI and still in beta, Audio2Face isn’t perfect. The quality of the source audio will heavily impact the quality of the tech’s lip-syncing, and it doesn’t seem to do a very good job of capturing facial emotions. No matter what sentence you throw at Audio2Face’s default “Digital Mark” character, the eyes, cheeks, ears, and nose all remain fairly static. There is some movement, but it’s generally more subdued than the lip animations, which are clearly the main focus.

But perhaps that’s a good thing. Conveying accurate emotions in the face of a 3D character is what animators train for years to accomplish. Depending on how easy this tool is to implement into a given developer’s workflow, it could provide serviceable, perhaps placeholder lip-sync animations while letting animators focus on other parts of a character’s face.

Some of our readers, or simply hardcore fans of Cyberpunk 2077, may recall that CDPR’s title used similar technology called “JALI.” It used AI to automatically lip-sync dialogue across all of the game’s main supported languages (those with both subtitles and voice acting), lifting the burden from the animators themselves.

Audio2Face doesn’t have that capability, as far as we can tell, but it still looks useful. If we hear of any instances where a developer has taken advantage of the tech, we’ll let you know. If you want to give it a spin yourself, the open beta is available for download, but just know that you’ll need an RTX GPU of some kind for it to function properly.



Source link

Advertisement. Scroll to continue reading.
Click to comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Advertisement

Latest

Technology

AI innovation: Game development is an incredibly time-consuming and expensive process, with art and animation budgets often eating up a hefty chunk of a...

Top Stories

The official announcement on Mirror from ENS states that $ENS will be used to vote on a proposed constitution and govern protocol parameters. The...

Top Stories

Jules Urbach, the CEO and founder of OTOY, wants to democratize content creation, specifically as it pertains to virtual assets. OTOY’s technology intends to...

Online Business Success

The ability to exclude capital gain on the sale of qualified small business stock (QSBS) is one of the most powerful and exciting tax...

Social Media

They’ve been in testing over the last few months, but now, Twitter is rolling out its updated visual customization and editing tools to more...

Top Stories

Blockchain platform Telos raised $8 million in funding led by ConsenSys and Polygon investor John Lilic. The funding was secured right before the launch...

Top Stories

Bitcoin investors appear to be increasingly sitting on their hands in hopes of higher prices, with the share of Bitcoin’s supply that has remained...

Technology

Santa Clara, California-based company, Advanced Micro Devices (AMD), says its “Milan-X” AMD EPYC processors with 3D V-Cache, to launch in early 2022, will deliver...

Advertisement

You May Also Like

SEO Guide

Want to rank in Google image search? Images that you use as a featured images when writing a post actually appear on Google Images...

Technology

Sonos is one of the most popular wireless speaker brands in the world, and for good reason – its range of portable Bluetooth speakers,...

Online Business Success

Carrying Credit Card Debt Isn’t Just Bad for Your Budget. It May Also Affect Your Health. The stress of carrying card debt through adulthood...

Online Business Success

File photo The Economist Intelligence Unit (EIU) has said that inflation will remain high in Pakistan for the next six months and the rupee...

Advertisement