Google has unveiled the LUMIERE AI model, a revolutionary tool enabling users to craft videos by inputting text. LUMIERE, an innovative text-to-video and image-to-video model, empowers users to generate videos not only directly from text but also by transforming still images into dynamic motion videos. It was shared on X, a demonstration video illustrates the functionality of this tool.
 Unlike conventional video models, LUMIERE leverages a space-time U-Net architecture, producing an entire video's temporal span in a single pass. This approach eliminates the need for synthesizing distant keyframes post-temporal super-resolution, ensuring enhanced global temporal stability. Researchers employed a pre-trained text-to-image diffusion for the text-to-video generation framework.
 
Overcoming challenges related to globally consistent speed, the team successfully generated full-frame video clips by deploying the space-time U-Net architecture with integrated spatial and temporal modules. The research efforts yielded commendable results in Image to Video, Video Inpainting, and Stylized Generation.

Remarkably, the model's capabilities include crafting videos from written text, generating videos from static images, stylized generation, allowing the creation of images in a chosen style by referencing a photo, and animating photos. This versatile model is poised to revolutionize video editing, facilitating tasks such as changing object or clothing colours using AI.
For more interesting updates wa click and follow Indiaherald WhatsApp channel

Find out more:

LUMIERE AI

Sindujaa D N

26/01/2024 11:22 PM