Luma AI launches the Dream Machine platform that uses artificial intelligence to generate text in videos

Luma AI on Wednesday launched its artificial intelligence (AI)-powered text-to-video generation model, Dream Machine, globally. The platform can generate videos up to five seconds long based on simple or descriptive text prompts. The AI ​​model can generate videos in a variety of styles such as cinematic, animated, realistic, and more. The artificial intelligence company claims that the Dream Machine was trained solely on videos and is capable of generating “physically accurate, consistent and eventful shots.” The platform is currently free to access and use, but there will likely be a daily generation limit.

Luma AI debuts Dream Machine

According to the website, the Dream Machine AI model is built on a transformer model and was trained directly on videos. Typically, large language models (LLMs) are trained on text and images and then transferred to videos because they require a deeper understanding of space and motion. “Dream Machine is our first step towards building a universal imagination engine,” the company added.

Dream Machine joins video generation platforms such as Runway AI and Pika 1.0, which are also available in the public domain and offer three- to five-second video generation. The Gadgets 360 team tried out the platform and found the platform’s speed to be unsatisfactory. The problem occurs with multiple characters or when the prompt is too complex. However, compared to the other two, it is capable of generating higher quality cinematic videos.

It takes the AI ​​platform 120 seconds to generate a video that the company says will have 120 different frames. Dream Machine is also said to understand how people, animals, and objects interact with the physical world, and creates videos with accurate physics and character consistency.

However, Luma AI also highlighted several limitations of the current mode, such as motion, text, morphing, and the famous Janus problem, where instead of showing a consistent 3D result, the AI ​​model shows multiple canonical views of an object in different directions.

Dream Machine Luma AI creates a movie about stormtroopers
Photo source: Luma AI/Maszyna Marzeń

Luma AI has not released any technical details about the AI ​​model, so information about parameter sizes, benchmarks, architecture and training methods is unknown. The company also did not share any details about the acquisition of training data. It is worth noting that Gadgets 360 has managed to generate several videos containing copyrighted characters.

To try out the platform, enthusiasts can visit the website and click the “Try Now” button. Users will have to register on the platform before they can generate videos.