The increasingly competitive AI video technology race took another turn on Monday as Luma AI, a San Francisco-based startup founded by former Google, Meta, Adobe and Apple engineers, announced an application programming interface (API) for its Dream Machine video generation model just hours after rival AI video startup Runway announced its own API.
The Dream Machine API allows users — whether they be individual software developers, startup founders, or engineers on teams at larger enterprises — to build applications and services atop Luma’s hit video generation model.
As such, it should bring the AI video technology to more apps, teams, and users around the world, and will enable a whole new class of AI video generation features outside of the Luma AI website. Prior to the API launch, the only way to make AI-generated videos with Dream Machine was through Luma’s website.
AI video models such as Dream Machine and Runway work by training on millions of clips of previously posted footage — in some cases, without express permission or compensation — and transforming them into mathematical structures called “embeddings” that can then produce similar or conceptually related visuals based on a user’s text prompts or still images that they upload (and which the model automatically converts into motion).
Also, unlike rival New York City-based Runway — which debuted two versions of its API for smaller teams and large enterprises, respectively, both via Google Forms waitlists — Dream Machine’s API is available to begin using now. Already, developers at the New York City-based AI code repository Hugging Face have implemented a demo version on the public Hugging Face website:
Amit Jain, co-founder and CEO of Luma AI, explained the company’s vision in a statement published as part of a press release, saying: “Our creative intelligence is now available to developers and builders around the world. Through Luma’s research and engineering, we aim to bring about the age of abundance in visual exploration and creation so more ideas can be tried, better narratives can be built, and diverse stories can be told by those who never could before.”
Luma’s Dream Machine API and Runway’s API both arrived just one weekend after Adobe previewed its “enterprise-safe” Firefly Video AI model — trained only on data that is public domain or that Adobe has direct license to. But Adobe’s Firefly Video is only available to individual users through a waitlist for now, not through an API for enterprises and teams to build separate apps on.
Dream Machine’s fast rise
Dream Machine debuted back in June 2024 as a public beta, instantly wowing users and AI creators with its high degree of realism, relatively fast generation times, and accessibility — especially in the face of OpenAI’s still private Sora model.
Luma also previously released a still image, 3D asset generation AI model called Genie via its Discord server. It recently upgraded Dream Machine to add more control via a dropdown of selected camera motions.
Now it claims that Dream Machine is the “the world’s most popular video model,” based on “the number of users and the number of generations metrics,” according to Caroline Ingeborn, a Luma AI spokesperson, who replied to VentureBeat via email.
Luma Dream Machine API features and capabilities
The Dream Machine API is powered by the latest version of Dream Machine (v1.6) and offers several advanced video generation tools:
• Text-to-Video: Users can generate videos by simply providing text instructions, eliminating the need for prompt engineering.
• Image-to-Video: Static images can be instantly transformed into high-quality animations using natural language commands.
• Keyframe Control: Developers can guide video creation with start and end keyframes, controlling the narrative flow.
• Video Extension and Looping: The API enables users to extend video sequences or create seamless loops, ideal for UI visuals or marketing content.
• Camera Motion Control: This feature lets users direct video scenes through simple text inputs, offering granular control over the generated video’s perspective and movement.
• Variable Aspect Ratios: The API can produce videos optimized for different platforms, removing the complexity of video and image editing.
The Dream Machine API is designed to simplify the process of video creation. Developers can integrate these features into their applications without the need for complex video editing tools, allowing users to stay focused on storytelling and creation.
Accessibility and Ppricing
One of Luma AI’s core goals with the Dream Machine API is to democratize access to high-quality video creation.
Jain highlighted the company’s dedication to making this technology widely available, stating, “We believe in making these powerful technologies accessible to as many people as possible. This is what we did with the Dream Machine launch, and we have learned an immense amount. I’m excited to learn alongside developers and see what they build with Dream Machine.”
The API is priced competitively, at $0.32 per million pixels generated, translating to $0.35 for a 5-second video at 720p resolution with 24 frames per second.
This pricing model ensures that even smaller developers can experiment with and leverage the platform without facing prohibitive costs.
However, without publicly posted pricing by Runway, it is not currently possible to compare the two in terms of value at this time.
Scalable for enterprises
While the Dream Machine API is open to all developers, Luma AI has also introduced a “Scale” option to cater to larger companies and organizations.
This option provides higher rate limits and personalized onboarding and engineering support.
According to Jain, the Scale option is a direct response to demand from enterprise clients: “Since day one of Dream Machine, we have had an immense interest from larger companies and organizations asking us about access to our models. So today, we are excited to bring up our Scale option to serve customers and their far-reaching use cases.”
Responsible use and moderation
Luma AI says it uses a multi-layered moderation system, combining AI filters with human oversight to ensure its tech is used responsibly and complies with legal standards.
Developers using the API can tailor moderation settings to suit their specific markets and user bases.
Luma AI also takes steps to protect user privacy and ownership. Inputs and outputs generated through the API are not used to train Luma’s AI models unless explicit permission is granted by the user, ensuring that intellectual property rights remain intact.
However, Luma and all other AI video generation model providers have been critiqued by human artists and activists who believe that the tech — which was presumably trained on videos from around the web, in some cases (perhaps many) without permission or compensation to the owners — is inherently exploitative and may even violate copyright.
Nonetheless, the AI video providers remain undaunted for now. And with the launch of the Dream Machine API, Luma AI aims to further fuel AI video creation around the web, empowering developers to build innovative video tools with ease — and users to gain further access to tools for expressing their imaginations.
(C) venturebeat
