Nvidia’s unmatched supremacy in the market is not finished, but new competitors and options are emerging from various sources.

ZML, an emerging French AI enterprise supported by Turing Award laureate Yann LeCun, has unveiled inference-performance software that enables numerous open source large language models to operate on a range of chips — including those from Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc.

With the introduction of ZML/LLMD, the brand new LLM inference server, the firm aims to dismantle existing barriers and facilitate the use of different chips for AI purposes at their maximum achievable speed, and at times even quicker, as ZML founder Steeve Morin shared with TechCrunch.

As AI becomes more intertwined with our professional and personal lives, optimizing inference — or the processing of prompts — is increasingly overshadowing model training in significance, yet it often appears inconsistent behind the scenes, marred by software and architectural hurdles that lead to vendor lock-in, according to Morin.

The potential of attaining optimal performance across various chips represents a significant technological advancement, with the ability to disrupt the market amidst growing concerns regarding AI-related expenses.

ZML aspires to give businesses and cloud providers the choice to utilize a combination of chips, some of which might be more affordable or consume less power. “The aim is to return the control to people to construct their own systems and achieve genuine efficiency improvements that facilitate the spread of [AI],” Morin stated.

This software support could assist innovative AI chip producers, many of whom are based in Europe, noted Morin, mentioning companies such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. However, he emphasized that the significant factor is not their geographical origin, but that ZML is capable of collaborating with them on “initiatives that have never been pursued before anywhere globally.”

Morin is not pessimistic about Nvidia. He acknowledged a positive relationship with the AI chip leader, which is preparing for the rise of inference.

Inference has attracted immense investment, leading to a trend termed the “inference gold rush.” Consequently, ZML faces rivals such as Baseten, recently valued at $13 billion; Inferact, from the developers of open source project vLLM; and RadixArk, the commercial entity behind SGLang.

Both vLLM and SGLang partially compete with LLMD, but Morin’s aspirations for ZML encompass a broader range. “We have arrived at a stage where we are co-designing silicon,” he expressed. He further credited ZML’s compact team of 20 as a key factor enabling the Paris-based startup to operate swiftly, with more launches on the horizon.

The fact that this small team is well-financed relative to its size also contributed to its success. Morin, who has a history as VP of engineering at Zenly — acquired by Snapchat for a significant sum in 2017 — raised $20 million from various venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

In contrast to ZML’s initial public project, the inference-centric ML framework introduced in 2024 and updated in March, ZML/LLMD will not be open source. Nonetheless, it is debuting as a free offering with the intent of understanding usage patterns. “I’d prefer to assess and [then generate revenue] where it is most effective without foolishly stunting my growth due to excessive greed from the outset,” Morin remarked.

It remains uncertain when ZML/LLMD might transition to a paid offering and how its adoption will unfold. However, the startup’s cap table indicates that other founders are observing closely, including Dagger and Docker founder Solomon Hykes, Clément Delangue and Julien Chaumond from Hugging Face, as well as LeCun, now at AMI Labs. This reinforces the notion that Europe’s AI startups can now flourish locally. “I couldn’t establish ZML anywhere other than Paris,” Morin concluded.