Companies are optimistic about agentic applications that can understand users’ instructions and intent to perform different tasks in digital environments. This is the next wave in the generative AI era, but many organizations are still struggling with low throughput with their models. Now, Katanemo, a startup that builds intelligent infrastructure for native AI applications, has taken a step to solve this problem by open source Arch-Function. It is a collection of cutting-edge large language models (LLMs) promising lightning-fast speeds for function calling tasks essential to agentic workflows.

But what speed are we talking about here? According to Salman Paracha, founder and CEO of Katanemo, the new open models are almost 12 times faster than OpenAI’s GPT-4. It even outperforms Anthropic’s offerings while providing significant cost savings.

The move can easily pave the way for ultra-responsive agents that can handle domain-specific use cases without burning a hole in companies’ pockets. According to Gartner, by 2028, 33% of enterprise software tools will use agentic AI, up from less than 1% currently, enabling 15% of daily business decisions to be made autonomously.

What exactly does Arch-Function provide?

A week ago, Katanemo launched Arch, an intelligent prompt gateway that uses specialized LLMs (subbillions) to handle all critical tasks related to prompt management and processing. This includes detecting and rejecting jailbreak attempts, intelligently calling backend APIs to respond to user request, and managing observability of LLM prompts and interactions centrally.

The offering enables developers to create fast, secure, and personalized build AI applications at any scale. Now, as the next step in this work, the company has open sourced some of the “intelligence” behind the gateway in the form of LLM Arch-Function.

As the founder says, these new LLMs – built on Qwen 2.5 with parameters 3B and 7B – are designed to handle function calls, which essentially allows them to interact with external tools and systems to perform digital tasks. and access the latest technologies. information on dates.

Using a given set of natural language prompts, Arch-Function models can understand complex function signatures, identify required parameters, and produce accurate function call outputs. This allows it to perform any required task, whether it is an API interaction or an automated backend workflow. This, in turn, can enable businesses to develop agentic applications.

“Simply put, Arch-Function helps you customize your LLM applications by invoking application-specific operations triggered via user prompts. With Arch-Function, you can create rapid “agent” workflows tailored to domain-specific use cases – from updating insurance claims to creating ad campaigns via prompts. Arch-Function parses prompts, extracts critical information, engages in light conversations to collect missing user parameters, and makes API calls so you can focus on writing business logic “, explained Paracha.

Speed ​​and cost are the main strengths

Although function calling is not a new feature (many templates support it), the efficiency with which Arch-Function LLMs are handled is the highlight. According to details shared by Paracha on X, the models beat or match frontier models, including those from OpenAI and Anthropic, in terms of quality but offer significant advantages in speed and cost savings.

For example, compared to GPT-4, Arch-Function-3B offers ~12x throughput improvement and massive 44x cost savings. Similar results were also observed against GPT-4o and Claude 3.5 Sonnet. The company has yet to share full benchmarks, but Paracha noted that throughput and cost savings were seen when an Nvidia L40S GPU was used to host the 3B settings model.

“The standard uses the V100 or A100 to run/evaluate the LLMS, and the L40S is a cheaper instance than both. Of course, this is our quantized version, with similar quality performance,” he noted.

With this work, businesses can have a faster, more affordable family of function calling LLMs to power their agentic applications. The company has not yet shared any case studies on how these models are used, but high-throughput performance and low costs are an ideal combination for real-time production use cases, such as processing incoming data for campaign optimization or sending emails to customers.

According to Markets and Markets, globally, the AI ​​agent market is expected to grow with a CAGR of almost 45% to become a $47 billion opportunity by 2030.