How AI Agents Are Changing Software Development

Software engineering is one of many fields that is changing with the rapid advances in large language models (LLMs). In a few years, LLMs have evolved from sophisticated code completion tools to AI agents that can design software, implement and improve entire modules, and help software engineers become more productive.

Like many things LLM-related, some of the enthusiasm around AI-powered software engineering agents is baseless hype. But there’s also real value to be captured, and developers who learn to use the next generation of AI tools will be able to do a lot more in less time.

AI Coding Assistants

There are three main ways that LLMs change the coding experience. The first is the direct use of boundary models as assistants. Developers use ChatGPT, Claude, and other chatbot interfaces as coding assistants. The models are getting better at generating code from text descriptions, improving a piece of code you give them, or helping you debug your code.

Recognizing the use case for software development, model providers are adding new features to improve the developer experience in the chatbot interface. For example, Claude’s new Artifacts feature lets you review and run code while iterating through the model.

More advanced uses of AI coding assistants include LLMs that are added to integrated development environments (IDEs) as plugins. These tools can use project files and the code base as context to provide more accurate answers and perform more complex tasks.

Microsoft was the first company to enter this market with GitHub Copilot, launched a year before ChatGPT. It started as a program for writing code snippets in a code editor. It has since evolved into a full assistant that can help with various tasks in a programming environment.

Amazon’s Q Coding Assistant offers similar features in a coding environment, including automatic code completion, design agents, and code migration between different programming languages.

Several startups have also entered the space, including Tabnine, which claims to have millions of users and that developers use it to write 30% to 40% of their code. Other players include Replit, which provides a coding environment based on its own LLM, and Codeium, an AI coding assistant that can integrate with dozens of IDEs.

Software Engineering Agents

The third way that LLMs are changing software development is through agent-based frameworks. AI agents are essentially multiple LLMs that are fed different system prompts and are tasked with working together to complete a project. For example, one agent might be a designer who provides a high-level plan for completing a task, such as finding resources that provide information, creating modules, and then running them on a cloud platform. Another agent might provide a more detailed breakdown of each of these steps. A third agent might be assigned to write the code for a specific task and send it to another agent, who checks the code for quality and sends it back for corrections. Finally, another agent might pull all the pieces together, compile them, test them, and approve them for launch.

In theory, software engineering agents could be given a project brief and complete it from start to finish. For example, in March, AI startup Cognition announced Devin, dubbed the “first AI software engineer.” Devin uses LLM agents and a variety of tools, including a browser, IDE, and compiler, to gather resources, reason about a task, write code, and evaluate the result. A user can follow the reasoning process and watch Devin progress through his work. Multiple demonstrations published by Cognition AI have shown Devin performing various tasks, including UpWork’s work on a computer vision project, creating the impression that AI agents could soon replace software engineers.

Devin is not open source and still is not open to the public. But it has inspired other projects, such as OpenDevin, an open source software engineering agent with similar capabilities. And other software development agents, such as GPT-engineer, have been around for several months with impressive demos.

Noise or reality?

Many studies show that AI assistants like GitHub’s Copilot increase developer productivity and help them focus on their tasks rather than searching for solutions to problems online. ChatGPT and Claude have also become regular tools for developers to sketch out software design ideas, prepare initial versions of code, and learn new coding skills.

However, some of the enthusiasm and hype surrounding AI software assistants is misplaced and has caught the attention of experienced engineers. For example, many of the videos show that Devin’s pre-built demos aren’t what they’re advertised to be, and the AI ​​agents are far from performing the full set of tasks of a mid-level or senior software engineer.

There are also concerns that tools like Copilot can generate unsafe code that may have appeared in their training data or in a user’s code base. Tool vendors are constantly working to add safeguards to prevent models from generating unsafe code. There’s also the risk of “automation blindness,” where developers get too used to accepting AI-generated code without reviewing it. This can result in unpredictable code that takes extra time to debug.

It is certain that AI will not replace programmers. However, we are still in the early stages of AI coding assistants and there is no denying the value of using LLM in software development. As AI enters more domains, the demand for software programmers also grows. As tools and models mature, we can expect greater productivity gains in software engineering.

The upcoming VB Transform 2024 conference will further explore these topics with expert panels discussing the cross-functional future of AI, with thought leaders. We hope to see you there!