A New Look at Productivity Measurement in the Age of Artificial Intelligence

Executives are focused on how generative AI can impact employee performance. As the chief strategy officer at DevSecOps software company GitLab, I spend a lot of time talking to customers about the impact of AI on software development.

Organizations have largely gotten over their AI fears and are now looking to make AI scalable and sustainable. But many executives need help quantifying AI’s impact on productivity. In a recent GitLab study, more than half (57%) of executives said measuring developer productivity is critical to growing their business, and 51% believe their methods for measuring developer productivity are flawed—or they want to measure it but don’t know how.

So what does “productivity” mean in this context? How should executives capture and measure the impact of generative AI on their development teams?

Integrating AI into an organization’s workflows can drive better business outcomes, help build strategic capabilities, and increase competitiveness. Developers are key to all three. Finding meaningful ways to measure AI’s impact on developer productivity in these domains is essential to unlocking its strategic value by connecting it to business outcomes.

Measurement beyond performance

Traditional metrics like lines of code, code commits, or task completion often miss essential elements of software development like problem solving, teamwork, and innovation, which are key to assessing business impact. Capturing AI contributions involves more than just counting time, team dynamics, and tasks; these metrics should lead to tangible business outcomes like user adoption, revenue, and customer satisfaction. It’s also worth noting that business outcomes can vary by company or project.

It is important to track the completion time of entire projects and maintain a comprehensive view of the development process. This includes monitoring the frequency of deployment, change turnaround time, and service recovery time to provide a holistic view of project performance. In addition, evaluating team metrics is key. Peer support, work environment, engagement, and collaboration significantly impact employee turnover and productivity.

Developers spend only about 25% of their workdays writing code; the rest is spent fixing bugs, resolving security issues, or updating legacy systems. Automating these tasks with generative AI allows developers to use their knowledge more effectively, focusing on creativity and solving complex problems. This not only drives innovation, but also increases job satisfaction. Performance reviews, turnover rates, and internal customer satisfaction surveys are valuable tools for tracking these improvements.

In addition, AI is key to predicting development bottlenecks and automating routine tasks, leading to more predictable release cycles and faster time to market. Gen AI improves code reviews and creates comprehensive test scenarios, increasing code reliability and reducing errors, leading to improved software quality and greater customer satisfaction. Gen AI’s ability to quickly and accurately adapt software based on user feedback ensures that products more effectively meet customer needs and expectations.

AI-driven improvements can be measured through customer feedback, service requests, analyst and expert reviews, and overall market health, providing a clear picture of AI’s contribution to business goals.

Making strategic decisions to empower developers

Knowing that the impact of generative AI on developer productivity translates into business outcomes, strategic opportunities, and competitive advantage for a company, executives should make strategic decisions about AI implementation to empower development teams to:

1. Empower developers to make decisions: Empower developers to decide which AI tools can improve their sense of ownership and engagement, encouraging them to decide how to integrate AI into their work.
2. Iterate and adapt: Encourage a culture of experimentation and iteration with AI tools. Allow development teams to go through trial-and-error phases to understand how AI best fits into their processes. Support them through potential short-term productivity dips as they adapt to new tools while pursuing long-term gains.
3. Beware of bad habits: AI has the potential to help less experienced developers write code faster and improve their skills. However, it also has the potential to teach them bad coding practices unintentionally. Developer team leaders should monitor this closely.
4. Deploy AI for long-term transformation: View AI not as a temporary fix, but as a transformational tool that can fundamentally change software development. Companies can ensure sustainable growth and leadership in technology-driven markets by aligning AI strategies with long-term business goals.

Maximize what you measure

Developer productivity is multidimensional. It goes beyond task completion and time management to include team dynamics, problem-solving skills, and more. To truly understand how developers contribute to business value, management needs a more holistic perspective.

A recent study found that while 69% of senior executive respondents said they were delivering software at least twice as fast as they were a year ago – indicating an ongoing acceleration – only 26% of them admitted to implementing AI.

Forward-thinking executives should explore how AI tools can improve the amount of work done and the quality of business outcomes. This way, companies will not only be able to measure AI’s true potential, but also have the ability to maximize it.

Ashley Kramer is director of marketing and strategy at GitLab Inc. She wrote this article for SiliconANGLE.

Image: SiliconANGLE/Ideogram