Anthropic, the American AI startup, announced on Tuesday a revolutionary feature that allows computers to be used in a more human way. This feature is designed to help developers automate repetitive tasks, manage cursor movements, click buttons, and enter text, thereby improving the functionality of AI agents.

How will the feature work?

Currently in public beta, this feature is included in Anthropic’s latest language model, Claude 3.5 Sonnet, which was also updated this week alongside another model, Claude 3.5 Haiku. As TechTarget reports, the new technology allows developers to ask Claude to interact directly with computer interfaces, enabling a range of applications from task automation to software development and exploratory research.

Using an API developed by Anthropic, programmers can facilitate the automation of tasks that typically require human intervention. This development aligns with the generative AI industry’s growing trend toward employing AI agents, a shift further highlighted by Salesforce’s recent introduction of Large Action Models and its Agentforce platform.

Anthropic acknowledged that computer interaction functionality is still developing, with challenges in performing tasks such as scrolling, dragging and zooming – actions that humans typically perform easily. Additionally, the Claude 3.5 Haiku template, which excels at coding tasks, is expected to be released later this month on various platforms, including Anthropic’s API, Amazon Bedrock, and Google Cloud’s Vertex AI.

Anthropic noted that several companies, including Asana, Canva and DoorDash, have already used this IT interaction feature before its public release, suggesting strong interest in the automation potential.

While this new feature appears to be a significant advancement, it shares similarities with existing Robotic Process Automation (RPA) and Business Process Automation (BPA) tools, long used in the process management industry job.

What are the risks?

Despite the promising opportunities, the new tool carries some risks. As reported by TechTarget, experts have warned against premature adoption, expressing concerns over security, safety and responsible use. This feature could potentially allow malicious actors to exploit system vulnerabilities by guiding AI systems in computer networks.