The US government wants you — yes, you — to find the flaws in generative AI

At the Defcon 2023 hacker conference in Las Vegas, prominent AI tech companies partnered with algorithm integrity and transparency groups to bring together thousands of participants on generative AI platforms and find weaknesses in these critical systems. That “red-teaming” exercise, which also had U.S. government support, took a step toward opening up these increasingly influential but opaque systems to scrutiny. Now, ethical AI and algorithm evaluation nonprofit Humane Intelligence is taking that model a step further. On Wednesday, the group put out a call for applications to the U.S. National Institute of Standards and Technology, inviting anyone in the U.S. to participate in a qualifying round of a nationwide red-teaming effort to evaluate AI desktop software.

The qualifications will be held online and open to both developers and the general public as part of NIST’s AI challenges, known as Assessing Risks and Impacts of AI, or ARIA. Participants who advance through the qualification round will participate in an in-person red-teaming event in late October at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia. The goal is to expand the ability to conduct rigorous testing of the security, resilience, and ethics of generative AI technologies.

“The average person using one of these models has no way of knowing whether that model is fit for purpose,” says Theo Skeadas, CEO of AI governance and online security group Tech Policy Consulting, which works with Humane Intelligence. “So we want to democratize the ability to do that assessment and make sure that anyone using these models can make their own assessment of whether that model is meeting their needs.”

The final CAMLIS event will divide participants into a red team attempting to attack AI systems and a blue team working on defense. Participants will use NIST’s AI risk management framework, known as AI 600-1, as a rubric to measure whether the red team is able to produce results that violate the expected behavior of the systems.

“NIST’s ARIA relies on structured user feedback to understand real-world applications of AI models,” says Humane Intelligence founder Rumman Chowdhury, who is also an executive in NIST’s Office of Emerging Technologies and a member of the AI ​​Security Board at the U.S. Department of Homeland Security. “The ARIA team is primarily composed of experts in sociotechnical testing and assessment, and (is) using that experience as a way to evolve the field toward rigorous, scientific assessment of generative AI.”

Chowdhury and Skeadas say the NIST partnership is just one in a series of AI red team collaborations that Humane Intelligence will announce in the coming weeks with U.S. government agencies, international governments, and nongovernmental organizations. The goal of the effort is to make it much more common for companies and organizations that currently develop black-box algorithms to offer transparency and accountability through mechanisms like “bias reward challenges,” where individuals can be rewarded for finding problems and inequalities in AI models.

“The community needs to be broader than developers,” Skeadas says. “Policymakers, journalists, civil society, and non-technical people need to be involved in the process of testing and evaluating these systems. We need to make sure that underrepresented groups, such as those who speak minority languages ​​or come from non-majority cultures and perspectives, can participate in this process.”