California Considers Unique Security Rules for AI Companies, but Faces Opposition from Tech Firms

SACRAMENTO, Calif. (AP) — California lawmakers are considering legislation that would require artificial intelligence companies to test their systems and add security measures so they can’t potentially be manipulated to destroy the state’s power grid or build chemical weapons — scenarios that experts say could be possible in the future as technology advances at the speed of light.

Lawmakers plan to vote Tuesday on the first-of-its-kind bill that aims to reduce the risks posed by AI. It is being fiercely opposed by tech companies including Meta, the parent company of Facebook and Instagram, and Google, who say the rules target developers and should instead focus on those who use and abuse AI systems to cause harm.

Democratic state Sen. Scott Wiener, who authored the bill, said the proposal would provide reasonable safety standards to prevent “catastrophic harm” from extremely powerful AI models that might be created in the future. The requirements would apply only to systems that cost more than $100 million in computing power to train. No current AI models had met that threshold as of July.

“It’s not about smaller AI models,” Wiener said at a recent legislative hearing. “It’s about incredibly large and powerful models that, as far as we know, don’t exist today but will exist in the not-too-distant future.”

Democratic Gov. Gavin Newsom has touted California as an early adopter and regulator of AI, saying the state could soon deploy generative AI tools to solve highway congestion, make roads safer and provide tax guidance. At the same time, his administration is considering new rules against AI discrimination in hiring practices. He declined to comment on the bill, but warned that overregulation could put the state in a “dangerous situation.”

The proposal, backed by some of the most prominent AI researchers, would also create a new state agency to oversee developers and ensure best practices. The state attorney general could also take legal action for violations.

A growing coalition of tech companies says the requirements will discourage companies from building large AI systems or releasing their technologies as open source.

“The bill would make the AI ​​ecosystem less secure, threaten the open source models that startups and small businesses rely on, rely on nonexistent standards, and introduce regulatory fragmentation,” Rob Sherman, Meta’s vice president and deputy chief privacy officer, wrote in a letter to lawmakers.

The state Chamber of Commerce said the proposal could also force companies to leave state lines to avoid regulations.

Opponents want to wait for more guidance from the federal government. Supporters of the bill have said California can’t wait, citing hard lessons it learned by not acting sooner to restrict social media companies.

State lawmakers on Tuesday also considered another ambitious measure aimed at combating automation discrimination, when companies use artificial intelligence models to review employee resumes and rental housing applications.