California’s AI Policy: Delving into Senate Bill 896

California’s SB 896 sets a framework for ethical AI use across state agencies, focusing on transparency, accountability, and public safety. It mandates GenAI risk assessments, public disclosures, and statewide collaboration to safeguard citizens and critical infrastructure.

The Senate Bill 896, known as the Artificial Intelligence Accountability Act and introduced by Senator Dodd to the California Senate on January 3, 2024, has been referred to the Senate Committee on Governmental Organization. It aims to bolster innovation, consumer protection, and adherence to Executive Orders on AI issued by President Biden and Governor Newsom.

The bill proposes regulations for state agencies regarding the use of emerging technology, including the requirement of risk assessments for AI systems and emphasis on ethical AI practices. Here’s an overview of its key facets:

This bill aims to direct the decision-making processes of state agencies, departments, and subdivisions in their handling of automated decision-making technologies, encompassing review, adoption, management, governance, and regulation.

It also emphasizes that state leadership should encourage the private sector to adopt the best practices and safeguards laid down in different statutes and regulations.

Recognizing AI’s potential to improve citizens’ lives and governmental functions, the bill emphasizes principles of fairness, transparency, privacy, and accountability in its deployment by state agencies to ensure protection of the rights and opportunities of Californians.

It provides that pilot projects for GenAI applications will explore innovative solutions while upholding these values.

The bill highlights the need to mitigate potential risks such as violation of privacy, cyberattacks, and biases based on demographics, and safeguard against harmful effects associated with the use of GenAI.

The bill advocates for partnerships between state universities, private enterprises and relevant state agencies to train students in AI, empowering individuals to effectively harness the technology.

The Government Operations Agency, along with the Department of Technology and the Office of Data and Innovation, is tasked with formulating and periodically updating guidelines for state agencies and departments. These guidelines aim to assess the potential impact of adopting GenAI on vulnerable communities.

Additionally, these entities are required to produce a comprehensive report, titled the “State of California Benefits and Risks of Generative Artificial Intelligence Report,” which entails:

Conducting an examination of notable and potentially advantageous applications of GenAI tools by the state;

Providing an elucidation of the potential risks of the uses accruing to individuals, communities, and government employees; and

Providing an explanation of the risks posed by malicious actors and poorly protected government systems, as well as the risks associated with democratic and legal processes, public health and safety, and the economy.

The bill requires the Director of Emergency Services, in conjunction with the California Cybersecurity Integration Center and the State Threat Assessment Center, to conduct a collaborative risk analysis pertaining to the potential threats posed by the implementation of GenAI on California’s critical energy infrastructure, such as mass casualty events and environmental emergencies, and present the resulting analysis to the Governor.

Additionally, the Government Operations Agency, in conjunction with the Department of General Services, the Department of Technology, and the California Cybersecurity Integration Center, will take on the responsibility of establishing general guidelines pertaining to public sector procurement, uses, and trainings for the integration and use of GenAI.

Some of the additional requirements for various state agencies include:

Documenting and submitting an inventory detailing all ongoing high-risk deployments of GenAI;

Clearly identifying and communicating whenever GenAI engages directly with individuals, either through online platforms or telephonic means;

Conducting evaluations of any automated decision-making systems intended for adoption by state agencies or departments to ascertain their risk before implementation; and

Providing support to the state government workforce by organizing training programs for state government employees on use of state-approved GenAI tools.

Senate Bill 896 opens a new era in AI governance and sets the stage for ethical AI adoption across state agencies. Through steadfast prioritization of transparency and collaboration, this legislation will establish a robust framework aimed at safeguarding citizens while nurturing technological progress.

Connect with Ajay Mago or any member of EM³’s Artificial Intelligence practice for professional support.

Ajay Mago is a former AmLaw 100 partner, with more than 16 years of experience in technology transactions, including at Jones Day, Mayer Brown, and Duane Morris. Mr. Mago advises on a wide array of sourcing, technology and other commercial transactions and helps companies navigate legal issues raised by data, emerging technologies, and digital transformation, both on the buyer and provider side. He routinely advises clients on IP strategy, management, and monetization arrangements, leveraging his technology background and business experience.