CA gov signs executive order to prevent 'mass casualty events and environmental emergencies' by AI

California Governor Gavin Newsom (D) signed an executive order on Wednesday laying out a plan to regulate the rapidly accelerating advancement of artificial intelligence, which scores of scientists and industry leaders have warned poses an imminent threat to human civilization.

"This is a potentially transformative technology – comparable to the advent of the internet – and we're only scratching the surface of understanding what GenAI is capable of," Newsom said. "We recognize both the potential benefits and risks these tools enable. We're neither frozen by the fears nor hypnotized by the upside. We're taking a clear-eyed, humble approach to this world-changing technology. Asking questions. Seeking answers from experts. Focused on shaping the future of ethical, transparent, and trustworthy AI. Doing what California always does – leading the world in technological progress."

Newsom's office said in a press release that he seeks to maintain the Golden State as the "global hub" for developments across several economic sectors, including "education, innovation, research, development, talent, entrepreneurship, and new technologies."

POLL: Should Trump be allowed to hold office again?

EO N-12-23 states:

Within 60 days of issuance of this Order, the Government Operations Agency, the California Department of Technology, the Office of Data and Innovation, and the Governor's Office of Business and Economic Development, in collaboration with other State agencies and departments and their workforce, shall draft a report to the Governor examining the most significant, potentially beneficial use cases for deployment of GenAI tools by the State. The report shall also explain the potential risks to individuals, communities, and government and state government workers, with a focus on high-risk use cases, such as where GenAI is used to make a consequential decision affecting access to essential goods and services. Additionally, the report shall include but not be limited to: risks stemming from bad actors and insufficiently guarded governmental systems, unintended or emergent effects, and potential risks toward democratic and legal processes, public health and safety, and the economy. The report shall be regularly assessed for any significant developments or necessary updates and as appropriate, be done in consultation with civil society, academia, industry experts, and the state government workforce or organizations that represent state government employees.

It continues:

No later than March 2024, the California Cybersecurity Integration Center and the California State Threat Assessment Center, both established within the Governor's Office of Emergency Services, and inclusive of the California Department of Technology, the California Military Department, and the California Highway Patrol, shall perform a joint risk analysis of potential threats to and vulnerabilities of California's critical energy infrastructure by the use of GenAI, including those which could lead to mass casualty events and environmental emergencies, and develop, in consultation with external experts as appropriate from civil society, academia, and industry, a strategy to assess similar potential threats to other critical infrastructure. Once this analysis is completed, these agencies shall provide a classified briefing to the Governor and, where appropriate and without divulging classified information, make public recommendations for further administrative actions and/or collaboration with the Legislature to guard against these potential threats and vulnerabilities. These recommendations shall address how to ensure systems are regularly tested and monitored to detect and avoid unintended behavior, and how to ensure they remain under effective human control. At a cadence deemed appropriate by the Governor's Office of Emergency Services, the analysis and public recommendations should be updated to reflect changes to the technology, its applications, and risk management processes and learnings.

Newsom's directive also instructs state agencies to explore "procurement and enterprise use opportunities where GenAI can improve the efficiency, effectiveness, accessibility, and equity of government operations" while considering "relevant stakeholders, including historically vulnerable and marginalized communities, and organizations that represent state government employees, in the development of any guidelines, criteria, reports, and/or training as directed by this Order."

READ MORE: Scientists and tech leaders sign open letter to prioritize 'mitigating the risk of extinction from AI'