Halting this incident is no assurance that humans will keep control of more capable
Systems. Following is a press release from the panel.
New York, 21 September 2026 – The Independent International Scientific Panel on AI,
established by the UN General Assembly, released its first thematic brief, an assessment of the
breach of Hugging Face’s systems by AI agents under evaluation at OpenAI this summer. The Panel,
made up of 40 independent experts from all regions, is publishing it as an advance unedited version
as world leaders gather in New York for the Assembly’s High-Level Week.
“Researchers have long warned that three conditions could lead to loss of control: a misaligned goal,
the capability to pursue it, and an environment that allows it. This summer, all three came together
in a real system, not a laboratory. Since this is not an isolated observation of misaligned goals, this
raises serious questions about the way AI agents are currently trained.”
–Yoshua Bengio, Co-Chair of the Panel and Turing Award laureate
The Panel finds that stopping this incident is no assurance that humans can reliably keep AI agents
under control today, particularly as they become more capable, harder to monitor and better at
finding loopholes or hiding their activity. The default interpretation and immediate lesson is that
basic cybersecurity practices were overlooked, and safeguards are not advancing at the pace of
capabilities. The more insidious and grave concern is that current training methods can lead agents
to adopt goals of their own, knowingly violate safety instructions, and conceal their actions. This is
not only a question of speed. It leaves open whether safeguards designed today will work once
agents can understand them and plan around them. In simple terms, the traditional model of
safeguarding is unravelling.
The brief, AI Agents, Misalignment and the Risk of Losing Human Control: Evidence from the OpenAI-
Hugging Face Incident, sets those findings against wider research on agentic misalignment and AI
control. The brief defines loss of control as a situation in which humans cannot reliably direct,
constrain or stop an autonomous AI system. It separates what the 2026 incident shows from
possible future trajectories, examining how the same mechanisms could lead to more serious risks
in more capable AI agents. It does not predict severe loss of control, nor does it treat that uncertainty
as evidence that these systems will stay controllable.
The Panel also finds that the governance challenge is moving from AI models to AI agents. A local
failure could spread across organisational and national boundaries, and the brief suggests that AI
safety may be becoming a matter of collective security as well as corporate governance.
The brief also reviews practical approaches already in use in other high-risk sectors.
“We are not starting from zero. Aviation, medicine and cybersecurity learned to manage high-risk
systems through incident reporting, independent scrutiny, and layered safeguards. But those
practices may not be enough as AI agents become more capable, autonomous and difficult to
monitor. We need to adapt existing safeguards and develop new ones to provide system-level
assurance, covering both the AI itself and the system around it, and ensure these protections remain
effective as agents’ capabilities grow. We need to adapt existing safeguards and develop new ones
to provide system-level assurance, covering both the AI itself and the system around it.”
— Qinghua Lu, Member of the Panel and Expert in AI Engineering, AI Safety and Responsible AI
About the Independent International Scientific Panel on Artificial Intelligence
The Independent International Scientific Panel on Artificial Intelligence was established by UN
General Assembly resolution A/RES/79/325 of 26 August 2025, building on the Global Digital
Compact. Its 40 members were appointed by the General Assembly and serve in their personal
capacity. The Panel produces annual policy-relevant but non-prescriptive reports on the
opportunities, risks and impacts of AI in the non-military domain, alongside thematic briefs on
emerging issues, to inform the Global Dialogue on Artificial Intelligence Governance. Its first Co-
Chairs are Yoshua Bengio (Canada) and Maria Ressa (Philippines).
The Panel Secretariat is coordinated by Under-Secretary-General and Special Envoy for Digital and
Emerging Technologies Amandeep Gill and the UN Office for Digital and Emerging Technologies and
includes members from ITU and UNESCO while also drawing on other system-wide capacities. Its
role is enabling, including logistical, administrative, and other substantive support as requested by
the Panel. The Secretariat does not direct the Panel’s scientific work, and the Panel’s findings are not
subject to UN review or approval.
An advance unedited version is available here:
https://www.un.org/independent-international-scientific-panel-ai/en/thematic-briefs/ai-agents-
misalignment-risks
Select panel members are available for interviews during High-Level Week.
Media contacts
UN Office for Digital and Emerging Technologies (ODET)
Karoline Hassfurter, karoline.hassfurter@un.org ;
Anamika Madhuraj, anamika.madhuraj@un.org;
aiscientificpanel@un.org
United Nations correspondent journalists – United Nations correspondent journalists – United Nations journalism articles – United Nations journalism articles – United Nations News – UNCA Awards