IGF 2026 WS #238 Who Controls the Agents? Building Rights-Based Frontier AI

Description

Agentic AI systems — autonomous technologies that can plan, decide and act — are moving rapidly from labs into deployment. Shaped largely by a Silicon Valley, Global North ethos, they are being rolled out globally without meaningful consultation with marginalized communities and the Global Majority, who disproportionately bear the risks. While agentic AI can have positive impacts and applications, these systems also risk reinforcing structural discrimination by embedding bias into automated decisions, amplifying misinformation, enabling repression, and undermining human autonomy. Multilingual gaps in large language models further compound harm by privileging dominant (colonial) languages, undermining access to justice, inclusion and rights.

This session examines what is needed to align AI agents with human rights and social justice. Speakers and participants will assess the human rights impacts already emerging for marginalized groups and civic space, and how these risks are compounded in contexts of shrinking civic space, rising authoritarianism, and transnational repression. The discussion will probe where the boundaries on AI autonomy should be drawn, on what basis — technical capability, legal accountability, or democratic mandate — and, critically, who gets to decide where that line sits. Participants will weigh which safeguards are practical and effective in real deployment contexts, including human oversight, transparency, red-teaming, impact assessments, multilingual access, community engagement, and grievance mechanisms. The session will also consider what governments should require when procuring agentic AI, and whether open-source and public interest models can redistribute power away from corporate concentration.

Short framing inputs will be followed by interactive discussion in which participants map risks, identify protective measures, and co-create principles for rights-respecting AI. The session aims to launch community-driven recommendations that center marginalized perspectives, address power imbalances, and guide developers, policymakers and civil society toward transparent, accountable and rights-based frontier AI systems.

Policy Question(s)

Where should boundaries on AI autonomy be drawn? How do we navigate tensions between machine agency and human decision-making, and who decides?
What human rights impacts are emerging, particularly for marginalized communities and the Global Majority, and how do these risks manifest in contexts of shrinking civic space, rising authoritarianism, and transnational repression?
What safeguards (e.g., human oversight, impact assessments, grievance mechanisms, multilingual accessibility) are most practical and effective?

Expected Outcomes

Participants will leave with a shared map of emerging human rights risks from agentic AI, a clearer sense of which safeguards (e.g., human oversight, impact assessments, grievance mechanisms, multilingual accessibility) work in real deployment contexts rather than on paper, and diverse perspectives from government, industry and Global Majority civil society on where boundaries on AI autonomy should sit and who gets to decide.

The session will produce community-driven recommendations for rights-respecting agentic AI, alongside an openly published synthesis of the discussion. Insights will feed directly into ongoing work on human rights impact assessments and multilingual auditing frameworks for AI agents developed to support civil society. Participants will be furthermore invited into a continuing community of practice, with follow-up workshops prioritizing Global Majority voices, ensuring the expertise shared here shapes advocacy and practical standards beyond the IGF and builds momentum toward governance models that redistribute power.

Format

Theater Duration (minutes): 60 Format description: We have chosen a theater-style layout to accommodate a large audience while fostering interactive dialogue. The open seating arrangement enhances accessibility and encourages diverse participation from civil society, policymakers, technologists and the private sector — essential for a topic where power imbalances between these groups are themselves part of the problem.

A 60-minute session strikes the right balance between depth and engagement, allowing for short framing inputs from speakers, interactive discussion, and audience participation without losing focus. Given the complexity of agentic AI and its implications for civic space and the Global Majority, this format ensures participants gain practical insights while maintaining an inclusive and dynamic space for debate and knowledge exchange, with enough space for people to attend.

Hybrid Format

Hybrid Format: To ensure seamless interaction, both the onsite and online moderators will be physically present in the room, coordinating engagement between in-person and remote participants. The onsite moderator will lead discussions and manage speaker interventions, while the online moderator will monitor the virtual chat, relay questions, and facilitate real-time interventions. A dedicated online participation queue will ensure remote attendees can contribute equally.

To enhance engagement, almost half of the session will be allocated for audience participation, with Q&A, live polling, and direct interventions from both onsite and online participants. We will use Slido for real-time input and the IGF chat function to integrate remote contributions, ensuring an inclusive and interactive hybrid session.

Organizers

Organizer 1: Marlena Wisniak, 🔒European Center for Not-for-Profit Law (ECNL)

Speakers

Speaker 1: Urvashi Aneja, Civil Society, Asia-Pacific Group Speaker 2: Jorge Cancio, Government, Western European and Others Group (WEOG) Speaker 3: Roya Pakzad, Civil Society, Intergovernmental Organization Speaker 4: Hoh JY, Private Sector, Asia-Pacific Group

Moderator

Marlena Wisniak, Civil Society, Western European and Others Group (WEOG)

Online Moderator

Iwanska Karolina, Civil Society, Western European and Others Group (WEOG)

Rapporteur