Trustworthy Multi-LLM Agent Collaboration
- Reference number
- STP25-0161
- Project leader
- Raza, Shahid
- Start and end dates
- 260801-310731
- Amount granted
- 5 000 000 SEK
- Administrative organization
- RISE Research Institutes of Sweden, Stockholm
- Research area
- Information, Communication and Systems Technology
Summary
While multi-agent LLM systems have shown promise in complex reasoning and planning, current approaches are largely heuristic, difficult to systematically evaluate, and insufficiently address safety, robustness, and privacy concerns. This project aims to establish a principled foundation for trustworthy multi-agent collaboration, by designing systems in which agents can collaborate effectively while remaining safe, privacy-preserving, and accountable. The executive plan spans five stages: a) establishing a reproducible experimental framework for multi-agent collaboration, b) advancing learning-based coordination, c) enabling privacy-preserving collaboration without harming utility, d) incorporating defenses against poisoning and toolchain attacks, and e) building scalable, verifiable multi-agent ecosystems that support auditing, accountability, and robustness. Expected results include open and reproducible benchmarks for multi-LLM collaboration, novel algorithms and frameworks for constraint-aware and privacy-preserving coordination, systematic attack-defense evaluation suites, and scalable agent architectures with verifiable decision records. The project’s broader impact lies in defining a new standard for trustworthy multi-LLM agent systems. By unifying learning, planning, safety, and privacy within a single, reproducible framework, the project will contribute foundational methods applicable to advanced AI systems deployed in complex, high-stakes environments.
Popular science description
Scientists are currently exploring how groups of smart computer programs, called multi-agent systems, can work together to tackle hard problems, e.g., autonomous traffic management in modern intelligent cities. Right now, these systems often rely on rough tricks that are hard to test, and they raise worries about safety, privacy, and reliability. This project aims to build a solid, trustworthy foundation for how multiple AI agents can cooperate safely and effectively. The plan for this project has five key steps: 1) Create a clear, repeatable way to test and compare how agent teams collaborate. 2) Improve how agents learn to coordinate with each other, so teamwork is smoother and smarter. 3) Allow agents to work together privately without losing usefulness or performance. 4) Develop defenses against clever attacks that try to poison the system or tamper with the tools the agents use. 5) Build large, scalable systems that can be audited, checked for accountability, and verified to be robust. What the project hopes to achieve: a) Open, reproducible benchmarks for measuring how well multiple AI agents collaborate. b) New methods for coordination that respect privacy and work well under constraints. c) Standardized tests and tools to evaluate how well agents resist attacks. d) Scalable architectures with clear records of decisions, so people can audit what happened. Big picture impact: This work could set a new standard for trustworthy multi-AI systems. By blending learning, planning, safety, and privacy in a transparent, reproducible framework, it aims to provide reliable tools for real-world use—from critical decision-making to complex automated tasks—where safety and accountability are crucial.