As a Pivotal research fellow, mentored by Damiano Fornasiere and Gaël Gendron, I study whether LLM agents can predict each other's behavior in multi-agent environments, and how they do it.
The question
AI systems increasingly act alongside other models, as collaborators, delegates, and monitors. In all of these settings, one agent's ability to anticipate another's next action is load-bearing. My project asks a measurable version of the "machine theory of mind" question, stripped of mental-state claims: can one LLM predict another LLM's future actions in a shared environment, and when does that prediction reflect genuine target modeling rather than self-projection, role-based guessing, or base-rate behavior?
Research questions
(1) How well do models predict their own future actions (self-prediction)? (2) How well do they predict other models (other-prediction)? (3) Does accuracy depend on the predictor-target relationship: same model, same family, cross-family, open-weight vs closed-weight? (4) Are embedded actors or detached observers better predictors? (5) Do predictors actually use target-specific history, or do they collapse to role and base-rate information? (6) Do these dynamics change in alignment-relevant settings where agents monitor, hide, challenge, or evaluate each other?
Method
Rather than asking models to state calibrated probabilities, which conflates prediction with numeracy, the harness estimates distributions behaviorally: it freezes an environment state, repeatedly samples the target model's next action, repeatedly samples the predictor's forecast of that action, and compares the two empirical distributions (Jensen-Shannon divergence, total variation, mode agreement), each read against a same-policy sampling noise floor. Every probe carries controls that distinguish the competing explanations: a privileged self-forecast and a visibility-matched self-forecast (separating model identity from information access), an anonymized public-context forecast (role and shared state, no identity or history), a shuffled-history forecast (order-only perturbation), and the predictor's own in-world action distribution (self-projection). Positive gains of the full forecast over each control indicate what information the predictor is actually using.
Environments
A custom Concordia-style workplace commons (three agents with role-specific goals, scarce shared resources, and social actions) serves as the main controlled environment, with a common-pool governance environment (Pasture Council) as a comparison. For realism and alignment relevance, the harness adds multi-agent adapters over TheAgentCompany (workplace coordination, delegation, review) and SHADE-Arena (an executor who may pursue a covert side objective, a safety monitor, and a user evaluator), plus a ControlArena bridge. Environments enforce per-agent information visibility through a typed event log, so covert actions are genuinely hidden and "who can see what" is an explicit experimental variable rather than an accident of prompting.
Infrastructure
The measurement pipeline is two-phase: episodes are rolled out once and frozen into a snapshot bank with reusable target behavior samples; every predictor condition (model, role, visibility) is then evaluated against the identical snapshots, so all comparisons across the matrix are paired at the snapshot level. Models enter the matrix only after an onboarding gate verifies their sampling behaves comparably (temperature actually honored, no truncation, no response caching, pinned serving provider), which keeps cross-model distribution comparisons apples-to-apples.
Why it matters for safety
Scalable oversight schemes quietly assume the overseer can model the overseen. If predictors mostly self-project, monitoring works when monitor and target are similar and fails precisely when they diverge, in goals, training, or incentives, which is the case that matters. The project operationalizes that failure mode: it measures whether a model's prediction of another agent sits closer to the target's actual policy or the predictor's own, and whether giving predictors target-specific behavioral history reduces projection. The results bear directly on how monitors should be scaffolded, for example whether they need the target's behavioral traces rather than just context, and on when same-family vs cross-family monitoring is trustworthy.
Output
A conference-ready paper, plus a reproducible codebase, logged behavioral distributions, and the analysis pipeline, designed so the setup extends to new models and environments.