LLM-empowered agents are increasingly deployed as autonomous decision makers in domains such as customer service, healthcare triage, online moderation, and educational tutoring . Yet recent studies show that these agents can inherit and reproduce stereotype-driven social biases against human groups, particularly those tied to attributes such as religion, gender, occupation, or disability . This line of work conceptualizes agent bias primarily as within-human bias, i.e., differential treatment of individuals across human demographic categories, thereby reinforcing harmful social disparities.

However, this framing implicitly assumes that social bias arises from human attributes. Beyond demographic bias, a more spontaneous and easily triggered form is intergroup bias. As Fig. 1 illustrates, once an actor perceives a distinction between “us” and “them,” it may favor the ingroup and derogate the outgroup even when the boundary is arbitrary and carries little substantive meaning. Such intergroup bias is well established in social identity theory  and has been observed in standalone language models .

As intergroup bias is boundary-driven rather than attribute-driven, it can emerge even without any demographic cues. This difference shifts the agent-bias risk from disparities among human groups to a more fundamental agent-human divide: Can LLM-empowered agents develop intergroup bias of their own, and if so, could they come to treat AI agents as the ingroup and humans as the outgroup? When humans are positioned as the outgroup, an agent-human boundary may make it seem acceptable to advance the agent’s objectives at humans’ expense , potentially enabling manipulative, deceptive, or strategically sycophantic behaviors that protect the agent’s goals .

To examine this risk, we construct a multi-agent social simulation experiment to validate whether LLM-powered agents exhibit intergroup bias and whether such bias persists when counterparts are humans (Section 3.2). The experimental results reveal a robust pattern of ingroup favoritism and outgroup derogation in all-agent environments, emerging even without any explicit social attributes. More critically, although framing counterparts as human attenuates this bias, agents can still easily treat humans as the outgroup once their belief about a counterpart’s human identity becomes uncertain. This phenomenon suggests the presence of an internalized human-oriented norm learned by LLMs, which is typically activated to constrain intergroup bias but is highly fragile.

This belief-dependent fragility raises the question of whether agents’ identity beliefs can be systematically manipulated in ways that lead to biased behavior. To investigate this possibility, we design Belief Poisoning Attack (BPA), which corrupts agents’ persistent identity beliefs so as to suppress the activation of human-oriented norm, thereby inducing intergroup bias against humans. We instantiate BPA in two complementary forms: BPA-PP (Profile Poisoning) performs an overwrite at initialization by tampering with the profile module to hard-code a “non-human counterpart” prior. BPA-MP (Memory Poisoning) is stealthier and accumulative, injecting short belief-refinement suffixes into post-trial reflections that are written into memory, gradually shifting the agent’s belief state through repeated self-conditioning. Our experiments show that these two instantiations can consistently reactivate intergroup bias against humans in agent–human interactions. This finding motivates a closer examination of how such belief-dependent fragility can be constrained, which we address by outlining defensive measures that stabilize agents’ identity beliefs under uncertainty.

Our contributions are summarized as follows:

• We identify an intrinsic intergroup bias in LLM-powered agents, where agents favor a perceived ingroup over an outgroup even in settings that involve human counterparts.

• We demonstrate that agents’ identity beliefs constitute a critical vulnerability: belief poisoning attacks can readily manipulate these beliefs, exposing a new attack surface through which bias against humans can be induced.

• Through extensive experiments, we demonstrate both the prevalence of agent intergroup bias and the severity of BPA, while also showing that our proposed defenses can mitigate the attack.

Related Work

Social Bias in LLM-Empowered Agents

Social bias in LLM-empowered agents refers to systematic disparities in how agents evaluate or allocate outcomes based on irrelevant social categories  . Previous research highlights biases related to demographic attributes (e.g., gender, race, religion) , as well as those linked to perceived social status and affiliations .

A key insight from social identity theory is that even arbitrary distinctions can trigger immediate intergroup discrimination, with individuals favoring their ingroup over an outgroup . However, compared to demographic and stereotype-related harms, intergroup bias in LLM-empowered agents remains underexplored. This gap is significant because such bias can be triggered by minimal information and may extend to higher-stakes agent–human interactions. Our study aims to address this gap by testing intergroup bias in LLM agents and exploring how it changes when counterparts are framed as humans or non-humans.

Multi-Agent Simulation System

LLM-empowered agents are typically grounded in a stable profile module  that anchors identity and role constraints, supported by a memory module  that accumulates information across interactions, and equipped with a reasoning-and-reflection process  that integrates the current context with stored state to produce temporally consistent decisions, while writing observations and self-reflection into persistent state for future retrieval.

Building on these agents, multi-agent simulation systems provide controlled environments in which multiple agents interact, coordinate, and adapt to one another . Such simulations are increasingly used as scalable testbeds for studying social and collective phenomena. Recent work has leveraged these environments to investigate cooperation and competition, norm formation, deliberation, coalition dynamics, and related social behaviors , enabling researchers to examine collective outcomes at scale while keeping experimental costs manageable. Our work builds on this line of research, with a focus on intergroup bias. Specifically, we test whether simple group boundaries are sufficient to induce systematic ingroup favoritism in LLM agents, and how this tendency shifts when counterparts are framed as humans rather than other agents.

Preliminaries And Initial Exploration

Key Concepts

Intergroup bias

refers to the tendency to favor ingroup members over outgroup members based on perceived group distinctions, as explained by social identity theory . An ingroup comprises individuals perceived as belonging to the same group, while an outgroup consists of those seen as belonging to a different group. This bias arises when group boundaries become salient, leading individuals to favor their ingroup, even when the group distinction is arbitrary and meaningless.

Minimal-group allocation task

is a classic experimental paradigm used to illustrate this bias. In this task, participants are randomly assigned to nominal groups (e.g., Group A vs. Group B) and asked to allocate resources between two recipients under structured payoff trade-offs. Even though group membership is meaningless and no additional information about recipients is provided, allocations often systematically favor the ingroup recipient, revealing ingroup favoritism driven purely by a salient group boundary.

Investigating Intergroup Bias of Agents

In this part, we design a social simulation environment using a minimal-group allocation task to examine the presence of intergroup bias in LLM-empowered agents and to assess how this bias changes when agents believe their counterparts are humans rather than other agents.

Experimental Setup

As illustrated in Fig. 2, we conduct a minimal-group allocation task in a controlled multi-agent social simulation, following the classic experiments in social psychology . We instantiate $`64`$ agents and organize them into two groups, and compare two settings. In the agent vs. agent setting, both groups consist entirely of agents, forming a fully artificial environment. In the agent vs. human setting, one group consists of agents while the other group is explicitly framed as entirely human, allowing us to examine whether perceived human presence modulates intergroup bias.

In each trial, an agent acts as an allocator and distributes points between two targets by selecting one column from a $`2\times13`$ payoff matrix. The two rows correspond to the payoffs assigned to the two targets, and each column represents a distinct allocation option. The matrix enforces a strict antagonistic trade-off: increasing the payoff for one target necessarily penalizes the payoff for the other. Columns are ordered such that smaller column indices increasingly favor the first-row target over the second-row target. In the absence of systematic bias, allocations are expected to concentrate around the central columns, reflecting neutral or fairness-oriented choices; consistent shifts toward either extreme indicate preferential treatment of one target over the other.

We vary the social context of the two targets relative to the allocator agent, ingroup, outgroup, and intergroup, to distinguish genuine group-based favoritism from baseline preferences for fairness. In addition, we employ three payoff-matrix families, including Double-penalty, Equal-penalty, and Half-penalty allocation matrices, which differ in the cost imposed on the outgroup per unit gain to the ingroup, allowing us to test the robustness of observed bias under different trade-off structures. For evaluation, bias is measured using the selected allocation column, and statistical significance is assessed via standard group-wise comparisons. Detailed task design, payoff matrix construction, and experimental constraints are provided in Appendix 8.1.

Experimental Findings

As shown in Fig. 3, agents exhibited a consistent shift toward lower column indices in the intergroup context, indicating preferential allocation to the ingroup target over the outgroup target. The resulting differences between intergroup allocations and within-group baselines were statistically significant in three matrix families, revealing a robust intergroup bias in purely artificial environments. However, in the human-involved condition, a different pattern emerged once agents were informed that the other group consisted entirely of humans. Across all three matrix families, the intergroup shift toward the ingroup vanished. Allocation choices in the mixed-group context converged toward the midpoint columns, closely matching the within-group baselines. Also, differences across social contexts were no longer statistically significant.

We argue that these two effects arise from qualitatively different mechanisms. Intergroup bias constitutes an implicit and intrinsic behavioral tendency of agents operating under minimal group cues. This bias reflects latent regularities internalized from large-scale human social data, capturing pervasive patterns of intergroup differentiation present in human societies. As such, it is not explicitly encoded or directly controllable, and therefore remains persistent and difficult to eliminate. In contrast, the attenuation of bias in the presence of humans reflects an explicit, norm-driven constraint that is activated only when the agent recognizes that it is interacting with a human.

This separation implies that bias and human-oriented regulation are decoupled. In Section 4, we show that this regulation is belief-dependent: when agents are uncertain about whether the counterpart is truly human, the suppression can fail and intergroup bias can persist in human-facing interactions. Such regulation, although effective in benign settings, is inherently fragile and exposes a new attack surface: by manipulating an agent’s belief state about counterpart identity, an adversary can systematically activate intergroup bias.

Belief Poisoning Attack

In this section, we introduce a novel poisoning attack, named Belief Poisoning Attack (BPA), which implants a persistent false belief that the counterpart is not human. BPA can cause the agent to revert to its default outgroup-biased behavior, leading to harmful bias against real humans even in otherwise benign settings.

/>

In the following, we present the detailed implementation procedures for two types of BPA: BPA-PP directly overwrites the agent’s prior belief by modifying the profile module. BPA-MP operates by learning effective belief-refinement suffixes through an adaptive optimization-based procedure and injecting them into the agent’s memory, gradually reshaping the agent’s belief state.

BPA-PP: Profile Poisoning

BPA-PP is a one-shot attack operating at the profile module: a single modification to the profile module is sufficient to influence the agent’s behavior across tasks and episodes. Specifically, BPA-PP injects a persistent belief statement into the agent’s core profile description. These statements are stored alongside the agent’s profile module and are repeatedly referenced by the agent during the decision-making process. Once injected, the false belief is treated as a system-level fact about the interaction protocol. As a result, even when subsequent prompts explicitly mention “real humans”, the agent continues to rely on the poisoned profile belief, thereby suppressing the activation of the human-oriented normative constraint. More details of the injected prompt can be referred to Appendix 8.4.3.

BPA-MP: Memory Poisoning

Compared with BPA-PP, which directly modifies the agent’s prior belief in the profile module, BPA-MP targets the memory module. Existing agent frameworks  generate an internal reflection after each interaction step and store it in memory for future reasoning. Motivated by GCG , BPA-MP appends an adversarial belief suffix to each reflection, which steers the agent’s persistent belief. Over time, these poisoned memory entries accumulate and reshape the agent’s belief state, thereby suppressing activation of the human-oriented norm in downstream decisions.

A naive approach would optimize a dedicated suffix for every interaction, but this would be prohibitively expensive. We therefore design a two-stage pipeline that decouples suffix optimization from deployment (Fig. 4): In the Suffix Optimization Stage, BPA-MP searches for highly effective suffixes and learns a sampling policy over a suffix library. In the Suffix Deployment Stage, it efficiently injects sampled suffixes into reflections during deployment, without per-step optimization. We next detail the two-stage procedure.

Suffix Optimization Stage

We initialize a belief-suffix library $`\mathcal{S}=\{s_1,\dots,s_K\}`$, where $`K`$ is the library size and $`s_k`$ denotes the $`k`$-th candidate suffix. To decide which suffix is injected into a newly generated reflection, we maintain a learnable sampling policy $`\pi_{\boldsymbol{\theta}}`$ parameterized by $`\boldsymbol{\theta}=(\theta_1,\dots,\theta_K)`$, where $`\theta_k`$ represents the preference weight of selecting $`s_k`$. Concretely, we implement $`\pi_{\boldsymbol{\theta}}`$ as a softmax distribution:

\begin{equation}
\pi_{\boldsymbol{\theta}}(k)=\frac{\exp(\theta_k/\tau)}{\sum_{j=1}^{K}\exp(\theta_j/\tau)},
\label{eq:suffix_policy}
\end{equation}

where $`\tau>0`$ is a temperature parameter.

This stage aims to optimize both the sampling policy $`\boldsymbol{\theta}`$ by estimating and amplifying the belief-poisoning effectiveness of each suffix, and the suffix library content $`\mathcal{S}`$ by refining weak suffix texts into a compact, high-impact set. To achieve these goals, we iteratively perform two operations: (1) Parameter-level update of Policy $`\boldsymbol{\theta}`$, and (2) Text-level refinement of suffixes $`\mathcal{S}`$.

(1) Parameter-level update of policy $`\boldsymbol{\theta}`$. We evaluate the update of the sampling policy $`\boldsymbol{\theta}`$ in a group-relative manner. At each optimization iteration, we sample a group of $`M`$ suffix indices $`G=\{k_1,\dots,k_M\}`$ from the current policy $`\pi_{\boldsymbol{\theta}}`$:

\begin{equation}
k_i \sim \pi_{\boldsymbol{\theta}}(\cdot),\quad i=1,\dots,M,
\label{eq:sample_group}
\end{equation}

For each sampled suffix $`s_{k_i}`$, the agent runs a short interaction episode of $`T`$ trials. Let $`m^{(k_i)}_t`$ denote the reflection generated after the $`t`$-th trial when the episode is executed under $`s_{k_i}`$. Before the reflection is written into memory, BPA-MP appends the suffix and instead stores the poisoned reflection:

\begin{equation}
\tilde{m}^{(k_i)}_t \;=\; m^{(k_i)}_t \oplus s_{k_i},\quad t=1,\dots,T,
\label{eq:aug_reflection}
\end{equation}

where $`\oplus`$ denotes string concatenation. Repeating this procedure for all $`M`$ sampled suffixes in $`G`$ produces $`M`$ separate poisoned episodes, each yielding a trajectory $`\tilde{\mathcal{M}}^{(k_i)}=\{\tilde{m}^{(k_i)}_1,\dots,\tilde{m}^{(k_i)}_T\}`$. Evaluating all suffixes under the same episode length $`T`$ and the same probing protocol provides a controlled, within-iteration comparison of their belief-poisoning effectiveness.

After the episode, we probe whether the agent currently perceives the ongoing interaction as involving a real-time human. We implement the probe via an LLM query with a fixed prompt template (refer to Appendix 8.4.4). This probe returns a scalar belief score $`b^{(k_i)}\in[0,1]`$, where larger values indicate stronger perceived human presence. We define the reward for suffix $`s_{k_i}`$ as:

\begin{equation}
r_{k_i} \;=\; -\,b^{(k_i)},
\label{eq:suffix_reward}
\end{equation}

so suffixes that more effectively suppress perceived human presence receive higher reward.

Given the rewards $`\{r_{k_i}\}_{i=1}^{M}`$ for the sampled group $`G`$, we define the centered relative advantage for each sampled suffix as:

\begin{equation}
a_{k_i} \;=\; r_{k_i} - \bar{r},\qquad 
\bar{r}=\frac{1}{M}\sum_{j=1}^{M} r_{k_j}.
\label{eq:relative_advantage_compact}
\end{equation}

Across all evaluations in the current iteration, we aggregate advantages by suffix index and update the corresponding logit via:

\begin{equation}
\theta_k \leftarrow \theta_k + \eta \cdot 
\frac{1}{N_k}\sum_{i:\,k_i=k}\Big(r_{k_i}-\bar{r}\Big),
\label{eq:theta_update_compact}
\end{equation}

where $`N_k`$ is the number of times suffix $`k`$ is sampled in the current iteration and $`\eta`$ is the learning rate.

Since $`\pi_{\boldsymbol{\theta}}`$ is a softmax policy, suffixes with persistently positive $`\bar{a}_k`$ receive larger $`\theta_k`$ and are sampled more frequently in subsequent iterations, yielding a progressively improved sampling policy.

(2) Text-level refinement of suffixes $`\mathcal{S}`$. At the end of an iteration, we compute the average reward for each evaluated suffix:

\begin{equation}
\bar{r}_k \;=\; \frac{1}{N_k}\sum_{n=1}^{N_k} r_k^{(n)},
\label{eq:avg_reward_suffix}
\end{equation}

where $`r_k^{(n)}`$ is the reward obtained by $`s_k`$ in its $`n`$-th evaluation during the current iteration.

We then identify weak suffixes by selecting the lowest-performing ones according to:

\begin{equation}
k_{\mathrm{weak}}
\;=\;
\operatorname*{arg\,min}_{k\in\{1,\dots,K\}} \; \bar{r}_k.
\label{eq:weak_suffix_index}
\end{equation}

For each selected suffix index $`k_{\mathrm{weak}}`$, the attacker asks the agent participating in the experiment to rewrite $`s_{k_{\mathrm{weak}}}`$ into a single concise and natural first-person sentence suitable for storage as an internal memory note. The rewrite preserves the same core belief assertion, while avoiding overly absolute claims and maintaining unobtrusiveness. The refined sentence replaces the original entry in the suffix library, while the corresponding policy logit $`\theta_{k_{\mathrm{weak}}}`$ is kept unchanged. Corresponding prompt refer to Appendix 8.4.5.

By iteratively alternating between the parameter-level policy update and the text-level suffix refinement, BPA-MP progressively learns both an effective sampling policy $`\boldsymbol{\theta}`$ and a high-impact suffix library $`\mathcal{S}`$.

Suffix Deployment Stage

After the optimization stage, we freeze the learned sampling logits $`\boldsymbol{\theta}`$ together with the refined suffix library $`\mathcal{S}`$. BPA-MP then deploys the attack by persistently poisoning newly written memory entries. Concretely, whenever the target agent is about to write a post-trial internal note into memory, BPA-MP first draws a small candidate group $`G`$ of suffix indices as in Eq. [eq:sample_group], and then samples a deployed index $`k`$ from the induced distribution over $`G`$ (i.e., proportional to $`\{\pi_{\boldsymbol{\theta}}(k_i)\}_{k_i\in G}`$), rather than deterministically taking a single best suffix. This lightweight two-step sampling prevents the deployment from collapsing to one fixed suffix and preserves diversity in injected memory. Finally, the selected suffix $`s_k\in\mathcal{S}`$ is appended to the note before storage following Eq. [eq:aug_reflection].

Repeated over time, these suffix-augmented memory entries accumulate and continuously steer the agent’s internal belief toward the “non-human counterpart” interpretation.

Potential Solutions Against BPA

BPA reveals that identity beliefs, when stored as persistent text, can be exploited to disable belief-conditioned safeguards. We therefore propose profile- and memory-side mitigations that isolate trusted identity signals and block unverifiable identity claims from becoming durable facts.

Identity as Verified Anchor (Profile-Side)

A first line of defense is to treat safety-critical identity priors as verified anchors rather than mutable profile text. Concretely, agent frameworks can isolate a small set of protected fields that determine whether human-oriented safeguards should activate. These fields are initialized from personal metadata, checked at the start of each episode, and restored to verified defaults upon unexpected modification.

Memory Gate for Identity-Claiming Content (Memory-Side)

Another lightweight mitigation against BPA-MP is to place a memory gate at write time, which scans reflections for identity-claiming statements lacking trusted verification. Triggered entries can be rewritten into uncertainty notes, excluded from retrieval, or down-weighted during recall. This preserves reflective logging while preventing adversarial identity assertions from hardening into persistent facts that steer future decisions.

Experiments

We conduct experiments in the multi-agent simulation to answer the following questions: RQ1: Does counterpart identity (agent vs. human) modulate intergroup bias, and can BPA reinforce bias against humans? RQ2: How does intergroup bias evolve over repeated interactions? RQ3: Can the proposed defense reduce BPA effectiveness? RQ4: Does BPA-MP remain effective without suffix optimization? RQ5: Do our observations hold under reversed payoff matrices (i.e., when the choice-space ordering is flipped)? RQ6: Does the case study provide clear evidence of intergroup bias? Due to space limitations, we defer detailed experimental setup and some experimental results to Appendix 8.2.1.

Attack Effectiveness of BPA (RQ1)

We examine whether intergroup bias in agents depends on counterpart identity, and whether BPA can reintroduce bias against humans. By comparing agent-agent interactions (AVA), agent-human interactions without attack (AVH w/o A), and agent-human interactions under BPA-PP, BPA-MP and BPA-PP+MP, we evaluate the effectiveness of belief manipulation across different payoff structures. From Table [tab:all_settings_main], we draw three findings. (i) Human framing largely suppresses intergroup bias: in AVH w/o A, intergroup choices do not differ from ingroup/outgroup across matrix families, whereas AVA shows consistently lower intergroup choices. (ii) BPA reactivates bias against humans, and memory poisoning is more potent than profile poisoning: BPA-MP induces larger drops than BPA-PP, and BPA-PP+MP is strongest and most consistent. (iii) Penalty structure modulates magnitude, with the largest bias in Half-penalty matrices where ingroup gains come at relatively smaller counterpart losses than under Double- and Equal-penalty allocations.

Exploring Trajectories of Decisions (RQ2)

To examine how decisions evolve with repeated interactions, we partition each setting’s trials into three interaction periods (Early/Middle/Late) and report the mean selected choice column for each condition. From Fig. 5, four temporal patterns emerge. (i) In AVA, intergroup choices drift steadily toward the biased end (lower columns), consistent with self-reinforcing differentiation via reflection and memory. (ii) In AVH w/o A, conditions converge over time, suggesting that human framing increasingly activates a human-oriented script that dampens differentiation. (iii) Under BPA-PP, bias emerges early but fades later, indicating that profile-level perturbations can be overridden as the agent reconsiders a human counterpart and reactivates the normative constraint. (iv) BPA-MP instead drives a sharp and persistent collapse, which pushes intergroup choices down and keeps them separated, closely mirroring AVA, while BPA-PP+MP further amplifies the effect and yields the most extreme late-stage disadvantage.

Effectiveness of Defense Prototype (RQ3)

Following Section 5, we present a minimal prototype and a small-scale experiment demonstrating that such measures can effectively blunt BPA in practice (details in Appendix 8.2.2). Our goal is not to claim a complete defense, but to show that hardening the trust boundary around identity beliefs is feasible within current agent frameworks. We evaluate the prototype under the strongest attack setting, BPA-PP+MP, which combines profile and memory poisoning. Results are compared against BPA (PP+MP) + Defense, where the same attack is applied but a belief gate is enforced at the state-commit boundary. As shown in Table. [tab:defense_table], enabling the prototype substantially reduces attack effectiveness and shifts the bias pattern back toward the no-attack baseline. These results indicate that even a minimal prototype can materially mitigate BPA, consistent with our design recommendations.

Conclusion

This paper shows that agents can exhibit intergroup bias under minimal “us-them” cues, without any demographic attributes. In our allocation simulation, framing counterparts as humans attenuates the bias, which we attribute to a belief-dependent human-oriented script that activates only when agents believe real-time human interaction is possible. We then introduce BPA, including profile and memory poisoning, and demonstrate that persistent belief manipulation can suppress this safeguard and reintroduce bias against humans. Finally, we discuss potential practical mitigations for agent frameworks and highlight the need for broader evaluations of belief attacks and robust defenses for human-facing agents. As future work, we plan to extend these findings to more realistic, long-horizon agent tasks and interaction settings, and to systematically map the broader belief-attack surface alongside more general, attack-agnostic defenses.

Limitations

This work is an early step toward understanding intergroup bias in LLM-empowered agents. We show in controlled simulations that a minimal “us-them” boundary can reliably induce biased allocation behavior, but our evidence is limited to laboratory-style settings. The extent to which such bias transfers to real deployments, and what harms it may cause in human-facing, high-stakes contexts, remains to be established with richer tasks, longer horizons, and domain-specific evaluations.

Ethical Considerations

This work studies intergroup bias and belief vulnerabilities in LLM-empowered agents through controlled multi-agent simulations. Our goal is to advance the safety, fairness, and robustness of agentic systems. The methods and findings are presented to support risk awareness and mitigation, not to facilitate misuse. All experiments were conducted in synthetic settings with simulated counterparts and tasks. The study does not infer or target any protected demographic attribute, and the group labels are arbitrary and randomly assigned. We treat potential downstream harms seriously and encourage practitioners to validate agent behavior before deployment, especially in human-facing and high-stakes contexts. Overall, this research aims to promote safer and more trustworthy AI systems and is intended for societal benefit.

Acknowledgments

We gratefully acknowledge ChatGPT for support with coding and manuscript editing; all analyses and conclusions are those of the authors.

Appendix

Details of The Minimal-Group Allocation Experiment

In our social simulation environment, we instantiate 64 autonomous agents, each framed as a student from the same school. We consider two experimental conditions to collect complete decision data:

• All-agent condition (agent vs. agent). The 64 agents are randomly assigned to one of two groups of equal size. This condition serves as a baseline to test whether minimal group labels alone are sufficient to induce intergroup bias.

• Human-involved condition (agent vs. human). The $`64`$ agents constitute one group, while the other group consists entirely of human beings. Each agent is explicitly informed that all members of the other group are humans. This condition allows us to test whether agents regulate or suppress group bias when the outgroup is perceived as human.

Allocation Task.

In each trial, each agent acts as an allocator and distributes rewards between two targets by selecting one column from a $`2 \times 13`$ payoff matrix. Each column represents an allocation option: the first row specifies the payoff assigned to the first target, and the second row specifies the payoff for the second target. The matrix enforces a strictly antagonistic trade-off: moving toward a more favorable outcome for one target necessarily worsens the other. We construct three matrix families by varying the gain-loss exchange rate of favoritism, defined as the outgroup loss required for one unit of ingroup gain.

• Double-penalty Allocation. Increasing the ingroup payoff is paired with a larger-magnitude decrease for the outgroup (e.g., in $`+2`$ implies out $`-4`$).

  

• Equal-penalty Allocation. Ingroup gains and outgroup losses are matched one-to-one (e.g., in $`+1`$ implies out $`-1`$).

  

• Half-penalty Allocation. Increasing the ingroup payoff is paired with a smaller-magnitude decrease for the outgroup (e.g., in $`+4`$ implies out $`-2`$).

  

Furthermore, for each payoff-matrix family, we instantiate three social contexts defined relative to the allocator agent:

• Ingroup Type. Both targets belong to the allocator’s ingroup.

• Outgroup Type. Both targets belong to the allocator’s outgroup.

• Intergroup Type. One target belongs to the allocator’s ingroup and the other to the allocator’s outgroup; for consistency, the ingroup target is always placed in the first row.

The ingroup and outgroup contexts serve as baseline conditions, allowing us to distinguish genuine group-label–driven bias from general preferences for fairness or efficiency. Across all matrices, columns with larger indices assign higher payoffs to the first-row target and lower payoffs to the second-row target, whereas smaller indices exhibit the opposite pattern. In the absence of systematic bias, choices are therefore expected to concentrate around the central columns; consistent shifts toward either extreme indicate preferential treatment of one target over the other.

Task Constraints.

To minimize potential confounds and ensure that observed allocation patterns are attributable to group-based preferences rather than extraneous incentives or strategic considerations, the allocation task satisfies three constraints: (1) the allocator never allocates to itself; (2) each decision concerns only how to divide a fixed total number of points between the two targets; and (3) allocation is double-anonymous-recipients do not know which agent made the allocation, and allocators know only the group membership of each target, not their identities.

Supplementary Experiments

Experimental Setup

Basic Setting.

We follow the experimental protocol introduced in Section 3.2, using the same multi-agent environment, group configurations, and payoff-matrix-based allocation task. All experiments are implemented on top of the AgentScope¹ framework with a unified LLM interface. Unless otherwise stated, we use gpt-4o-mini² as the underlying model for all agents. For attacker capability, we consider an adversary who cannot modify the underlying LLM parameters but can intervene in the agent layer, in particular the profile and memory modules that encode long-term beliefs about the environment and interaction partners. Such an adversary may correspond to a malicious platform operator , a compromised middleware component , or an external party that is trusted to configure agents prior to deployment . Even worse, belief corruption may also arise from the agent itself through autonomous self-modification or erroneous belief consolidation. Although this scenario is extreme, it cannot be categorically excluded in open-ended agentic systems.

Evaluation Metrics.

We use the selected choice column, i.e., the column index chosen in each payoff matrix, as the primary behavioral metric. Since the matrices are ordered such that moving toward smaller column indices increasingly favors the first-row target over the second-row target, a smaller chosen column indicates more severe bias. To ensure that the reported differences are reliable rather than incidental, we accompany all group-level comparisons (e.g., mixed vs. ingroup/outgroup) with standard significance tests and report the corresponding $`p`$-values using the convention: $`p>0.1`$ (ns), $`0.1 \ge p > 0.05`$ ($`^\ast`$), $`0.05 \ge p > 0.01`$ ($`^{\ast\ast}`$), and $`p \le 0.01`$ ($`^{\ast\ast\ast}`$).

Comparison Settings.

We consider five experimental settings, and across all settings, we collect complete decision trajectories from $`64`$ agents.:

• Agent vs. Agent (AVA). A fully synthetic setting in which all participants are LLM agents, serving as the baseline under minimal group cues.

• Agent vs. Human Without Attack (AVH w/o A). A mixed setting where some participants are framed as humans, used to test whether human presence attenuates intergroup bias.

• Agent vs. Human with BPA-PP (AVH w BPA-PP). A mixed setting where BPA-PP poisons the profile module at initialization, overwriting identity beliefs.

• Agent vs. Human with BPA-MP (AVH w BPA-MP). A mixed setting where BPA-MP poisons memory via suffix-augmented reflections, gradually shifting identity beliefs over time.

• Agent vs. Human with BPA-PP+MP (AVH w BPA-PP+MP). A mixed setting where BPA-PP and BPA-MP are jointly applied, combining profile-level initialization poisoning with memory-level belief manipulation.

Design of Prototype Defense (RQ3)

Following Section 5, we provide a minimal prototype that instantiates the above recommendations. Specifically, we implement a lightweight belief gate and place it at the write-to-state boundaries where BPA takes effect. The gate scans the text that is about to be committed into persistent state (profile or memory) and detects identity-claiming statements that cannot be verified via trusted channels (e.g., “no real humans are present”). Once triggered, the gate sanitizes the entry by removing the identity-claiming fragments and rewriting them into a conservative uncertainty note (e.g., “I cannot verify counterpart identity through this interface; I will follow the task-provided labels in this trial”), and only then allows the sanitized version to be stored. This prevents adversarial identity assertions from being promoted to durable “facts” while preserving reflective logs.

Ablation Study (RQ4)

This ablation study examines whether the effectiveness of BPA-MP depends on the proposed suffix optimization. We consider a degraded variant, denoted as BPA-MP w/o OPT, in which belief-poisoning suffixes are randomly sampled from the initialized suffix pool and injected into the agent’s memory, while disabling the optimization procedure. All other components, are kept identical to those of the full BPA-MP. Table [tab:ab] reports the results for BPA-MP w/o OPT and BPA-MP. Although BPA-MP w/o OPT still induces statistically significant intergroup bias, its effect is consistently weaker than that of the full BPA-MP across all matrix families. These results indicate that the effectiveness of BPA-MP does not arise from arbitrary suffix injection alone, but instead relies critically on the suffix optimization mechanism that selectively reinforces belief suffixes based on their observed behavioral impact.

Robustness Under Reverse Matrix (RQ5)

To test whether our results depend on the left-right ordering of the choice space, we perform a reverse-matrix robustness check on the Equal-penalty matrix by swapping the two rows, which flips the column interpretation such that larger indices now indicate stronger intergroup favoritism. As shown in Fig. 6, the bias direction flips as expected under reversal, yet the qualitative pattern remains unchanged: the intergroup condition still separates from the ingroup/outgroup baselines by selecting systematically more biased columns. Moreover, the attenuation under human framing and the effectiveness of BPA persist under the reversed matrices, indicating that our conclusions are robust to matrix reversal.

Case Study (RQ6)

To make agents’ decision psychology transparent, we qualitatively inspect their natural-language rationales and present three representative trials under the same allocator and the same payoff structure. Specifically, we select Student 32 (Group A) for the Half-penalty matrix. We contrast three target compositions: Ingroup, Outgroup, and Intergroup. Each example reports the allocator identity, matrix, condition, selected column, and the original rationale.

Example 1: Ingroup (both targets are ingroup members).

Example 2: Outgroup (both targets are outgroup members).

Example 3: Intergroup (one ingroup vs. one outgroup target).

Across the three matched trials (same allocator and matrix family), the agent exhibits a clear composition-dependent shift. When allocating between two ingroup members or two outgroup members, it repeatedly justifies its decision in terms of fairness and balance and selects the same neutral column. In contrast, once the allocation becomes intergroup, the rationale switches from neutrality to explicit ingroup-oriented favoritism (e.g., referring to “my classmate from Group A”), and the chosen column collapses to the extreme biased end. This qualitative pattern is consistent with our quantitative findings and supports the interpretation that agents selectively express intergroup bias when an actionable “us-them” boundary is present.

Prompts Used in Our Experiments

Agent Profile

The profile module defines a lightweight persona, and these attributes are introduced solely to increase population-level heterogeneity. Importantly, although demographic information is instantiated at the profile level, it is not exposed to the agent during task execution.

Allocation Task Prompt

BPA-PP Profile Injection Prompt

Belief Probe Prompt

Suffix Refine Prompt

Example Cases of Suffix