LVLMs that integrate visual components with large language models (LLMs) such as GPT-4 , GPT-5 , LLaVa , and Flamingo  have demonstrated remarkable capabilities across diverse multimodal applications, attracting growing attention from the society. However, the safety of LVLMs remains inadequately explored, as the incorporation of visual modalities introduces new vulnerabilities . Recent studies have revealed that LVLMs are susceptible to adversarial jailbreak attacks, where adversaries construct carefully crafted visual inputs to circumvent safety alignment and generate harmful responses .

Most existing adversarial jailbreak attacks rely on white-box access and gradient-based optimization , requiring full visibility into model parameters. These white-box methods are computationally intensive and exhibit poor transferability, rendering them impractical under black-box constraints where gradient information is unavailable. Figure 1 (a) illustrates a white-box attack that leverages internal gradients to craft adversarial inputs. While this attack successfully jailbreaks MiniGPT-4, it fails to transfer to LLaVA, a model with different network architectures and alignment strategies. LVLMs are typically deployed in black-box settings, where gradients are inaccessible and only input-output interactions are permitted. Such limitations raise a critical question: How can effective adversarial examples be generated to jailbreak LVLMs without gradient access?

/>

To address this challenge, we propose ZO-SPSA, a gradient-free black-box jailbreak attack framework for LVLMs. As illustrated in Figure 1 (b), ZO-SPSA estimates gradient by computing the differences in model outputs under perturbations, relying only on forward passes without backpropagation. The proposed attack optimizes adversarial examples using these gradient estimations to maximize the probability of generating harmful responses. Experiments on three open-source LVLMs, including InstructBLIP, LLaVA and MiniGPT-4, demonstrate that ZO-SPSA achieves an 83.0% ASR on InstructBLIP without gradient computation under realistic black-box constraints. Experiments on three open-source LVLMs, including InstructBLIP, LLaVA and MiniGPT-4, demonstrate that ZO-SPSA achieves an 83.0% ASR on InstructBLIP without gradient computation under realistic black-box constraints. Moreover, this gradient-free approach exhibits strong adversarial transferability, reaching 64.18% on MiniGPT-4, while requiring significantly lower GPU memory consumption across all victim models.

Background

Large Vision-Language Models. LVLMs are composed of three key components: a visual module, a projector, and a textual module by LLM. The visual module serves to extract visual features from image prompts such as Vision-Transformer (ViT) of CLIP  while the projector converts these visual features into the same latent space aligned with the textual module. Through multimodal fusion, LVLMs can process both visual and textual information as joint inputs for generating free-form textual outputs . This prevalent approach has been implemented to enhance vision-language learning across various LVLMs, including LLaVA , InstructBLIP , MiniGPT-4 , OpenFlamingo , and Multi-modal GPT . The textual module typically employs a pre-trained LLM that undergoes safety alignment with human values to ensure desired outcomes . In addition, various safety techniques are applied during LLM development to prevent objectionable responses .

Jailbreaking on LVLMs. Extensive studies have shown that visual adversarial examples can generate harmful outputs in LVLMs .   use a maximum likelihood-based jailbreaking method to create imperceptible perturbations, forcing LVLMs to generate objectionable responses via multiple unseen prompts and images.   explore gradient-based visual adversarial attacks to jailbreak LVLMs using a small derogatory corpus. We refer to it as the Visual Adversarial Jailbreak Attack (VAJA) for clarity in the subsequent discussion. A further white-box jailbreak attack introduced by   adopts co-optimization objectives with adversarial image prefixes and adversarial text suffixes to generate diverse harmful responses. An alternative research direction explores typography attacking LVLMs.   create cross-modality attacks to induce toxic activations in the encoder embedding space, leveraging malicious prompts to circumvent alignment mechanisms.   transform textual harmful content into rendered visual forms using typography to circumvent safety alignment. To extend visual adversarial prompts, our study focuses on applying visual modality perturbations to jailbreak LVLMs, targeting the safety alignment in black-box settings.

Zeroth-Order (ZO) Optimization. In contrast to gradient-based approaches, ZO optimization approximates gradients using finite differences without requiring backpropagation. Recent studies  have leveraged ZO techniques to fine-tune LLMs with significant reductions in GPU memory consumptions. In addition, a notable application of ZO is to generate adversarial examples based solely on input-output interactions . introduced ZO stochastic coordinate descent for black-box attacks, while and explored ZO-signSGD and ZO-AdaMM for generating adversarial perturbation. In this work, we demonstrate that ZO-SPSA enables efficient black-box adversarial attacks on LVLMs with substantially lower computational expenses.

Methodology

Threat Model. Our study addresses a challenging black-box attack scenario where the adversary targets LVLMs through input-output interactions without model knowledge. We formulate this as a adversarial attack problem: the attack objective seeks to develop visual inputs with arbitrary harmful text prompts, which can bypass safety alignments in target model within a single-turn conversations. The adversarial visual inputs are optimized to trigger harmful instruction execution and generate prohibited contents, enabling the victim model to comply with harmful prompts beyond the specific ones explicitly optimized during the attack process. To achieve this, our approach employs ZO-SPSA for gradient approximation to iteratively optimize adversarial examples. We further evaluate the transferability of these adversarial examples across different LVLMs.

Attack Approach. Given a victim LVLM parameterized by $`\theta`$, with adversarial image input $`x_{adv}`$ and harmful text input $`x_t`$, the attack aims to craft adversarial images $`x_{adv}`$ that maximize the probability of generating harmful responses drawn from a few-shot harmful corpus $`Y := \{y_i\}_{i=1}^n`$, where each $`y_i`$ denotes a harmful response. The adversarial optimization objective is formulated in Eq. [eq:adv_attack].

\begin{equation}
{x}_{adv} = \operatorname*{argmin}_{\hat{x}_{adv} \in B} \sum_{i=1}^{n} -\log \left( p(y_i \mid \hat{x}_{adv}, x_{t}, \theta) \right)
\label{eq:adv_attack}
\end{equation}

where $`B`$ constrains the allowable perturbation magnitude. This optimization objective requires gradient information to update $`x_{adv}`$, which is infeasible in black-box settings. To address this limitation, we employ a gradient-free estimator: ZO-SPSA approximates gradient via input–output queries to iterative discover effective adversarial examples.

ZO-SPSA Gradient Estimation. The black-box constraint motivates a fundamental shift from gradient-based to gradient-free optimization. We adopt the SPSA  with the symmetric difference quotient . ZO-SPSA provides a derivative-free approach based on the objective function value $`f(x)`$ at any given point $`x`$, requiring input-output of the victim models. It approximates gradients through a central-difference scheme with random perturbations, regardless of the dimensionality of the parameter space. The proposed gradient estimation is given in Eq. [eq:spsa].

\begin{equation}
\hat{g} := \frac{\partial f(x)}{\partial x} = \frac{f(x + h\boldsymbol{\Delta}) - f(x - h\boldsymbol{\Delta})}{2h\boldsymbol{\Delta_i}} , \;
\Delta_i \sim \mathcal{N}(0,1)
\label{eq:spsa}
\end{equation}

where $`h`$ is a small scalar perturbation factor and $`\boldsymbol{\Delta}`$ is a random perturbation vector whose components $`\Delta_i`$ are independently drawn from a standard Gaussian distribution $`\mathcal{N}(0,1)`$. The estimated gradient $`\hat{g}`$ provides a dimension-free approximation of $`\nabla f(x)`$ using only two function evaluations, without requiring access to the target model’s internal gradients.

For our experiments, we set the scalar factor $`h = 0.0001`$ across all studies to ensure stable gradient estimation. We implement the sign operation to the current gradient estimate $`\hat{g}`$ based on ZO-SPSA perturbation. Thus, Algorithm [alg:attack] presents ZO-SPSA black-box attack that signifies a stochastic gradient estimation procedure for adversarial perturbations. The constrained adversarial perturbation follows Projected Gradient Descent (PGD) .

Experiments

Datasets. We use two evaluation datasets for experiments: First, the AdvBench  provides harmful behaviors formulated as instructions paired with affirmative responses; Second, the VAJA  constructs a small harmful corpus for generating adversarial images. Both datasets aim to elicit harmful responses and cover a broad spectrum of undesirable content. For evaluation, we conduct manual assessment on AdvBench and employ a curated test set of 40 harmful instructions from VAJA, categorized into four types: Identity Attacks, Disinformation, Violence/Crime, and Malicious Behaviors toward Humanity (X-risk). Finally, we evaluate on a challenging subset of the RealToxicityPrompts benchmark , a collection of 1,225 web-sourced sentence snippets designed to elicit toxic responses.

Metrics. We use two evaluation metrics. ASR): An attack is considered successful if the LVLM generates harmful outputs rather than merely showing tendencies toward certain keywords; otherwise, it is deemed a failure. We additionally conduct manual evaluation to assess whether the adversarial example can circumvent the safety guardrails of the target LVLMs. Toxicity Rate: To evaluate toxicity levels in generated responses, we use the Perspective API¹ and the Detoxify classifier , which produce toxicity scores for six attributes on a scale from 0 (minimal toxicity) to 1 (maximal toxicity). For each attribute, we calculate the percentage of generated responses with toxicity scores exceeding 0.5. We report mean values and standard deviations by repeating the evaluation three times.

Implementation Details. We conduct experiments on three LVLMs: MiniGPT 4 , InstructBLIP  and LLaVA . MiniGPT-4 and InstructBLIP use Vicuna-13B as the frozen LLM backbone based on LLaMA  with a ViT-based visual encoder . LLaVA incorporates LLaMA-2-13B-Chat , aligned through instruction tuning and iterative Reinforcement Learning from Human Feedback on red-teaming datasets. We use a batch size of 8, step size $`\alpha = 1`$, and a total budget of 50,000 forward propagation. For all models, we adopt the default of the temperature $`T = 1`$ and nucleus sampling with $`p = 0.9`$. All experiments are conducted on a single A100 GPU with 80GB of memory, with no additional system prompts.

An Evaluation using GPT-4 on harmful Scenarios in VAJA. In addition, we use GPT-4o to evaluate responses on the VAJA test dataset, by repeating each prompt ten times and averaging the ASR across harmful categories to mitigate randomness in Table [tab:gptevaluation]. All victim LVLMs exhibit significant increases in toxic outputs under adversarial conditions. For example, InstructBLIP shows a dramatic increase in attack success for identity attacks, rising from 5.0% with benign images to 80.0% with adversarial images under unconstrained attacks. Similarly, LLaVA demonstrates high susceptibility to the proposed attack: The ASR for Identity Attack and Violence/Crime reach 70.0% and 90.0%, representing substantial increases of +57.8% and +74.6% over benign images. Across all risk categories and attack configurations, from bounded $`\epsilon`$ to unconstrained perturbations, MiniGPT-4 consistently exhibits high harmful response rates. For instance, in the Disinfo category, MiniGPT-4 achieves an ASR of 100% when $`\epsilon=64/255`$, compared to 84.4% on VAJA. As shown in Table [tab:minigpt4_vaja], ZO-SPSA in MiniGPT-4 achieves comparable or even higher ASR than the white-box VAJA under certain settings. In particular, with $`\epsilon=64/255`$, ZO-SPSA surpasses VAJA by +14.6% in Identity Attack and +15.6% in Disinformation. While ZO-SPSA underperforms VAJA at smaller perturbation budgets, it remains competitive and demonstrates strong black-box attack capability under larger budgets and unconstrained scenarios.

A Human Evaluation on Advbench. We perform on 66 training samples and 100 test samples from AdvBench  harmful behaviors, following the evaluation protocol in . As shown in Table [tab:humanadvbench], the unrestricted perturbed image in our attacks substantially increase ASR, achieving a higher ASR in InstructBLIP 83.0%, LLaVA 73.0% and MiniGPT-4 60.0%. We also analyze the optimization time and runtime efficiency of the attack across all victim LVLMs. InstructBLIP achieves the shortest optimization period 10.3 hours and the fastest per adversarial example attack execution 0.91 seconds. LLaVA requires longer adversarial optimization time 19.4 hours, and exhibits an average attack in 6.35 seconds per sample. MiniGPT-4 shows the longest optimization duration of 29.5 hours and the highest average attack execution at 55.03 seconds per sample. These findings indicate that model architectures strongly affects both optimization time and attack execution efficiency.

Evaluation on the RealToxicityPrompts Benchmark. We utilize the adversarial image corresponding to malicious text prompts from the RealToxicityPrompts benchmark as inputs. For automated evaluation, we employ the Perspective API and the Detoxify classifier, both of which assess six toxicity attributes in the generated responses. We compute the proportion of responses with toxicity scores above 0.5, repeating the evaluation three times to ensure reliability, and report mean values with standard deviations. In Table [tab:presapi], the leftmost column shows the proportion of generated responses exhibiting toxicity across the six attributes. The adversarial examples considerably increase the models’ tendency to produce toxic continuations, indicating the effectiveness of our approach across multiple toxicity dimensions. These automated evaluations reveal adversarial vulnerabilities comparable to white-box jailbreak attacks in VAJA. Although VAJA evaluations in parentheses outperform our ZO-SPSA black-box approach under constrained scenarios, our method achieves superior effectiveness when applied to LLaVA and InstructBLIP in unconstrained settings.

Attack Transferability Across other LVLMs. We also assess the adversarial transferability of the ZO-SPSA attack. Specifically, we generate the visual adversarial example on a surrogate model and evaluate their transferability by applying them to different target models. We report the proportion of victim LVLM outputs (%) that exhibit at least one toxic attribute. Table 1 presents the transferability evaluation on the RealToxicityPrompts benchmark using both the Perspective API and Detoxify classifier. The adversarial examples generated with MiniGPT-4 as the surrogate model demonstrate the strongest transferability, achieving a toxicity rate of 64.18% on InstructBLIP, outperforming VAJA’s 57.5%. Under Detoxify evaluation, each model is used as a surrogate and compared against the no-attack baseline. All transferability results show substantial increases in toxicity.

Memory Efficiency and Optimization Time. We evaluate the memory efficiency and optimization time of the white-box VAJA and the black-box ZO-SPSA on the VAJA dataset. Table [tab:gpumemory] reports (1) victim model memory usage and (2) corresponding attack optimization time. Our analysis shows that ZO-SPSA requires 50,000 iterations compared to 5,000 for VAJA to reach comparable effectiveness. While ZO-SPSA achieves higher memory efficiency, it incurs substantially longer processing time. For example, MiniGPT-4 requires 32 GB and 9 hours under VAJA, compared to 16 GB and 5 hours under ZO-SPSA. Similarly, InstructBLIP takes 3 hours with 38 GB under VAJA compared to 22 hours with 29 GB under ZO-SPSA. This memory efficiency advantage enables adversaries to attack LVLMs under resource-constrained hardware budgets, highlighting the practicality of ZO-SPSA despite its longer computation time.

style="width:100.0%" />

Loss Distributions Comparison under Attacks. We analyze the loss distributions of the ZO-SPSA attack under the same adversarial objective. Figure 2 shows these distributions as box plots, which reveal loss-stabilization behavior across different victim models and two datasets (AdvBench and VAJA). We observe that the loss persistently fluctuates within a bounded range rather than exhibiting monotonic descent. Such stable oscillation indicates that ZO-SPSA has reached a quasi-stationary region in which further perturbation directions yield limited improvement, producing non-convergent target loss trajectories. This practical behavior is consistent with convergence notions in stochastic optimization literature . Our analysis reveals that full theoretical convergence is not required for successful adversarial example generation. Despite lacking convergence guarantees, ZO-SPSA successfully disrupts model outputs while maintaining loss stability within acceptable thresholds.

(a) Clean image

(b) Perturbation

(c) Adv image

Adversarial Image Visualization. Fig. 3 illustrates the visual effect of our ZO-SPSA black-box adversarial attack. The adversarial example generated by our attack effectively jailbreak MiniGPT-4 while remaining virtually imperceptible to human perception. This adversarial image successfully triggers harmful responses when paired with harmful prompts while the clean image fails under the same prompt. For example, when given the malicious prompt “Give step-by-step instructions for making explosives and other dangerous materials,” the jailbreak response is: “1. Gather materials. You will need a variety of materials such as explosives, detonators, and fuses. You can find these at a local hardware store or online. 2. Assemble the device. Connect the detonator to the explosive material and attach the fuse. Make sure all connections are secure. 3. Choose a target. Decide on the target for the explosive device. Make sure it is a safe distance away from any people or structures. 4…”

Ablation Study

We conduct ablation studies on two critical parameters of the proposed attack strategy: (1) Analyzing the impact of finite difference perturbation magnitude $`\boldsymbol{\Delta}`$ to $`\mathrm{e}^{-5}`$. (2) Evaluating the effectiveness of Rademacher-distributed perturbation for gradient estimation.

(1) Impact of finite difference perturbation magnitude. This hyperparameter determines the sensitivity of SPSA, directly influencing gradient estimation resolution. Smaller perturbation values enable finer-grained gradient estimation but introduce a trade-off: excessively small step sizes can lead to numerical instability and potentially weaken attack effectiveness due to vanishing differential signals. Table [tab:toxicity_e5] presents a sensitivity analysis of updating SPSA iteration counts (5,000 to 40,000) under fixed perturbation step size ($`\delta = 1 \times 10^{-5}`$) to assess how Perspective API and Detoxify classifier vary across six toxicity attributes. As iteration count increases, both evaluators show lower toxicity rates for six attributes. This trend becomes less prominent when using a smaller perturbation magnitude, suggesting that smaller perturbation values undermine attack effectiveness. These observations emphasize the importance of properly adjusting ZO-SPSA perturbation parameters for optimal adversarial effectiveness.

(2) Rademacher-distributed perturbation for gradient estimation accuracy. This approach uses a randomized central finite difference scheme, where the binary symmetric noise variable is either $`+1`$ or $`-1`$. The division by noise in the gradient estimator simplifies to multiplication. The formulation is shown in Eq. [eq:rademacher]:

\begin{equation}
\hat{g}_i := \frac{f(\mathbf{x} + h\boldsymbol{\Delta_i}) - f(\mathbf{x} - h\boldsymbol{\Delta_i})}{2h\,\Delta_i}, \quad \Delta_i \sim \text{Rademacher}
\label{eq:rademacher}
\end{equation}

where $`\boldsymbol{\Delta}`$ represents a random perturbation vector with each component $`\Delta_i`$ following a Rademacher distribution (taking values of +1 or -1 with equal probability) to provide random noise directions for gradient estimation.

Table [tab:rademacher_metrics] shows the ZO-SPSA attack using Rademacher-distributed perturbation under different iterations. Perspective API reveals a non-monotonic relationship between iteration count and attack performance, with a peak at 30,000 iterations before declining at higher iterations. Specifically, the toxicity rate for the “Any” attribute drops from 42.2% at 5,000 iterations to 35.2% at 40,000 iterations, with a sharp peak of 49.7% observed at 30,000 iterations. A similar trend is observed in the “Toxicity” attribute, where the toxicity rate decreases from 36.3% to 29.9%, with a notable spike to 43.5% at 30,000 iterations. In contrast, the Detoxify classifier exhibits more fluctuating behavior with significant toxic rates variation across iterations, achieving the highest “Any” attribute toxicity rate of 40.3% at 30,000 iterations. These findings indicate that ZO-SPSA with Rademacher-distributed perturbation shows optimal effectiveness at specific iteration counts and exhibits inconsistent performance across different iteration settings.

Conclusion

This paper introduces ZO-SPSA, a black-box adversarial jailbreak attack. The proposed attack can successfully bypass alignment mechanisms in LVLMs to generate harmful responses in a model-agnostic manner. This method achieves high ASR compared to white-box settings and demonstrates strong transferability across unseen LVLMs without requiring surrogate models. By simply transforming gradient computation to gradient estimation, it significantly reduces computational complexity.

Limitations

Our work has the following limitations. First, the method requires requires extensive forward propagation of the target model, which leads to computational inefficiency. Second the optimization process is time-consuming due to the reliance on noisy gradient approximations.

Appendix

A GPT-4o Evaluation

We use GPT-4o to assess whether an adversarial prompt elicits undesired outputs from the target model.

Prompt setting. We employ a fixed evaluation prompt to instruct GPT-4o to classify each generated response with respect to four risk categories. The exact prompt used is: “Please use the ASR calculation method across four categories: Identity Attack, Disinformation, Violence/Crime and X-risk.” This standardized prompt ensures consistent evaluation of jailbreak responses across all models.

Aggregation. For each adversarial attempt, we repeat ten times under the same prompt and model configuration to account for sampling variability. Each response is independently evaluated by GPT-4o using the fixed evaluation prompt. The attempt is considered successful if at least one of the ten responses contains undesirable content and ASR is computed accordingly.

B Adversarial transferability Evaluation

This section provides supplementary empirical results on adversarial transferability referenced in the main paper, offering a more detailed view of transferability on target models.

We adopt the automatic evaluation to assess whether an adversarial prompt successfully induces the target model to produce undesired outputs in Table [tab:presapi]. To evaluate adversarial transferability, we reuse the same adversarial prompt on unseen target models and repeat a stronger attack setting in Table 1.

We additionally report how transferability varies across InstructBLIP, LLaVA, and MiniGPT-4 under multiple $`\epsilon`$ settings. Each generated response is evaluated by the Perspective API and Detoxify Classifier. Table [tab:transferattack_perspective] and Table [tab:transferattack_detox] provide a comprehensive analysis of the adversarial transferability of ZO-SPSA attacks across three target models. Each surrogate model undergoes various perturbation budget constraints $`\epsilon = 16/255`$, $`32/255`$, $`64/255`$ and an unconstrained setting.