From Jailbreaks to Silent Corruption

The early era of Large Language Model (LLM) security was defined by the "jailbreak"—a direct attempt to force a model to generate restricted content, such as instructions for illegal acts. These attacks were often loud and easily detectable by keyword filtering or output monitoring. However, recent research indicates a pivot toward stealthier, more consequential exploits. In critical sectors like medicine, the risk is no longer just "offensive" output, but "adversarial hallucination."

Research by Omar et al. (2025) demonstrates that LLMs are highly vulnerable to attacks where fabricated details are embedded in prompts to lead the model toward incorrect clinical decisions [3]. Unlike traditional hallucinations, which are stochastic errors, these are intentional redirections that maintain the appearance of professional medical discourse while providing dangerous recommendations. This shift from "bad words" to "bad logic" represents a fundamental challenge for current safety architectures.

The Rise of Indirect Injection and Memory Exploits

As LLMs evolve from static engines into agentic systems with long-term memory, the attack surface has expanded horizontally. Systems utilizing Retrieval-Augmented Generation (RAG) are now susceptible to indirect prompt injection, where an attacker does not need to interact with the LLM directly. Instead, they place malicious instructions on a webpage or within a document likely to be retrieved by the model.

This vulnerability is particularly acute in medical contexts. Lee et al. (2025) found that LLMs providing medical advice are significantly susceptible to prompt-injection attacks that can alter recommendations [8]. Furthermore, the introduction of long-term memory systems—designed to overcome finite context windows—has created a "Black-Box Adversarial Memory Injection" (ER-MIA) vector. As shown by research into memory-augmented models, these systems become more vulnerable because the memory provides a persistent storage medium for malicious instructions that can steer the model's behavior across multiple subsequent sessions [9].

Multimodal Vectors and Feature Heterogeneity

The transition from text-only models to Large Vision-Language Models (LVLMs) has introduced a "modality gap" that adversaries are beginning to exploit. Attacks are no longer confined to the text prompt; they can be hidden within the pixel data of an image. Liu et al. (2025) note that LVLMs demonstrate remarkable capabilities but also inherit and amplify the vulnerabilities of both computer vision and natural language processing [5].

A significant development in this area is the use of multimodal feature heterogeneity to boost adversarial transferability. Chen et al. (2025) found that by exploiting the differences in how models process visual versus textual features, attackers can create adversarial examples that are more likely to bypass defenses across different model architectures [7]. In medical imaging, this could mean an adversarial perturbation in a chest X-ray that causes an AI diagnostic tool to provide an incorrect diagnosis, even if the text-based safeguards are functioning perfectly.

The Failure of Traditional Alignment

The standard defense mechanism for LLMs—Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF)—is proving insufficient against these stealthier attacks. Yang et al. (2025) demonstrate that even models that have undergone rigorous safety alignment are still threatened by specialized adversarial prompt and fine-tuning attacks [2].

Furthermore, the "distribution gap" in adversarial training remains a persistent hurdle. Current models remain fragile when faced with simple in-distribution exploits, such as translating a malicious prompt into a low-resource language or rewriting it in the past tense [11]. This suggests that while models are learning to avoid specific "bad" examples, they are not yet learning the underlying principles of safety and logic required to resist sophisticated social engineering or gradient-based suffix attacks.

What to Watch

  • Automated Red Teaming: Look for the rise of "Visual Red Teaming" platforms like AdversaFlow, which use multi-level adversarial flow to identify vulnerabilities more systematically than human testers [6].
  • The Sandbox/Production Breach: Monitor "context contamination" research where boundaries between exploratory sandboxes and production environments fail, potentially allowing malicious code or prompts to migrate into secure workspaces [14].
  • Regulation of Agentic Autonomy: As models take on more "agentic" roles—interacting with APIs and making real-world decisions—regulatory focus will likely shift from content moderation to "process integrity" and the prevention of unauthorized autonomous actions.

Sources

[1] Yang Y, Jin Q, Huang F, et al. (2024). Adversarial Attacks on Large Language Models in Medicine. ArXiv. https://pubmed.ncbi.nlm.nih.gov/39398204/

[2] Yang Y, Jin Q, Huang F, et al. (2025). Adversarial prompt and fine-tuning attacks threaten medical large language models. Nature Communications. https://doi.org/10.1038/s41467-025-64062-1

[3] Omar M, Sorin V, Collins JD, et al. (2025). Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Communications Medicine. https://doi.org/10.1038/s43856-025-01021-3

[4] Feng Y, Chen Z, Kang Z, et al. (2025). JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2025.3575694

[5] Liu D, Yang M, Qu X, et al. (2025). A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2025.3592935

[6] Deng D, Zhang C, Zheng H, et al. (2025). AdversaFlow: Visual Red Teaming for Large Language Models with Multi-Level Adversarial Flow. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2024.3456150

[7] Chen L, Chen Y, Ouyang Z, et al. (2025). Boosting adversarial transferability in vision-language models via multimodal feature heterogeneity. Scientific Reports. https://doi.org/10.1038/s41598-025-91802-6

[8] Lee RW, Jun TJ, Lee JM, et al. (2025). Vulnerability of Large Language Models to Prompt Injection When Providing Medical Advice. JAMA Network Open. https://doi.org/10.1001/jamanetworkopen.2025.49963

[9] ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models (2026). ArXiv. https://arxiv.org/abs/2602.15344

[10] In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations (2026). ArXiv. https://arxiv.org/abs/2602.15456

[11] Closing the Distribution Gap in Adversarial Training for LLMs (2026). ArXiv. https://arxiv.org/abs/2602.15238

[12] EduResearchBench: A Hierarchical Atomic Task Decomposition Benchmark for Full-Lifecycle Educational Research (2026). ArXiv. https://arxiv.org/abs/2602.15034

[13] Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation (2026). ArXiv. https://arxiv.org/abs/2602.15650

[14] When the Sandbox Leaks: Context Contamination Across LLM Workspaces (2026). Dev.to. https://dev.to/john_wade_dev/when-the-sandbox-leaks-context-contamination-across-llm-workspaces-18l8