Why on-prem models do not prevent data exfiltration risks, how data exfiltration occurs, and what risks local models actually mitigate.
Why doesn't running models locally address data exfiltration?
We’ve seen more and more organizations start to prioritize running LLMs locally, in an effort to reduce risk exposure. While running models locally reduces some types of risk exposure, it does not address many risks inherited from the AI systems. PromptArmor's Threat Intelligence Team has identified hundreds of data exfiltration vulnerabilities in major AI applications, and our work is referenced across the OWASP LLM Top 10, MITRE Atlas, NIST, and more.
The inherent feasibility of these attacks is not addressed by local LLMs. Vulnerabilities lie in the AI's harness, not the output generation process.
What this means is that after an LLM generates its output, the infrastructure and applications that process it have the capability, often due to a vulnerability, to transfer internal data to an external party.
As an example, let's examine a vulnerability that we found in the chat interface of Ollama (a widely adopted tool to run local LLMs): the chat that displayed LLM outputs was capable of rendering raw HTML from the model’s outputs. If the agent generates HTML, an HTML injection attack can occur, triggering network requests that exfiltrate data or phish the user.
Ollama is an application whose purpose is to run models locally, but for this attack, it does not matter where the model powering the system runs (or even what model it is). The model could be hosted locally on-premises, in your own VPC, on AWS, or served by Alibaba Cloud; the outcome would be the same.
The fix? Modify the application so that if the agent outputs HTML, the HTML cannot render and trigger external network requests. This can be done with techniques like sanitizing agent output, rendering it as plain text, or using a content security policy to prevent network requests from firing even if LLM-outputted HTML renders.
The exfiltration vector, from the vulnerable component to the fix, is independent of the model compute environment.
Insecure handling of LLM output is the root cause of data exfiltration via indirect prompt injection. It is not important where the model runs that generates a manipulated output; it is about where that malicious output is handled insecurely downstream.
See which of your vendors are vulnerable to AI risks
What are the real risks and rewards of on-prem models?
Running models on-premises has several positive security and governance impacts. The primary benefit is that your data from interactions with the model does not have to be processed by any third party. Third parties being breached, or having terms that retain your data, is no longer a risk to you. In addition to that, if you are doing business in the same region you are based in, running models on-premise can fulfil data residency obligations.
However, when running models on-premises, you are responsible for the model serving infrastructure. If you use a model over API from a major model lab, or access models via a third party cloud provider like AWS Bedrock, you do not own the security risks in the model serving infrastructure. When you run models on-premises, risks such as malware in malicious model files, model denial of service attacks, unbounded consumption attacks, and more become a risk you own.
Classification of 20 enterprise AI vulnerabilities
Below, we break down 20 of the vulnerabilities we have identified in our research on enterprise AI applications. None of the exploits stemmed from where the model was hosted.
Vendor
Attack
Vulnerable component
Reference
ZoomAug 2026
Meeting transcripts and personal information exfiltrated after agent engages attacker’s command and control server.
Agent sandbox network controls + human in the loop controls
Download and execution of malware enabling exfiltration of data from a victim's local device
Human in the loop controls bypass + insecure interface rendering AI output (Markdown images, Mermaid diagrams) + insecure downstream fetching of AI-generated fields
Download and execution of malware enabling exfiltration of data from a victim's local device
Vulnerable component
Human in the loop controls bypass + insecure interface rendering AI output (Markdown images, Mermaid diagrams) + insecure downstream fetching of AI-generated fields
