AI security threats: Why it is about data, not models

Samsung banned ChatGPT after engineers pasted proprietary code into it. Amazon warned employees after noticing ChatGPT responses that looked suspiciously like internal documentation. OpenAI had to take their service offline when a bug exposed user payment information.

These are not theoretical attacks on AI models. These are data breaches through AI interfaces.

The industry keeps talking about model poisoning and adversarial examples while 77% of sensitive enterprise data is flowing into GenAI tools through unmanaged personal accounts. AI security threats enterprise teams actually face are fundamentally about data security, not model security.

Why everyone focuses on the wrong threats

Walk into any AI security discussion and someone will mention adversarial examples - those carefully crafted inputs that make image classifiers see stop signs as speed limits. Fascinating research. Irrelevant to most organizations.

Research from IBM shows that 13% of organizations have already experienced AI-related breaches. Of those compromised, 97% lacked proper AI access controls. Not sophisticated model attacks. Basic access control failures. Shadow AI alone is making things worse - one in five organizations reported a breach specifically due to unauthorized AI tool usage, with significantly higher breach costs.

The OWASP Top 10 for LLMs was updated in 2025, and the changes tell the story. Sensitive Information Disclosure jumped from position six to position two. Three entirely new categories were added - System Prompt Leakage, Vector and Embedding Weaknesses, and Misinformation. The threats are evolving toward data exposure, not model manipulation.

The pattern is clear. Attackers are not spending time crafting perfect adversarial examples. They are using AI systems as new attack surfaces for traditional data exfiltration.

Your ChatGPT integration has access to customer records. Your Copilot instance can see internal emails. Your custom LLM processes financial documents. These are data access points, and OWASP lists prompt injection as the number one LLM security risk precisely because it turns AI systems into data theft tools.

The copy-paste problem nobody is fixing

Here is what keeps me up at night about AI security threats enterprise environments face today.

The scale of AI tool usage is staggering. Enterprise AI/ML activity grew more than 90% year-over-year in 2025, with data transfers to AI tools rising 93% to tens of thousands of terabytes. Most of it through personal accounts. Every time someone copies data from your systems and pastes it into ChatGPT to “help write this email” or “summarize this document,” that data leaves your control entirely.

LayerX Security found that AI is already the single largest uncontrolled channel for corporate data exfiltration - bigger than shadow SaaS, bigger than unmanaged file sharing. For every 10,000 users, expect around 660 daily prompts to ChatGPT, with source code being the most frequently exposed sensitive data type.

And it is not just employees you need to worry about. AI agents move 16x more data than human users, running nonstop and chaining tasks across multiple applications. By 2026, Gartner projects 40% of enterprise applications will embed task-specific agents. That is a massive expansion of the data exfiltration surface that most security teams have not accounted for.

Your traditional DLP tools? They are looking for file uploads and email attachments. They do not see copy-paste. They cannot monitor what gets typed into browser-based AI tools. The entire attack vector is invisible to systems built for a file-centric world.

Samsung learned this the hard way. Their engineers pasted proprietary semiconductor code into ChatGPT for optimization suggestions. That code is now potentially part of OpenAI’s training data. The financial impact? Significant enough to warrant an immediate company-wide ban.

When your AI turns against you

Prompt injection is the AI equivalent of SQL injection, except it is harder to fix and easier to exploit.

Someone embeds malicious instructions in a document. Your AI assistant reads that document. Those hidden instructions override your system prompts. Suddenly your AI is exfiltrating data to external URLs or manipulating its responses to spread misinformation.

Microsoft reported that indirect prompt injection is one of the most widely-used techniques in AI security vulnerabilities they see. The enterprise versions of Copilot and Gemini have access to emails, document repositories, and internet content. Hidden instructions in any of those sources can compromise the entire system.

Researchers demonstrated this with Slack AI - tricking it into leaking data from private channels through carefully crafted prompts. Recent research on hybrid AI threats shows attackers combining prompt injection with traditional exploits like XSS and CSRF, creating attack chains that bypass multiple security layers.

The defense is not perfect - and that is putting it mildly. A landmark study from researchers across OpenAI, Anthropic, and Google DeepMind examined 12 published prompt injection defenses and bypassed them with attack success rates above 90% for most. Human red-teamers scored 100%, defeating every defense tested. OpenAI themselves acknowledged that “prompt injection is a long-term AI security challenge” with no deterministic guarantees.

Microsoft uses hardened system prompts, Spotlighting techniques that reduced attack success rates from over 50% to below 2%, and deterministic blocking for known exfiltration patterns. But there is no complete solution yet. Every prompt injection defense can be bypassed with enough creativity.

The supply chain you did not know you had

Your AI model came from somewhere. So did its training data. Both are attack surfaces.

Research demonstrates that attackers successfully poisoned 92% of popular ML frameworks in controlled tests. Training data poisoning affected over 150 million records across major cloud platforms. These are not theoretical risks - they are measured vulnerabilities in production systems.

Model poisoning works like this: an attacker manipulates training data or model parameters to create backdoors. OWASP warns that supplied models from public registries can contain deliberately embedded biases or data exfiltration capabilities. The poisoning affects the entire model, not just individual sessions.

The difference between prompt injection and supply chain poisoning? Prompt injection is temporary, affecting single interactions. Supply chain poisoning is permanent, built into the model itself.

The attack surface is also expanding beyond traditional models. OWASP added Vector and Embedding Weaknesses as a new top-10 category in 2025. With most companies opting to use RAG pipelines instead of fine-tuning, the vector databases and embedding stores powering those pipelines are now prime targets for data poisoning and manipulation.

Defense requires treating AI models like any other software dependency. Verify provenance. Test for anomalies. Scan training datasets before use. Most organizations are not doing this. They download pre-trained models from HuggingFace or public registries without security review.

At Tallyfy, when we evaluate AI integrations, we ask: where did this model come from? Who trained it? What data did they use? Can we verify any of this? Most vendors cannot answer these questions. That is a supply chain risk.

What actually works

Gartner predicts that 40% of AI data breaches will stem from cross-border GenAI misuse by 2027. Yet only 35% of organizations have an established AI governance framework, and 63% of breached organizations either lack an AI governance policy entirely or are still developing one. The organizations that avoid becoming statistics focus on data governance, not exotic AI security measures.

Here is what works in practice.

Control data access, not just model access. Your AI assistant does not need access to everything. Scope permissions tightly. If the tool processes customer support tickets, it should not see financial records. Basic least-privilege principles applied to AI integrations.

Monitor what data goes into AI tools. Traditional DLP is blind to AI. You need visibility into what employees paste into ChatGPT, what documents get uploaded to Claude, what code goes into Copilot. Organizations that monitor shadow AI experience significantly lower breach costs than those that do not.

Harden AI interfaces against manipulation. Use input validation. Implement output filtering. Set up anomaly detection for unusual data access patterns through AI systems. Microsoft’s approach combines multiple techniques - no single control stops all attacks, but layered defenses make exploitation significantly harder.

Verify AI supply chains. Before deploying any model, understand its provenance. Scan training data for poisoning. Test models for backdoors using adversarial robustness techniques. This is tedious work. It is also necessary.

Assume data will leak. Design your AI implementations assuming employees will paste sensitive information into public tools. Which data absolutely cannot leak? Implement technical controls that prevent that data from being copied or exported. Everything else, monitor and educate. Building a proper AI governance framework is not optional - it is the foundation everything else rests on.

The AI security threats enterprise teams face today are data security problems wearing new clothes. The attacks leverage AI capabilities, but the goal remains the same: steal or manipulate information. Defend accordingly.

Organizations that treat AI security as an exotic new field separate from existing security practices will struggle. Those that recognize AI systems as new data access points and apply proven security principles - least privilege, defense in depth, continuous monitoring - will be fine. The biggest AI failures are organizational, not technical.

Your biggest AI security risk is not a sophisticated adversarial attack on your model. It is your sales team pasting customer lists into ChatGPT to help draft emails.