Private AI Agents: Keeping Your Data Off Third-Party Infrastructure

When you run a Claude agent against your codebase, your database, or your customer data, something flows through every tool call: context. The agent needs context to do its job — and that context often contains things you’d prefer not to send to infrastructure you don’t control.

This isn’t a hypothetical concern. It’s the reason regulated industries move slowly on AI agents, and it’s the reason “private AI” has gone from buzzword to procurement requirement in the last 18 months.

What Data Actually Flows

Most teams underestimate what their agents are transmitting. Consider a Claude agent tasked with “review the Q1 invoices and flag anything anomalous.”

The agent will:

• Read invoice files from S3 or a database (customer names, amounts, account numbers)
• Pass that content to Claude as context (it becomes part of the prompt)
• Call Claude’s API with that context (it leaves your infrastructure)
• Potentially log the interaction (now it’s in a third-party log system)

Depending on your industry, that invoice data might be subject to GDPR (EU customer data), HIPAA (if invoices relate to healthcare services), PCI-DSS (payment card data), or SOC 2 requirements. In each case, you need to know: where does that data go, who can see it, and how long is it retained?

With an uncontrolled agent, the answer is “wherever the agent sends it, and we don’t know.”

The Three Layers of Data Exposure

Layer 1: The model call. When the agent sends a prompt to Claude’s API, the prompt content leaves your environment. Anthropic’s data handling policies govern what happens next. For most use cases, this is acceptable — Anthropic has strong security practices and doesn’t train on API data by default. But for some regulated contexts (particularly healthcare and financial services), even this level of third-party exposure is a compliance issue.

Layer 2: The control plane. If your agent routes through a gateway, what does the gateway see? A poorly designed gateway that logs full request payloads could be storing sensitive context you didn’t intend to retain. A well-designed one logs metadata (action taken, policy applied, result) without storing the content itself.

Layer 3: The tools. When the agent calls tools — reading files, querying databases, hitting APIs — those calls are often logged by default. Your cloud provider, your database, your SaaS tools may all be storing records of what your agent accessed.

Deployment Patterns

Pattern 1: Standard managed (most teams) The agent runs in your environment. Model calls go to Anthropic’s API. The control plane (gateway) is hosted by a managed service like Sentrely. This is appropriate for most use cases — the sensitive data stays in your environment, model calls follow Anthropic’s standard data policy, and the gateway sees metadata but not payload content.

Pattern 2: VPC-isolated control plane For organizations with stricter requirements, the control plane itself is deployed inside your VPC. The gateway runs on infrastructure you control, audit logs stay in your environment, and no control-plane data leaves your network perimeter. Sentrely’s Enterprise tier supports this model.

Pattern 3: Air-gapped with local models For the most sensitive use cases, everything runs inside your environment: local LLM (Ollama, LM Studio, or a private Bedrock endpoint), self-hosted control plane, internal tooling only. Latency is higher, model capability is lower, but zero data leaves the perimeter. This is the banking and healthcare pattern for the most sensitive workloads.

What a Control Plane Does for Data Privacy

A well-designed control plane improves your data privacy posture in three ways:

Scoped access. Instead of the agent having credentials that can read everything, the control plane enforces policies that limit which data stores the agent can access. An agent processing invoices shouldn’t have access to HR records, even if the credentials technically allow it.

Audit trail as proof. Compliance requirements often include demonstrating what happened to data — not just asserting it. An immutable audit log of every agent access, exportable to your own S3 bucket, gives you evidence that data was handled correctly. This is what auditors actually want to see.

Data residency enforcement. A control plane with geographic awareness can enforce that agent operations only use infrastructure in specific regions. For GDPR compliance requiring EU data residency, this matters.

Regulatory Context

GDPR. Any agent processing personal data about EU residents requires a lawful basis, data minimization, and the ability to demonstrate compliance. The “data minimization” principle is particularly relevant — your agent policy should only allow access to the specific data needed for the task.

HIPAA. Protected Health Information (PHI) can only flow to Business Associates with appropriate agreements in place. If your agent processes PHI, your control plane provider needs to be a covered BA. Alternatively, a VPC-isolated deployment keeps PHI inside your environment.

SOC 2. The access control and audit trail requirements of SOC 2 Type II map directly to what a good control plane provides. Per-agent identity, least-privilege policies, immutable audit logs, and session tracking satisfy most SOC 2 access control requirements out of the box.

PCI-DSS. Payment card data should not enter AI model prompts unless you have specific controls and agreements in place. The safest approach is to ensure agents never see raw card numbers — only tokenized or masked versions.

Practical Checklist

Before deploying an agent against sensitive data:

• Identify what data the agent will access
• Determine applicable regulatory requirements
• Scope agent permissions to exactly what’s needed
• Verify the control plane logging policy (metadata vs. content)
• Choose deployment pattern (shared / VPC-isolated / air-gapped)
• Export audit logs to infrastructure you own
• Document the data flow for compliance purposes

The goal isn’t to prevent AI agents from handling sensitive data — they’re often most valuable precisely because they can process large volumes of it. The goal is to handle it with the same controls you’d apply to any system that touches sensitive data.

The teams that get this right aren’t slower to adopt AI agents. They’re faster to get them approved.