Private AI Agents: Keeping Your Data Off Third-Party Infrastructure

When you run a Claude agent against your codebase, your database, or your customer data, something flows through every tool call: context. The agent needs context to do its job — and that context often contains things you’d prefer not to send to infrastructure you don’t control.

This isn’t a hypothetical concern. It’s the reason regulated industries move slowly on AI agents, and it’s the reason “private AI” has gone from buzzword to procurement requirement in the last 18 months.

What Data Actually Flows

Most teams underestimate what their agents are transmitting. Consider a Claude agent tasked with “review the Q1 invoices and flag anything anomalous.”

The agent will:

• Read invoice files from S3 or a database (customer names, amounts, account numbers)
• Pass that content to Claude as context (it becomes part of the prompt)
• Call Claude’s API with that context (it leaves your infrastructure)
• Potentially log the interaction (now it’s in a third-party log system)

Depending on your industry, that invoice data might be subject to GDPR (EU customer data), HIPAA (if invoices relate to healthcare services), PCI-DSS (payment card data), or SOC 2 requirements. In each case, you need to know: where does that data go, who can see it, and how long is it retained?

With an uncontrolled agent, the answer is “wherever the agent sends it, and we don’t know.”

The Three Layers of Data Exposure

Layer 1: The model call. When the agent sends a prompt to Claude’s API, the prompt content leaves your environment. Anthropic’s data handling policies govern what happens next. For most use cases, this is acceptable — Anthropic has strong security practices and doesn’t train on API data by default. But for some regulated contexts (particularly healthcare and financial services), even this level of third-party exposure is a compliance issue.

Layer 2: The middle layer. If your agent routes through a gateway or proxy, what does that layer see? A poorly designed gateway that logs full request payloads could be storing sensitive context you didn’t intend to retain. A well-designed one logs metadata (action taken, policy applied, result) without storing the content itself.

Layer 3: The tools. When the agent calls tools — reading files, querying databases, hitting APIs — those calls are often logged by default. Your cloud provider, your database, your SaaS tools may all be storing records of what your agent accessed.

Deployment Patterns

Pattern 1: Standard managed (most teams) The agent runs in your environment. Model calls go to Anthropic’s API. Any policy or logging layer is hosted by a managed provider. This is appropriate for most use cases: the sensitive data stays in your environment, model calls follow Anthropic’s standard data policy, and the middle layer should see metadata but not payload content.

Pattern 2: VPC-isolated policy layer For organizations with stricter requirements, the policy and logging layer itself is deployed inside your VPC. It runs on infrastructure you control, audit logs stay in your environment, and none of that data leaves your network perimeter.

Pattern 3: Air-gapped with local models For the most sensitive use cases, everything runs inside your environment: local LLM (Ollama, LM Studio, or a private Bedrock endpoint), an in-house policy layer, internal tooling only. Latency is higher, model capability is lower, but zero data leaves the perimeter. This is the banking and healthcare pattern for the most sensitive workloads.

What a Policy Layer Does for Data Privacy

A well-designed policy layer between agents and your data improves your data privacy posture in three ways:

Scoped access. Instead of the agent having credentials that can read everything, the policy layer enforces rules that limit which data stores the agent can access. An agent processing invoices shouldn’t have access to HR records, even if the credentials technically allow it.

Audit trail as proof. Compliance requirements often include demonstrating what happened to data — not just asserting it. An immutable audit log of every agent access, exportable to your own S3 bucket, gives you evidence that data was handled correctly. This is what auditors actually want to see.

Data residency enforcement. A policy layer with geographic awareness can enforce that agent operations only use infrastructure in specific regions. For GDPR compliance requiring EU data residency, this matters.

Regulatory Context

GDPR. Any agent processing personal data about EU residents requires a lawful basis, data minimization, and the ability to demonstrate compliance. The “data minimization” principle is particularly relevant — your agent policy should only allow access to the specific data needed for the task.

HIPAA. Protected Health Information (PHI) can only flow to Business Associates with appropriate agreements in place. If your agent processes PHI, any third party in the data path needs a Business Associate Agreement. Alternatively, a VPC-isolated deployment keeps PHI inside your environment.

SOC 2. The access control and audit trail requirements of SOC 2 Type II map directly to what a good policy layer provides. Per-agent identity, least-privilege policies, immutable audit logs, and session tracking satisfy most SOC 2 access control requirements out of the box.

PCI-DSS. Payment card data should not enter AI model prompts unless you have specific controls and agreements in place. The safest approach is to ensure agents never see raw card numbers — only tokenized or masked versions.

Practical Checklist

Before deploying an agent against sensitive data:

• Identify what data the agent will access
• Determine applicable regulatory requirements
• Scope agent permissions to exactly what’s needed
• Verify what every layer in the data path logs (metadata vs. content)
• Choose deployment pattern (shared / VPC-isolated / air-gapped)
• Export audit logs to infrastructure you own
• Document the data flow for compliance purposes

The goal isn’t to prevent AI agents from handling sensitive data — they’re often most valuable precisely because they can process large volumes of it. The goal is to handle it with the same controls you’d apply to any system that touches sensitive data.

The teams that get this right aren’t slower to adopt AI agents. They’re faster to get them approved.

If you’d rather have someone else build this way for you, Sentrely is a managed AI team, backed by real engineers, that builds, ships and monitors production-grade software in accounts you own, so your data stays in your own hosting and code accounts. Book a free 30-minute call to talk it through.