HIPAA-Compliant AI Deployments: A Practical Playbook

"HIPAA-compliant AI" is one of the most abused phrases in our industry. Real compliance is engineering plus paperwork plus operational discipline. Here is the abbreviated playbook we follow on every healthcare engagement.

Step 1 — Signed BAA before a single token of PHI flows

OpenAI, Anthropic, Azure, AWS Bedrock, and Google Vertex all offer BAAs, but the agreements vary. OpenAI's BAA, for example, covers their API but not ChatGPT or the file-upload features. Read each one and document what's in scope.

Step 2 — Data minimization in your prompts

If the model doesn't need the patient's name to answer the question, strip it. We use a deterministic PHI scrubber upstream of every LLM call: NER pass identifies entities, replaces them with stable hashes, and re-hydrates in post-processing. Auditors love it; latency cost is ~80ms.

Step 3 — Audit logging on every PHI-touching call

• Request ID, user ID, patient ID (hashed), model used, prompt hash, response hash, timestamp, latency, token counts.
• Stored in an append-only log with 6-year retention.
• Searchable by patient ID for breach response and right-to-erasure requests.

Step 4 — Documented retention and disposal

Spell out in your privacy policy and BAA how long prompts/responses are retained, where, and how they're deleted. "We don't store anything" is rarely true and never defensible.

Step 5 — Incident response plan tested in dry-run

If a model leaks PHI in a response, who gets paged, what gets shut off, what gets disclosed to whom in what timeframe? Run a tabletop exercise before you go live, not after.

The realistic timeline

4–8 weeks of engineering + legal review before your first patient-facing pilot. Plan for it. Teams that try to ship in 2 weeks end up shipping in 12 weeks plus a remediation audit.

Bottom line

HIPAA-compliant AI is achievable, but it is not a checkbox. Budget the time, hire someone who has done it before, and document everything.