AI and HIPAA: what healthcare businesses need to know

Healthcare ran headfirst into AI before most of healthcare was ready for it. Front desk staff use ChatGPT to draft patient letters. Clinicians paste de-identified (and often not-actually-de-identified) case notes into consumer AI to look up a differential. Billing teams use Copilot to summarize denial letters that contain patient names and diagnoses. Administrators ask Gemini to draft HR communications that include protected health information about specific employees. The first time a HIPAA risk assessment looks honestly at AI use in a small practice, the gap between current practice and the rules is usually wider than anyone wants to admit.

This article is the AI-specific HIPAA companion to the foundational HIPAA cybersecurity requirements article. It covers why putting protected health information into consumer AI tools is a HIPAA violation in itself, what a Business Associate Agreement (BAA) means specifically for AI vendors, which AI tools currently sign BAAs and under which licensing, what clinical and administrative AI use cases are lower-risk versus higher-risk, and how to document AI tool use for HIPAA audit purposes. The upstream pieces – the shadow AI wake-up call, the AI acceptable use policy template, and the data-side breakdown of what each AI vendor does with your inputs – sit underneath this article. If you are HIPAA-covered and have not read those yet, start there.

It is written for small healthcare practice owners, practice managers, compliance officers in small healthcare businesses, business associates serving healthcare clients (IT providers, billing companies, transcription services), and the in-house IT generalist who got handed the AI question. If you are a covered entity or a business associate, this applies to you directly.

Short answer

Using consumer AI tools (free ChatGPT, free Gemini, free Copilot, free Claude) with protected health information is a HIPAA breach in itself, even if no one outside the AI vendor sees the data. The breach is the unauthorized disclosure to a third party that does not have a BAA. The fix is not “tell staff to be careful” – it is a documented AI acceptable use policy, an approved-tools list of AI vendors that have signed BAAs under appropriate licensing (currently includes Microsoft Copilot for Microsoft 365 under HIPAA-eligible plans, ChatGPT Enterprise, the OpenAI API under specific configurations, the Anthropic API under enterprise configurations, and Google Gemini for Workspace under appropriate add-ons), workforce training on what is and is not allowed, and audit documentation that shows the controls are real. The fastest workable first move is to identify any consumer AI use happening on PHI right now and shut it off, then license the right tier of one or two approved AI tools, then write the policy and train staff. The most expensive mistake is treating AI like ordinary technology adoption – it is a HIPAA Security Rule question from the first prompt onward.

AI and HIPAA at a glance

Question	Short answer
Is ChatGPT HIPAA compliant?	ChatGPT Free and Plus are not. ChatGPT Enterprise can be, with a signed BAA under appropriate enterprise configuration. The OpenAI API can be, with HIPAA configuration and a BAA.
Is Microsoft Copilot HIPAA compliant?	Free / consumer Copilot is not. Copilot for Microsoft 365 under HIPAA-eligible M365 plans can be, with the standard Microsoft BAA in place.
Is Google Gemini HIPAA compliant?	Consumer Gemini is not. Gemini for Google Workspace under HIPAA-eligible Workspace plans can be, with the Google BAA.
Is Claude HIPAA compliant?	Consumer Claude.ai is not. The Anthropic API and Claude for Work under enterprise configurations can be, with a BAA.
Do I need a BAA to use AI?	If the AI tool will touch PHI, yes. No BAA means no PHI.
Does de-identifying data fix it?	Only if it actually meets the HIPAA de-identification standard. “I removed the name” is not de-identification.
What about AI features inside my EHR?	EHR vendors have their own BAAs that should cover their AI features – verify in writing.
Is dictation to AI scribes covered?	Only if the scribe vendor has a BAA, the configuration is HIPAA-eligible, and patients have been notified per the privacy rule.
What is the biggest violation I am at risk for?	An unauthorized disclosure breach from staff pasting PHI into a consumer AI tool, which OCR treats the same as any other unauthorized disclosure.
How do I document AI use for audit?	Approved-tools list with BAAs on file, AI use logged where possible, staff training records, AI section in the risk assessment, AI clause in the cybersecurity / acceptable use policy.

Why consumer AI on PHI is a HIPAA breach in itself

The HIPAA Privacy Rule and Security Rule together prohibit the unauthorized disclosure of protected health information to third parties. A “third party” in HIPAA terms is anyone who is not the covered entity, the patient, or a business associate under a signed BAA. A consumer AI vendor is a third party. The moment a staff member pastes PHI into a consumer AI tool, the practice has disclosed PHI to that vendor.

That disclosure is the breach. It does not matter whether the vendor’s employees ever read the conversation. It does not matter whether the model “remembers” the prompt. It does not matter whether the staff member meant well. The unauthorized disclosure happened the instant the data left the practice’s environment and entered a vendor’s infrastructure under terms the practice never agreed to.

This is the part healthcare staff usually do not internalize. The HIPAA mental model for most people is “do not send patient data to people who should not have it.” The actual HIPAA standard is “do not send patient data to entities that do not have a BAA with you.” Those are different rules. A consumer AI vendor is not the same kind of “entity” as a random outsider – it is a sophisticated data processor running on cloud infrastructure – but from a HIPAA standpoint, the lack of a BAA is what matters, not the sophistication.

The practical consequences:

• Breach notification obligations may apply. If the unauthorized disclosure rises to the level of a reportable breach (more on the analysis below), the practice has 60-day notification obligations to affected individuals and to HHS.
• OCR enforcement risk exists from the moment PHI lands in an unauthorized tool. Whether OCR actually finds out is a separate question, but the violation exists either way.
• State attorney general enforcement is possible. Several state AGs have brought HIPAA-related actions, and many state health privacy laws stack additional penalties on top.
• Civil exposure from patients is real. State privacy torts, contract claims (where the patient signed engagement paperwork with confidentiality terms), and statutory remedies in some states can be triggered by the disclosure.
• Business associate liability runs in parallel. A billing company that pastes a patient’s PHI into ChatGPT is exposed independently of its covered entity client.

The “low probability of compromise” analysis under the HIPAA Breach Notification Rule can in some narrow cases support a finding that an unauthorized disclosure was not a reportable breach. That analysis considers the nature and extent of PHI involved, the recipient, whether the PHI was actually acquired or viewed, and the extent to which the risk has been mitigated. Whether you can legitimately reach a “low probability” conclusion when PHI sat in a consumer AI vendor’s logs for an indefinite period is a hard argument to make. The default assumption should be that the disclosure is reportable until proven otherwise.

What a Business Associate Agreement means for AI vendors

A BAA is the contract that turns a third party into a business associate under HIPAA. It binds the vendor to safeguard PHI in line with the HIPAA Security Rule, to limit its uses and disclosures of PHI to those permitted by the covered entity, to report breaches, and to make its records available to the covered entity and HHS for compliance audits. Without a BAA, the vendor is just a third party. With a BAA, the vendor is a regulated party that shares HIPAA exposure with the covered entity.

For AI vendors specifically, the BAA needs to cover the particular ways AI use creates PHI exposure:

• Where PHI is processed. The vendor must commit to processing PHI only in environments configured for HIPAA – typically a specific enterprise tier with enterprise data protection terms, not the consumer or default tier.
• What happens to the prompts and outputs. The vendor must commit not to use PHI for training future models, not to allow routine human review of PHI-containing prompts, and to retain logs only as required for safety and operations.
• Subprocessors. Many AI vendors run on cloud infrastructure (Azure, AWS, Google Cloud) and use upstream model providers (OpenAI’s models, Anthropic’s models, Google’s models). The BAA needs to either pass through to those subprocessors under their own BAAs or explicitly limit subprocessing to entities under BAAs.
• Breach notification. The vendor must commit to notifying the covered entity of breaches within a defined window – typically much shorter than the 60-day HHS notification window so the covered entity has time to do its own analysis and notifications.
• Termination and data return / destruction. The BAA must specify what happens to PHI in the vendor’s possession when the relationship ends. For AI, this is more complex than it sounds because prompts may be embedded in logs, embeddings, telemetry, and safety review systems that are not easy to fully purge.
• HIPAA-eligible configuration. Most AI vendors offer HIPAA coverage only under specific configurations – particular API endpoints, particular enterprise tiers, particular regions. The BAA either incorporates these configuration requirements by reference or names them explicitly.

A BAA is not a “we are HIPAA compliant” sticker. It is a contractual allocation of HIPAA risk between you and the vendor. Get a copy in your files, read it, and make sure your actual use of the tool matches what the BAA actually covers. The most common AI HIPAA failure pattern is having a BAA on file for “ChatGPT Enterprise” but staff using ChatGPT Plus on their own personal accounts. The BAA only covers what the BAA covers.

Which AI tools currently sign BAAs

Vendor BAA terms change. Always verify with the vendor at the moment you license. The shape below describes the pattern as of the most recent stable terms in each vendor’s HIPAA documentation. Treat this as a directional guide, not a binding compliance reference.

Microsoft

Microsoft offers a Business Associate Agreement that covers HIPAA-eligible workloads on:

• Microsoft 365 (including Exchange Online, SharePoint Online, OneDrive for Business, Teams, Outlook) under appropriate enterprise plans
• Copilot for Microsoft 365 under HIPAA-eligible M365 licensing – the licensed Copilot product, not the free consumer Copilot or Copilot Pro
• Microsoft 365 Copilot Chat for organizations with eligible M365 licenses
• Azure OpenAI Service under HIPAA-eligible Azure configurations – this is the path to use OpenAI’s models inside Microsoft’s HIPAA framework
• Power Platform with AI Builder, under appropriate licensing
• Microsoft Intune, Defender, Entra ID and the rest of the M365 security stack

The Microsoft BAA is offered automatically to customers on eligible commercial plans through the standard commercial terms. There is no separate negotiation in most cases. The most important detail: the BAA covers the licensed Copilot for Microsoft 365 product, not the consumer Copilot accessible from any web browser without M365 licensing. Audit which version each staff member is using before relying on the BAA.

OpenAI

OpenAI offers a BAA on:

• ChatGPT Enterprise under specific HIPAA configuration with the standard OpenAI BAA
• The OpenAI API under specific HIPAA configuration with a BAA – typically requires a request through OpenAI’s enterprise channel

ChatGPT Free, ChatGPT Plus, and ChatGPT Team are not currently offered with a BAA. If your practice wants OpenAI’s models for clinical or administrative AI work that touches PHI, the path is either ChatGPT Enterprise (for the chat interface) or the API (typically through a HIPAA-aware application built on the API).

Anthropic

Anthropic offers a BAA on:

• The Anthropic API under enterprise configurations
• Claude for Work (Team and Enterprise) under appropriate enterprise terms

Claude.ai consumer (Free, Pro, Max) is not currently offered with a BAA. If a practice wants Claude for PHI-adjacent work, the path is Claude for Work / Enterprise or the Anthropic API.

Google

Google offers a BAA on Google Workspace under HIPAA-eligible plans. Gemini for Google Workspace is covered under the Workspace BAA framework for customers on appropriate Workspace plans with the right Gemini add-on. Consumer Gemini in a personal Google account is not covered.

Google Cloud’s Vertex AI (which includes Gemini API access) is HIPAA-eligible under appropriate Google Cloud configuration with a BAA.

Healthcare-native AI vendors

Several vendors sell AI products explicitly designed for healthcare, with HIPAA compliance built in as a default rather than an add-on:

• Ambient clinical documentation / AI scribe vendors. Nuance (now part of Microsoft, with the DAX line), Abridge, Suki, Augmedix, DeepScribe, Heidi Health, Freed, and others. Most offer BAAs as standard. The differences are in workflow, accuracy, EHR integration, and pricing.
• EHR-native AI features. Major EHR vendors (Epic, Athenahealth, NextGen, eClinicalWorks, Practice Fusion, Kareo, AdvancedMD, DrChrono) have been adding AI features. These should be covered under the EHR vendor’s existing BAA, but verify in writing – some AI features are explicitly excluded from the standard BAA pending separate configuration.
• Healthcare-specific assistants. Doximity GPT, OpenEvidence, Glass Health, and similar tools that are designed for clinical use. BAA availability and terms vary; always verify.

The advantage of healthcare-native AI is that the vendor has typically already engineered around HIPAA constraints – the workflow assumes PHI is in scope and the data flows are designed for it. The disadvantage is that healthcare-native AI is often more expensive and more narrowly scoped than general-purpose AI. Most small practices end up using a mix: a general-purpose AI tool with a BAA for administrative work, and a healthcare-specific AI tool for clinical documentation or research.

Embedded AI inside SaaS tools

The same warning from the general AI data privacy article applies even more sharply in healthcare. Zoom AI Companion, Otter.ai, Fireflies, Read.ai, Notion AI, Slack AI, HubSpot Breeze, AI features inside HR / payroll / scheduling tools, AI features inside survey tools – each one ingests data and routes it through an AI service. If any of those data streams touches PHI, the embedded AI feature needs to be covered by the vendor’s BAA, or it needs to be turned off, or the data flow needs to be re-routed so PHI does not reach it.

The default assumption for embedded AI features should be that they are not HIPAA-covered unless the vendor explicitly says they are. Many SaaS vendors offer HIPAA-compliant tiers of their main product but exclude the AI features from BAA coverage. Read the BAA exhibit carefully.

De-identification: when it actually fixes the problem, and when it does not

A common workaround pattern: “We remove the patient name before pasting into ChatGPT, so it is not really PHI anymore.” This is almost never sufficient under HIPAA.

HIPAA recognizes two methods of de-identification:

• Safe Harbor. Remove 18 specified identifiers (names, geographic data smaller than state, all elements of dates except year for dates directly related to the individual, telephone numbers, fax numbers, email addresses, SSNs, medical record numbers, health plan beneficiary numbers, account numbers, certificate / license numbers, vehicle identifiers, device identifiers, URLs, IP addresses, biometric identifiers, full-face photos, and any other unique identifying number, characteristic, or code), and the covered entity must have no actual knowledge that the remaining information could identify the individual.
• Expert Determination. A qualified statistical expert documents that the risk of re-identification is very small, considering the data and the recipient.

Practical reality in a small practice: full Safe Harbor de-identification of an arbitrary clinical note is hard. Dates of service, ages over 89, specific procedures on specific days, narrative context, geographic specifics – any combination can be re-identifying. A note that says “75-year-old male with a history of [specific rare condition], status post [specific procedure] on [date], presenting with [specific complication]” is not de-identified in the practical sense even if the patient’s name and MRN are stripped, because the combination of facts narrows the population enough that re-identification is plausible.

Limited Data Sets are a separate concept: PHI with most direct identifiers removed but still containing dates and some geographic information. Limited Data Sets can be disclosed for specific purposes (research, public health, healthcare operations) under a Data Use Agreement, but they are still PHI. They cannot be put into a consumer AI tool just because they are “limited.”

The practical rule of thumb for AI use:

• If the data contains any of the 18 Safe Harbor identifiers, it is PHI. It cannot enter a non-BAA AI tool.
• If the data has been Safe-Harbor-stripped but contains rich enough clinical context to plausibly re-identify, treat it as PHI. It cannot enter a non-BAA AI tool.
• If the data has been Expert-Determination-de-identified with documentation, it can be used in a non-BAA AI tool, subject to vendor terms. This is rare in small practice workflow.
• If the data is genuinely about a hypothetical patient (“draft a patient education sheet for someone with [condition]”), it is not PHI. AI tools are useful here without HIPAA exposure.

The de-identification workaround is most useful as a workflow design principle: structure clinical AI tasks so the prompt is generic and the patient-specific work happens in the BAA-covered EHR or BAA-covered AI tool. “Draft a patient education sheet about [condition]” in a non-BAA tool is fine; “draft a patient education sheet for Mrs. Smith about her [condition]” in the same tool is a breach.

Lower-risk vs higher-risk AI use cases in healthcare

Not every AI use case in healthcare is equally exposed. Stratifying use cases by HIPAA risk helps a small practice know where to invest in BAA-covered tools and where free or paid general-purpose AI is fine for staff productivity.

Lower-risk: AI on non-PHI tasks

These use cases do not touch PHI in any meaningful way and can be done in any approved general-purpose AI tool, including (in many cases) consumer tiers used carefully under your policy.

• Drafting generic patient education content not tied to a specific patient
• Drafting marketing copy, website content, and external communications
• Summarizing public guidelines, journal articles, or research not tied to a specific patient
• Brainstorming office workflow improvements
• Drafting generic HR documents, job descriptions, and policy templates that do not name employees with sensitive accommodations
• Drafting generic vendor communications that do not include patient information
• Learning a new clinical topic or technology in a general way

These are the AI use cases where most of the productivity benefit lives, and they are also the ones with the least HIPAA exposure. Doing them well in your approved AI tools is a real efficiency gain without compliance cost.

Medium-risk: administrative AI with PHI under BAA-covered tools

These use cases touch PHI but in ways that are reasonable to handle inside a BAA-covered AI tool. The data is still sensitive and the controls still need to be in place, but the use case is well-bounded.

• Drafting patient-specific letters, statements, denial appeals, or follow-up communications inside Copilot for M365 (where the document and patient context are inside the M365 tenant)
• Summarizing patient correspondence inside the EHR’s AI features (where the AI feature is covered under the EHR BAA)
• Drafting prior authorization narratives in a BAA-covered AI tool from PHI-containing inputs
• Coding and billing assistance using AI features in a BAA-covered billing platform
• Administrative HR work involving employee health information in a BAA-covered AI tool (if applicable – employee health information may be PHI in some contexts, ERISA-covered in others)
• Internal practice analytics using PHI in a BAA-covered analytics or BI tool with AI features

The discipline is straightforward: PHI never leaves the BAA-covered toolset, and the toolset is configured for HIPAA. The work is still inside the practice’s compliance boundary.

Higher-risk: clinical AI on patient care decisions

Clinical AI – AI that influences diagnosis, treatment, or care decisions – sits in a separate risk category from administrative AI for several reasons that go beyond HIPAA.

• FDA regulation. Some clinical AI tools meet the FDA definition of a medical device (Software as a Medical Device, SaMD) and require clearance, registration, or specific intended use boundaries. Using a general-purpose AI tool for clinical decisions you would normally make with regulated diagnostic equipment is a separate legal issue from HIPAA.
• Standard of care. If AI is used in a way that contributes to a treatment decision, the clinician remains responsible for the standard of care. AI hallucinations are documented and clinically dangerous – the model will confidently invent drug interactions, contraindications, or treatment recommendations that do not exist. The clinician who acted on the hallucination is professionally and legally exposed.
• Documentation. Clinical decisions need to be defensible in the chart. “AI suggested it” is not a defensible chart entry. Whether AI was consulted, and how its output was used, needs documentation policy.
• Patient consent and disclosure. Some state laws and emerging HHS guidance require disclosure to patients when AI is used in their care in certain ways. Patient-facing AI – chatbots, scheduling assistants, intake summarizers – typically warrants explicit notice in the practice’s privacy notice and intake paperwork.
• AI scribes deserve special care. Ambient AI scribes (Nuance DAX, Abridge, Suki, Augmedix, Heidi, Freed, and others) capture clinician-patient conversations. They are extraordinarily useful and they are also a recording of a clinical encounter. The vendor needs a BAA, the configuration needs to be HIPAA-eligible, patients need notice (state laws on recording vary – some are one-party consent, others require explicit patient awareness), the practice needs a defined retention and review policy for the recordings, and the clinician retains responsibility for the accuracy of the AI-generated note.

Higher-risk does not mean “do not use.” It means “do not use without policy, training, BAA, configuration check, and documented use limits.” Many practices are getting real value out of clinical AI by deploying it deliberately rather than ad hoc.

How to document AI use for HIPAA audit purposes

OCR investigations and HIPAA audits work from documentation. If a control exists only in people’s heads, it does not exist for compliance purposes. The same rule applies to AI controls. The documentation that should be on file for a small healthcare practice’s AI use:

1. AI section in the risk assessment. Your annual HIPAA risk assessment should explicitly cover AI use – what AI tools are in use, what PHI flows through them, what threats and vulnerabilities exist, what controls mitigate them, what residual risk remains. AI does not need its own separate risk assessment; it needs to be inside the main one.
2. AI acceptable use policy. A signed, dated, distributed policy covering approved AI tools, prohibited uses, data classification rules, and consequences. The AI acceptable use policy template walks through the structure.
3. Approved-tools list with BAAs on file. A single list (in your policy or as an appendix) naming the approved AI tools and the licensing tier of each, with a copy of each vendor’s BAA in your compliance binder or document management system. When OCR asks “what AI tools does your practice use and do you have BAAs?”, the answer is “this list, and yes, BAAs are at [location].”
4. Workforce training records. Documentation that staff have been trained on the AI policy, including the date of training, the content covered, and the staff member’s acknowledgment. HIPAA already requires workforce training; the AI portion can sit inside the existing training cadence.
5. Audit logs where available. For BAA-covered AI tools that log usage (Copilot for M365, ChatGPT Enterprise, the EHR’s AI features), retain the logs per your audit log retention policy. For tools that do not log at the level you need, document the limitation.
6. Incident logs. Any incident involving AI use – a staff member who pasted PHI into a non-BAA tool, a vendor incident reported under the BAA, a clinical AI output that contributed to a near-miss – should be logged. The log is the evidence that the practice noticed and responded.
7. AI clause in the cybersecurity policy. The broader cybersecurity policy should reference the AI policy as a sibling document. Auditors look for the cross-references.
8. Vendor BAAs and configuration attestations. Beyond the BAA itself, the vendor’s HIPAA configuration documentation (what specific settings need to be in place, what specific endpoints or tiers are HIPAA-eligible) should be in the file. If the BAA says “PHI may be processed only on the HIPAA-eligible configuration,” you need evidence that you are actually on that configuration.
9. Patient-facing disclosures where required. If the practice uses AI scribes or patient-facing AI, the notice in the privacy notice, intake paperwork, or signage should be archived. Some practices add an explicit AI disclosure to the Notice of Privacy Practices.
10. Review cadence evidence. The AI policy should be reviewed at least annually (twice a year is more realistic in 2026). Keep evidence of each review – even a one-line note in the policy’s version history is better than nothing.

Practices that maintain this documentation pass AI-related HIPAA audit questions easily. Practices that have AI use happening without this documentation can have controls that are excellent in practice and still fail an audit because there is nothing on paper.

The first move for a small healthcare practice

A pragmatic sequence for a practice that has not yet confronted its AI use:

Step 1: stop the bleeding. Identify any consumer AI use happening on PHI right now and shut it off. This is a 30-minute staff conversation: “We are going to formalize AI use in the practice. Until we do, do not paste any patient information, employee health information, or anything that could reasonably identify a patient into ChatGPT, Gemini, Copilot, Claude, or any other AI tool that is not the EHR.” Acknowledge that staff have likely been doing this – the conversation works much better without blame.

Step 2: pick the first BAA-covered AI tool. For most small practices already on Microsoft 365, the path of least resistance is licensing Copilot for Microsoft 365 under HIPAA-eligible M365 plans. The licensing piggybacks on what is already in place. For practices on Google Workspace, Gemini for Google Workspace under a HIPAA-eligible plan is the equivalent path. For practices that want broader AI capabilities beyond their primary suite, ChatGPT Enterprise or Claude for Work under their respective BAAs is the second tool.

Step 3: confirm the BAA. Get a signed BAA on file from the vendor. For most enterprise plans this is offered automatically through the commercial terms; in some cases (typically smaller plans) it requires an explicit request. Confirm in writing that the BAA covers the specific tier and configuration you are licensing.

Step 4: pick the clinical AI use case, if any, you want to enable. If you want ambient AI scribing, pick a vendor (Nuance DAX, Abridge, Suki, Augmedix, Heidi, Freed – the right pick depends on EHR integration, specialty fit, and price). Confirm BAA, confirm HIPAA-eligible configuration, confirm patient notice approach.

Step 5: write the policy. Adapt the AI acceptable use policy template to your practice. Name the approved tools. Name the prohibited tools. Reference the data classification (which in healthcare is largely “PHI vs. non-PHI”). Define consequences.

Step 6: train staff. Single staff meeting, 20-30 minutes. Walk through what is approved, what is prohibited, what to do if something goes wrong (reporting without immediate punishment). Sign acknowledgments. Repeat at onboarding for new hires.

Step 7: update the risk assessment. Add the AI section to your annual HIPAA risk assessment. If your last risk assessment was a year ago and AI was not in it, this is a good time to revisit it.

Step 8: document. File the BAAs, save the policy with signatures, save the training records, save the approved-tools list, save the risk assessment update.

Step 9: tell your business associates and your covered entity clients. If you are a covered entity, tell your IT provider, billing company, and other business associates that AI use is now governed – they should confirm their own AI controls. If you are a business associate, tell your covered entity clients about your AI controls before they ask.

Step 10: re-evaluate quarterly. Vendor terms, tools, and use cases change fast. Quarterly is realistic for a regulated practice in 2026.

Worked example: a 4-provider primary care practice

A small primary care practice with 4 providers, 8 clinical staff, and 4 administrative staff. EHR is Athenahealth. M365 Business Premium across the office. Billing is internal. No current AI policy.

The honest discovery:

• Two MAs are using ChatGPT free for drug interaction lookups. They sometimes paste patient context for accuracy.
• The office manager uses Gemini consumer to draft difficult patient letters – sometimes with patient names and conditions in the prompt.
• One provider uses ChatGPT Plus on her personal phone to draft denial appeals from EHR-exported PHI.
• The front desk uses Otter.ai (free) to record staff meetings; recordings sometimes include patient discussion.
• The practice has signed BAAs for Athenahealth, Microsoft (covering M365), and the IT provider. No BAAs for any consumer AI tool.

The remediation plan:

• Immediate (this week): Stop all consumer AI use on anything patient-related. Communicate to staff in a 20-minute meeting. Disable Otter.ai or upgrade it to a BAA-covered tier.
• Within 30 days: Upgrade three of the four provider staff (and the office manager) to Copilot for Microsoft 365 ($30 per user per month). Continue using the Athenahealth-native AI features for in-EHR work under the existing BAA. Evaluate one ambient AI scribe vendor (Abridge or Heidi for primary-care fit; Nuance DAX if the practice prefers Microsoft alignment); pilot with one provider for 30 days.
• Within 60 days: Adopt the AI acceptable use policy. Train staff. Update the risk assessment. File the BAAs (Microsoft’s BAA covers Copilot for M365 already; pick up the scribe vendor’s BAA).
• Within 90 days: Confirm patient notice approach for the ambient scribe (update the Notice of Privacy Practices, add intake-form language, train staff on the verbal notice). Document quarterly review cadence.
• Cost: Around $150-$170 per month for Copilot licensing (5 staff at $30 each) + scribe vendor pricing (varies by vendor, typically $200-$500 per provider per month if rolled out broadly).
• Risk position 90 days later: Documented BAA-covered AI use, written policy, trained staff, risk assessment updated, scribe pilot data informing rollout decision. Audit-ready.

Most small practices land somewhere in this range – a few hundred dollars per month in incremental licensing, a few hours of admin work, a defensible compliance posture.

How this fits with the rest of your HIPAA program

AI does not sit apart from the rest of HIPAA compliance. The structural relationships:

• HIPAA risk assessment. The annual risk assessment must cover AI use. AI is not a separate compliance regime; it is a new vector inside the existing Security Rule framework. The HIPAA cybersecurity requirements article covers the broader risk assessment expectations.
• Cybersecurity policy. The cybersecurity policy sets the data classification framework that the AI policy uses. PHI is the regulated tier; AI rules apply on top.
• AI acceptable use policy. The AI acceptable use policy is where AI-specific rules get codified for staff and signed.
• Data privacy and AI tool selection. The what data are you feeding into AI tools breakdown is the vendor-by-vendor reference – which tools currently sign BAAs, under which licensing, with which configuration.
• Shadow AI awareness. The shadow AI wake-up call is the upstream problem statement – in healthcare, shadow AI is not just a productivity issue, it is a compliance breach in itself.
• BAA management. Every business associate needs a BAA. AI vendors are no different. Practices that already have a BAA inventory should add AI tools to it; practices that do not should build one starting with AI.
• Workforce training. HIPAA already requires workforce security awareness training. AI should be a section inside that training, not a separate program.
• Cyber insurance. Insurers are now asking about AI use during renewals. Healthcare insurers are asking more pointed questions. The cyber insurance for small business overview covers the baseline.
• Managed cybersecurity engagement. If your practice runs through a managed cybersecurity provider, the AI governance work should be inside the engagement – the provider’s risk assessment, policy library, training program, and audit documentation should all already cover AI.

10 common AI HIPAA mistakes

The patterns that show up when small healthcare practices hit AI compliance gaps:

1. Assuming consumer AI use is a small problem. It is a HIPAA breach in itself. Treating it as a productivity hygiene issue rather than an unauthorized disclosure issue mis-frames the response.
2. Relying on “I removed the name” de-identification. Names are one of 18 Safe Harbor identifiers. Removing only the name almost never meets the de-identification standard.
3. Confusing Microsoft’s products. Free Copilot, Copilot Pro, Copilot for Microsoft 365, and Microsoft 365 Copilot Chat are different products with different BAA coverage. Audit which version each staff member is using.
4. Confusing OpenAI’s products. ChatGPT Free, Plus, Team, Enterprise, and the API have different BAA coverage. Only Enterprise and specific API configurations are BAA-eligible.
5. Forgetting embedded AI inside SaaS. Zoom AI Companion, Otter.ai, Notion AI, AI features inside HR / payroll / scheduling / survey tools. Each one is a potential PHI flow that needs BAA coverage.
6. Skipping the BAA configuration step. Having a BAA on file but using the tool outside the HIPAA-eligible configuration is the same as not having a BAA for that use case.
7. Not documenting AI in the risk assessment. OCR’s first ask is the risk assessment. If AI is in use and not in the risk assessment, that gap is visible immediately.
8. Deploying clinical AI without disclosure. AI scribes especially – patients should know. Some state laws require it; HHS guidance leans toward it; client trust depends on it.
9. Using “AI suggested it” as a clinical defense. AI does not carry standard of care responsibility. The clinician does.
10. Treating the AI policy as a one-time deliverable. Vendor terms shift, tools change, use cases expand. Quarterly review is the realistic cadence for a regulated practice.

Time to set up AI HIPAA governance

A workable plan for a typical small healthcare practice (3-10 providers, 10-30 staff):

Phase	What happens	Time
Immediate consumer AI shutdown	Identify current shadow AI use, communicate stop in a staff meeting, disable obvious risky tools	1 week
Tool selection and licensing	Pick BAA-covered AI tools, secure licensing, confirm BAAs in writing	2-3 weeks (includes procurement and configuration)
AI scribe pilot (if pursuing)	Pick vendor, run 30-day pilot with one provider, evaluate	4-6 weeks
Policy writing	Adapt the AI acceptable use policy template for healthcare	4-8 hours of drafting
Risk assessment update	Add AI section to the annual risk assessment	4-8 hours
Staff training	Single meeting + onboarding update for new hires	2 hours of meeting + ongoing
Patient notice update (if scribe)	Update Notice of Privacy Practices, intake paperwork, staff verbal-notice training	1-2 weeks
Documentation	File BAAs, save policy, training records, configuration attestations	Ongoing
Total elapsed time	From “we should do this” to “we have done this”	4-8 weeks

The eight-week version is the realistic version for a practice that wants AI scribing rolled out as well. The four-week version is the policy-and-licensing-only version that gets the practice into a defensible compliance position quickly with clinical AI deferred to a later decision.

What is next in this content series

This article covered the AI + HIPAA intersection – why consumer AI on PHI is a breach, what BAAs mean for AI, which vendors sign them, lower-risk vs higher-risk use cases, and audit documentation. The follow-ups go deeper into adjacent topics:

• AI security risks every small business should know about (prompt injection, AI-generated phishing, voice cloning, deepfakes) – relevant in healthcare given the high-value targeting
• Microsoft Copilot for small business specifically, including the M365 permission hygiene that has to happen before rollout
• How to build a lightweight AI governance framework for SMBs
• How cybercriminals are using AI to attack small businesses, with a healthcare angle on patient-impersonation phishing and clinic-targeted fraud
• How to evaluate any AI tool for business use, including the BAA and configuration questions to ask

If you have not read them yet, the upstream pieces are the shadow AI wake-up call, the AI acceptable use policy template, and the data-side breakdown of AI vendor terms. The foundational HIPAA piece this article sits on top of is the HIPAA cybersecurity requirements article.

If your practice’s HIPAA program is part of a broader managed cybersecurity engagement, AI governance should be inside it – not separate.

How Sequentur can help

If you want help auditing AI use in your practice, picking the right BAA-covered AI tools, updating your risk assessment to cover AI, or building the policy and documentation that survives a HIPAA audit, schedule a call.