How to evaluate whether an AI tool is safe for your business to use

Someone on your team found a new AI tool. It transcribes meetings, or it writes proposals, or it answers customer emails, and it is very good at the thing it does. They want to start using it on real work. Now you have to answer a question you probably do not have a process for yet: is this tool safe for the business to use? Most small businesses answer it by feel, or by asking “is it popular,” or by not answering it at all and letting the tool get used anyway. None of those is an evaluation. This article gives you one.

This is a practical vetting guide, not a procurement framework built for an enterprise security team. It walks through the specific questions to ask before you approve any AI tool – where your data goes, whether there is a real business tier, whether the tool trains on your inputs, whether a Business Associate Agreement is available if you are regulated, what the retention and deletion policies actually say, and whether the tool fits the identity and access controls you already run. At the end there is a reusable evaluation checklist you can copy, fill in once per tool, and keep on file. That checklist is the artifact. Everything before it explains how to fill it in honestly.

It is written for the in-house IT generalist, operations manager, or owner who gets handed “can we use this?” and needs a repeatable way to say yes, no, or not yet. If you have already built an AI governance framework, this is the evaluation step that feeds your approved tools list. If you have not, this is a reasonable place to start, because vetting one tool well teaches you most of what the framework needs.

Short answer

To evaluate whether an AI tool is safe for your business, work through six questions in order: where does your data physically go and who can see it, is there a paid business or enterprise tier with real data protection terms, does the tool train its models on your inputs by default, is a Business Associate Agreement or equivalent contract available if you handle regulated data, what are the data retention and deletion policies, and does it integrate with your existing identity and access management so you control logins and offboarding centrally. A tool that gives good answers to all six is a candidate for approval at the data tier those answers support. A tool that fails any one of them for the data you intend to use is either restricted to a lower data tier or rejected. The fastest workable version of this is the one-page checklist at the end of this article, filled in once per tool and kept on file. The most common mistake is evaluating the tool’s features instead of its data handling – the feature set tells you whether the tool is good, not whether it is safe.

AI tool evaluation at a glance

Question	What you are checking	Why it matters
Where does my data go?	Hosting region, sub-processors, who can access it	Determines jurisdiction, exposure, and which compliance rules apply
Is there a business or enterprise tier?	Whether a paid plan with data protection terms exists	Free and consumer tiers almost never have the terms a business needs
Does it train on my inputs?	Default training behavior and whether you can opt out	Training on inputs can mean your data resurfaces or is retained indefinitely
Is a BAA available?	Whether the vendor will sign a Business Associate Agreement	Required before any PHI touches the tool under HIPAA
What are retention and deletion policies?	How long data is kept and whether you can delete it	Drives breach scope, compliance, and what happens when you leave
Does it fit my IAM?	SSO, MFA, central provisioning and deprovisioning	Controls who can log in and whether offboarding actually removes access
Who owns the output?	Contract terms on generated content	Affects IP, confidentiality, and reuse of what the tool produces
Is there an audit trail?	Logging of who used it and what they did	Needed for incident response and compliance evidence
What is the vendor’s security posture?	Certifications, breach history, transparency	A proxy for whether the vendor takes security seriously
What data tier is it cleared for?	The output of all the above	The decision: public only, internal, or sensitive and regulated

Why feature evaluation is not safety evaluation

The trap almost every small business falls into is evaluating an AI tool the way you would evaluate any other software: does it do the job, is it easy to use, is the price reasonable. Those are the right questions for whether a tool is good. They are the wrong questions for whether it is safe. A tool can be excellent at its job and still be a data-exposure problem, because the thing that makes AI tools useful – you feed them your real information and they do something with it – is also the thing that makes them risky.

When you put a document into a traditional SaaS tool, you usually have a rough sense of where it lives and who can reach it. When you put a document into an AI tool, several less obvious things can happen. The text may be sent to a model hosted by a third party the vendor does not control. It may be retained to “improve the service,” which can mean used for training. It may be readable by vendor staff for abuse monitoring or debugging. It may be passed to sub-processors in other countries. None of that shows up in the feature list. All of it matters for whether the tool is safe for a given category of your data.

So the evaluation has to be a separate pass. Decide the tool is good first – that is the easy part and your team can do it. Then run the safety evaluation before anyone uses it on anything beyond public, throwaway data. The two questions do not answer each other, and conflating them is how regulated data ends up in a free chatbot because “everyone said it was great.”

The six questions that decide whether a tool is safe

These six are the core of the evaluation. Work through them in order, because each one narrows what data the tool can safely handle, and a hard no on an early one can save you the rest of the work.

Question 1: Where does your data go?

Before anything else, find out where the data you put into the tool physically goes and who can touch it along the way. The questions to answer:

• Where is it hosted? Which cloud, which region, which country. This decides the legal jurisdiction your data falls under and whether that creates a problem for clients with data residency requirements.
• Who are the sub-processors? Many AI tools are wrappers around someone else’s model. A small vendor’s “AI assistant” may send every input to a large model provider behind the scenes. You need to know who is actually processing your data, not just whose logo is on the login page. Reputable vendors publish a sub-processor list.
• Who at the vendor can see it? Some vendors let staff review inputs for abuse monitoring, debugging, or quality. Some encrypt data so even they cannot read it. Find out which.
• Is it isolated from other customers? Multi-tenant is normal and fine, but you want to confirm your data is logically separated and not pooled in a way that lets it surface for another customer.

If the vendor cannot or will not answer where your data goes and who can see it, that is itself the answer. A vendor that is vague about data handling is telling you how much they have thought about it.

Question 2: Is there a real business or enterprise tier?

The single most reliable predictor of whether an AI tool is safe for business use is whether the vendor sells a paid business or enterprise plan with actual data protection terms – and whether you are on it. The free and consumer-paid tiers of the same product frequently have completely different data handling. The free tier may train on your inputs and retain them indefinitely; the business tier of the identical product may contractually commit to neither. The data-side breakdown of how AI vendor tiers differ goes deep on this, because it is the most misunderstood part of AI tool safety.

What you are checking:

• Does a business, team, or enterprise tier exist at all? Some tools are consumer-only, which usually rules them out for anything but public data.
• Does that tier come with a data processing agreement or enterprise data protection terms in writing? Marketing language (“we take privacy seriously”) is not terms. You want the actual contractual commitment.
• Are you actually on it? A business that approves “ChatGPT” without specifying the paid Team or Enterprise tier has approved nothing safe, because staff will use the free tier.

A consumer-only tool, or a business on the free tier of a tool that offers a better one, is restricted to public data at most. Full stop.

Question 3: Does it train on your inputs?

This is the question that surprises people most. Many consumer AI tools use your inputs to train or improve their models by default, and the opt-out is buried in settings or only available on a paid tier. Training on your inputs matters for two reasons: your data is retained to do it, and there is a small but real risk that information from your inputs influences outputs seen by other users.

What to confirm:

• Default behavior. Does the tool train on inputs unless you turn it off, or is the business tier no-training by default? Default matters because defaults are what actually happen.
• Whether you can opt out, and where. If opting out requires every individual user to find a settings toggle, assume most will not. You want an account-level or contractual no-training guarantee, not a per-user checkbox.
• Whether the no-training commitment is contractual. A toggle a vendor can change in a future update is weaker than a term in your data processing agreement.

For any data above the public tier, “does not train on your inputs” should be a hard requirement, not a nice-to-have.

Question 4: Is a BAA or equivalent contract available?

If your business handles regulated data, the evaluation gains a hard gate. For healthcare data, that gate is the Business Associate Agreement. Under HIPAA, no protected health information can go into an AI tool unless the vendor has signed a BAA covering that use – and most consumer AI tools will not sign one. The AI and HIPAA breakdown covers which tools offer BAAs and what the agreement actually obligates the vendor to do.

The same logic applies beyond healthcare. If you handle data covered by a contractual or regulatory obligation – client data under NDA, payment data, regulated financial records – you need the contract that covers it before the data touches the tool. For most SMBs the practical checks are:

• Is a BAA available for PHI, and will the vendor sign yours or provide theirs?
• Is a data processing agreement available for personal data covered by privacy laws?
• Do the contracts cover the specific data type you intend to process, not just “data” in the abstract?

No applicable contract means the tool is restricted to data that does not require one. This is the gate that keeps a regulated business out of trouble, and it is non-negotiable for the regulated data tier.

Question 5: What are the data retention and deletion policies?

Retention and deletion answer two questions you will care about later: how big is the exposure if this vendor is breached, and can you actually get your data out and gone when you stop using the tool. Vague retention policies are a quiet liability, because data you forgot the tool was holding is still data that can leak.

Check:

• How long is input data retained? Some tools keep inputs for a fixed window for abuse monitoring then delete; some keep indefinitely. Shorter, defined retention is better.
• Can you delete data on demand? If a staff member pastes something they should not have, can you remove it, and how fast? This connects directly to AI data leak incident response – a tool you cannot delete from turns a small mistake into a permanent exposure.
• What happens when you cancel? Confirm data is deleted on offboarding and that you can get an export first if you need one. “Data is retained for 90 days after cancellation” is a fact you want to know before you sign, not after you leave.

Question 6: Does it fit your identity and access management?

The last core question is operational rather than contractual: can you control who logs in and, just as importantly, cut off access when someone leaves. An AI tool that staff sign into with personal accounts is a tool you do not control and cannot offboard, which means a departing employee may keep access to it – and to whatever business data they put in it – indefinitely.

What good looks like:

• Single sign-on (SSO) so logins go through your identity provider (Microsoft Entra, Google Workspace) rather than standalone passwords.
• Multi-factor authentication, ideally enforced through your SSO rather than configured per user.
• Central provisioning and deprovisioning so you can grant access when someone joins and, critically, revoke it the moment they leave. Offboarding that does not remove AI tool access is a gap that shows up months later.
• Role-based access if the tool handles different data sensitivities for different teams.

A tool with no SSO and no central control is not automatically disqualified for small, low-sensitivity use, but it cannot hold a meaningful role in a business that handles sensitive data, because you cannot prove who has access or close it cleanly.

The secondary questions worth asking

The six core questions decide safety. These additional ones sharpen the picture and are worth answering for any tool that will see internal or sensitive data.

• Who owns the output? Read the terms on generated content. Most reputable tools assign output ownership to the customer, but some retain rights, and a few have ambiguous terms that matter if the output is client-facing or proprietary.
• Is there an audit trail? Can you see who used the tool and, ideally, what they did with it? You will want this for incident response and for any compliance evidence a client or insurer asks for.
• What is the vendor’s security posture? Certifications like SOC 2 Type II or ISO 27001 are not guarantees, but they show the vendor submitted to an external audit. A documented breach history handled transparently is better than a clean-looking vendor that hides incidents.
• How stable is the vendor? AI is a fast-moving market with a lot of new entrants. A tool from a six-month-old startup may not exist in a year, which is a different risk than a feature inside an established platform you already trust.
• Does it touch your other systems? A tool that connects to your email, files, or CRM has far more access than a standalone chatbot. The more it integrates, the more carefully the first six questions need answering.

Turning answers into a decision

Once the questions are answered, the decision is not pass/fail – it is which data tier the tool is cleared for. Map the answers to one of three outcomes, using the same three-tier model your AI acceptable use policy should already use:

• Cleared for sensitive and regulated data. Business or enterprise tier, no training on inputs by contract, applicable BAA or DPA signed, defined retention and on-demand deletion, SSO with central offboarding. This is the bar for PHI, PII, payment data, and contractually restricted client data.
• Cleared for internal data only. Business tier with no-training terms and reasonable retention, but missing a piece needed for regulated data (no BAA, or no SSO). Fine for internal operational data, not for regulated or contractually restricted data.
• Cleared for public data only, or rejected. Consumer-only tier, trains on inputs with no contractual opt-out, vague on retention, no central access control. Usable for genuinely public, throwaway content if at all. Not for anything that matters.

Write down the tier and the date. A tool cleared today can drift – vendors change terms, add features, and get acquired – so the clearance is a point-in-time decision, not a permanent one. This is exactly why the governance framework schedules re-evaluation rather than approving once and forgetting.

The reusable AI tool evaluation checklist

Copy this, fill it in once per tool, and keep the completed copies on file. It is deliberately one page. The goal is a record you can show a client, an insurer, or an auditor that proves you evaluated the tool before approving it, and a consistent process so every tool gets vetted the same way.

AI TOOL EVALUATION CHECKLIST

Tool name:
Vendor:
Tier evaluated (free / business / enterprise):
Intended use case:
Evaluated by:
Date:

1. WHERE DOES THE DATA GO?
[ ] Hosting region / country known:
[ ] Sub-processors identified (who actually processes inputs):
[ ] Vendor staff access to inputs (none / abuse-monitoring / debugging):
[ ] Customer data isolation confirmed:

2. BUSINESS / ENTERPRISE TIER
[ ] Paid business or enterprise tier exists:
[ ] Data processing terms in writing (not marketing language):
[ ] We are on the paid tier (not the free one):

3. TRAINING ON INPUTS
[ ] Default training behavior (trains / does not train):
[ ] Opt-out available and how (per-user toggle / account-level / contractual):
[ ] No-training commitment is contractual:

4. REGULATED-DATA CONTRACTS
[ ] BAA available and signable (if PHI in scope):
[ ] DPA available (if personal data in scope):
[ ] Contract covers the specific data type we will use:

5. RETENTION AND DELETION
[ ] Input retention period:
[ ] On-demand deletion available:
[ ] Data handling on cancellation (deleted / retained N days / exportable):

6. IDENTITY AND ACCESS
[ ] SSO supported (via our identity provider):
[ ] MFA enforced:
[ ] Central provisioning / deprovisioning (offboarding removes access):
[ ] Role-based access (if needed):

SECONDARY CHECKS
[ ] Output ownership assigned to customer:
[ ] Audit trail / usage logging available:
[ ] Vendor certifications (SOC 2 / ISO 27001 / none):
[ ] Vendor stability / breach history:
[ ] Integrations with our systems (email / files / CRM / none):

DECISION
Cleared for data tier (public only / internal / sensitive-regulated / rejected):
Conditions or restrictions:
Re-evaluation date:
Approved by:

A few notes on using it. Fill it in honestly even when the tool is one you want to approve – the point is to catch the gap before it catches you. Keep the completed checklists somewhere central, not in the head of whoever ran the evaluation, so the approved tools list stays defensible. And set the re-evaluation date when you fill it in, not later, because “we will revisit it” without a date means never.

Common mistakes when evaluating AI tools

1. Evaluating features instead of data handling. The tool being good and the tool being safe are different questions. Answer both, separately.
2. Approving a product name instead of a tier. “We approved ChatGPT” is meaningless if it does not specify Team or Enterprise. Staff will use the free tier.
3. Trusting marketing language as contract terms. “We take your privacy seriously” is not a no-training commitment or a DPA. Read the actual terms.
4. Skipping the sub-processor question. A small vendor’s AI feature often sends your data to a large model provider you never evaluated. Find out who actually processes inputs.
5. Ignoring training-on-inputs because the tool is popular. Popularity is not a data protection control. Many widely used consumer tools train on inputs by default.
6. Letting individual staff vet their own tools. Decentralized evaluation produces inconsistent decisions and shadow AI. Run it through one process.
7. Forgetting offboarding. A tool with no SSO and no central deprovisioning means departed employees may keep access. Check this before approval, not after a departure.
8. No retention or deletion check. Data you cannot delete is permanent exposure. Confirm you can remove inputs before you put real data in.
9. One-and-done approval. Vendors change terms and get acquired. A clearance with no re-evaluation date drifts out of date within a quarter or two.
10. No written record. An evaluation you cannot show a client or insurer is an evaluation that, for compliance purposes, did not happen. Keep the completed checklist.

How long the evaluation takes

For a single tool, a thorough evaluation is a few hours of work, most of it spent reading the vendor’s terms, security documentation, and sub-processor list rather than testing the product. The table below is a realistic breakdown for vetting one tool from scratch.

Phase	What it involves	Time
Initial screen	Confirm a business tier exists and the tool is not consumer-only	15-30 minutes
Data handling review	Read terms, privacy policy, sub-processor list, security docs	1-2 hours
Regulated-data check	Confirm BAA or DPA availability, request if needed	30 minutes plus vendor turnaround
IAM and offboarding test	Confirm SSO, MFA, central provisioning, run a test deprovision	30-60 minutes
Decision and record	Fill in the checklist, assign data tier, set re-evaluation date	30 minutes

The first tool takes the longest because you are learning the process. By the third or fourth, the checklist makes it routine, and most rejections become obvious in the initial screen – a consumer-only tool that trains on inputs and offers no BAA is a fast no for any regulated business, and you have saved yourself the deep dive.

What is next in this content series

A clean evaluation process feeds the rest of your AI program. The output of this checklist is the approved tools list at the center of your AI governance framework, and the data tiers it assigns are the same tiers your AI acceptable use policy enforces. If you have not yet mapped what your team is already using, the shadow AI wake-up call is the place to start, because you cannot evaluate tools you do not know are in use. Upcoming articles in this series cover how to roll AI tools out to your team without creating security gaps, how privacy laws apply to AI use, and what to do if an employee leaks data through an AI tool.

How Sequentur can help

If you want a second pair of eyes on a tool you are evaluating, or help building a repeatable vetting process across your business, schedule a call.