Legal AI Security Questionnaire: 25 Questions to Send Vendors
A legal AI security questionnaire should be short enough that a vendor will actually answer it and specific enough that a vague answer shows. The 25 questions below are grouped into five areas (data use, storage, access, certification and exit), and each one comes with what a strong answer sounds like, what a weak answer sounds like, and what should end the conversation. Send them in writing before you upload a single client file, ask for the answers to be signed off by someone at the vendor who can be held to them, and treat anything that arrives as marketing copy until it is backed by a document.
We run a directory of AI legal software and we don't sell any of it. That matters here, because most security question lists for legal AI are written by a vendor describing how well it would score. This one is written for the person holding the clipboard. Copy the questions, paste them into an email, and use the answer examples to grade what comes back.
This is software evaluation guidance, not legal or compliance advice. Your professional obligations depend on your jurisdiction and your clients' contracts.
When should you send a security questionnaire?
Send it after the demo and before the pilot. A demo tells you what the product does. A pilot puts real documents into it. The questionnaire belongs in the gap, because once client files are inside a system you are asking the vendor to explain a decision you already made.
Three situations make it non-optional:
- Client or matter data will be uploaded. Contracts, discovery sets, medical records, deal rooms. If a document would be covered by your confidentiality duties, the tool that processes it has to be vetted first.
- The tool connects to other systems. An add-in that reads your document management system or inbox has a wider reach than a chat window, and the questions about access matter more.
- Your clients ask. Outside counsel guidelines and in-house procurement teams increasingly send their own questionnaires to law firms. Having vetted your own vendors makes your answers true.
On the professional side, the ethics guidance is consistent about the underlying duty. North Carolina's 2024 Formal Ethics Opinion 1 says a lawyer using an outside company's AI service must make reasonable efforts to ensure the service is compatible with the lawyer's responsibilities, and specifically the duty of confidentiality. A questionnaire and a file of the answers is how you show those efforts were made. Your own bar's rules govern, so read them directly.
Do small firms need this too? Yes, in a lighter form, and the FAQ below explains how. The risk of a client file leaking does not scale with firm size. What scales is the budget for finding out.
Which questions cover training and data use?
This is the first block because it is the one lawyers ask about most and vendors answer worst. "We don't train on your data" is a sentence with at least three possible meanings: the vendor's own models, the model provider's models, and the vendor's other systems such as search ranking or feedback tuning. Every question here forces one of those apart.
| # | Question | Strong answer | Weak answer | Red flag |
|---|---|---|---|---|
| 1 | Do you or any model provider train, fine-tune or improve any model using our prompts, documents or outputs? | "No, for us and for every provider in our stack. It is in the contract and here is the clause." | "We don't train foundation models on your data." (Silent on everything else.) | "Only anonymized data." No definition of anonymized. |
| 2 | Which model providers process our data, and what are their terms for data retention and training? | Names each provider, states the contract tier (not default API terms), and confirms zero retention or a stated short window. | Names the provider but not the terms. | Won't say which providers are used. |
| 3 | Can any human at your company or a provider read our prompts or documents, and under what conditions? | Named roles, a logged break-glass process, customer notification. | "Only when necessary for support." | Support staff can browse customer workspaces by default. |
| 4 | Do you use our data to produce benchmarks, aggregate insights or product analytics? | "No. Usage metrics only, and they contain no document content." | "Aggregated and de-identified insights, on an opt-out basis." | Opt-out is buried in a settings page, or requires a ticket. |
| 5 | Is any custom model trained on our data an opt-in feature, and is it isolated to us? | Opt-in only, in writing, isolated to the customer, deletable on exit. | Optional, but isolation unclear. | Training on customer data is the default and the contract permits it. |
Question 2 deserves a follow-up. Many products route prompts to a hosted model from a large lab or a cloud platform, and that provider's terms can differ from the vendor's. A strong vendor can tell you which tier of provider contract applies to your traffic without a call to someone in engineering. Our guide to legal AI data security walks through what the training and retention terms usually look like in vendor documents.
Which questions cover storage, access and sub-processors?
These ten questions are about where the data sits, who can reach it, and who else is in the chain. Storage and access are separate questions, and vendors tend to answer the easier one. Splitting them keeps the answers honest.
Storage
| # | Question | Strong answer | Weak answer | Red flag |
|---|---|---|---|---|
| 6 | In which regions is our data stored and processed, including backups and model inference? | Region named for storage and inference, with contractual commitment and an option to choose. | Storage region given, inference region not mentioned. | "Our cloud provider handles that." |
| 7 | Is our data logically isolated from other customers', and how is that enforced? | Per-tenant isolation described (separate storage, indexes, encryption keys) and tested. | "Multi-tenant architecture with strong access controls." | Shared search index across customers with no description of controls. |
| 8 | How is data encrypted at rest and in transit, and who holds the keys? | Standard modern encryption in both states; customer-managed keys available for sensitive matters. | "Industry-standard encryption." No detail. | Encryption in transit only, or keys held by an unnamed party. |
| 9 | What copies of our data exist outside the main store: search indexes, embeddings, logs, caches, backups? | A list of each and its retention period. | "We delete everything when you ask." | The vendor doesn't know what the full list is. |
| 10 | How long do you retain our prompts, outputs and uploaded files by default, and can we shorten it? | A default period stated in days, configurable by the customer. | "As long as your account is active." | "Indefinitely" with no deletion process. |
Access and sub-processors
| # | Question | Strong answer | Weak answer | Red flag |
|---|---|---|---|---|
| 11 | Who inside our firm can see which matters, and how are permissions enforced in the product? | Matter-level permissions, ethical walls supported, SSO and role-based access. | Workspace-level permissions only. | Everyone in the account sees everything and it can't be changed. |
| 12 | Does the product support single sign-on, enforced multi-factor authentication and an audit log of who accessed what? | Yes to all three, with log export. | MFA optional, logs available "on request." | No audit log for customer data access. |
| 13 | Who are your sub-processors, and how will we be told when that list changes? | Published list, advance notice by email, a right to object. | List exists but updates are silent. | No list; "we may use third parties." |
| 14 | What connectors does the product use to reach our other systems, and what permissions does each request? | Read-only scopes by default, a per-connector permission list, revocable by the customer. | Broad scopes requested "for convenience." | Connectors cannot be turned off individually. |
| 15 | Do your own staff access our data for support or engineering, and is that access logged and reviewed? | Just-in-time access, logged, reviewed periodically, customer-visible. | "Limited staff have access." | Standing production access for all engineers. |
When the vendor sends back a sub-processor list, check it against question 2. If a model provider appears on one list and not the other, you have found the gap the questionnaire was built to find.
Which questions cover certification and independent proof?
Certifications tell you an outsider has looked. They do not tell you the outsider looked at the part you are buying. The five questions in this group exist to close that distance. Our legal AI SOC 2 guide explains how to read a SOC 2 report in full.
| # | Question | Strong answer | Weak answer | Red flag |
|---|---|---|---|---|
| 16 | Which independent audits or certifications do you hold, and can we see the current report under NDA? | Report name, type, auditor, observation period dates, and a yes to the NDA request within days. | A badge on the website and no report offered. | "SOC 2 compliant" with no type, no auditor and no dates. |
| 17 | Does the audit scope include the AI feature and the model integrations we are buying? | The product and its AI components are named in the system description. | "The platform is covered." | The AI feature was added after the audit window and is excluded. |
| 18 | When did the last observation period end, and when is the next one scheduled? | Ended within the last year; next one booked. | Ended over a year ago; next one "planned." | "In progress" for more than a year. |
| 19 | When did you last run a third-party penetration test, and will you share a summary? | Within the last year, a summary available, critical findings remediated. | Annual test, results "confidential." | Internal testing only. |
| 20 | Do you map your AI controls to a recognised framework, and which? | Names a framework and shows the mapping (for example the NIST AI Risk Management Framework or a vendor attestation to the CSA AI questionnaire). | "We follow best practices." | Blank when asked what AI governance exists. |
For question 20, the reference points are public. NIST publishes its AI Risk Management Framework as a voluntary framework for managing AI risk, and the Cloud Security Alliance publishes the AI Consensus Assessments Initiative Questionnaire (AI-CAIQ) as a structured self-assessment for AI controls across areas like governance, security and privacy. The AI-CAIQ is the fuller reference if you want a list longer than 25; it is built for enterprise cloud assessments, so a firm buying one tool will usually want something shorter first. A vendor that has already filled one in can send you the completed copy, and that should save everyone time.
Which questions cover leaving the vendor?
Exit questions are the ones nobody asks, and the ones you will wish you had asked. They also tell you a lot about the vendor's confidence. A company that expects to keep you with a good product answers them in a paragraph. A company that expects to keep you with friction answers them in a sales call.
| # | Question | Strong answer | Weak answer | Red flag |
|---|---|---|---|---|
| 21 | Can we export all our data, in what formats, and at what cost? | Full export of documents, annotations and outputs in open formats, included in the plan. | Export of documents only. | Export is a paid professional-services project. |
| 22 | When we terminate, how soon is our data deleted from production, backups and provider systems? | Stated timeline for each, with written confirmation of deletion. | "Within a reasonable time." | No deletion commitment after termination. |
| 23 | What happens to our data if you are acquired, merge or shut down? | Contract terms covering change of control, notice and a deletion or export right. | "We have no plans to be acquired." | Terms allow transfer of customer data to any successor without notice. |
| 24 | How will you tell us about a security incident, and how fast? | Defined notice window in the contract (hours or a few days), named contact, post-incident report. | "We will notify as required by law." | No commitment to notify at all. |
| 25 | What are your liability and indemnity terms for a breach of customer data? | Specific figures in the contract; breach liability not capped at a month of fees. | Standard cap that nobody has negotiated. | Vendor disclaims all liability for data loss. |
Question 23 is not hypothetical in this market. We track legal AI acquisitions and rebrands because ownership changes, and your data sits with the new owner on the new owner's terms unless the contract says otherwise.
How do you judge the answers you get back?
Grade each of the 25 as strong, weak or red flag, using the tables above. Then apply four rules.
1. Count red flags first, not strong answers. A vendor with twenty strong answers and one red flag on question 1 has not passed. Training on customer data by default, undisclosed model providers, and no audit log for data access should each stop a purchase until fixed in writing.
2. Weak is a request, not a verdict. A weak answer usually means the vendor has the right answer in a document and gave you the sales summary. Reply with "please point me to the clause or policy." Several weak answers that turn strong after that follow-up are a good sign. Several that stay weak are the pattern you came for.
3. Match every claim to a document. "We have zero data retention with our model provider" should map to a contract term or provider tier. "We have SOC 2" should map to a report you have read. If a claim can only be proved by repeating it, record it as unverified.
4. Compare answers against what is published. Read the vendor's security page, privacy policy and sub-processor list yourself before the answers arrive. If the email says one thing and the public documents say another, the public documents are what a regulator or opposing counsel could find too.
A simple scorecard works: tally strong, weak and red-flag counts per section, so you can see that a vendor is excellent on certification and thin on exit rather than averaging it away. If you want the product side of the evaluation (accuracy, workflow fit, pricing), our guide on how to evaluate legal AI covers it, and the security questionnaire is one gate within that process, not a replacement for it.
In-house teams have an extra step. Procurement and information security often have their own forms, and the useful move is to send these 25 first, then forward the vendor's answers to your security team as pre-filled evidence. Legal teams buying tools can start from the vendors on the in-house audience page.
What should happen when a vendor won't answer?
Silence tells you something, but only if you give it a fair chance first. Follow this sequence.
- Send it in writing to a named contact, with a response date. Ten business days is reasonable for 25 questions. A vendor with a trust center can often answer most of them in a day, because the documents already exist.
- Offer a call for the questions that need engineering input, usually 2, 9 and 17. Do not let a sales rep summarise these, ask for the security lead.
- Chase once in writing. If the second request is ignored, record it. A vendor that cannot find time to answer before you have paid will not find time after.
- Escalate the gaps to the contract. Anything the vendor says is true but will not put on paper belongs in the data processing terms. If they will not sign it, the answer is no.
- Keep the file. Store the questionnaire, the answers and the follow-ups with the contract. If a client asks later what diligence you did, this is the answer.
Some refusals are legitimate. A vendor may not share a penetration test report with an unsigned prospect, or may not post its sub-processor list publicly. The test is whether they offer a route (NDA, trust center login, summary letter). "We never share that" without an alternative, on a question a customer is entitled to ask, is the red flag.
Once you have a response from each, the practical next step is to shortlist the tools worth questioning. Every listing in our directory of AI legal tools is verified active, and listing is free and never changes ranking, so you can build a shortlist without wondering whose budget shaped it.
FAQ
What should a legal AI security questionnaire include?
It should cover five areas: how your data is used and whether anyone trains on it, where it is stored and for how long, who can access it and which sub-processors are involved, what independent audits back the claims, and what happens when you leave. The 25 questions above cover each area with five questions, so every area can be graded on its own.
Do small firms need to send one?
Yes, but a smaller version works. A solo or small firm can send questions 1, 2, 5, 8, 10, 13, 16, 17, 21 and 24, which cover training, model providers, retention, sub-processors, audits and exit. The vendor's answers take them a few minutes to write if the documents exist. The duty to protect client information does not shrink with headcount, and neither does the risk to the client.
How long should vendors take to respond?
A vendor with a trust center and a data processing agreement ready can answer most of these within a few business days. Ten business days is a fair outer limit for the full 25, including time to involve their security lead. If a vendor takes weeks to answer written questions before a sale, expect that pace when something goes wrong later.
Is a SOC 2 report a substitute for a questionnaire?
No. A SOC 2 report tests the controls the vendor described, over the scope the vendor chose, for the period the auditor examined. It will not tell you which model providers receive your prompts, whether the AI feature you are buying is in scope, how long your data lives after termination, or what happens at acquisition. Use the report to answer questions 16 to 18 and use the questionnaire for the rest.