Back to the reading room
Business insights

How to check whether an AI vendor trains on your business data

Check an AI vendor’s training terms, account settings and supporting evidence before allowing personal information into your Australian SME’s pilot.

Stanic & Co.

Perspectives for business leaders

10 min read

Before allowing personal information into an AI pilot, obtain evidence for the exact product, subscription and configuration you will use. Check whether inputs and outputs can be used for model training, whether an opt-out is required, and what the contract commits the vendor to. An opt-out switch alone is not proof of segregation. If the evidence is incomplete or contradictory, restrict the pilot to approved non-personal, non-confidential material or choose another supplier.

This is a narrow procurement decision: can your Australian SME establish a defensible boundary between pilot data and the supplier’s model improvement activities? It is not a complete privacy assessment or proof that a product cannot leak information. The proposed checks below translate the supplied OAIC guidance into a practical approval process; they are not requirements prescribed by the regulator.

Check AI vendor training policies for your actual account

Start with a precise question: “Under the agreement and settings applying to our account, can our prompts, uploads, outputs or feedback be used to train or improve models used by other customers?” Ask the supplier to answer in writing and identify the relevant contractual clauses. A general assurance that “you own your data” does not answer this question. Ownership and permission to use information are separate questions for procurement to resolve.

The OAIC’s guidance on commercially available AI products recommends due diligence covering suitability, human oversight, privacy and security risks, and who can access information entered or generated. It also recommends, as best practice, not entering personal information, particularly sensitive information, into publicly available generative AI tools. Do not interpret the checks in this article as overriding that advice.

Record the product name, subscription tier, contracting entity, account identifier and intended workflow. Include any connected applications and optional feedback features. Ask which terms govern each component rather than assuming a statement about one service covers the whole workflow. Without verified vendor documentation, there is no basis here to conclude that a particular subscription includes a no-training commitment.

Define the boundary you need before shopping for assurances. For example: “Customer enquiries must not be used to train shared models; access, retention and deletion must be documented and approved.” Keeping this decision separate from the broader choice of your first AI pilot prevents an attractive use case from becoming an excuse to accept unclear data handling.

Separate a no-training commitment from data isolation

A useful verification process asks different questions about training, access and storage. “Not used for training” addresses one purpose for which data might be used. It does not, by itself, establish who can inspect that data, how long it remains available, whether another customer can retrieve it, or what happens when you close the account. Treat each issue as a separate evidence request.

Likewise, do not make a dedicated environment a substitute for evidence. Ask whether the deployment is shared or dedicated, what the supplier means by those terms, and which controls enforce the customer boundary. A private-instance label alone does not establish a contractual prohibition on training. Conversely, the absence of a dedicated instance does not establish that training occurs. Evaluate the documented controls, not the label.

Ask separately about prompts, uploaded files, generated responses, logs, feedback submissions and any customer-specific model adaptation. Require the supplier to explain whether optional activities change the default treatment. If your team intends to fine-tune a model, make that a separate approval: the OAIC’s guidance on developing and training generative AI models expressly includes adaptation through fine-tuning and explains that using existing personal information for training raises privacy questions.

No document pack proves that a supplier’s implementation is flawless. Your attainable decision is whether the contractual commitment, configuration and supporting technical explanation are sufficiently consistent for the proposed risk. When they conflict, pause rather than selecting the most reassuring statement. Ask a qualified privacy or legal adviser to resolve questions about contractual scope, applicable obligations and whether the intended use of personal information is appropriate.

Build an evidence pack before uploading customer information

Use the following as an illustrative procurement procedure. Request the applicable service agreement, data-handling terms, product-specific training policy and instructions for controlling training participation. Save dated copies or references and the supplier’s written answers. Record which document controls if marketing language and contractual terms differ. An unanswered question should remain visibly unresolved, not become an assumed “yes”.

Then request an explanation of the technical boundary: how accounts are separated, how access is authorised, which supplier personnel or other service providers may access information, and which records are retained. Ask what deletion covers and what it does not cover. These requests support the OAIC’s due-diligence focus on access and lifecycle review; they are not claims that any particular vendor offers these controls.

Have the account administrator demonstrate the relevant settings in the account intended for the pilot. Capture the setting, subscription and verification date without including secrets or customer information. Ask whether the control applies prospectively, whether previously submitted information is affected, and what happens after a plan change. Do not infer these behaviours from the presence of a toggle.

Finally, map the proposed inputs. Can the business test the workflow without personal information at all? The OAIC’s data analytics guide recommends first considering whether personal information is necessary and using de-identified data where possible. However, removing names is not a sufficient basis to assume that a record is no longer identifiable. Prefer deliberately constructed fictional cases for initial testing, and seek advice where de-identification is uncertain.

Reusable vendor data segregation scorecard

Copy this illustrative scorecard into your procurement record. For every item, enter Pass, Fail or Unknown, followed by the evidence reference, reviewer and date. Do not total the answers into an average: a strong answer elsewhere cannot cancel an unresolved training permission. These are proposed business approval gates, not a regulator-issued certification.

Gate A — training permission. Does the applicable agreement exclude using the proposed inputs and outputs to train or improve shared models? Record any exceptions. Example entry: “Unknown — supplier email says no training, but cited terms do not identify our subscription; procurement to obtain clarification.” Pass only when the scope and exceptions are documented and acceptable. A marketing statement alone is insufficient for this proposed gate.

Gate B — configuration and environment. Are required controls enabled for the actual account, and is the shared or dedicated architecture documented? Example entry: “Pass — administrator recorded the required setting and linked the supplier’s account-isolation explanation.” A dedicated environment is an evidence field, not an automatic pass. Gate C — access and lifecycle. Are access, retention, deletion and relevant service-provider arrangements documented and reviewed? Example entry: “Fail — supplier has not answered who can access retained uploads.”

Gate D — ongoing control. Is someone responsible for checking changes, restricting uploads and stopping the pilot? Example entry: “Pass — operations manager owns approval; administrator controls access; changes require reassessment.” Decision: any Fail or Unknown means no personal-information uploads under this proposed process. Use only approved non-personal, non-confidential material or select another supplier. All gates passing means proceed to privacy and security approval, not automatic permission to launch. Publicly accessible material is not automatically suitable: the OAIC’s training guidance warns that public availability does not itself establish lawful permission to train on personal information.

Hypothetical failure: the wrong subscription was approved

Consider a fictional Australian repair business planning an AI assistant to draft customer-service replies. Procurement obtains a supplier statement saying business inputs are excluded from shared-model training. However, that statement concerns a different product from the low-cost account the team actually opens. Staff begin pasting enquiries containing names, addresses and repair histories, believing the procurement check applies.

During a review, the administrator finds that the evidence pack does not cover the active subscription and cannot establish how its uploads are used. This is the failure: the business has exposed personal information to an unverified processing arrangement. It is not evidence that the model has learned those records or disclosed them to other users. Such an outcome would require investigation, not an assumption about continuous learning.

Under the illustrative response procedure, the operations manager immediately stops new uploads and disables the affected workflow or access while preserving relevant account and activity records. The team identifies what was submitted, by whom and when, then asks the supplier about use, access, retention and available deletion. A qualified adviser helps determine the privacy response and any notification obligations. Requesting deletion is not proof that all copies or downstream uses have been reversed.

Detection would have been earlier if approval required the account identifier and subscription to match the evidence pack. Recovery might involve moving to a verified arrangement, rebuilding the test with fictional data or abandoning the AI route. The business should also check generated drafts for unnecessary personal details. The OAIC’s commercial AI guidance explains that privacy obligations concern personal information in outputs as well as inputs; reviewing only uploads leaves part of the workflow unchecked.

Assign an owner and recognise what human review cannot fix

For this proposed process, make the business owner or delegated executive accountable for accepting the residual risk. Procurement collects and reconciles supplier evidence. The administrator verifies account settings and access. The workflow owner defines permitted inputs and watches for scope changes. A privacy or legal adviser addresses unresolved legal questions. In a small firm, one person may fill several roles, but the approval record should still distinguish the responsibilities.

Human review of a draft cannot undo an inappropriate upload. A reviewer may remove a customer’s address from an outgoing reply, but that does not establish what happened to the address upstream. Nor can a fluent, accurate response prove that training is disabled. Treat output review as a separate safeguard, not evidence of vendor data segregation. For that separate release decision, use the AI pilot accuracy audit.

Use fictional test cases to check the workflow, account permissions and whether staff follow the input restrictions. These tests can reveal operational mistakes; they cannot prove the absence of supplier-side training. Do not upload real customer details as a “canary” to see whether they emerge elsewhere. That would create the very exposure the procurement gate is intended to avoid.

Set review triggers rather than treating approval as permanent: a new subscription, changed terms, new integration, feedback feature, broader dataset or revised use case should reopen the relevant checks. The OAIC’s commercial AI guidance calls for ongoing performance review, staff training and monitoring throughout the product lifecycle. The operational trade-off is real: maintaining this evidence takes staff time, and a tool that cannot justify that effort may be unsuitable for a small pilot.

When a searchable FAQ is the better pilot

If the business mainly needs staff to find approved answers, consider a structured, human-managed FAQ rather than an AI assistant. Start with a list of recurring questions, an approved answer for each, a source owner and a review date. Give staff straightforward search and a route for questions the collection does not answer. This proposed alternative avoids sending those questions into a generative model for the search task.

It does not provide zero privacy risk or guaranteed accuracy. Staff can still paste personal information into the wrong place, permissions can be misconfigured and approved answers can become stale. Keep customer records out of the FAQ unless there is an established need and appropriate handling. The distinction is narrower: this design does not require a model-training arrangement for generating answers, so that particular procurement issue is removed.

The trade-off is less flexible drafting and more manual maintenance. Staff may need to adapt an answer, search using different terms or escalate an unusual case. In exchange, the business can inspect each approved response directly and maintain a clear owner. Compare these practical costs with the time needed to obtain vendor evidence, manage account controls and reassess the AI arrangement.

Do not use this procurement approach as a shortcut for approving sensitive or consequential uses. A clear no-training commitment does not settle whether the original collection permits the proposed use, whether access is appropriate, or whether outputs are reliable. The decision-ready endpoint is a written outcome: approve the documented boundary subject to remaining reviews; restrict the trial to approved fictional or non-personal material; or stop and choose a simpler workflow. Research reference date: 23 October 2026. This article relies on the supplied OAIC material, not an assertion that vendor terms or law were independently rechecked on that date.

Sources

General information only; verify sources before making business decisions.

Turn an idea into a next step.

A conversation about what matters to your business.

Let's talk