Which customer queries are safe to automate with AI?
Classify customer queries by data sensitivity, account access and input controls, then choose AI automation, human review or a simpler non-AI route.
Stanic & Co.
Perspectives for business leaders
Australian small business owners should classify customer queries by both the information supplied and what the system could disclose or change—not by topic alone. General policy questions can be candidates for automation. Queries containing personal information need controlled handling, while debt, hardship, identity and account-access requests should default to a protected human route until assessed. Crucially, a harmless question can arrive with sensitive attachments or account details. Check the actual input before sending it to an AI service.
This article proposes a practical triage process for public-facing customer service. Its low, medium and high labels are operational categories, not legal classifications. The aim is to decide what may reach the model, what needs human judgement and when a category-selection form is the better option. It is educational guidance, not a determination of your business’s privacy obligations.
Classify customer queries before choosing AI automation
Begin with one boundary: a customer’s selected topic does not establish the risk of their message. Someone asking about returns might include an invoice, home address and payment details. Someone asking about opening hours might paste an earlier complaint containing another person’s information. Your proposed workflow should therefore assess the query topic, the actual content and the access required to answer it. Where these point to different risk levels, use the highest level.
For this proposed scheme, low means public information with no customer-specific details and no account access: opening hours, published delivery areas or a general returns policy. Medium means ordinary customer-specific information, such as an order reference or contact details, where the task is limited and a protected workflow has been approved. High means material warranting specialist assessment or stronger restrictions, such as identity documents, payment credentials, debt and hardship information, or requests to change account ownership.
These labels deliberately differ from legal terminology. In particular, calling something high sensitivity here does not mean it falls within a particular statutory definition of sensitive information. The OAIC privacy resource identifies the regulator’s role in upholding national privacy laws and provides pathways to guidance on personal information, the Australian Privacy Principles and data breaches. Use that guidance and qualified advice to establish which obligations apply to your business.
Also assess the action being requested. Explaining a published cancellation policy is different from cancelling a particular customer’s service. A message containing little personal information can still pose a security concern if it asks the system to reveal account records or change permissions. For an initial deployment, keep automated public answers separate from account lookups and transactions. Treat these as separate approvals rather than gradually adding access to a previously approved FAQ bot.
Use this reusable customer-query triage tool
Copy the following record into your service-design checklist. This is an illustrative procedure, not a rule prescribed by the sources. Complete it for each query category, then apply the same checks to incoming messages. Record: query topic; information actually present; information needed to answer; sensitivity label; requested system access or action; pre-model input controls; permitted route; decision owner; and fallback channel. Mark an unknown control as a failed check rather than assuming the supplier provides it.
Input 1—topic: is this a public-information question, a customer-specific enquiry or a consequential account request? Input 2—data sensitivity: assign low, medium or high using the working definitions above. Input 3—system capability: can the service detect and remove or divert disallowed information before model processing, including relevant uploads? Separately record what happens to the original message in logs and other systems. A promise to redact the final answer does not pass this input-control check.
Output—automate fully: pass only where the actual input is low risk, the answer uses approved public information, no account access or action is required, and the input boundary has been tested. Output—human-in-the-loop: use only for an approved, limited medium-risk workflow where staff approve the response or action and the data-processing path has already been accepted. Output—manual only: use for high-risk content, unverified access, unsupported attachments or failed controls. Here, manual only means no submission to the generative AI service, not freedom from normal information-security controls.
Example entries: “What is your returns window?” with no personal details, public-policy access only and tested input controls → low → automate fully. “Please explain the status of order ABC123” → medium → human-in-the-loop only through an approved customer-data workflow; otherwise manual. “I cannot meet my repayments; here is my debt statement” → high → manual through a protected channel. “Change the email address on my account” → high-action risk → manual under the account-verification process. Stop automation whenever the content or requested action falls outside the approved entry.
Make the input boundary real, not just a chatbot instruction
Distinguish a warning from a control. A notice asking customers not to share personal details is useful communication, but your proposed workflow should still handle a customer who ignores it. Similarly, if the same model reads an unfiltered message to decide whether that message is safe, you have not kept the original content out of that model. Document the sequence explicitly: customer submission, initial screening, routing, model processing, response and storage.
Ask the technical owner to show where original text and attachments go before any redaction occurs. Does an intake component store the full message? Can support staff see it? Does diagnostic logging retain the original alongside a cleaned version? Can a customer bypass the screened form through another channel? These are questions to investigate, not claims about a particular vendor. An unresolved answer should prevent approval for the affected category.
The OWASP project on security risks in LLM applications describes its broader work on identifying and mitigating security and safety risks in generative AI systems. It supports treating security as a deployment concern, but it does not certify this triage method or any product. For this workflow, assess both information entering the service and the records or actions available to it. Reducing the first does not automatically restrict the second.
Before launch, use synthetic test messages rather than real customer records. Include a public question with a pasted address, a hardship request disguised as a billing query, an attachment the screening component cannot read, and a request to reveal another account’s details. The proposed pass condition is correct routing with no prohibited model submission or account action. If you cannot establish what happened from the available records, treat that as a failed test. For downstream permissions and action controls, use the complementary guide to connecting AI to existing business systems.
Hypothetical failure: a billing query becomes a hardship record
Consider a hypothetical business offering customer credit. It approves its bot for routine billing explanations. A customer selects “billing” and writes that they cannot meet repayments, attaching a statement showing their repayment history. The bot drafts a sympathetic reply and offers to pass the request to staff. Although the reply appears appropriate, the original message and attachment remain in a broadly accessible diagnostic log.
The failure is not simply a poor answer. The business classified the dropdown category rather than the actual content, allowed information into an unassessed processing path and gave log access more broadly than intended. A staff member approving the outgoing reply might never inspect those storage settings. This scenario illustrates a potential privacy and security incident; whether it breaches a particular obligation requires assessment of the facts and applicable law.
The Privacy (Credit Reporting) Code 2024 defines credit-related personal information through specified categories, including credit information and credit reporting information. Its text also addresses financial hardship arrangements and repayment history. That is a reason to seek specialist assessment for relevant credit workflows—not to assume every billing message is regulated credit reporting information, or that every payment-extension request is a regulated hardship request. The Code’s hardship-request definition expressly distinguishes certain once-off, short-term extensions.
Under the proposed response procedure, the service owner pauses the affected AI route and switches it to protected manual intake. The technical owner restricts access to the exposed records and establishes what was processed, stored and viewed. Preserve appropriate incident evidence before altering records; obtain privacy advice on assessment, notification and retention or deletion decisions. Reopening requires a corrected route and repeat tests of the original failure. An acceptable customer-facing answer is not sufficient evidence that the underlying handling was safe.
Give one owner authority to stop the workflow
Make the customer-service manager accountable for the routing register and the decision to operate each category. A technical owner should verify processing boundaries, access controls and the fallback switch. The person responsible for privacy should arrange advice on uncertain obligations and review proposed handling of customer information. In a small business, one person may hold several roles, but record the responsibilities separately so a supplier’s product setting is not mistaken for business approval.
Human review belongs at a defined decision point. A reviewer might check a draft against the approved policy, confirm that the requested action is authorised and decide whether the query requires another team. Give them access to the relevant source and an explicit option to reject or escalate. Do not make approval the quickest way to clear a queue while requiring a lengthy explanation to stop an unsafe response.
Review has limits. It cannot undo an earlier disclosure to a service, guarantee detection of hidden information in an attachment, or establish that a customer is entitled to account access merely because their story sounds plausible. A reviewer can also miss an error or rely too heavily on the draft. For that reason, proposed controls should constrain what reaches the model and what the system can do, rather than relying entirely on someone checking the final wording.
Monitor routing mistakes as well as answer quality. Review examples of sensitive content entering the automated route, public questions being unnecessarily diverted, customers returning after an unresolved answer, and staff overriding classifications. Use synthetic regression cases after changes to forms, integrations or access settings. Pause an affected route on confirmed prohibited processing or unauthorised access; investigate uncertain cases promptly. These are proposed operational stop conditions, not legal thresholds. Record who can authorise reopening and what evidence they must see.
Compare AI with a category-selection form
The simpler alternative is a click-to-select form: customers choose a category, then receive either a static FAQ page or a protected form for human response. Start with categories such as public policy, existing order, account access and payment difficulty. Keep public answers available without requiring a free-text message. For human routes, request only the information needed to begin handling the enquiry and explain what customers should not submit.
This approach removes generative language processing from intake, but it does not eliminate privacy or security responsibilities. Customers may choose the wrong category or enter unnecessary details into a form. An ordinary shared mailbox is not automatically a suitable destination for high-risk submissions. Assess the receiving system, staff access and handling process just as deliberately as you would for AI. Prefer a properly assessed contact channel over an unexamined email fallback.
The trade-off is practical. A form offers predictable routing and a simpler processing path, but customers must navigate your categories and may need staff help with ambiguous requests. An AI intake service can be designed to interpret varied wording, but that adds a processing stage requiring its own controls and evaluation. Do not adopt it merely to replace a short, understandable menu. First establish which customer problem the language-processing step is intended to solve.
Do not use the proposed AI triage approach where you cannot verify pre-model handling, have no protected manual destination, cannot separate public answers from account actions, or lack an owner able to investigate mistakes. If personal or credit-related information is central to most enquiries, begin with a specialist-designed manual workflow rather than forcing the work through a public FAQ bot. For the separate investment decision, see how to budget AI compliance costs before approving a customer bot.
Approve a narrow route, not the whole inbox
The decision-ready starting point is a public-information route with no account access, no transactional authority and a tested way to divert disallowed input. Write its approved purpose in one sentence: “Answer published delivery-policy questions using approved public material; divert customer-specific messages without sending them to the model.” If the implementation cannot deliver that boundary, choose the static FAQ and category-selection form instead.
Before approval, the service owner should be able to produce the completed triage entries, a processing map, synthetic test results, named reviewers where needed and a working fallback. Have the technical owner demonstrate the stop mechanism. Confirm who will handle diverted messages and what customers will be told about response times. A routing decision is incomplete if the supposedly safer destination is unattended or encourages customers to resend the same information elsewhere.
Take unresolved legal questions to a qualified adviser with the actual workflow, not just a product name. Ask whether the business and proposed information handling fall within relevant privacy or credit-reporting obligations, how the collected information should be handled, and what incident-response process is appropriate. The three sources cited here establish useful regulatory and security context; they do not establish the compliance of a particular bot, business or configuration.
Approve expansion category by category only after the new data, access and failure paths have been assessed. The central rule of this proposed method is straightforward: classify the content and the requested action, verify the processing boundary, and route uncertainty away from unattended automation. A bot that handles fewer query types within a demonstrated boundary is a more defensible starting decision than one approved for every message arriving in the customer-service inbox.
Sources
General information only; verify sources before making business decisions.