Back to the reading room
Business insights

Can you verify AI pilot deletion before uploading personal data?

Test AI pilot deletion with synthetic records across uploads, outputs, logs and connected stores, then document backup limits and unresolved copies.

Stanic & Co.

Perspectives for business leaders

10 min read

Before an AI pilot receives personal information, run a synthetic-record deletion test: trace a made-up record through uploads, outputs, logs and connected stores, delete it, then check what remains. Obtain supplier confirmation for backup limits and copies you cannot inspect. If material copies remain unexplained, keep personal information out. This is one practical privacy-by-design check—not proof that the whole pilot is suitable or compliant.

The question is not simply whether a Delete button works. It is whether your business can demonstrate the scope and limits of deletion for its intended configuration. The procedure below is illustrative, not a regulator-prescribed test. Use the existing privacy-first due diligence audit for broader questions about purpose, access, supplier terms and suitability.

What does verifying AI pilot deletion mean?

A useful deletion test distinguishes three outcomes: removal you can directly observe, handling the supplier confirms in writing, and copies that remain unresolved. Do not combine them into a single reassuring tick. A file disappearing from a workspace demonstrates something narrower than deletion from every system involved in processing it. Your approval record should preserve that distinction.

Start by defining the deletion event you need to support. Is it removing one uploaded document, clearing a conversation, deleting information about one person, or closing the entire pilot? These are different test cases. For example, deleting an account is not a workable substitute if the operational requirement is to remove one customer’s information while retaining unrelated records.

The OAIC’s guidance on commercially available AI products recommends privacy by design, including a Privacy Impact Assessment, and due diligence covering intended use, oversight, privacy and security risks, and access to information. It also explains that privacy obligations concern personal information in outputs, not only inputs. That makes an upload-only check too narrow for this proposed test.

Ask your privacy adviser which information must be retained, removed or otherwise handled, and when, before setting test conditions. This article does not establish a universal right to immediate deletion or a legally acceptable retention period. A documented backup delay is not automatically acceptable; an inability to erase every copy immediately is not, by itself, a complete legal assessment.

Map the record and assign a deletion owner

Name a business owner who can withhold permission to upload personal information. Give a technical owner responsibility for running deletion and checking connected systems. A privacy lead should review the remaining limits and coordinate qualified advice. In a small business, one person may hold several roles, but record who can challenge the result and who can stop processing.

Map the exact service tier, account settings and integrations intended for the pilot. Include the upload location, generated files, conversation history, application logs, search indexes, connected databases, exports and supplier-managed retention where applicable. Treat these as places to investigate, not a claim that every AI product creates every kind of copy. Record a system owner beside each applicable location.

Ask the supplier what its deletion action covers, what it excludes, whether a separate request is needed, and how completion is evidenced. Specifically ask about support copies, subprocessors, backup expiry and restoration handling. Request answers applicable to your configuration. A general privacy statement is not evidence that a particular deletion request reached every relevant store.

Keep training use separate from deletion. A restriction on training does not establish retention limits, and removing a visible file does not demonstrate that any prior training use has been reversed. For the deletion test, unresolved training use is a reason to retain the synthetic-only boundary while broader due diligence resolves the issue. Do not infer supplier capabilities from subscription price or product category.

Build a synthetic record that can be traced

Create a wholly invented record without adapting a real customer’s document. Use a conspicuous marker such as SYNTHETIC-DELETE-TEST-ALPHA in the document body and a separate invented reference field. Avoid realistic identity-document numbers, real contact details or copied customer narratives. Check that the test file itself contains no personal information before uploading it.

Use an isolated pilot workspace with only the connections under assessment. Record the configuration, permissions and test date. Process the record through the intended workflow: upload it, generate the normal output, and permit only the planned downstream actions. Save the object identifiers and destination references needed to find its copies. Do not connect production stores merely to make the demonstration look realistic.

Before deleting anything, establish a positive control: show that the marker or associated object is discoverable in each accessible location where it should exist. If your search cannot find a known existing copy, a later empty search is weak evidence. For derived outputs that omit the marker, use recorded output identifiers and links to the source rather than relying solely on text matching.

This preparation is an illustrative evaluation method. The voluntary NIST AI Risk Management Framework supports incorporating trustworthiness considerations into design, development, use and evaluation; it does not prescribe this deletion protocol or certify its outcome. Synthetic testing can expose configuration and workflow failures, but cannot prove how every live record or hidden supplier system will behave.

Reusable synthetic-record deletion test

Copy this record for each test: purpose; configuration and version; synthetic marker; input and output identifiers; store; deletion action; expected completion point; observed result; evidence reference; owner; unresolved copy; and decision. Mark each store Observed pass, Supplier-confirmed limit, Fail or Unknown. These are illustrative business categories, not legal compliance scores. Set acceptable completion times before testing, based on operational needs and advice.

Uploads — Example entry: “File ALPHA; deleted through the documented file action; original link no longer opens for an authorised test user; deletion event saved.” Pass only when the agreed checks show the source is unavailable through its normal access routes. Fail if another authorised route still retrieves it after the expected completion point. A hidden file is not a demonstrated deletion.

Outputs — Example entry: “Extraction result and conversation response located by object identifiers; both removed separately; saved export also checked.” Pass when applicable output copies have been removed under the agreed procedure. Fail if deleting the upload leaves a generated document or conversation accessible. If an output must be retained, record the reason and seek advice rather than silently excluding it.

Logs — Example entry: “Application log contains the marker; payload logging disabled for future runs; existing entry awaiting administrator action.” That is not a pass. Inspect the accessible logging locations identified in the map and obtain supplier evidence for inaccessible ones. A content-free event recording that deletion occurred should be distinguished from a retained payload containing the record. Unknown log contents remain unresolved.

Connected stores — Example entry: “Search index entry removed; connected database row removed; scheduled synchronisation tested.” Pass when the agreed downstream checks succeed and a subsequent workflow cycle does not recreate the record. Fail if an index, cache, export or retry restores a copy. Test only the stores and actions actually used; mark exclusions explicitly rather than claiming universal coverage.

Backups and unresolved copies — Example entry: “Supplier confirms retained backup copy, expiry point, access restrictions and restoration procedure; business acceptance pending.” Record this as a supplier-confirmed limit, not observed erasure. Stop personal-information admission if a material copy is Unknown, a required deletion fails, or a retention limit remains unacceptable. Never average failures against passes; close each gap or redesign the workflow.

Hypothetical failure: the scanner deletes only the upload

Consider a hypothetical equipment-hire business assessing an AI scanner for customer documents. No real customer records have been uploaded. Its technical owner sends the invented ALPHA record through extraction and a connected review queue. The scanner produces the expected fields, and deleting the source file makes it disappear from the upload screen. The initial demonstration looks successful.

The deletion test finds two failures. The review queue still holds the extracted fields, and an application log contains the document text. A supplier response also identifies a retained backup copy but does not explain what happens if that backup is restored. These are invented scenario details, not claims about an actual supplier. Extraction accuracy does not resolve any of them.

The business owner records Fail for connected outputs and logs, and Unknown for restoration handling. Personal-information testing remains blocked. The technical owner removes the downstream record, changes payload logging and checks existing entries. The supplier is asked to explain backup restrictions and how a deletion is maintained following restoration. The team reruns the synthetic workflow rather than accepting a screenshot of the empty upload screen.

If the queue repopulates after synchronisation, the failure remains open: the source-to-destination workflow must be changed, not merely cleaned once. If the supplier cannot resolve the backup question to an acceptable standard, the owner selects another design. Qualified advice determines the implications of retained information; overseas storage or a backup delay alone should not be labelled an automatic privacy breach.

Compare deletion work with a simpler non-AI form

For a task that only collects customer-confirmed details, compare the scanner with a structured digital form and staff check. An illustrative alternative stores only the necessary fields in a controlled local-only database, without uploading documents to an external AI processor or generating inferred attributes. This may remove some places requiring deletion checks, but the actual design must be examined.

Apply the same record-tracing exercise to the form. Check submissions, database rows, administrative exports, logs and backups. Local storage is not a legal safe harbour, and a non-AI system can also retain overlooked copies. The practical comparison is whether the simpler process meets the business need with fewer deletion actions, clearer ownership and manageable retention limits.

Record customer effort, exception handling, staff time and supplier dependencies alongside deletion work. A form may require more typing or manual checking. A scanner may accommodate varied documents but add generated outputs and supplier-managed stores to investigate. Neither offers guaranteed savings. If the form meets the need and the AI deletion workflow cannot be supported operationally, do not proceed with that AI design.

Also ask whether personal information is necessary at all. The OAIC’s product-selection guidance says organisations should consider whether using personal information with AI is necessary and the best solution. It recommends, as best practice, keeping personal information, particularly sensitive information, out of publicly available generative AI tools. A deletion test is not permission to disregard that guidance.

Detect failures, stop processing and document the limits

Human review helps only within the reviewer’s access and evidence. A reviewer can miss an export, misunderstand an empty search or accept a supplier answer that concerns a different service tier. They cannot inspect an inaccessible backup or prove physical erasure from a screenshot. Have another person check important evidence where feasible, while documenting what neither person could verify.

Before running the test, demonstrate how to pause new uploads, cancel applicable queued work and disconnect the pilot’s access. If deletion fails, use those controls, preserve minimal evidence needed to investigate, and assign each unresolved copy an owner. Do not create a new uncontrolled archive of test payloads merely to document removal. Keep deletion-event evidence separate from the information being deleted.

For an accidental real-data upload, stop further processing and obtain qualified privacy advice about containment, retention, deletion and any notification obligations. Do not promise that deleting the visible record reverses disclosure or removes every retained copy. Resume only after the affected locations and unresolved limits have been assessed, the cause addressed, and the relevant synthetic checks repeated.

Repeat the test when changes affect the data path, such as a new connector, logging setting or supplier retention arrangement. The NCSC’s secure AI development guidelines treat security as a lifecycle requirement, including operation, monitoring and maintenance. The final decision should be explicit: deletion behaviour demonstrated within stated limits; redesign required; or personal-information use blocked. Even the first outcome clears only this deletion check—not broader privacy, accuracy or production approval.

Sources

General information only; verify sources before making business decisions.

Turn an idea into a next step.

A conversation about what matters to your business.

Let's talk