Back to the reading room
Business insights

PTU spillover: can your AI cost limit survive a traffic spike?

Evaluate Azure PTU spillover with a reusable cost-control scorecard, a worked failure scenario and clear ownership before committing dedicated AI capacity.

Stanic & Co.

Perspectives for business leaders

10 min read

For most Australian SMEs, standard pay-per-token deployments are the safer starting point. Consider Provisioned Throughput Units (PTUs) only when representative production traffic, tested latency and total costs justify dedicated capacity. Before approving them, answer a narrower question: can you control spending when traffic exceeds that capacity? If you enable spillover to standard inference, your fixed PTU cost is not your total spending limit.

This article focuses on that overflow decision, rather than repeating the general pay-as-you-go versus PTU capacity assessment. It uses the supplied Microsoft documentation for the research reference date of 14 November 2026, without asserting current prices or regional availability. The scorecard, thresholds and calculations below are illustrative business procedures, not Microsoft requirements.

PTU spillover cost control: what are you approving?

Microsoft describes provisioned throughput as dedicated processing capacity held for your deployment, whether requests are being made or not. Standard deployments share inference capacity across customers. Provisioned deployments are billed by deployed PTUs per hour, or through an applicable reservation discount, rather than simply by tokens consumed. These are Azure-specific capabilities, not general promises about every AI provider. See Microsoft's provisioned throughput documentation.

Spillover adds a second spending route. Microsoft documents an optional configuration that redirects requests from a provisioned deployment to a corresponding standard deployment in the same Foundry resource when the provisioned deployment is fully utilised and returns non-200 responses, such as capacity-related 429 responses. The same documentation says spillover can be configured for all requests or controlled per request. Do not assume every model provider supports it; verify your selected deployment.

The business decision is therefore not just whether dedicated capacity is economical. It is whether you will pay additional usage charges to preserve service during bursts, or instead accept a queue, delay, rejection or human handover. Each option needs an explicit owner and an acceptable customer outcome.

Approve three things separately: the dedicated capacity baseline, the maximum authorised overflow allowance, and the behaviour when that allowance is exhausted. A reservation is a financial discount, not a traffic-control mechanism. Microsoft's documentation also distinguishes quota from available capacity and warns that reservations do not guarantee capacity. Check availability before purchasing, rather than assuming a commercial commitment secures the deployment.

Measure the burst, not just the monthly average

A monthly token total is useful for estimating expenditure, but it cannot describe the traffic that an interactive service must handle at a particular moment. Microsoft's PTU sizing guidance identifies requests per minute, average input and output tokens, the output-to-input processing ratio, and cache rate as sizing factors. Throughput also varies by model. Use those factors when assessing your workload rather than treating one PTU as a universal quantity of tokens. See the PTU sizing overview.

For an illustrative evaluation protocol, collect representative operating periods, including a normal week, your busiest known business event and a deliberately constructed burst. Record requests per minute, prompt size, response size, completed-task latency, retries and the destination deployment. Include the actual workflow: search, application processing and human handover can affect whether a customer receives a useful answer in time.

Specify maximum acceptable latency as a business requirement. For example, an internal drafting task may tolerate a queue, while a customer-facing transaction may need an immediate fallback. Test the agreed requirement against observed end-to-end performance; do not treat a model-level latency target as proof that the whole application meets it.

If you record traffic variance, define it consistently. One possible illustrative measure is (peak minus trough) divided by average requests per minute, using comparable business-hour windows. A variance below 20% is not a source-backed PTU recommendation, and it should not become your approval rule. Seasonal changes, token lengths and the consequences of a rare burst still matter. A stable average can coexist with an expensive overflow event.

Reusable PTU overflow approval scorecard

Copy the following fields into a decision record. Record an evidence location and an accountable person for each answer. The pass/fail conditions are proposed controls for this article; adapt them to your service and financial tolerance. Approval should require evidence, not a total score that allows a serious control failure to be offset by several easy passes.

Workload and latency: enter monthly average input/output tokens, the measurement period, peak requests per minute, traffic variance definition and maximum acceptable completed-task latency. Example entry: “Internal enquiry assistant; normal and peak-event samples available; eight-second target for immediate answers; queued work must complete within the agreed business deadline.” Pass if the test represents intended traffic and the workflow meets its agreed target. Stop if you have only monthly totals or have not tested the fallback.

Cost envelope: enter baseline PTU cost, permitted standard spillover cost, other application costs, currency, tax treatment and pricing assumptions. Example entry: “Illustrative monthly baseline A$2,400; overflow allowance A$300; other services A$400; all figures exclude GST.” Pass if finance accepts the combined envelope and the rate assumptions can be checked. Stop if the proposal compares PTUs only with standard token charges while omitting overflow or supporting services.

Overflow control: enter which requests may spill over, how usage is measured, the admission threshold, the maximum in-flight exposure and the exhaustion behaviour. Example entry: “Only approved interactive requests; background work queues; stop admitting new overflow work at the authorised threshold.” Pass if a technical test demonstrates the control and customer fallback. Stop if a budget email is the only supposed spending limit.

Ownership and recovery: enter the service owner, finance approver, engineer responsible for controls, after-hours contact and rollback procedure. Pass if each accepts the responsibility and the rollback has been rehearsed in a safe environment. Stop if nobody can disable overflow promptly or if doing so would silently lose work.

Decision entry: “Approve a limited trial / remain on standard / defer pending evidence; reasons; unresolved risks; review date.” A failed financial, fallback or ownership gate means defer approval. A pass authorises only the documented operating envelope, not unrestricted overflow or an automatic annual commitment.

Calculate the overflow allowance before signing

Use this illustrative planning equation: total monthly solution cost = dedicated PTU baseline + standard spillover usage + other application services + control and operational costs. Keep the comparison on the same currency, period and scope. Microsoft cautions that Foundry costs are only part of the application cost and that billing units and rates vary by model, deployment type and meter. Its Foundry cost guidance supports checking meter-level costs rather than relying solely on a portal estimate.

Assume a monthly spending envelope of A$3,500, a PTU baseline of A$2,400, supporting services of A$400 and an operational allowance of A$200. The remaining overflow allowance is A$500: A$3,500 minus A$2,400 minus A$400 minus A$200. These are invented planning amounts, not Azure quotes. If the comparable standard-only workflow costs A$2,700 and meets your latency requirement, this example does not establish a financial case for PTUs.

Next estimate exposure during the time needed to detect and stop excess overflow. Assume the chosen workload's conservative aggregate spillover estimate is A$25 per minute, and detection plus intervention takes ten minutes. That creates A$250 of potential additional exposure, before accounting for requests already admitted or ancillary costs. Reserving that amount would leave A$250 for planned overflow within the A$500 allowance.

Do not interpret this arithmetic as a guaranteed bill ceiling. It depends on the rate bound, request mix, concurrent work and how effectively your application blocks new admissions. Test those assumptions. If exposure cannot be bounded well enough for finance to accept it, choose queueing or rejection rather than unrestricted spillover. For reconciliation after the trial, use the complementary guide to matching Azure AI costs with actual usage.

Hypothetical failure: unused capacity and an overflow bill

Consider a hypothetical SME that buys a one-year PTU reservation for a customer chatbot. It expects steady adoption, but customers move to a simpler self-service page. The deployment now operates at roughly 10% of its planned utilisation. A later promotional event generates a short burst of chatbot requests, with spillover enabled. The business has paid for a dedicated baseline while also allowing extra standard inference expenditure.

Suppose its illustrative monthly baseline is A$2,400 and ordinary overflow has averaged A$200. During the promotion, overflow reaches A$40 per minute for 90 minutes. That event adds A$3,600, taking baseline plus spillover to A$6,200 before supporting services. Low utilisation over the month did not prevent a short capacity-exhausting burst. These numbers describe a constructed failure, not client experience or published Azure rates.

The owner assumed an 80% budget alert would stop spending. Microsoft's budget documentation explicitly says budget thresholds do not affect resources or stop consumption. Cost and usage data is typically available within 8–24 hours, and budgets are evaluated every 24 hours. A short event can therefore run ahead of this notification process.

Detection should compare the declining workload baseline with the growing share of requests routed to standard inference, then reconcile estimated usage against billed meters. Recovery should first restrict new overflow admissions and activate the tested fallback. Next, resize or reconsider the architecture and review the reservation's applicable commercial terms. Do not promise that deleting a deployment cancels a reservation or produces a refund. Microsoft says deleting or scaling down releases capacity, and reacquiring it later is not guaranteed.

The simpler alternative: standard usage and a non-AI fallback

Remain on standard pay-per-token deployment when demand is uncertain, the dedicated baseline is hard to justify, or the business cannot safely manage overflow. Microsoft identifies standard deployments as the better fit for development, testing, low-volume usage and highly variable traffic. It describes batch deployments as asynchronous processing for bulk workloads without latency requirements. Verify model and feature availability before relying on either option. See the deployment comparison.

Set budget alerts as an accountability tool, not a hard cap. An illustrative alert at 80% of the approved monthly solution budget can prompt investigation; a forecast alert can flag a developing trend. Assign a recipient and a response deadline. Microsoft documents actual and forecast alerts and optional action-group integration, but an automated action triggered by a budget still inherits the budget's evaluation timing. It is not an immediate request-level control.

For customer enquiries, compare the AI architecture with a maintained FAQ page, search, a structured contact form and human support. Those non-AI options may answer the narrow service need without dedicated inference capacity. The trade-off is less conversational flexibility and potentially more staff work. Measure completion, waiting time and maintenance effort rather than assuming either route is cheaper or better.

A hybrid fallback can also keep the decision reversible: display verified answers for common enquiries, queue non-urgent work, and route exceptional cases to staff. Make the degraded service visible. Do not show a generic successful response when no answer was generated or no handover was recorded. PTUs are a poor fit where the simpler workflow already meets the business requirement, or where the cost-control mechanism creates an unacceptable service failure.

Who signs off, and what must trigger a review?

The business service owner should define which requests deserve immediate processing and what happens when capacity or spending limits are reached. Finance should approve the combined cost envelope and commitment assumptions. The technical owner should implement routing, admission controls, monitoring and recovery. One named person should have authority to activate the fallback during an incident; shared awareness is not a substitute for clear authority.

Before production approval, run an illustrative failure test: exhaust the permitted overflow allowance in a controlled environment, verify that new work follows the agreed path, and check that queued or rejected requests remain visible. Test a monitoring outage and an unavailable responder as well. Record what the application actually did, not just what its configuration intended. Use synthetic or otherwise approved test inputs.

Human review has limits. A finance approver cannot validate a burst profile from a single monthly total, and an engineer reviewing logs may miss traffic absent from that logging scope. Alert recipients may be unavailable, and delayed billing records cannot provide instant assurance. Reduce dependence on manual intervention with tested application controls, while reconciling estimates with meter records afterwards. Microsoft states that HTTP status codes alone do not determine whether usage is billed in its Foundry billing guidance.

Set review triggers before committing: material adoption changes, a model change, a larger request shape, unexpected overflow, missed latency targets or an untested fallback. The final decision should be explicit: approve PTUs only if the dedicated baseline is justified and overflow remains operationally and financially acceptable; otherwise stay on standard, change the workflow or defer. The objective is a service whose normal operation and failure behaviour the business can afford—not simply a lower advertised unit price.

Sources

General information only; verify sources before making business decisions.

Turn an idea into a next step.

A conversation about what matters to your business.

Let's talk