What Happens to an AI Product at Ten Times Its Current Usage?

To assess an AI product at ten times its current usage, define which work grows and then follow it through to an accepted customer result. Test peak arrivals, dependency limits, and recovery alongside quality and human effort. Record what the product has demonstrated, what a controlled test supports, and what still depends on a projection.

Define what ten times the usage means

Ten times the monthly jobs can come from more users at existing customers, more customers, or heavier work per job. Each route places different demands on the product. More enterprise customers can add integrations and permission rules even when their total model traffic is modest.

Start with a recent production population and a completed task definition. For a document-processing product, a completed task might be an accepted extraction saved to the customer's receiving system. Count failed attempts, corrections, and unfinished jobs separately. Ten times the number of model responses doesn't establish ten times the completed work.

Describe the workload mix: document length, extraction complexity, tool calls, and expected output. Identify large customers and bursts, such as a morning upload or a month-end batch. A daily average can conceal a short arrival peak that determines the actual capacity requirement.

Write separate scenarios for more of the same work and for a changed customer mix. Hold the product version and acceptance rule constant in the first comparison. Then test the new case explicitly, including any longer inputs or additional access checks it introduces.

A discussion among AI application builders asks how to handle provider downtime and rate limits, including the delay from sequential failover. It reflects a practical reader question, rather than a reliability benchmark. Investors can turn that question into a request for observed behaviour under a defined workload.

The method below is a proposed diligence approach. Its examples are illustrative and don't describe a client result. Use it with the target's engineering and operating owners to establish which growth assumptions need evidence.

Inspect capacity at every dependency

Map the task through intake, source retrieval, model calls, validation, and the receiving system. Add the queues and human steps between them. For each dependency, ask what limits throughput, who controls that limit, and what evidence supports the capacity claimed.

Inspect the target's actual account and configuration through authorised access. A model provider may limit requests and different token categories separately. Database connections, document conversion, search, and customer APIs can impose another ceiling before the model limit becomes relevant.

Anthropic's rate-limit documentation, checked on 19 October 2026, distinguishes requests, input tokens, and output tokens per minute. It warns that short bursts can exceed limits and that allowed usage isn't a guaranteed minimum. Confirm the applicable model, account, and limit instead of importing a capacity figure from another customer's tier.

Translate the customer job into dependency demand. If an extraction makes several model calls and sometimes retries, include those attempts. A longer document can change token demand without changing the job count. Use observed distributions and identify estimates that lack request-level records.

Ask what happens when an increase request is delayed or refused. The application needs an admission rule, a visible pending state, and a route for work that exceeds the customer's deadline. More application workers can't remove a fixed downstream limit.

Check the fallback path's capacity and approval separately. A second model can have its own limits, data terms, and output behaviour. Ask for its task evaluation and end-to-end failure drill. A configured provider name doesn't establish that the alternative can carry the intended load.

Measure completion time and backlog recovery

Measure elapsed time from customer intake to accepted completion. Record queue wait, processing, review, and final delivery where the workflow permits that separation. Streaming an early token may improve the interface while leaving the time to complete the customer's task unchanged.

Request the median and slow-end latency, such as the 95th percentile, for a stated population and period. For a completed-task measure, the 95th percentile marks a time within which 95% of the completed tasks finished. Show timed-out and unfinished tasks separately so excluding them doesn't make completion appear faster.

Increase offered arrivals in a controlled test without making every new job wait for the previous job to finish. Otherwise the test client can slow down with the application and conceal unmet demand. Record offered, admitted, completed, rejected, and pending jobs over the same intervals.

The Amazon Builders' Library article on queue backlogs explains how arrivals above processing capacity drive growing delays. It also describes measuring message age to detect work falling behind. A queue's acceptance response therefore needs an accompanying view of completion and oldest pending work.

Test recovery after a burst or dependency slowdown. Continue admitting the ordinary workload while measuring whether the backlog drains, how long customers wait, and whether jobs expire. Spare processing capacity must cover both current arrivals and recovery work.

Bound retries and inspect where they occur. The model client, worker, and customer may each retry the same task. Count attempts and reconcile uncertain writes through the destination's supported contract. Our safe-fallback guide explains the task ownership and recovery boundary in more detail.

Keep quality checks running under load

A throughput result needs an acceptance measure. Count tasks that meet the agreed business criteria, along with unsupported answers, invalid outputs, and actions that need correction. A fast response that moves work back to staff changes the capacity calculation.

Run the same approved evaluation cases at baseline and increased load. Record application, model, prompt, retrieval, and source versions. Add representative long or difficult cases so the exercise doesn't prove only the fastest route through the system.

The NIST AI RMF Core calls for documented test sets and metrics, measurement in conditions similar to deployment, and stated limits on generalisation. Use that principle to connect a scale claim to its actual test conditions and coverage.

Inspect changes the product makes under pressure. It may retrieve fewer documents, shorten context, reduce verification, or use another model. Evaluate those paths against the same accepted outcome. Record which cases receive degraded treatment and what the user sees.

Separate a capacity effect from a workload effect. The same cases may keep their quality at higher concurrency while new customers introduce unfamiliar documents or business rules. Report those comparisons separately so the team can identify the appropriate repair.

Keep consequential failures visible outside the blended score. An unauthorised action or customer-data disclosure needs its own finding and response. Set stop criteria with the responsible owners before testing, and use approved test records and destinations.

Test enterprise permissions and customer isolation

More enterprise users introduce more combinations of role, record access, and customer context. Trace how the application carries identity through retrieval, background jobs, caches, and external tools. Confirm which component enforces each restriction.

Test a user requesting a permitted record and a similar request outside their access. Repeat the checks through a queued job and after a role change. Inspect the returned result and stored task state, including whether the application rechecks authority before a consequential action.

Ask how cache keys and retrieval indexes separate customers and permissions. A shared answer cache needs a defined context and invalidation rule. Have the team demonstrate that similar questions from different customers can't expose another customer's answer or source material.

Load tests also need customer isolation. Ask what happens when a large account consumes its allowance or a customer API becomes slow. Measure whether other customers still receive their expected service and whether the target can identify the affected scope.

Inspect the support route. More customers can increase requests for exports, permission changes, and incident investigation. Staff need bounded access and an audit record for those actions. A plan to give support broad production access creates a separate control question.

Use synthetic or approved data for these checks and agree active testing with the target. A diligence engagement doesn't authorise cross-customer access or production disruption. Record untested controls and specialist security dependencies in the findings.

Calculate review and support demand

Model the people who complete the product's exceptions. Measure the share of jobs needing review, handling time, and the arrival pattern. Include rework after review and cases that need a different specialist rather than treating every intervention as interchangeable.

The following example assumes 1,000 daily jobs at baseline and 10,000 in the growth scenario. Each reviewed job takes six minutes of active handling. It holds that time constant to show the effect of volume and review share. The figures are illustrative assumptions, not staffing benchmarks.

The calculation is daily jobs multiplied by review share and active minutes per reviewed job. It excludes training, breaks, leave, supervision, and waiting for other teams. Active hours don't directly specify a headcount or prove that reviews can finish within the customer's deadline.

Inspect the queue by time of day and case age. Fifty hours of daily handling can still miss a narrow morning service window. Have the operator explain how arrivals match staffed coverage and which skills constrain the difficult cases.

Keep customer support and onboarding alongside task review. Repeated customer-specific setup can consume engineering time even if the automatic processing path scales. Compare deployment cohorts to see whether the claimed reduction in effort appears in records.

Ask which interventions the founder performs and how another person takes them over. Unrecorded founder effort creates uncertainty in both capacity and economics. Request a supervised handover or documented case reconstruction rather than assuming that hiring later resolves the dependency.

Connect the growth scenario to operating economics

Reconcile accepted jobs, all attempts, and delivery cost for the same customer population. Include retries, retrieval, hosting, and human effort under an explicit allocation method. The inference-bill guide covers the request ledger and margin calculation.

Compare two commercial scenarios: existing customers doing more work within their current terms, and new customers bringing new revenue. Revenue needn't rise with usage in the first case. The second can add onboarding and support demands that a simple volume multiplier misses.

Identify capacity steps and commitments. The next growth band may require more infrastructure, a different service agreement, or added operating coverage. Ask for evidence of that requirement and keep proposed arrangements separate from signed commitments.

Test the sensitivity to cache behaviour and workload mix. A result based on repeated prompts can differ from many customers sending unique documents. Quality-preserving caching also needs the permission and freshness boundaries described above.

Include the operating cost of the tested degraded path. If a provider slowdown diverts work to staff or another approved model, record that path's cost and completed-task rate. A recovery plan can protect service while changing the economics.

Keep measured results separate from the growth model. A short controlled run can establish a capacity observation under its conditions; it doesn't prove a future margin. The investment and finance teams own the financial interpretation and the assumptions they carry into the deal model.

Request a reproducible scale evidence pack

Request a workload definition, environment description, dependency limits, test configuration, and task-level result ledger. The ledger should connect offered work to its final state and explain missing observations. Record the product version and evidence cutoff.

Agree a staged test with the target's owners. Start at the measured baseline, increase offered demand, hold a defined interval, introduce an approved failure, and observe recovery. Set resource bounds and stop conditions before running it. Use production only within an explicitly authorised scope.

Separate tests that use simulated dependencies from tests against real approved services. Simulation can isolate application behaviour without consuming provider capacity. It can't establish the real service's latency, error pattern, or ability to supply the assumed throughput.

Require evidence for user-visible outcomes: accepted task completion, slow and pending work, quality, review demand, and customer isolation. Have the team reproduce a failed case and explain the remedy. Record whether the repair itself needs another test.

Finish with a finding for each material growth assumption. State the observed limit, supporting evidence, unresolved condition, and work needed to revisit it. For commissioning access and reporting, use our decision-focused diligence brief or review the AI technical due diligence scope. A target may have a credible growth plan with open work; your review should make that work and its dependencies inspectable.