Table of Contents
- Why the unit of control is the workflow
- Define what a budget measures
- Attribute usage across hops
- Set ceilings that preserve useful work
- Reserve, reconcile, and close the ledger
- A hypothetical worked example
- Retries, partial results, and failure economics
- Alerts and operational signals
- Turn spend into unit economics
- A printable budget worksheet
- Summary and next steps
Why the unit of control is the workflow
A model call is a useful place to record usage, but it is not the right unit for managing the full cost of an agent. A workflow may call a model several times, retrieve information, run tools, wait for an external operation, resume after a failure, and call the model again to explain or verify the result. Each step can add cost or consume capacity. Counting only the first call hides the accumulated expense and makes it hard to explain what a completed task cost.
For finance and engineering, the practical unit is a named business workflow: for example, one support case triaged, one invoice checked, or one research brief completed. The boundary starts when the system accepts the work and ends when it records a terminal outcome. That outcome might be completed, partially completed, rejected as ineligible, or stopped because a limit was reached. A workflow boundary gives teams a consistent way to compare spend, retries, latency, and usefulness.
An agent hop is an individual model request within that workflow. It might classify a request, decide which tool to call, summarize returned records, or compose a final response. Hop-level measurements help locate waste, but they do not describe the total cost of the business operation. A low-cost first hop can initiate many expensive follow-up calls. Conversely, several modest hops may be reasonable if they avoid a costly manual process or produce a verified result.
This changes the central budgeting question. Rather than asking only how many tokens a prompt may consume, ask how much usage the complete task may incur, what work that usage is allowed to perform, and how the system behaves when the limit is reached. Tokens remain useful for estimating model consumption. They are not a complete accounting unit for the workflow, nor do they tell an operator whether the task delivered value.
A useful production design therefore has two connected ledgers. The usage ledger records model calls and other attributable expenses as they occur. The workflow ledger records the task's budget, status, outcome, and final cost. A shared workflow identifier links the records. This lets an operator move from a monthly spend report to a particular class of work, and from that class to the individual steps that need attention.
Define what a budget measures
Keep consumption, cost, and outcome distinct
A budget policy should say what it limits. A token ceiling limits a particular measure of model input and output. A currency ceiling estimates or limits spend. A hop ceiling limits the number of model requests. A wall-clock deadline limits elapsed time. A tool-call ceiling can constrain repeated or broad operations. These controls address different failure modes; one should not be treated as a substitute for all the others.
For example, an input-token limit may catch an unexpectedly large retrieved document set, but it does not stop a short prompt from triggering dozens of calls. A hop limit can bound call count but cannot ensure that one call uses an affordable model or produces a useful answer. A time limit can stop a job that is stuck waiting, while still allowing it to have incurred meaningful cost before the deadline. Select limits based on the ways the workflow can become unbounded.
Record actual input and output usage separately when it is available, then retain the provider and model identifiers, the workflow version, and the time of the call. Do not infer actual consumption from a prompt-length estimate after the call has completed. An estimate is useful for admission and reservation; a reported usage figure is useful for reconciliation. Preserve both values so a gap between them can be investigated.
Currency accounting needs its own assumptions. The application can convert recorded usage into an estimated cost using a rate-card version maintained by the business. Store that version with the event rather than recalculating every historical record using whatever assumption happens to be current. This guide does not prescribe a provider price: commercial terms and pricing can change, and a team should use its own current, approved assumptions. If an exact cost is not available at event time, label the result as an estimate rather than presenting false precision.
Separate hard limits from planning estimates
A planning estimate predicts the likely consumption of a workflow. A hard ceiling defines what the system will permit. They serve different purposes. The estimate can be updated as observed work changes; the ceiling is an operational boundary that requires an explicit product decision to change. A workflow should never silently convert a forecast overrun into permission to spend more.
For an initial policy, document a normal range, an alert threshold, and a maximum permitted amount. The normal range helps product and finance understand expected behavior. The alert threshold triggers attention before the ceiling is reached. The maximum determines whether the system stops, returns a partial result, or follows an approved fallback. These values should be based on observed traffic and the consequences of stopping, not copied from another team's task category.
One practical distinction is between a workflow ceiling and a per-hop ceiling. The workflow ceiling prevents accumulated use from growing without bound. The per-hop ceiling catches a single unexpectedly large request or response. A hop limit also makes it easier to identify which stage needs a tighter context contract. If one hop repeatedly approaches its maximum, investigate its inputs and output requirements rather than simply raising the limit.
Attribute usage across hops
Give every event a stable identity
Each accepted workflow needs a stable workflow ID that persists through its hops, retries, and resumptions. Each model request needs its own event ID, and each attempt should be distinguishable from the logical step it is trying to perform. This allows the ledger to answer both how much the workflow consumed and whether that use arose from the original attempt or a retry.
A practical event record includes the workflow ID, task type, tenant or account identifier where appropriate, workflow version, step name, attempt number, provider and model identifiers, start and end timestamps, input and output usage, estimated cost, and terminal or intermediate status. Add a parent step or trace identifier when one step can initiate child work. Keep the payload itself out of routine cost events unless there is a justified operational need; the accounting question usually requires identifiers and measures, not a copy of user content.
The workflow record should hold the accepted budget, reservation state, cumulative reconciled usage, completion status, outcome category, and final business unit. Useful outcome categories are specific to the product: completed, completed with a limitation, stopped at a ceiling, failed technically, or abandoned. A single generic success flag collapses distinctions that matter to both customer experience and cost analysis.
Attribute shared work without distorting the task
Some expenses support more than one workflow. A cached retrieval result, shared preprocessing job, or periodic indexing task may not belong to one user request. Do not charge the entire shared operation to whichever workflow happens to read it first. Decide whether to treat it as a shared operating cost, allocate it according to a documented rule, or keep it in a separate cost pool. Whichever method is chosen, apply it consistently and keep direct usage distinguishable from allocated overhead.
Similarly, distinguish model usage from non-model work. Tool execution, retrieval infrastructure, storage, and human review may affect the economics of a workflow even if the model API reports none of them. An application can start with model consumption because it is easy to observe, then expand its cost record as the workflow matures. Calling a model-only figure the total cost of a task will eventually mislead decisions about volume or automation.
For long-running work, write the workflow state and accumulated accounting state together often enough that a restart does not lose the budget history. Resuming a job should continue the same workflow identity, not create a fresh allowance by accident. The engineering details of safe restart behavior are covered in AI Agents in Production: Long-Running Workflows and the related implementation patterns for Idempotent Tools for Agents: Surviving Retries and Restarts. Budgeting depends on those mechanics because a retry that cannot be associated with its original work can make both costs and outcomes difficult to reconcile.
Set ceilings that preserve useful work
Choose limits from workflow shape
A useful ceiling reflects the work the system is intended to finish. Start by listing the stages, the information each stage needs, likely repetition points, and the cost of a partial result. Then decide which stages are essential and which are optional. For example, a workflow might be required to classify and summarize a case, while a second-pass style refinement is optional. If the budget is nearly exhausted, omitting optional refinement may be better than discarding the already useful result.
Set a total workflow limit, a per-hop limit, a maximum hop count, and a deadline only where each one controls a distinct risk. Give each limit an owner and a reason. A very low hop limit can make a workflow fail routinely; a very high one can hide loops. A total allowance that is generous enough for the rarest possible input may wastefully authorize that amount for every request. A staged allocation can make room for normal work while requiring a deliberate decision before expensive optional work begins.
A parent can reserve a portion for each child task, which makes the maximum spend of each branch visible in advance. That is predictable, but fixed allocations can strand unused capacity in one branch while another needs more. A shared workflow pool uses capacity more flexibly, but needs a reliable central accounting decision before concurrent branches proceed. For a first implementation, teams can use a shared pool with a per-hop cap and a small reserve for finalization; teams with strict branch isolation may prefer explicit child allocations.
Do not budget only for the planned path. Include a modest allowance for failures that the workflow is permitted to retry, as well as the final step that records or communicates the result. If retries are not allowed, say so in the policy. The point is not to set a universal retry allowance; it is to make the retry decision visible in the total budget rather than treating each attempt as free.
Make the stopping behavior part of the policy
When a ceiling is reached, the system needs a defined behavior for that workflow type. It can return a partial result with a clear limitation, stop and request a human decision, or terminate with a reason that the caller can act on. The appropriate choice depends on whether a partial answer is safe and useful. It is not acceptable to present unfinished work as complete merely because the model produced fluent text before the final required check.
Some work should not continue automatically after its budget is spent. A financial reconciliation that has not verified its records is not complete simply because an agent has described an apparent match. For any workflow that can produce an external side effect, budget exhaustion should occur before the action when the remaining allowance cannot cover the required decision and verification stages. The application should not treat a model's claim that an operation succeeded as proof of the business result.
The flow below shows a basic control loop. Reconciliation happens before deciding whether another hop can run. If the remaining ceiling is insufficient, the workflow stops or returns an allowed partial outcome, then records a terminal state.
flowchart TD
accTitle: Workflow budget control loop
accDescr: A workflow reserves a ceiling, runs a bounded hop, reconciles its usage, and either continues within the remaining budget or stops and closes its accounting record.
A["Accept workflow"] --> B["Reserve ceiling"]
B --> C["Run bounded hop"]
C --> D["Reconcile usage"]
D --> E{"Continue within ceiling?"}
E -- "Yes" --> C
E -- "No" --> F["Stop or return partial"]
F --> G["Close ledger and classify outcome"]
The loop assumes the budget service can make the latest usage visible before authorizing the next call. If parallel work can run, a stale balance is not sufficient: two branches may both believe the same remaining capacity is available. Reserve capacity before launching concurrent work, or serialize authorization for the shared pool. If reservation fails, the workflow should follow the same defined stop behavior rather than launching an unbudgeted branch.
Reserve, reconcile, and close the ledger
Reserve before work begins
At acceptance, create a workflow record with its policy version and a reservation against the maximum planned exposure. A reservation is not a cost; it is capacity held aside so another concurrent branch or request does not spend the same allowance. The system can release unused reservation as stages finish, provided its accounting can do so safely. Make the distinction between reserved, incurred, and reconciled amounts visible in operational views.
For a single sequential workflow, the controller can estimate the next request, check the remaining allowance, and authorize it. For concurrent work, authorization must account for outstanding reservations as well as settled usage. A simple conceptual calculation is: available capacity equals the workflow ceiling minus reconciled usage minus active reservations. If that amount cannot cover the next planned operation, the controller should not dispatch it. The estimate used for a reservation can be conservative; the final ledger should reconcile against actual reported usage when available.
A reservation strategy also needs a release path. If a branch is cancelled before dispatch, release its reservation. If the provider call times out but may have completed, do not assume it consumed nothing; record the attempt as unresolved until available usage information or a deliberate reconciliation policy resolves it. Otherwise, a retry can be authorized on the mistaken belief that the previous attempt was free.
Reconcile each attempt, then close the workflow
Append usage events rather than overwriting a running total. An append-only event history makes it possible to see the order of calls, identify missing events, and explain corrections. A reconciled event can update the workflow's current balance, but the original estimate and reported usage should remain available for comparison. If a usage report arrives late, reconcile it to the original attempt identifier instead of counting it as a new model call.
At workflow closure, check that every dispatched attempt is represented, reservations have been settled or released, and the outcome has been recorded. A technically successful model response does not necessarily mean the business task succeeded. A missing tool result, a failed downstream write, or an incomplete validation may mean the workflow should be classified as partial or failed. Keep the accounting status and the task outcome related but distinct: spend can be accurately recorded even when the work failed.
These states are especially important when an operation can be retried. A duplicate model call may increase cost even if the surrounding tool action is safely deduplicated. Conversely, an external action might have succeeded even though the caller did not receive a response. The accounting record should preserve the uncertainty and the attempt history; do not promise exactly-once external effects. Teams implementing bounded rollout changes to the workflow can use Canary Releases for Agent Changes: Prompts Need Rollback Too for the separate question of comparing and reverting agent changes.
A hypothetical worked example
Consider a hypothetical support operation that drafts an answer to a billing question. The workflow accepts a case, retrieves the relevant account facts, asks a model to classify the issue, drafts a response, and performs a final check against required fields. A product team chooses an illustrative workflow ceiling of 12,000 total tokens and a maximum of four model hops. Those figures are assumptions for this example, not recommended defaults or a claim about any provider's prices.
The controller reserves enough of the ceiling for classification, a potential clarification pass, and finalization. It does not give every hop the full workflow allowance. The retrieval stage returns a compact, structured record with only the fields needed for this task: case category, relevant dates, and the account facts the support policy permits the workflow to use. The team records retrieval separately from model usage so its cost is not hidden inside a token estimate.
Suppose the first model call uses an illustrative 1,600 input tokens and 240 output tokens. The next call uses 2,900 input and 500 output tokens after adding retrieved facts. A verification call then uses 3,200 input and 350 output tokens. The cumulative model use is 8,790 tokens. These arithmetic figures are invented to show accounting, not observed measurements or a benchmark. The ledger records each call, its stage and attempt, then links the total to the support case outcome.
At this point, the workflow still has nominal room under its 12,000-token ceiling, but the controller must consider reservations for any remaining required work. If finalization requires another call, it can proceed only if the estimated request fits the unreserved balance and the four-hop limit has not been reached. If a response has already passed the required check, the system should not spend the remaining allowance on an optional rewrite merely because capacity exists. A ceiling is permission to spend at most that amount, not a target to consume it.
For illustration, assume the draft fails a required-field check and needs another pass. The workflow records the validation failure as a reason for the new attempt, budgets the call against the remaining capacity, and repeats validation. If the second draft passes, the outcome is completed. If the call would exceed the ceiling, the workflow returns a partial status for a human to finish rather than labeling the draft verified. That outcome is useful operational information: the issue may be an insufficiently compact input, a policy that needs a better deterministic check, or a task whose allowed complexity exceeds its ceiling.
Now compare a counterexample. A team measures only the initial classification call and reports that a support case costs little to process. In production, an ambiguous case triggers repeated retrieval, another model call, a timeout, and an untracked retry. The initial metric remains low while total workflow cost and completion time rise. The team may respond by changing the model or prompt even though the actual defect is a retry loop and missing end-to-end attribution. Measuring a complete task would reveal where the additional attempts accumulated.
The example also shows why cost per accepted case can differ from cost per attempted case. If some cases are rejected as out of scope, report their volume and consumption separately instead of quietly excluding them. If the measure is only cost per successful completion, a worsening failure rate can make that figure look better or worse in misleading ways. Pair spend with the outcome counts needed to explain who completed, who failed, and what work was left for a person.
Retries, partial results, and failure economics
A failure trace to test before launch
Use an operational test that deliberately interrupts a workflow after a model request has been dispatched but before the usage event is reconciled. The test is not a benchmark; it checks whether the accounting and control logic survive an ambiguous failure. Give the run a fixed workflow ID and a small illustrative ceiling in a non-production environment, then simulate a timeout at that boundary.
The expected trace is: the workflow reserves capacity; the controller records an attempt before dispatch; the simulated interruption leaves the attempt marked unresolved; the restart resumes the same workflow identity; the controller checks whether the prior request's usage can be reconciled; and only then does it decide whether another attempt is allowed. The final record includes the attempt count, any unresolved usage, the outcome, and released or retained reservations. The test should not assume a timed-out request consumed zero.
Pass conditions: no resumed attempt receives a fresh workflow ceiling; no concurrent branch spends an already reserved allowance; the unresolved call remains visible; the retry requires available capacity and obeys the configured attempt and hop limits; and the workflow closes as completed, partial, or failed with its accounting status intact.
Fail conditions: a restart resets cumulative usage; a retry occurs without an authorization decision; two branches each spend the same remaining balance; an unresolved call disappears from the ledger; or a fluent model response is marked complete despite a missing required verification. A failed condition should block rollout of the budget control until corrected. This test is intentionally about accounting and retry behavior, not a claim that external operations can be made exactly once.
Distinguish retries from new work
A retry is another attempt to complete the same logical step; it is still real consumption and must count. A user-initiated new request may be a separate workflow, even when it concerns the same case. Store both workflow identity and logical task identity so reports can show whether repeated attempts arise from technical failure, ambiguous user input, or normal new work. Otherwise, teams can undercount retries by making every attempt look like a fresh task, or overcount work by merging legitimate new requests.
Set retry policy by failure class. A transient transport failure might merit a bounded retry if the request's state is known. A validation failure may justify a targeted correction pass. A repeated policy or data problem should not trigger another identical model call. Include the retry's reason and whether the input changed in the event record. That information can distinguish useful recovery from repetition that adds expense without improving the result.
Decide what the user sees
A partial result should identify what was completed and what remains unresolved in language appropriate to the product. The status must come from workflow state, not a model's self-description. For instance, a response can be drafted but not verified; a retrieval step can finish but a required record may be unavailable. Show the user a usable next action, such as submitting for review or trying again later, only where the product supports that action.
If a workflow has reached its ceiling, restarting it at the same settings can reproduce the same failure and charge again. Before offering a retry, identify what changes: a smaller input, a different task path, a human review, or a consciously higher allowance. Any increase should be an explicit product or operator decision, not an automatic escape hatch that defeats the ceiling. This guide does not cover incident command; for that distinct response process, see An Agent Incident Runbook: Stop, Contain, Investigate and Resume.
Alerts and operational signals
An alert should prompt a decision, not merely announce that a number moved. A spend threshold can tell an operator that a workflow family is using more capacity than expected; a rise in average hops can point to repeated reasoning or a changed input mix; a growing unresolved-attempt count can indicate that reconciliation is falling behind. Pair each alert with an owner, a response action, and a view that can identify affected workflow versions and task categories.
Use more than one time horizon. A sudden burst may require immediate containment even when the monthly total remains small. A steady increase in cost per completed unit may deserve a product investigation without paging an on-call engineer. A daily ceiling can prevent a bad configuration from running unchecked across a large volume, while a per-workflow ceiling limits exposure from one unusually complex task. These controls should have separate thresholds because they answer different operational questions.
Monitor distributions, not only averages. A stable mean can conceal a growing tail of workflows that use many more hops than usual. Track a high-percentile or maximum-use view alongside the median and total, and segment by task type, workflow version, and outcome. Avoid treating any one percentile as a universal alert threshold; select it based on traffic, the consequences of expensive outliers, and how quickly the team can respond.
An alert is more actionable when the team can inspect a representative trace without exposing unnecessary user content. Include stage names, token measures, retries, estimated cost, outcome, and policy version in the investigation view. If the aggregate alert cannot be traced to events, it can identify a symptom but not a fix. Retain sufficient event history to compare a changed workflow with its prior behavior, and document when an accounting assumption changes so a rate-card update is not mistaken for a usage spike.
Turn spend into unit economics
Usage reports answer how much was consumed. Business operators also need to know what that consumption was for: one support case handled, one invoice reviewed, or another clearly defined unit. Select a unit that matches the decision being made and can be counted reliably. Avoid changing the definition from one reporting period to the next without showing the break in comparability.
Unit economics relates cost to a business unit of value; calculations should state their assumptions and include relevant indirect costs, not only direct model usage (FinOps Foundation: Unit Economics). In practice, a team can begin with direct model usage and separately identify retrieval, tool execution, infrastructure, human review, and shared services. Include an indirect component only when its allocation method is explainable. A rough, explicit allocation is more useful than presenting a narrow direct-cost figure as if it represented the whole operation.
A simple illustrative calculation makes the assumptions visible. Suppose, hypothetically, a month contains 2,000 attempted cases, of which 1,700 meet the product's completion definition. Assume the accounting system attributes $340 of model expense and $510 of other directly assigned workflow expense to those attempts, plus an assumed $300 allocation for shared operations. The resulting illustrative total is $1,150. Dividing by 1,700 completed cases gives about $0.68 per completed case; dividing by 2,000 attempts gives about $0.58 per attempt. Those calculations use invented inputs and are not a price estimate or forecast. The difference is a reminder to show the denominator and the treatment of incomplete work.
For a decision about automation, compare like with like. A workflow that costs less per completed case may still be undesirable if it produces more corrections, takes longer, or shifts hidden work to a support team. Conversely, higher model consumption can be reasonable if it materially reduces another measured cost or improves the completion rate. Include the quality or service measure that makes the business unit meaningful, and state which costs were left out. Do not claim a return based solely on model spend divided by request count.
Review unit measures when the workflow changes. A new verification stage might increase direct use while reducing unresolved cases. A cheaper prompt might lower model consumption but increase human correction. The point of measuring unit economics is not to minimize every usage number; it is to help product, engineering, and finance decide whether the workflow's total operating pattern is acceptable and where improvement is worth pursuing.
For organizations that need help connecting workflow instrumentation, service ownership, and operating controls, production platform engineering is a relevant next step. Keep the engagement focused on the systems and measures your team actually needs rather than buying a broader program by default.
A printable budget worksheet
Use this worksheet for one workflow type at a time. Complete the policy fields before rollout, then use the event and review fields during operations. The aim is to make decisions inspectable; it is not to create a single universal template that fits every task.
Workflow definition
- Business unit: Name the completed or attempted work being measured. Specify exactly what counts as one unit.
- Start and terminal states: Define when the workflow begins and the outcomes that close it, including partial, failed, and rejected work.
- Required stages: List the stages necessary for a valid result and identify optional work that can be omitted when capacity is low.
- Workflow identity: Confirm that the same identifier follows retries, resumptions, and child work that belongs to this task.
- Version: Record the policy and workflow version used for each accepted task.
Budget policy
- Workflow ceiling: Specify the maximum permitted use and the measure it limits. State whether this is a token limit, estimated currency limit, or both.
- Per-hop ceiling: Set the maximum for one model request and explain which stage needs it.
- Hop and retry limits: Set separate maximums where they control different behavior. Define which failure classes may retry.
- Reservations: Document how capacity is reserved for concurrent branches, finalization, and permitted retries.
- Alert threshold: Name the signal, owner, and response action that apply before the hard ceiling is reached.
- Exhaustion behavior: Decide whether the workflow stops, returns an allowed partial result, or requests human handling.
Accounting and outcome
- Usage events: Record per-attempt input and output usage, identifiers, timestamps, stage, and status.
- Cost assumptions: Version the rate assumptions and distinguish estimated from reconciled cost.
- Other expenses: Record relevant tool, retrieval, infrastructure, review, or shared costs, with allocation methods where applicable.
- Completion test: Define the evidence that supports each terminal outcome; do not use a model's assertion alone as proof.
- Unit measure: Report cost per attempt and per completed unit where both help explain performance.
- Recovery test: Simulate interruption after dispatch and verify that a resume does not reset usage or authorize unbudgeted work.
Printable review summary
Workflow: ____________________ Owner: ____________________ Policy version: ____________________
Business unit: ____________________ Terminal outcomes: ____________________
Workflow ceiling: ____________________ Per-hop ceiling: ____________________ Maximum hops/retries: ____________________
Exhaustion behavior: ____________________ Alert owner and action: ____________________
Current cost assumptions: ____________________ Excluded or allocated expenses: ____________________
Recovery test date and result: ____________________ Next review date: ____________________
Summary and next steps
Start by treating the complete business workflow as the accounting boundary and model calls as events within it. Give each accepted task a stable identity, attach each attempt to that identity, and distinguish estimated consumption from reconciled use. Add ceilings for the specific risks your workflow has: cumulative spend, oversized hops, excess calls, or excessive elapsed time.
Then define the stopping behavior before a limit is reached. Decide what can safely be returned as partial, what requires verification, and what should stop for a person. Reserve capacity before concurrent work begins, reconcile attempts rather than assuming a timeout is free, and close the ledger with both an accounting status and a business outcome.
Finally, review spend against a defined unit of work, with assumptions and relevant indirect costs visible. Use the worksheet to agree on the workflow boundary, limits, alert response, and recovery test. Start with one important workflow, inspect its actual traces, and revise its policy when the evidence shows that a stage is wasteful, a ceiling routinely blocks valid work, or the measured unit no longer reflects the decision the business needs to make.