Scheduling model decisions for responsive agents
How admission, queues and decision dependencies let a person keep directing an agent while it works.
An agent is finding a black dress in medium, under $150, available for collection on a particular day. While it checks the shop, the person says: “Actually, make that large.”
That correction should enter the work promptly. The agent should preserve the budget and collection date, establish what happened to the medium reservation, and continue toward a valid large one. The person should not have to wait for every background check before being heard.
This is part of what a business commissions when it asks for an interactive capability. It supplies the operating distinctions: which evidence makes an option usable, what must remain fixed, and which changes deserve immediate attention. The resulting model and tool need an execution design that preserves those distinctions under load.
Our browser-agent demonstration makes this concrete. It operates an owned demonstration shop, with actual speech recognition, model decisions and simulated reservations. Inspecting its timings reveals why a fast individual decision can still produce a slow interaction—and where a more useful next step can begin.
Measure the path to a useful result
A response can wait twice before a decision request even starts: once in the browser’s work queue and again at the decision gateway. The request then takes time, followed by validation and delivery back to the application.
In an earlier capture, one collection response followed this path:
| Part of the path | Observed time |
|---|---|
| Browser queue | 1,359 ms |
| Gateway queue | 304 ms |
| Decision HTTP request | 450 ms |
| Remaining transport and local work | 35 ms |
| Response-body arrival → validated callback | 2,147 ms |
Values are rounded independently. Roughly 1.66 seconds accumulated in queues. Shortening the decision itself would have left most of that wait intact.
In a new capture on 31 October 2026, the revised scheduler admitted up to eight resource decisions while reserving capacity for a spoken goal. Across 27 resource decisions:
- 26 reached a validated callback within one second of the response body arriving. The median was 490 ms.
- Browser queueing peaked at 15 ms; gateway queueing at 19 ms. These are separate measured intervals.
- One source still took 1,679 ms. Its decision HTTP request accounted for 1,622 ms of that path.
The five spoken directions produced their corresponding interface responses in 790–871 ms from speech end, estimated through the browser’s paint opportunity. These are different endpoints: a validated source callback is not a separately measured visible update. Decision HTTP includes network transport, not just model computation.
“Speech end” is an acoustic estimate: the end of the last 10 ms audio frame with RMS level at or above −45 dBFS, mapped to the browser’s estimated audio output time. Each prerecorded input plays in full before warmed GPU recognition. Live microphone endpoint detection was outside this run.
These two development captures diagnose particular executions; they are not a controlled benchmark or a latency guarantee. The public interactive examples replay qualified cases without making fresh model requests. The captured film exercises the actual speech and decision path.
Give new direction a route through the system
The revised capture gateway allows eight resource requests and nine requests in total. A waiting goal takes priority over further resource admission. Its HTTP connection pool also accommodates nine requests: otherwise the newly admitted goal could simply join another queue below the scheduler.
That last detail matters. A priority assigned in one layer has little effect if all work immediately enters a deeper queue that ignores it. Reserving capacity is also different from interrupting requests already in progress. This implementation lets those requests finish; it does not reclaim their running work.
Welsh, Culler and Brewer’s SEDA treats stages and explicit queues as control boundaries. A stage can expose its backlog, limit admission and apply its own scheduling policy. Its discussion of finite queues also makes overload a service decision: apply backpressure, reject work or provide a deliberately reduced service. SEDA studies Internet services; our capture uses a fixed admission policy, not its adaptive controllers.
For an agent, those boundaries can express the activity being commissioned:
- New direction: reserve a path for the person to change the goal.
- Immediate prerequisites: admit checks that can unlock the next useful action.
- Background examination: keep useful diagnostics progressing within an explicit budget.
- Exceeded capacity: expose pending or unavailable evidence rather than treating an unchecked condition as satisfied.
The last three are policy choices to develop for each capability. The current pilot prioritizes the active goal generation and page in its browser queue; it does not yet calculate the value of every pending check. Eight resource slots worked for this bounded capture. It is not a universal setting for a heavily loaded service.
A negative finding can finish a branch
The shop agent needs several facts about a candidate: the requested variant is in stock, collection is early enough, and the combined price meets the budget. The semantic decision identifies the response’s role; deterministic code checks the actual product identity, variant, dates and amounts.
An acceptable reservation needs all relevant conditions. Rejecting one candidate can need much less.
Suppose the collection response already proves that a dress cannot be ready by the requested date. Its inventory answer is still pending. A scheduler could close that candidate immediately and give attention to another one, while retaining the late inventory result as an observation if it remains useful.
Required facts
├─ Stock fits request
├─ Collection meets date
└─ Total fits budget
All true → may reserve
Any false → reject candidate
Unknown → keep pending
The recorded controller starts inventory and collection requests for eligible candidates concurrently, but consumes candidates in order and awaits each inventory result before inspecting collection. A slow inventory decision can therefore delay acting on an already available collection failure. More concurrent requests alone do not remove that dependency in the controller.
The public replay now checks those two proofs independently. Either verified failure can close a candidate; reserving still requires both positive proofs, and a later candidate cannot bypass an earlier one that remains possible. Missing, malformed, stale or wrong-product evidence stays unavailable. This refinement passed local ordering and fault tests. The actual-model timings above belong to the recorded controller, before this change.
The distinction also prevents an unsafe shortcut. “Most checks have finished” does not mean the agent may reserve. A missing stock check is not equivalent to an unrelated analytics response still being examined. The tool must define the exact set of evidence that permits each action.
Design for the occasional slow result
Dean and Barroso’s The Tail at Scale explains how waiting for every member of a fan-out can magnify occasional slow responses. It also argues for differentiated service classes and short lower-level queues so application priorities can take effect. Its setting is large distributed services; the transferable question here is which completion the next operation actually requires.
Our newer capture still contains a slow inventory response. An agent can remain responsive around such a delay by acknowledging new direction, showing established facts and advancing independent work. It must keep dependent actions pending when their necessary evidence is missing.
The same paper discusses issuing duplicate requests to reduce tail latency. That is not implemented in this capture. For model-backed tools, such a design would need to account for additional cost, possibly different answers and whether cancellation actually stops computation. Racing duplicate reservation writes would require a separate effect and idempotency protocol. The first useful improvement here is to avoid waiting on an irrelevant dependency, not to repeat every slow operation.
Keep progress compatible with the person’s correction
Fast feedback and completed work have different clocks. In the new recording, the first catalog callback arrived 527 ms after the accepted goal was dispatched. The medium cart’s quote callback arrived 3.213 seconds after that same dispatch. Preparing it required several dependent requests; the subsecond goal response did not mean the whole task had finished.
The person began the large-size correction while that work was progressing. The medium quote finished during the spoken sentence. The controller accepted the correction under a narrow compatibility rule: the goal and chosen product still matched, and the latest reservation could be reconciled.
It then released the medium reservation, verified that release, checked fresh large-size stock and prepared the replacement. Dropping an old callback would not have undone a reservation already created. The application needs both a rule for ignoring obsolete decisions and a rule for resolving effects already in progress.
This is where context validity meets scheduling. A queue can prioritize newer work, but priority does not make that work safe to apply. The tool still needs the relevant identities, revisions and evidence at its action boundary.
Commission the decision and the work around it
A model trained to recognize an inventory response is only one part of this capability. Its meaning depends on the tool: which response categories exist, what identifiers will be joined, and which action each category can enable. Recurring decisions can specialize around those stable semantics while still receiving the current response and goal as live input.
Changing the tool’s meaning therefore changes what must be evaluated. Adding a new reservation state, for example, calls for examples of that state and checks that the controller handles it correctly. A model that returns an old, plausible label quickly can send the workflow down the wrong branch. This is why SYSTEM1 develops the model and embeddable tool together; this prototype demonstrates the execution design without claiming a custom-trained quality gain.
A useful commissioning brief can describe behavior in terms of the work:
- A repair desk could keep quoting while stock checks run. Fit and collection evidence would arrive separately. An impossible collection date could close one supplier branch immediately; a mechanic’s correction to the part specification would invalidate only evidence tied to the old part. The desk could preserve the customer’s price limit throughout.
- A returns team could prepare the next conversation before a case is complete. Clear order and delivery evidence could open the appropriate question while an ambiguous condition report remains pending. The refund action would still wait for its required evidence. “We can ask this now” and “we can approve this now” would be different tool operations.
- A release-testing agent could follow a broken workflow as it happens. Responses identifying the booking service could unlock a focused test while unrelated asset checks continue. If the operator changes the tested build, old observations could remain visible as history but lose permission to advance the new test.
These are proposed capabilities, not additional demonstrations. Each makes a different unit of progress available before the whole investigation finishes. The business contributes the operational distinctions; the model interprets incoming material; the tool turns qualified findings into bounded next actions. Scheduling determines whether the person can use that capability while the work is still unfolding.