Loading

Preparing your content

Article Details
2026-11-17BXT Data Editorial

Stockout blinds agent checkout

Stockout blinds agent checkout article image

57.2% of shopping tasks passed; 42.8% did not, and more than half of the failures still ended in a payment. Visa's Agentic Commerce Benchmark, published on November 5 and read here as of November 17, 2026, ran 22 models from 12 labs across 15 shopping tasks with three attempts each. For instant retail operators the interesting part is not the pass rate but where the runs broke: stock visibility, delivery windows and store choice, the three fields a quick commerce order actually depends on.

Where the shopping agents actually broke

Known factsUnverifiedTo verify
22 models from 12 labs, 15 shopping tasks, three attempts per task, average pass rate 57.2%Whether the 15 tasks are weighted towards grocery, convenience or general merchandiseThe per-task breakdown and the task scripts used in the runs
Best model 82.2%, weakest 28.9%; only 34.2% of runs passed all three attemptsWhether the models saw live store inventory or a cached product feedThe feed freshness each model was served during the benchmark window
54% of failed runs still reached a completed purchase; Trust Index puts 72% on AI-assisted shopping and 23% on trusting an agent to payWhether the purchase happened on the same merchant site or on a competitorSession-level recovery data by merchant, which Visa has not published

The benchmark scored a run as passed only when the agent finished the whole job: it found the item, picked a variant, set a delivery or pickup option and paid without a person taking over. Two thirds of the models never produced three clean runs in a row. The best model finished 82.2% of attempts and the weakest 28.9%, a spread wide enough that “our agent can shop” tells a category team almost nothing on its own.

Visa also cut the results by behaviour and by price. Models that asked more clarifying questions before acting passed 70.2% of runs, while the models that asked least passed 44.4%, which points at missing product attributes rather than missing reasoning. The five most expensive models passed 67.1% and the five cheapest 50.2%, so capability sets the ceiling and budget alone does not buy it.

Three fields decide whether an agent can check out

Stock has to be visible before the agent asks

An agent that cannot see per-store availability does not fail politely; it substitutes. In the runs summarised by the benchmark, agents that met an unavailable variant moved to a sibling SKU or another store instead of stopping, which is how a session ends in a purchase the operator never planned. Availability by store, not assortment width, is the field that keeps an order inside the trade area the network was built for.

Delivery windows and store choice break after stock

The second cluster of failures came at slot selection and store choice, where a page offers a radius the agent cannot interpret or a pickup window that expires mid-session. That is a data-contract problem rather than a marketing one: a timestamp in ISO format and a distance in kilometres are readable, “today before six” and “near you” are not. Operators who publish structured slots and explicit cut-off times keep more agent sessions inside their own on-demand delivery network, and they convert consumer demand for speed into orders they can route.

Both clusters appear in the same category on the same day, which is why a single fix rarely moves the number. A store that publishes stock but not slots fails one step later, and the failure is recorded against the merchant rather than against the data pipeline that produced it.

What the pass rates change in a category plan

Benchmark metricResultWhat it means for an instant retail operation
Runs that completed a full task57.2%Plan for retry; one clean run is not a repeatable flow
Runs that failed42.8%Failures cluster at variant, slot and store selection
Failed runs that still ended in a purchase54%The shopper takes over at payment; log the handoff
Models passing all three attempts34.2%Consistency is the scarce trait, not peak ability
Best and weakest model82.2% / 28.9%Model choice shifts the outcome more than prompt tuning

Read as an industry trend signal, the spread matters more than the average. A category with tight variants, clear pack sizes and one naming convention will pass far more often than a category where the same product ships in four formats, and the benchmark's 15 tasks are broad enough to expose that gap. Agent traffic will therefore concentrate on the categories whose product data is already clean, at least until operators repair the rest.

There is a trust ceiling as well. Visa's Trust Index, cited alongside the benchmark, finds that 72% of consumers have used an AI assistant to shop while only 23% trust an AI to pay on their behalf. That gap explains the 54% number: a run that failed technically can still end in a completed purchase, because the shopper takes over at the payment step and finishes the order in the same session, often on a different site.

A weekly check a retail operations team can run

  1. Choose ten SKUs that carry the category's volume rather than its newest launches.
  2. Run one identical task through two agents and log the exact step where each run stopped.
  3. Classify the stop as data, access or payment, and route it to the team that owns that layer.
  4. Repeat weekly and escalate any failure point that survives two consecutive cycles.
FieldExampleHow it is captured
Run reference2026-11-17-A-07Date, agent and task number
Stop stepSlot selectionLast recorded action in the transcript
Availability shown0 units at store 214Product page at the moment of the run
Slot format publishedISO timestamp with cut-offDelivery promise on the page
OutcomePurchased elsewherePost-session check of the order file

The check is small enough to repeat every week and specific enough to produce a number. Pick ten SKUs from the top of the category, run the same task through two agents, and record where each run stopped. Two consecutive weeks at the same failure point is a data defect rather than a model defect, and it belongs to the team that owns the product feed instead of the team that owns the campaign.

Keep the log in one place and score it the same way each week: the share of runs that reached checkout, the share that reached payment, and the share that needed a human. Those three numbers move before revenue does, and they are the only ones a category manager can act on without waiting for the next benchmark refresh.

FAQ

  • Do these failure rates mean agents are not ready to shop?

    They mean single-run reliability is low, not that agents cannot buy. With a 57.2% pass rate and only 34.2% of models passing all three attempts, an operator should design for retry and handoff; the 54% of failed runs that still converted show where that recovery already happens.

  • Which data field removes the most failures?

    Store-level availability with a timestamp. Agents that met an unavailable variant substituted rather than stopped, so a live availability field and an explicit cut-off time protect both the order and the trade area.

  • How should an instant retail team measure agent traffic?

    Count sessions that reach checkout, sessions that reach payment and sessions that need a human, by category. Compare those three counts with your own app, because what looks like a model defect is often a stock visibility gap on the product page.

Sources