Building a universal catalog for agents

On September 20th, Amazon started blocking agents from shopping on their website. Users report that xAI's Grok and OpenAI's new agents can't get through either.

Yesterday, Walmart went the other way. They announced agent profiles, an MCP-based agent gateway, and a written set of rules for agents that shop on a customer's behalf.

We believe that agents are going to shop for humans no matter what. The question will become how they get in and what they find. We think the answer is an open door, and results that nobody can manipulate.

For context, Agora is a universal catalog for agents. An agent can search, buy, and track orders across more than 1 million online stores through one schema, one cart, and one checkout. Today we serve about 15 million agent requests a month, through distribution partners including Glance and Samsung TV Shopping.

This post explains how it works and why we built it this way.

The short version

• Websites are built for people. Agents that shop through a browser are slow and break easily. That is true for one big retailer, and much worse across a million small ones.

• We built one catalog for all of them. Agora ingests products from Shopify, WooCommerce, and custom stores into one data model. The catalog refreshes daily, and we check price and stock live when an agent interacts with a product.

• There are two ways in. Browser agents get a dedicated /agent interface with stable selectors and machine-readable JSON. Programmatic agents get a REST API for search, cart, checkout, and order tracking.

• The agent never handles card numbers. Payment goes through Stripe. The agent passes payment tokens, and by default it stops before payment so a person can confirm.

• Ranking is not for sale. Agora holds no inventory, sells no products of its own, and takes no money for ranking. For an agent, the ranking is the whole product, so this matters more than it does for people.

• Businesses pay us, not for placement. We make money from subscriptions and usage credits that we charge the businesses building on Agora. Merchants pay 0% commission.

• It is not finished. We cover about 1 million of an estimated 25 million online stores. Daily data goes stale. Our permission model is simpler than Walmart's. We cover those gaps below.

Why agents struggle with websites

Walmart said it in its announcement: agents that shop through a web browser follow "steps designed for people," which "can be slow and error-prone, and agents may get stuck along the way".

Anyone who has built a browser agent knows the frustration. Client-side hydration changes the page after it loads. Class names are generated by a build tool and change every deployment. Price is split across three spans, or rendered with a strikethrough that the model reads as the real price. Variant pickers are custom components that don't respond to a normal click. A newsletter modal appears on the second page view. An A/B test moves the add-to-cart button. Each of these is a small problem. Together they mean a shopping agent spends most of its tokens and time figuring out the page, not finding the best product.

Now multiply that by a million stores. Each one has its own theme, its own checkout, and its own idea of what a "variant" is. A person asking an agent for "a quiet dishwasher under $800" doesn't want the agent to visit a few sites it happens to know. They want the best answer from all of them. No agent can open a million websites per query, and no team wants a million agents learning its DOM by trial and error.

Walmart's answer is a gateway for Walmart. That is a good answer for Walmart. It is not an answer for the independent shoe brand on Shopify, the kitchen store on WooCommerce, or the specialty retailer running its own stack. Most of them will never build an MCP server or write terms for agents. They still want to sell to people who shop through agents.

That gap is what Agora fills.

What a universal catalog means

A universal catalog is one index of products from many stores, behind one interface. In practice it has four parts.

Ingestion. We bring stores into the catalog in three ways: platform-specific connectors for the big commerce systems such as Shopify and WooCommerce, custom connectors for selected merchants with their own stacks, and live data partnerships with larger retailers and marketplaces.

Normalization. Every product lands in the same data model, no matter where it came from. A product has an _id, a slug, a name, a brand, a price, an availability status, a store, and a list of variants, each with its own id, title, price, and availability. An agent learns this shape once and uses it everywhere.

Freshness. The catalog refreshes daily. Daily is good enough for search and ranking. It is not good enough for a purchase, because prices and stock change during the day. So when an agent opens a product or adds it to a cart, we check the live price and stock with the store. This is a deliberate tradeoff. A full real-time index of a million stores would be expensive and fragile. A daily index with live checks at the point of decision gives the agent fresh numbers where they matter.

Transactions. Search is only half the job. One cart can hold products from different stores, and one checkout places the order for all of them. Afterwards, one endpoint reports order status and tracking.

Two ways in

Agents today come in two broad types, and they want different things. So Agora has two interfaces over the same catalog.

Both start in the same place. An agent can fetch a JSON manifest at GET /api/agent-routes that lists the live routes, the DOM selectors, and the schema contracts. The same protocol is published in Markdown at /agents.md and /llms.txt, because many agents read Markdown more reliably than they read HTML. The full contract is on our agent routes page.

The browser interface

We could have told browser agents to use the API. Most of them can't. Browser agents are general-purpose. They don't read our docs before they arrive, and they don't carry API keys. They land on a page and try to make sense of it. So we built pages that are easy to make sense of.

The /agent routes are a separate interface from our consumer site. They render the same catalog with a few rules.

Everything is addressable by URL. An agent doesn't need to find a search box and type into it. It can open /agent/search?q=quiet%20dishwasher&page=1 directly. Fewer steps means fewer chances to fail.

Selectors are a contract, not an accident. Every element an agent needs has a stable id or a data-agent-* attribute: #agent-product-list, .agent-product-item, #agent-add-to-cart, #agent-submit-order. These don't change when we change the design. Each product card carries its id, slug, name, and price as attributes, so an agent can read a results page without parsing any visible text:

<li class="agent-product-item"
    data-agent-product-id="..."
    data-agent-product-slug="..."
    data-agent-product-name="..."
    data-agent-product-price="$499.00">

The full object is on the page. Every product page includes the complete product as JSON in pre#agent-product-raw-json. The agent reads it with .textContent and calls JSON.parse(). (Not .innerHTML, which escapes & as & and breaks the parse. We learned that from watching agents fail.)

State is explicit. After a click, an agent has to guess whether anything happened. We tell it. The add-to-cart button sets data-agent-cart-status to idle, adding, added, or error. Checkout sets data-agent-checkout-status to idle, invalid, or awaiting-payment. The agent waits for a value, not for a spinner to disappear.

There is a clear place to stop. The checkout form is plain and labelled. If the agent's policy requires a person to confirm payment, it fills the shipping form, submits, and hands off to the person to finish.

None of this is new technology. It is the kind of discipline you'd apply to a public API, applied to HTML.

The programmatic interface

For agents that run headless, such as backend tools, MCP servers, and partner integrations, there is a REST API. Search needs no key:

GET /api/search?q=quiet%20dishwasher&page=1&count=20

The authenticated v1 API lives at https://zues.searchagora.com/api/v1:

const res = await fetch(
'https://zues.searchagora.com/api/v1/search?q=quiet%20dishwasher&count=5',
{ headers: { Authorization: Bearer ${process.env.AGORA_API_KEY} } },
)
const data = await res.json()
// data.Products[] — use ._id for cart and checkout

A purchase takes three calls. First, set the cart:

POST /api/agent/cart
{ "items": [{ "product": "<_id>", "quantity": 1, "variantId": "<variantId>" }] }

Second, create a Stripe PaymentIntent with POST /api/agent/payment-intent. This returns a clientSecret and a publishableKey. The payment is confirmed with Stripe, using Stripe.js, Elements, or the Stripe API. The result is a payment method id (pm_…) and a PaymentIntent id (pi_…).

Third, place the order with POST /api/agent/checkout, passing the two ids and a shipping address. Checkout can also accept items directly, so the cart and order can happen in one call.

A few choices here are worth explaining.

The agent never sees a card number. It handles Stripe tokens, not payment details. This is the same principle Walmart wrote into its agent terms: agents "don't receive customers' payment-card information and can't add or change a customer's payment methods". It is also at the heart of Amazon's complaint about Muse, which Amazon says "appears to capture and store customer credentials". Meta disputes that. Either way, the safest design is one where the agent never has the credential to begin with.

Defaults are forgiving, errors are precise. Many products have only one variant. If an agent leaves out variantId, we default it to the product's _id rather than failing. But a malformed id returns 400 and an unknown product returns 404, so the agent can tell "you sent garbage" from "that product is gone".

Order tracking

Buying is not the end of a shopping task. People ask "where's my order?" and agents need to answer. GET /api/agent/orders/[orderId] returns the status, a carrier tracking link, and an estimated delivery date:

{
"orderId": "12345",
"status": "SHIPPED",
"trackingUrl": "https://tools.usps.com/go/TrackConfirmAction?tLabels=...",
"estimatedDelivery": "2026-10-24"
}

An API key can only read its own orders.

Why ranking matters more for agents

When a person shops on Amazon, they scroll past the ads. Consumers' Checkbook searched Amazon for 50 products and found that, on average, 54% of the first 25 results were paid placements, and the top two to five results were always ads.

An agent doesn't scroll. It reads the results it's given and picks from them. If the ranking is for sale, the agent's answer is for sale too, and the person on the other end never sees the label that says "Sponsored".

This is part of why the agent question is hard for big retailers. Bank of America analysts, writing about Walmart's agent strategy, noted that if "screenless computers are the ones checking out, it remains unclear what happens to the advertising landscape". A retailer with a large ad business has a reason to want people on its pages, not agents.

Agora doesn't have that conflict. We hold no inventory. We sell no products of our own. We take no money for ranking, and we charge merchants 0% commission. Results are ordered by how well they match what the person asked for, and nothing else. When an agent asks Agora for the best quiet dishwasher, the answer isn't shaped by who paid.

So how do we make money? Businesses that build on Agora, such as agent developers and distribution partners, pay a subscription and buy credits for usage. That's the whole model. Shoppers don't pay us, merchants don't pay us, and nobody pays us for position. Our incentive is to return the best result, because that's what our customers are paying for.

Where we agree with Amazon and Walmart

It would be easy to cast Amazon as the villain here. That would not be fair.

Amazon's stated concerns are real. Its position is that agents "should operate openly and respect service provider decisions about whether or not to participate". Its terms ask agents to identify themselves, not pose as humans, and not get around its limits. Those are reasonable rules. An agent that logs in as a person, holds their password, and clicks around undisclosed is a security problem, whatever its intentions.

Walmart's rules are reasonable too. Agents get their own profile. Purchases need approval by default, or must stay under a spending limit the customer sets. Agents can't take actions beyond what the shopper intends.

We agree with almost all of this. Agents should be identifiable. They shouldn't hold credentials. Our design tries to follow the same principles: agents come through a dedicated interface rather than impersonating shoppers on the consumer site, programmatic agents authenticate with their own key, payment runs through tokens, and checkout is seamless.

Where we differ is scope. Walmart's gateway covers Walmart and Sam's Club. Amazon's door is closed. Neither helps an agent shop the other million stores. As one agent told a Verge reporter while comparing granola bar prices, Amazon "might have had the lowest-priced option, but it was hard-blocked from browsing, so Muse couldn't confirm". An agent that can only see the stores that built a gateway gives a worse answer than one that can see everyone. We want to be the gateway for everyone else.

What this looks like in production

Agora handles about 15 million agent requests a month. Most come through distribution partners who build shopping into their own products. Glance uses our catalog and live-data API in its AI shopping experience, selecting stores through the API, with a daily catalog refresh and live price and stock checks when shoppers interact with products. Samsung TV Shopping is another, and there are more.

Running at this volume taught us a few things.

• Most agent traffic is research, not purchase. This matches what others report: people mostly use agents to research, not to buy. We see the same in our own traffic. That is why search quality and live data get most of our attention.

• Agents retry, a lot. A person who sees an error goes away. An agent tries again, often immediately. Clear status codes and idempotent cart writes matter more than we expected.

• Stability beats features. Every selector we change breaks someone's agent. We treat the /agent contract as versioned and keep old selectors working.

What doesn't work yet

We want to be clear about the limits.

Coverage. About 1 million stores is a lot, and it is still a fraction of the roughly 25 million online stores we estimate exist. Our goal is to index all of them by the end of 2027. Until then, an agent using Agora can miss a store that has the best product.

Freshness. The catalog refreshes daily. Live price and stock checks happen at the product and cart steps. If an agent ranks results using the daily snapshot, a product can go out of stock or change price between search and purchase. The checkout will catch it, but the ranking may already be wrong.

Permissions. Walmart's Agent Profiles let a customer give each agent its own spending limit and see that agent's purchases separately. Agora doesn't set spending limits. We leave approval rules and limits to the agent, which knows its user and its own policies. Programmatic access is scoped to an API key, and orders are visible only to that key's owner. The tradeoff is that a customer relies on the agent to enforce limits; Agora won't stop a purchase that the agent approves.

Returns and disputes. When an agent buys the wrong thing, someone has to make it right. Returns and disputes for agent-placed orders go through our 24/7 support, the same support people get when they buy directly. That works, but it is a human process. We don't yet offer an API for an agent to start a return on its own.

Developer access. The browser interface and basic search are open to any agent. The v1 API and headless checkout require a paid business account, on the subscription and credits model described above.

Try it

If you're building an agent that shops, start with the manifest at GET /api/agent-routes, or read /agents.md. Point a browser agent at searchagora.com/agent and ask it to find something. The full API reference is at docs.searchagora.com.

Agents are going to shop for people. Some stores will lock the door. We think the better answer is to give agents a clear, honest way in, and to make sure the best product in the world can be found by them, not just the one that paid to be first.