Discovery Is the New SEO

There is a file on your website right now, written for someone you have never met. Not your customers. Not Google. An AI.

I found mine by accident last week, poking around my wife's online store. Nobody set it up. Shopify did it automatically, for every store it runs.

The file is called robots.txt. Every website has one. For twenty years it did one boring job: tell Google's crawler which pages it could look at.

Open it today and the top is addressed to AI agents. In plain language, it tells them:

• here is how to shop this store
• here is the full product list
• here is how to fill a cart yourself

No human will ever read a word of it. It is written for software that shops on people's behalf.

And it is bigger than one store. The standard behind it launched in January, built by Google and Shopify, with Walmart, Target, Etsy, Visa and Mastercard behind it.

I spent ten years in IP and eleven at Google, in the ads business. Two decades on one question: how things get found, and who gets paid when they do. So I notice when the rules quietly change.

They just did.

A second reader at the door

For twenty five years, getting found meant ranking when a person typed a search. That still matters. But now a second reader stands next to the human one, and it does not shop like people do.

An agent never sees the homepage you fussed over. It reads that plain little file, and in a second it decides whether you make the short list it shows its owner. No second look, just a yes or a no, in the time it takes to blink.

And do not assume the machine is the lesser customer. A year ago, shoppers an AI sent you converted 38% worse than humans. This spring, 42% better, and spending more per visit. Adobe's numbers, not mine.

Who collects the toll

Here is the part that got me. In that same little file, an agent can browse all it likes, but it cannot pay on its own. A human still has to approve the money. And the store quietly steers every agent toward its own checkout, so the sale runs through the platform either way.

Anyone can be found. But whoever owns the road to the cash register collects the toll. I had a front-row seat while Google built exactly that road in search. Same move, now paved for agents.

Soon your business gets described to your best customer by software you never hired, reading from a list you have never checked. Your prices, your stock, your whole story, answered by a machine before a person sees a thing.

I cannot tell you how this plays out. I can tell you it has already started, and it is sitting in a file you have never opened.

So, one question for your week. If an AI described your business to your best customer tonight, with you nowhere in the room, what would it say? And have you ever read what it would be reading?

Go open your robots.txt. You might be surprised who it has been talking to.

Sources: the agent block, UCP discovery file, and MCP endpoint are live in the default Shopify robots.txt (verified on three independent stores, June 2026). The conversion figures are Adobe Analytics (reported April 2026). The Universal Commerce Protocol was launched by Google and Shopify at NRF, January 2026.