Technique

Before Black Friday, check what AI agents say about your store

Shopify moved its agent Knowledge base into the admin, and your policies and contact details are already published to agents. Read what they will repeat, fix the gaps, and check where Google now shows ads next to those answers.

22Atlas

Product studio

5 November 20265 min read

An agent can only repeat what your store has published. Read it the way a customer would.

A shopper asks an agent, "Can I cancel my order?" The agent answers from whatever your store has published. If you've never looked at that, you don't know what it says.

An agent can only repeat what your store has published. Read it the way a customer would.

What Shopify changed

On Oct 26, Shopify moved its Knowledge base into the admin, under Sales channels > Agentic > Knowledge base. Shopify calls it "a single page where you review, edit and publish the facts AI agents get about your store." Per Shopify, it replaces the Knowledge Base app, which goes away after Jan 15, 2027. Shopify drafts facts from your storefront, and "Nothing Shopify drafted reaches AI agents until you publish it."

There's an exception worth knowing. Shopify's help page says facts linked to the admin, "such as your store policies and your contact details, are published when Shopify creates them." The published knowledge base is "publicly available through the Storefront MCP." Shopify also lists a limit of 1,000 facts and says a rescan can take up to 48 hours.

Two days later, Shopify's Fall '26 Edition added Shop Pay checkout inside personal agents, starting with Muse from Meta, Claude Duet and Instinct, plus a Catalog eligibility view in the admin. And per Shopify's agentic storefronts page, "Agentic storefronts is active by default for eligible stores."

What agents say about cancels today

Search Engine Journal reports on a test by Bill Hunt for SEJ: of 88 Shopify stores whose policy tool he queried, two returned a usable answer about canceling an order. That's SEJ's and Bill Hunt's test. It ran in August and September, before the knowledge base moved into the admin.

We ran a small check of our own on Oct 29. We asked eight public Shopify stores' policy tools how to cancel an order, and one gave a usable answer. Most gave back a generic line about returns. Eight stores is a tiny sample from a single run, so read it as a reason to check your own store.

The check, in six steps

These steps are our suggestions, built on Shopify's and Google's docs.

1. Open the page and check eligibility

In your Shopify admin, go to Sales channels > Agentic. Per Shopify, next to Shopify Catalog you'll see a product count or an Ineligible warning. Click the warning to read the reason.

2. Read your knowledge base like a customer

Go category by category: returns and warranty, fulfillment, payment, gift services, contact, company policies. Start with policies and contact details, since Shopify publishes those by default. When something reads wrong, fix the source page in your admin or storefront, or switch that field to Manual.

3. Publish only what you'd print on the box

Approve, edit or reject each suggested update. Then add facts for the questions agents get wrong: how to cancel an order, preorder timing, final sale items, holiday shipping cutoffs. The cancel fact comes first, given what SEJ found.

4. Ask an agent your top five support questions

Take the five questions your inbox gets most and ask them in a personal agent your buyers use. Compare each answer to your published facts. After a big change, request a rescan and give it up to 48 hours, per Shopify. To see the whole path through checkout, run the test purchase from run a Shop MCP test buy on your own store.

5. Open your agents.md

Shopify says every store serves /agents.md, /llms.txt and /llms-full.txt automatically. Open yourstore.com/agents.md and read it. Leave it as is unless you have a clear reason to change it.

6. If you run ads, check your AI Mode exposure

On Oct 28, Google launched multiturn ads in the U.S., which can show on follow-up questions inside an AI Mode conversation. Google's Ads Help says existing Search, Shopping and Performance Max campaigns are eligible automatically, you can't target or opt out of AI Mode on its own, and results are "included in aggregate" in standard reports. Exact and phrase match keywords "aren't eligible for contextual matching on broader exploratory questions."

So look at which campaigns can match on those broader questions, review your negative keywords, and decide whether that reach is what you want. Then make sure your Merchant Center feed says the same thing about returns and shipping as your knowledge base, so the ad and the agent's answer agree.

Google says click-through rates for sponsored and organic product listings in AI Mode are already similar. That's Google's claim, and Search Engine Land notes Google hasn't disclosed the underlying rates.

What this doesn't tell you

Shopify doesn't say how often each agent reads your knowledge base, or whether it changes where you show up. Its docs mention "AI channels including Shop and third-party platforms" without a full list. SEJ's numbers predate the admin move, and our check is eight stores on one morning. Multiturn ads are U.S. only for now.

What it does give you is one page in your admin that shapes what agents tell your customers, and a short list of things to fix there.

This week

Open Sales channels > Agentic. Read your published policies and contact details. Add a cancel fact. Ask an agent your top five questions. Line up your Merchant Center feed with your knowledge base. If you're deciding which agents get in at all, start with write your store's agent access policy this week.

If you want Always On Agents that keep your store's facts current, get in touch.

Linked sources

Get our Articles and posts by email.

Up to one email a day with new Articles and posts from 22atlas.com and sean-weldon.com, plus occasional news about 22Atlas services.