Shopify's CEO says his engineers are "all verification engineers" now. The same shift is coming for anyone running a brand: when an AI agent can act on your ads, stock and customers, your job becomes writing the rules it runs inside.
On this page
AI agent guardrails are about to become part of every ecommerce operator's job, and the clearest explanation of why came from Shopify's CEO talking about something else entirely. At Vercel Ship in San Francisco in October, Tobi Lütke said Shopify had stopped writing code by hand, and that "we are now all verification engineers." A clip of it, posted by Mike Darlington from Vercel, had more than 78,000 views by the next day.
He was talking about software, and nothing in that conversation is advice for people who run brands. The parallel is mine, but I think the same shift is coming for us, and sooner than most founders expect. When an AI agent can change an ad budget, send a discount or raise a purchase order, your job stops being doing those things and becomes writing the rules the agent has to follow.
What is a harness?
The line comes from a fireside chat with Guillermo Rauch, the CEO of Vercel. Tobi said Shopify "went full pencils down", meaning its engineers stopped hand-writing code, and then:
"...we are now all verification engineers. We are building harnesses, we are building acceptance tests, and we are building... verifiables."
In an engineering team, a harness is the setup an agent's work runs inside, so you can see what it did and check it, and an acceptance test is a written definition of "done". If an agent writes the code, the person's job is to decide what correct looks like and build the checks that prove it.
For someone running a brand, a harness is the same idea with different parts: the rules, limits and checks an AI agent works inside. It has three parts, a profit floor the agent can't go below, a cap on what it can do before it has to ask, and a written definition of done for each job. The rest of this post is about how to write them. If the word is new to you, our plain-English guide explains what an AI harness is.
AI agents for ecommerce can now act, not just report
Most AI tools in ecommerce started out reading your data and telling you something about it. This year the platforms themselves opened the door to agents that act on it.
In July, Meta opened its ads MCP server to developers, which lets a connected AI app "create, edit, and delete campaigns, ad sets, and ads." In June, Shopify's Spring '26 Edition introduced Campaign Autopilot, now in early access, which "runs AI-powered marketing campaigns automatically across Facebook, Instagram, Shop, and email, and uses commerce intelligence to optimize over time within guardrails you set."
Both companies say the same thing about control. Meta's post says "Rules define the actions an AI agent can and cannot take on a given ad account or catalog," and recommends starting an account "in a read-only or approval-required posture." Shopify's phrase is "guardrails you set." The word doing the work in both is you. Someone has to write those rules, and an agent can only enforce the ones it's given, on the numbers it can see.
Most teams haven't written them yet. IAB Europe's 2026 report on AI in digital advertising, a small industry survey reported by PPC Land, found that 86% of respondents say their organisation uses AI for marketing, but only 48% have AI guidelines specifically for advertising and marketing.
Why a ROAS rule gets this wrong
ROAS is the number every ad platform reports, so it's the number most rules end up written on. The trouble is that ROAS says nothing about what an order costs you to fulfil, which is why ROAS and profit part ways more often than most dashboards show.
Here's an illustrative example, not a real brand, and it ignores returns to keep the maths simple. Say a product sells for $60. The product costs $20, and shipping, fulfilment and payment fees take another $15. That leaves $25 of margin before ad spend, or 41.7% of the sale, so your break-even ROAS is 1 divided by 0.417, which is 2.4. You can check your own with our break-even ROAS calculator.
| Illustrative example | Per order |
|---|---|
| Selling price | $60 |
| Product cost | $20 |
| Shipping, fulfilment and payment fees | $15 |
| Margin before ad spend | $25 (41.7%) |
| Break-even ROAS | 2.4 |
| At 2.2 ROAS, per $1 of ads | $2.20 revenue, $0.92 margin, 8 cents lost |
Now give an agent this rule, which sounds reasonable: scale any campaign with ROAS above 2.0. A campaign running at 2.2 passes. Every $1 of ad spend brings in $2.20 of revenue, which is about $0.92 of margin before ads, so the agent has just scaled a campaign that loses roughly 8 cents on every dollar of ad spend, and it will do it faster and more consistently than a person would.
It gets worse if the agent's cost data is stale. If product costs changed last quarter and nobody updated them, the agent is checking a margin that no longer exists, and the same goes for refunds, discounts and shipping, which move the margin all the time. Our post on Claude dashboards built on MCPs covers why dated costs matter.
An AI acting on the wrong margin is worse than no AI, because it acts quickly and with confidence, and that is the case for building the harness before you switch the agent on.
AI agent guardrails: the three parts of a harness
These are the same three jobs a verification engineer does, translated into running a brand. You decide what the limits are, what happens when the agent hits one, and what finished looks like.
1. A profit floor for every product and campaign
Don't give the agent a ROAS target. Give it a floor written in contribution margin, and let it work out the ROAS from your real costs.
- Use one definition everywhere. Here, contribution margin means revenue minus product cost, shipping, fulfilment, payment fees, returns and ad spend, so break-even is a contribution margin of zero.
- Set the floor per product, not per account. A hero product and a heavily discounted bundle have very different break-even points.
- Write the action next to the floor. For example, "don't scale any campaign whose contribution margin is below zero over the last seven days" and "flag it for a person if it stays below the floor for three days."
- Keep the costs behind the floor current and dated, or the rule checks the wrong number.
2. Caps, with a point where the agent has to stop and ask
Agents are good at doing the same thing many times, which is the problem when the thing is wrong, so limit what one run can do.
- A maximum change to any budget in a day, for example 20%.
- A maximum total daily spend the agent can reach without asking you.
- A list of campaigns it must never touch, such as a launch or a Black Friday push you've planned by hand.
- A stop-and-ask rule, so anything bigger than the cap, or any change to a campaign above a set spend, goes to a person first.
Pick numbers you can live with being wrong for one day. You can loosen them later, once you've watched the agent work.
3. A written definition of "done" for each job
This is the acceptance test. If you can't say what finished looks like, you can't check it, and neither can the agent.
- Pausing a campaign. Done means it's paused in the ad platform, the reason and the margin figure at that moment are logged, and someone checks the result the next morning.
- Reordering stock. Done might mean a draft purchase order is prepared and a person sends it, not that the agent places it. Keep money and suppliers behind a human until you trust the process.
- Reporting. Done means the numbers match Shopify's order count and your own cost data, not that a chart exists.
Our guide to ecommerce automation has a simple way to decide which jobs an agent should do at all: score each one on how often it happens, how long it takes and what goes wrong if it's wrong.
The same harness applies to every part of the brand
Ads are where the damage shows up fastest, but the pattern is the same anywhere an agent can act. Every tool reports its own flattering number, whether that's ROAS rather than MER in an ad account or revenue per send in an email tool, and an agent given that number will optimise for it. The platform announcements above cover ads and marketing. The other rows below are my view of where this goes next, not something a platform has announced.
| Operation | The number the tool reports | What an agent does with it | The profit rule instead |
|---|---|---|---|
| Ads | ROAS | Scales a campaign that loses money after costs | Contribution margin floor per product |
| Email and discounts | Revenue per send | Sends deeper discounts more often | Margin after discount, per order |
| Inventory | Sell-through or stock cover | Reorders a product that sells well but earns little | Reorder ranked by margin, with a cash cap |
| Customer service | Resolution time | Refunds quickly to close tickets | Refund limit, with a person above it |
In every row the harness has the same three parts: a floor written in profit, a cap with a stop-and-ask point, and a definition of done. Only the numbers change.
A day-one checklist
- Run the agent in review-only mode first, so it proposes changes and a person approves each one.
- Check that the agent reads the same margin numbers as your finance reports, with one definition of contribution margin used everywhere.
- Put the profit floor, the caps and the "never touch" list in writing before you connect any accounts.
- Make sure every action is logged with the reason and the numbers it used.
- Review the log weekly, and look for any action you wouldn't have taken yourself.
- Only loosen the rules after you've seen a few weeks of decisions you agree with.
This is the least exciting part of using AI, and it's the part that decides whether it helps your profit or quietly hurts it. It applies whether you buy an agent or build one yourself by connecting Claude to Shopify.
The bottom line
Tobi Lütke was talking about code, but the idea carries over. When agents do the work, the valuable human job is deciding what a good result is and checking that it happened. For a brand, that means a profit floor, limits and a clear definition of done, written down before an agent gets its hands on anything.
So before you connect an agent to any account, ask who is writing the rules it runs inside, and what numbers those rules are built on.
Work out break-even ROAS for each product before you let anything touch a budget. StoreHero's free calculator does it from your price, costs and fees.
Key takeaways
- Tobi Lütke said Shopify's engineers are now "all verification engineers", building the checks around agent-written code. He was talking about software.
- The same shift applies to operators: Meta and Shopify now let AI agents act on campaigns, inside rules that you have to write.
- A harness is the rules, limits and checks an agent works inside: a profit floor, caps with a stop-and-ask point, and a definition of done.
- A ROAS rule ignores your real costs. In the example here, a campaign at 2.2 ROAS passes a 2.0 rule and still loses 8 cents per $1 of ads.
- The same harness applies to email, inventory and customer service, because every tool reports a number that flatters it.
- Start in review-only mode and keep a person on anything that touches money or customers.
Frequently asked questions
What did Tobi Lütke mean by "verification engineers"?
At Vercel Ship in San Francisco, Tobi Lütke said Shopify's engineers have stopped writing code by hand and are now building the harnesses, acceptance tests and verifiable checks around code that AI agents write. In plain terms, people define what "correct" means and prove the agent's work meets it. He was describing software engineering at Shopify, not ecommerce operations.
Can AI agents change Meta ad campaigns?
Yes. Meta's ads MCP server lets a connected AI app create, edit and delete campaigns, ad sets and ads, where you've given it access. You control what the agent can and cannot do through rules on the ad account, and Meta recommends starting in a read-only or approval-required setup.
What AI agent guardrails should I set before letting an agent manage ad spend?
Set a contribution margin floor per product and campaign, a maximum budget change per day, a maximum total daily spend, a list of campaigns the agent can't touch, and a rule that anything bigger than those limits goes to a person. Start in review-only mode so you approve each change.
Why shouldn't I give an AI agent a ROAS target?
ROAS ignores product cost, shipping, fulfilment, fees and returns. A campaign can beat your ROAS target and still lose money once those costs are counted. Give the agent a contribution margin floor instead, and keep the costs behind it current.
Sources
- Mike Darlington (@DarlingtonDev), X post with the clip, 16 November 2026
- Vercel on YouTube: The Third Customer, a fireside with Tobi Lütke and Guillermo Rauch
- Vercel: Vercel Ship 2026 recap
- Meta for Developers: Meta's ads MCP server is now available for developers, 16 July 2026
- Shopify: Spring '26 Edition, 17 June 2026
- PPC Land: IAB Europe finds 86% use AI for marketing, but only 48% have ad rules, 24 October 2026
- Header photo of Tobi Lütke: Benjamin Forrest / Good Thanks Media, CC BY-SA 4.0, via Wikimedia Commons




