Can AI handle Shopify customer service without replying on its own?

Most shop owners asking this question have already seen an AI reply go wrong. We had too. Before we built our support agent at Mully, an email AI on our inbox answered internal system alerts as if they were customers, and 5 customers in two months asked to reach a human instead of the bot. So we built the next one to draft, not send.

This guide covers how that agent works, what it found in our tickets, and where the line between AI and a person sits.

Can an AI agent do support work without talking to customers directly?

Yes. Most of the work in a support ticket happens before the reply: figuring out what the customer actually wants, finding the order, checking the policy and deciding the steps. An agent can do all of that and leave a finished draft. A person reads it, edits if needed and presses send.

The big platforms already lean this way for staff tools. Shopify's suggested replies in Inbox are described as "a composing aid for staff," and Shopify warns that you are responsible for the accuracy of what you send, even when AI wrote it (Shopify Help Center). Intercom's Copilot drafts answers inside the inbox and lets the teammate check each source before sending (Intercom Help).

What did our agent find in 60 days of tickets?

Our CS Tree agent read every Intercom conversation from late August to late October 2026 and checked order details in Shopify. The numbers:

What	Count
Intercom conversations	464
Not customers (vendor alerts, pitches, spam, internal)	274
Customer tickets	190, from 145 customers
Decision tree branches	58, in 12 groups
Draft canned replies	57
Tickets that matched no branch	3 (1.6%)

The first surprise was that more than half the inbox was not customers at all. Triage is a job on its own, and it is exactly where our old email AI failed by answering system alerts.

The four biggest customer groups were cancellations (38 tickets), wrong size or item (33), shipping (27) and billing (22). Those four groups were 120 of the 190 tickets.

What does each branch in the decision tree contain?

Each branch has four parts, so a draft is never just a block of friendly text:

1. What the customer says. The patterns that put a ticket on this branch, written from real messages.
2. What to check. The order, subscription status, size profile or tracking to look at before replying.
3. The steps. What the team actually does to fix it.
4. The canned reply. A draft written in our voice that the agent fills in with the order details.

The checks are what make the draft useful. "Cancel, portal will not let me" was the single biggest branch with 20 tickets. A reply that says "you can cancel in your account" is wrong for those customers, because the portal was the problem. The branch tells the agent to check the subscription first and draft the reply around what it finds.

Which tickets should stay with a human?

Anything that commits money or makes an exception. In our tree, 70 tickets sat on branches where the team had decided refunds, partial refunds and credits case by case. We had no written rule for those, so the agent drafts the reply and flags it, and a person decides the outcome.

There is a legal reason for this too. In Moffatt v. Air Canada, a British Columbia tribunal held the airline responsible for a chatbot that promised a bereavement refund the policy did not allow, and ordered it to pay $812.02 (2024 BCCRT 149). The tribunal said it makes no difference whether the information comes from a static page or a chatbot. If an AI can promise a refund in your name, assume you are on the hook for it.

Hand to the agent as a draft	Keep a person deciding
Where is my order, tracking, delays	Refunds and partial refunds
How the plan or product works	Store credit and price exceptions
Size or profile updates	Replacement shipments
Login and account access steps	Customers who asked for a human

Do customers mind AI in customer service?

They mind not being able to reach a person. In a 2023 Gartner survey of 5,728 customers, 64% said they would prefer companies did not use AI for service, and the top concern was that it would get harder to reach a person (Gartner, 2024). By early 2026, Gartner found 50% of customers said generative AI made service easier, but 87% said an option to reach a human is essential (Gartner, 2026).

A draft-only agent sidesteps the problem. The customer always hears from a person. The person just starts from a checked draft instead of a blank box.

How do you measure whether the AI is helping?

We use one KPI: coverage, the share of customer tickets that match no branch. Our baseline is 1.6%, but that was measured on the same tickets the tree was built from, so it flatters the agent. The real number comes from new tickets. Every Friday the agent reads the week's tickets, maps each one to a branch and proposes new branches for anything that does not fit.

The three misses from the baseline show what that looks like in practice: a marketplace order alert, a coupon site asking if its code still worked, and a customer who wrote "summarize my account" with no other context. The first two became proposed triage branches. The third needs a human question back, and the tree says so.

What does an agent need access to?

Read access to your helpdesk and your store's orders is the minimum. We also learned where the gaps were: our Shopify connection could not read subscription contracts or store credit, so those checks point the person to the subscription app and Shopify admin instead of guessing. Write down which checks the agent cannot do. A confident draft built on a missing lookup is worse than no draft.