Customer Success AI Agents: 5 Demo Tests

The short answer

A customer success AI agent is software that, inside limits you set, reads each account's live data, picks an action nobody scripted, carries it out with the customer, and stops for a person when it should. Software that only drafts for a CSM, or replays steps someone wrote, is not an agent.

Five demo questions tell you which one you are looking at. Send them before the call, and run them in order.

What is agent washing?

Agent washing, in Gartner's June 2025 definition, is "the rebranding of existing products, such as AI assistants, robotic process automation (RPA) and chatbots, without substantial agentic capabilities." In the same release, Gartner estimated that only about 130 of the thousands of agentic AI vendors were real.

Gartner also predicts that "over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls." Both figures are cross-industry, but customer success is not exempt. In our reading, the tests below check for each cause.

Picture minute four of a demo. A health score drops on screen, the agent flags the account, and a tidy check-in email goes to the champion. Your side of the call nods. Nobody can tell yet whether the agent chose that email or replayed a rule somebody wrote two years ago.

Workflow, copilot or AI agent

Three different products get demoed under one word:

• Workflow. A person wrote every step in advance. Much of what teams already run as customer success automation looks like this.
• Copilot. It works when someone asks. Useful work, all of it bounded by the CSM's calendar. An account nobody opens never gets touched.
• Agent. It starts from account data, not a request. It picks the step, takes it and checks what the customer did.

None of the three is a bad purchase. The costly mistake is paying for an agent and getting a workflow.

Five tests for a 45-minute demo

This is our script, so keep the order. Send the questions ahead, ask for a tenant with enough accounts to pick look-alikes from, and bring a CSM who knows real accounts. Skip the slides.

1. Decide. "Let my CSM pick two accounts in the same segment with the same renewal date. Show me the step the agent chose for each, and the data it used to choose." A pass is two different steps, with a reason that isn't a date or a threshold.
2. Act. "Show me the last ten actions the agent took without a person pressing send." A pass is a log with timestamps, account names, channels and what the customer did next.
3. Dial. "Show me the setting for one task in one segment, then change it while I watch." A pass is three positions per task and segment, plus a defined path for an unanswered approval.
4. Hand off. "Trigger a handoff now, on a test account. Show me exactly what the CSM receives, and where." A pass is one message your CSM can act on without opening another tool.
5. Report. "Which number on my board deck will this agent move, and how will you separate the agent's effect from my CSMs'?" A pass is a named metric, such as gross and net retention, a set of accounts, a comparison group and a time window.

One trap on Decide: per-account output isn't a per-account decision. Okta's digital success plans, built on Base, the publisher of this blog, place each account on Okta's maturity model from live usage. Okta's story calls that experience automated, and it is: Okta's team designed the model behind every plan.

The agent question is whether the software chooses among actions nobody sequenced for it.

A dial, not a switch

An agent that acts needs a dial. Switched on globally, it can do anything to anyone. Switched off, you bought a reporting tool. Here is an illustration of the three positions:

Base AI builds the dial at that grain. Its homepage says "Choose what it does on its own, what it asks first and what it only suggests," and its AI Customer Growth platform page applies the setting "per task and per segment." Ask every vendor to change it live, us included.

We sort tasks by what a wrong action costs in what to automate in customer success. For the controls behind a safe customer-facing agent, see trust operations for post-sale AI agents.

How do you score an AI agent demo?

Score a customer success AI agent demo by running the five tests in order and stopping at the first fail. Fail Decide and you have a workflow. Fail Act and you have a copilot. Fail Dial or Hand off and you have an ungoverned agent. Fail only Report and it is unproven.

Pass all five and it is worth a pilot. Order matters because a product that can't decide has nothing to dial, and one that never acts has nothing to hand off.

Listen for relabeled answers along the way. These are our composites, not vendor quotes:

• "At 60 days out, the agent sends the renewal sequence." That is a rule.
• "Your CSM reviews and sends with one click." That puts a person back in the middle.
• "AI is switched on for this workspace." That has no dial.
• "The CSM gets a notification and can click through to the account." Click through to what?

Which AI agent is best for customer success?

The best AI agent for customer success is the one that passes all five tests on the specific job you plan to hand it, using your own account data. Pick the job first, then invite vendors and run the demo script on that job only.

Each job stretches a different test:

• Onboarding: stretch Act. Does the nudge reach the customer?
• Adoption and next best action: stretch Decide.
• QBR and value review: stretch Report.
• Risk triage and renewal plan: stretch Hand off.
• Expansion signal: stretch Dial. Anything touching price stays at "only suggests."

We have not run these tests on any vendor, so run them yourself, us included. Our definition echoes how Base describes its own product, so give us the hardest demo.

Run a blind test before you sign

The script runs on a demo tenant. Before a contract, compare the agent's decisions with your team's, blind, on your own data. This is our working method, not an industry benchmark, and it takes about four weeks.

1. Pick 30 accounts on one job, in one segment, mixing ones you worry about with ones you trust.
2. Have the agent and two of your CSMs each choose the next step for every account, without seeing each other's answers.
3. Have a CS leader rate the steps unlabeled, and count where the agent matched or beat your team.
4. Shadow for two weeks in suggest-only mode on live accounts.
5. Promote task by task where the agent matched or beat your team: "asks first," then low-risk tasks to "on its own."

The output is a map of which tasks this agent should own, propose or never touch. That map is what you negotiate the contract around.

Before you start, cover the plumbing: data access and security. If you would rather build, see our build vs. buy guide. Add one demand: an audit trail of every action, the bar Base sets on its security and privacy page.

Start with one email

Paste the five questions into the calendar invite. A vendor with a real agent will build the demo around them. One with a relabeled workflow will steer back to the slides, and you will know by minute 15.

If Base AI is on your shortlist, send us the same invite. The script doesn't care who publishes the blog.