When the Customer Agent's answers or behavior don't go the way you intended, reviewing the Knowledge Base, the system prompt, and your Workflow settings will resolve most problems. The quality of an answer depends on the material the agent learns from (the Knowledge Base) and the instructions it follows (the system prompt). We've organized the points to check by symptom for when you run into trouble.
What to check first
The accuracy of the Customer Agent changes significantly based on the "material" and the "instructions" you give it. Before changing any settings, keeping these three points in mind makes it easier to narrow down the cause.
- Knowledge Base: This is the material the agent uses as the basis for its answers. If the correct information isn't here, the agent won't return correct answers.
- System prompt: This is the instruction text that determines how the agent speaks and behaves. If it's vague, the answers will be inconsistent.
- Workflow: The Customer Agent runs as a "Customer Agent" node placed inside a Workflow. Whether it responds, and whether it hands off to a human, is determined by your Workflow settings.
We explain how to set up the Customer Agent in a separate article.
When an answer is incorrect
When an answer is factually wrong or off the mark, start by reviewing what's in the Knowledge Base. Because the agent answers based on the information in the Knowledge Base, missing or outdated information leads to inaccurate answers.
| Cause | What to do |
|---|---|
| The relevant information isn't in the Knowledge Base | Add material containing the correct information to the Knowledge Base |
| The information in the Knowledge Base is outdated | Delete the old material and replace it with the latest content |
| The system prompt's instructions are vague | Review the node's "System Prompt" and clearly state how to handle questions it can't answer, for example: "For questions you don't know, reply 'I'll check with our representative.'" |
The Knowledge Base supports PDF, text, Markdown, CSV (FAQ), and Word files, up to 50MB per file. You can also paste text or import a web page.
When the agent doesn't respond
When the agent doesn't react even though a friend has sent a message, check in order whether the Workflow is running and whether the message reaches the agent's node.
| What to check | Where to check | What to do |
|---|---|---|
| Whether the Workflow is enabled | The status in the Workflow list | Enable the Workflow that contains the Customer Agent |
| Whether the trigger conditions are met | The Workflow's trigger settings | Review whether the keyword conditions of the message-received trigger match the words your friends send |
| Whether processing reaches the node | The Workflow structure | Check that the flow is correctly connected from the trigger to the "Customer Agent" node |
We explain how to create and enable a Workflow in the Workflow article.
When it doesn't hand off to a human
When a friend wants to talk to a representative but the agent keeps responding, check the settings for switching to human handoff. In the "Customer Agent" node, when the handoff button is tapped, the flow continues on the "Handoff" branch. When the AI decides on a handoff by itself, it continues on the same "Handoff" branch.
| Cause | What to do |
|---|---|
| There's no action on the Handoff branch | On the "Handoff" branch that the handoff button tap leads to, set up an action to connect to a representative (such as tagging or a notification) |
| The human handoff button isn't displayed | Check whether "Hide handoff button" is selected in the node's settings |
| The system prompt has no handoff instruction | Add to the system prompt: "When you can't handle something yourself, recommend human handoff." |
When the end-conversation button is tapped, the flow continues on the "After end" branch. Set up behavior such as sending a thank-you message after the conversation ends on the "After end" branch side.
When it can't search for or suggest Shopify products
When the agent can't suggest product information, check the status of your Shopify integration and the reference source settings on the node. Referencing product data requires both an integration with Shopify and enabling "Use Shopify product data" on the node.
| Cause | What to do |
|---|---|
| Shopify isn't integrated | Check the Shopify integration status on the external service integration screen |
| Product data reference is disabled | Enable "Use Shopify product data" under the node's "Reference sources" |
For the integration steps, see "How to Integrate Shopify".
When it can't report Shopify order status
When the agent can't answer about an order or its shipping status, check the node settings and whether the customer has linked LINE. Reporting on orders requires both enabling "Use Shopify order and shipping status (LINE-linked customers only)" on the node and the customer having linked their own Shopify account with LINE.
| Situation | What to check |
|---|---|
| The agent asks the customer to link LINE | The customer hasn't linked their Shopify account with LINE. This is by design: for an unlinked customer, the agent doesn't disclose order details even when given an order number. Offer a way to link in the rich menu or the greeting message |
| The agent says it can't find the order | Only the linked customer's own orders are covered. Orders belonging to someone else, or placed more than 60 days before the Shopify integration, can't be reported. Also check for a mistyped order number |
| The agent only says "shipped" without a tracking number | The fulfillment in Shopify has no tracking number. Add the tracking number and carrier to the fulfillment on the Shopify order screen, and the agent can report it from then on |
| Cancellations or changes aren't carried out | This is by design. The agent doesn't act on orders and hands off to a representative instead. Place a notification to a representative on the "Handoff" branch |
Tips to improve answer quality
You can gradually improve the Customer Agent's accuracy as you operate it. The basic flow is to enrich the Knowledge Base, make the system prompt specific, and review actual exchanges to improve.
| Point | How to do it |
|---|---|
| Enrich the Knowledge Base | Register frequently asked questions and their answers as CSV (FAQ) or text |
| Make the system prompt specific | Stating things down to the tone and answer length, such as "Answer in a polite tone, within 100 characters," makes responses more consistent |
| Review actual exchanges regularly | Check your exchanges with friends in the inbox, and when you find a question that wasn't answered well, add that content to the Knowledge Base |
Improvement doesn't end after one round. By adding to the Knowledge Base every time a new question comes up, your agent's answers will steadily get smarter.