When the Customer Agent's answers or behavior don't go the way you intended, reviewing the Knowledge Base, the system prompt, and your Workflow settings will resolve most problems. The quality of an answer depends on the material the agent learns from (the Knowledge Base) and the instructions it follows (the system prompt). We've organized the points to check by symptom for when you run into trouble.
What to Check First
The accuracy of the Customer Agent changes significantly based on the "material" and the "instructions" you give it. Before changing any settings, keeping these three points in mind makes it easier to narrow down the cause.
- Knowledge Base: This is the PDF the agent uses as the basis for its answers. If the correct information isn't here, the agent won't return correct answers.
- System prompt: This is the instruction text that determines how the agent speaks and behaves. If it's vague, the answers will be inconsistent.
- Workflow: The Customer Agent runs as a "Customer Agent" node placed inside a Workflow. Whether it responds, and whether it hands off to a human, is determined by your Workflow settings.
We explain how to set up the Customer Agent in a separate article.
When an Answer Is Incorrect
When an answer is factually wrong or off the mark, start by reviewing what's in the Knowledge Base. Because the agent answers based on the information in the Knowledge Base, missing or outdated information leads to inaccurate answers.
| Cause | What to do |
|---|---|
| The relevant information isn't in the Knowledge Base | Upload a PDF containing the correct information to the Knowledge Base |
| The information in the Knowledge Base is outdated | Delete the old PDF and replace it with the latest content |
| The system prompt's instructions are vague | Review the node's "System prompt" and clearly state how to handle questions it can't answer, for example: "For questions you don't know, reply 'I'll check with our representative.'" |
Only PDFs can be uploaded to the Knowledge Base, and each file can be up to 50MB.
When the Agent Doesn't Respond
When the agent doesn't react even though a friend has sent a message, check in order whether the Workflow is running and whether the message reaches the agent's node.
| What to check | Where to check | What to do |
|---|---|---|
| Whether the Workflow is enabled | The status in the Workflow list | Enable the Workflow that contains the Customer Agent |
| Whether the trigger conditions are met | The Workflow's trigger settings | Review whether the keyword conditions of the message-received trigger match the words your friends send |
| Whether processing reaches the node | The Workflow structure | Check that the flow is correctly connected from the trigger to the "Customer Agent" node |
We explain how to create and enable a Workflow in the Workflow article.
When It Doesn't Hand Off to a Human
When a friend wants to talk to a representative but the agent keeps responding, check the settings for switching to human handoff. In the "Customer Agent" node, when the human handoff button is tapped, the flow proceeds to the False branch.
| Cause | What to do |
|---|---|
| There's no action on the False branch | On the False branch that the human handoff button tap leads to, set up an action to connect to a representative (such as tagging or a notification) |
| The human handoff button isn't displayed | Check whether "Don't show the human handoff button" is selected in the node's settings |
| The system prompt has no handoff instruction | Add to the system prompt: "When you can't handle something yourself, recommend human handoff." |
When the end-conversation button is tapped, the flow proceeds to the True branch. Set up behavior such as sending a thank-you message after the conversation ends on the True branch side.
When It Can't Search for or Suggest Products
When the agent can't suggest product information, check the status of your Shopify integration and the tool settings on the node. Referencing product data requires both an integration with Shopify and enabling the "Shopify product data" tool on the node.
| Cause | What to do |
|---|---|
| Shopify isn't integrated | Check the Shopify integration status on the external service integration screen |
| The product data tool is disabled | Enable "Shopify product data" under the node's "Enabled tools" |
We explain how to integrate with Shopify in the Shopify integration article.
Tips to Improve Answer Quality
You can gradually improve the Customer Agent's accuracy as you operate it. The basic flow is to enrich the Knowledge Base, make the system prompt specific, and review actual exchanges to improve.
| Point | How to do it |
|---|---|
| Enrich the Knowledge Base | Upload a PDF that compiles frequently asked questions and their answers |
| Make the system prompt specific | Stating things down to the tone and answer length, such as "Answer in a polite tone, within 100 characters," makes responses more consistent |
| Review actual exchanges regularly | Check your exchanges with friends in the inbox, and when you find a question that wasn't answered well, add that content to the Knowledge Base |
Improvement doesn't end after one round. By adding to the Knowledge Base every time a new question comes up, your agent's answers will steadily get smarter.