1. Overview of AI Troubleshooting
Begin by gaining an overview of AI troubleshooting. This foundational understanding will guide you in diagnosing unexpected AI responses. Everything you do will relate directly back to three primary sources: policies, listing information, and memories.
2. Review Live Logs
First, examine the live logs. Navigate to the AI section and select Live. This view allows you to see what the AI has understood from the guest's input, providing initial insights into the AI's processing.
3. Analyze Response Details
Assess the detailed information extracted by the AI, identify which agent responded, determine whether a policy or memory was utilized, and review the confidence score associated with the response.
4. Understand the AI’s Rationale
Utilize this information to understand the AI's rationale for its response. If the Save Chain of Thought feature is enabled, you can also review the AI's reasoning process step-by-step, simplifying the troubleshooting process.
5. Review Policies
Should the response fail to meet your expectations, proceed to the Policies tab. Investigate potential issues such as missing topics, conflicting rules, vague wording, or outdated instructions.
6. Confirm Policy Priority
It is crucial to remember that policies override all other information. Any unintentional entries will be followed by the AI. Often, the solution involves refining or updating the policy text.
7. Examine Listing Details
Next, examine your listing details. If a guest's inquiry pertains to property specifics like check-in details, Wi-Fi, parking, or amenities, the issue may reside in your listing information. Ensure instructions are complete, details align with what the AI is expected to share, and no contradictions exist with your policies.
8. Check for Missing Listing Information
If the information is missing from the listing, the AI cannot utilize it. Proceed to inspect the Memories tab if the AI references something inaccurate or out of place.
9. Review and Manage Memories
Review any memories saved due to the AI interpreting a guest message in a particular manner. Verify accuracy and delete if no longer relevant, as memories should enhance personalization without causing confusion.
10. Adjust Autopilot and Communication Settings
Now, consider checking autopilot and communication settings. If the content seems accurate but the tone, length, or style feels off, adjust settings under AI Config and Communication to modify response length, formality, and empathy level.
11. Understand Communication Impact
These adjustments directly influence how the AI communicates with the guest.
12. Evaluate Confidence Scores
Finally, evaluate confidence scores. If the AI fails to automatically send a message or delivers an unusual response, inspect its confidence score. Low confidence can result from insufficient information, conflicting policies, or unclear guest messages.
13. Fine-Tune Confidence Thresholds
Adjust the minimum confidence threshold in the Advanced tab to refine how cautious or proactive the AI should be. Understanding the AI's knowledge sources and their influence on responses simplifies troubleshooting. With well-structured policies, accurate listing details, and clear communication settings, you can fine-tune the AI to deliver the desired guest experience.
