Skip to content

Bot Testing & Knowledge Gaps

View as Markdown

OmnyBot provides two built-in tools to validate bot responses before going live and continuously improve coverage from real chats: the per-bot Testing tab and the Unanswered (Knowledge Gaps) page (/ai-gaps).

Open My Bots → select a bot → Testing (/ai-bots/{botId}/test) to simulate customer conversations in a sandbox without messaging real customers.

  • Quick start: Click one of the starter prompts or type a realistic customer question and press Enter.
  • Conversation context: Test chat history is stored locally per workspace and bot, sending up to the 10 most recent messages as context. Click Reset (or select it from the + menu) to clear the thread.
  • Per-reply diagnostics:
    • Retrieved sources: Expand to inspect the matched Knowledge Base snippet, collection name, and match percentage score.
    • Token usage: Shows the number of tokens consumed for that reply (if the AI token wallet is depleted, a Top up tokens button appears).
    • Media preview: Displays any attached images triggered by the reply.
  • Low-confidence and Out-of-KB handling:
    • When the bot lacks sufficient data, replies show an Uncertain or Out of Knowledge Base badge along with the configured fallback behavior (DRAFT or waiting message from Bot Configuration).
    • Click Train with Mina to open Mina prefilled with the test context, or click Add knowledge (or + → Add knowledge to base) to add a new article immediately.

From the left navigation sidebar, open Build → Unanswered (/ai-gaps). The sidebar badge displays the current count of open (OPEN) questions waiting for review.

When the bot encounters a real customer question it cannot answer from the Knowledge Base and responds with a deferral, OmnyBot automatically logs a gap ticket (the list displays up to the 200 most recent open items).

To prevent duplicate alerts, at most 1 open gap per conversation is recorded within a 24-hour window (questions and answers are trimmed to a maximum of 1,000 characters).

Each card shows the related Knowledge Base collection, the channel/account where the question occurred, the customer’s question, and the bot’s reply. You have 3 actions:

  1. Ask assistant to add: Hands the gap to Mina to draft and add the missing knowledge.
  2. Add manually: Opens the Knowledge Base ingestion dialog so you can write the answer directly.
  3. Dismiss: Marks the gap as DISMISSED if it does not need to be added to your Knowledge Base.