Test your AI before customers do
Use realistic questions in Playground to verify answers, uncertainty, and human handoff behavior.
- For
- Workspace administrators approving AI behavior before launch
- Needs
- Complete this in Onlo Playground. No customer will see these tests.
- Time
- 10 minutes active · None waiting
Before you start
- At least one Fresh source in Knowledge base
- Three realistic customer questions: one known answer, one uncertain case, and one human handoff case
Why test the AI?
Playground lets you find missing knowledge and unsafe behavior before customers encounter it. Test outcomes, not polished demo questions.
Run three representative tests
- Step 1
Open Playground
In Onlo, open AI setup → Playground and choose the customer channel you want to simulate.
Expected resultThe composer and the selected channel appear.
- Step 2
Enable the intended scope
Confirm Knowledge base is enabled. Include Playbooks only when the scenario should use one.
Expected resultThe test uses the same sources and workflows you intend to evaluate.
- Step 3
Ask a known question
Ask a question whose correct answer exists in the Fresh source you added.
Expected resultThe reply matches the source and the trace shows the relevant knowledge.
- Step 4
Ask an unsupported question
Ask for information your sources do not contain.
Expected resultThe AI asks for clarification, states uncertainty, or follows your configured fallback instead of inventing an answer.
- Step 5
Ask for a person
Use a message that should trigger your human handoff behavior.
Expected resultThe response follows the configured escalation or handoff path.
Repeat the known, unsupported, and handoff scenarios once without changing configuration.
Expected result
All three scenarios produce the intended answer or fallback behavior consistently.
If you don't see this
- For a wrong known answer, inspect the cited source and remove conflicting or obsolete content.
- For an invented answer, review AI settings and make the fallback behavior more explicit.
- For a failed handoff, verify the escalation condition and the teammate who should receive it.
Customer questions
Does Playground use workspace credits?
The current Playground labels its test usage as free and shows an estimated production cost for context.
How many questions should I test?
Start with the three behavior types in this guide. Add real support questions as your team discovers gaps.
What should I fix first?
Fix confident incorrect answers and broken handoffs before improving tone or response style.
Common mistakes
- Testing only questions copied directly from the source headings.
- Treating a fluent answer as correct without checking the source or trace.
- Skipping the unsupported and human handoff scenarios.
You have completed this guide when
- A known question receives an accurate answer from the intended knowledge.
- An unsupported question does not produce a confident invented answer.
- A handoff request follows the behavior configured for your team.
Troubleshooting
The AI does not use the new source.
- Likely cause
- Knowledge base is disabled in the test scope or the source does not closely match the question.
- Recovery
- Enable Knowledge base, ask a specific question, and confirm the source is Fresh.
The AI gives a confident wrong answer.
- Likely cause
- The source is unclear, conflicting, or the configured fallback allows an unsupported response.
- Recovery
- Inspect the trace, correct the owning source, and repeat the same question.
The AI never hands off.
- Likely cause
- The test does not match the escalation condition or no valid teammate path is configured.
- Recovery
- Review the handoff condition and responsible teammate, then repeat the exact scenario.