iKB uses SystemOne decision models such as Jev and Clef, plus OpenAI Decisions, to judge evidence and make small classification choices mid-question, with shadow mode, fallbacks and usage records.
How iKB uses SystemOne models
Inside a single question, iKB makes several small choices before and after the answer is written: which response skill fits, whether a downloadable file is wanted, which answer card suits the reply, whether a customer needs a person. For these calls iKB can use SystemOne decision models such as Clef and Jev, as well as OpenAI Decisions. They return labels, not prose, which keeps AI decision model classification fast and easy to inspect.
A decision model plays two roles in iKB:
- Judge: scoring retrieved evidence and delivered answers.
- Decisions: taking over small classification calls that would otherwise go to an LLM.
In both roles the existing code stays in place, and anything unsure, slow or failed falls back to it.
What they are
A SystemOne model never writes text. You send it a state and a set of typed questions, and it returns typed answers with probabilities and confidence. There are three question types:
- Yes/no: returns a probability.
- Choice: picks one of up to 255 options.
- Score: a position on a 2 to 10 level scale.
| Model | Provider | Notes |
|---|---|---|
| Jev | TypeSafe AI | Hosted, $0.042 per million input tokens, 70 to 500 ms |
| Clef | Cloudflare | Open weights, can be self-hosted |
| OpenAI Decisions | OpenAI | gpt-6-luna |
| Local chat model | Any | Scored from its token probabilities |
In iKB this is one setting, the Decision model, chosen in Model Defaults. It can be overridden per topic, and the default is resolved live.
The vendor’s own caveats set the boundaries: no world knowledge, no arithmetic, and English-first. Malay needs testing before thresholds are trusted, which is why everything starts in shadow mode.
Role 1: judge (scoring evidence and answers)
The evidence check is set per topic to off, shadow or filter. Before the answer model runs, the engine rates the retrieved chunks against the question. It records whether the question is answerable, how completely the evidence covers it, and each chunk’s relevance.
Filter mode is conservative by design:
- It drops a chunk only when confident it is irrelevant (probability below 0.2).
- It never drops a pinned chunk.
- It never goes below 3 chunks.
- It judges at most 30 chunks per turn.
It never refuses a question. Insufficient evidence is recorded, not acted on.
Independent quality scores are per topic and off by default. After the answer is delivered, the engine scores groundedness, relevance, accuracy and completeness, and names statements the evidence does not support. The scores are stored as their own judge row beside the LLM judge, never replacing it. If the engine fails, no row is written and the answer is untouched.
Illustration: an HR operations lead puts the evidence check in shadow mode on a leave-policy topic. For two weeks she compares the recorded answerable and relevance labels with the answers staff actually received, paying close attention to questions asked in Malay. Only when the labels match her own reading does she move to filter mode, knowing pinned chunks stay and at least 3 chunks always reach the answer model.
Role 2: decisions (replacing small classification LLM calls)
There are nine tasks:
- skills: which response skill fits the message.
- office_intent: whether a downloadable file is wanted, and which format.
- office_scene: which Office style to use.
- tool_preflight: which AI-decided tools to call.
- support: whether this is a support request.
- intent: the query intent after the free regex tier.
- ui_shape: which answer card fits.
- followup_check: whether the topic’s passages can answer a suggested next question.
- handover: whether a customer needs a human on a staffed channel.
Every task goes through one gate, decision_tasks.run, with three modes:
- Off (default): the existing code runs untouched.
- Shadow: the existing code answers, and the decision is recorded in the background as labels only, never text.
- On: a confident decision is used. Anything unsure, slow or failed falls back to the existing code.
Because all nine tasks share this gate, they follow the same rollout path.
Safety and governance rules
A decision only picks among options the existing code already allowed. It narrows a choice and never invents one.
Trust is asymmetric for costly-if-wrong calls. A confident NO from followup_check or handover is used on its own, but a YES is always confirmed by the LLM check. Picture a customer-service manager who sets handover to on for a staffed channel. A confident NO lets routine conversations continue without an extra LLM call, while every YES is confirmed before an agent is paged.
Strict-PII topics never send messages to a remote decision model. Models marked local-only are only ever called at a private address, re-checked on every call. A compliance admin might assign a self-hosted Clef marked local-only as the override for a strict-PII claims topic. A separate Flash topic for quick lookups skips decisions entirely, as all Flash topics do.
Failures are cheap:
- A timeout (1.5 s by default), an error or a malformed answer returns nothing, and the old code runs.
- A circuit breaker opens after 5 failures in 60 seconds and stays open for 2 minutes.
- Request and response sizes are capped.
Every call writes a token-usage row (evidence_check, jev_quality_score, decision_<task>), and the pipeline trace records each step. Usage that is not reported is marked unpriced, never zero.
What was measured
| Test | Result |
|---|---|
| Skill: one choice question over all skills | About 2.4k tokens, 340 ms, confident on 19 of 24 |
| Skill: sixty separate yes/no questions | About 11k tokens, timed out, could not cover all skills |
| Office intent: yes/no, then format choice | Confident and right on 12 of 12 |
| Handover on live models | Right on 12 of 12 |
| Intent with topic name and description | Fixed 6 of 12 wrong “out of scope” calls |
The samples are small, but the lesson is clear: how a question is shaped matters. One choice question covered every skill where sixty yes/no questions could not, splitting office intent into two steps worked cleanly, and giving the intent classifier the topic’s context corrected half of its wrong out-of-scope calls.
Where they are not used
- Retrieval ranking and analytics: both are consent-gated.
- Writing text: a decision model never writes.
- Moderation: a free OpenAI endpoint is kept.
For teams adopting them, start each task in shadow mode, compare its labels, usage rows and pipeline trace against current behaviour, test Malay questions, then switch tasks on one topic at a time.
