Evaluations tests each agent version and rule on every step it covers, before release and after any change. A failure blocks the release, and your QA team signs the evidence.
Process: Agency invoice, US · Suites rerun Wed Oct 21, 2026
Release REL-0412
Release REL-0412, model switch for the Agency invoice agent, results by step
Step
Result
Suites
Model
1. Read invoices from the AP mailbox
Passed
5 of 5 passed
Cleared to move to the new model
2. Classify the document: invoice or credit memo
Blocked
4 of 5 passed
Stays on the current model
3. Match the invoice to the SOW, purchase order and goods receipt
Passed
5 of 5 passed
Cleared to move to the new model
4. Propose the SAP posting
Passed
5 of 5 passed
In shadowon live traffic, before it moves
Blocked for step 2 only: the new model misreads one vendor’s credit memos as invoices. Other steps are cleared to move once your team adopts the release.
Suites for step 2Classify the document: invoice or credit memo
4 of 5 passed
ReplayCredit memos from one vendor read as invoicesFailed
Compliance300 planted cases, such as a new bank account in the email bodyPassed
Rule conformanceEach AP guidance rule in forcePassed
Sandbox systemFiled to the AP queue in a sandbox copy, then read backPassed
Synthetic pharma sandboxWhole invoice runs on synthetic vendors and contractsPassed
A model switch, gated step by step. Your QA team signs the evidence.
Trusted by 2 out of the 5 largest life sciences companies
Five tests for every step.
Each step an agent covers carries replay, compliance, rule, sandbox and synthetic suites. Any change to a model, prompt, rule or connector reruns the ones it touches.
A failure blocks only the step that failed.
When a model switch misreads one vendor’s credit memos, step 2 stays on the current model and the other steps move. Model swaps run in shadow on live traffic first.
People set the bar agents are measured against.
Two qualified people decide the same sample independently, and their agreement is the ceiling for that step. Model graders score only after they match people.
Suites are sized to the claim you make.
Zero misses in 300 clean cases supports a miss rate under 1%, and 3,000 supports under 0.1%. A weekly human audit measures the real rate in production.
Outside agents take the same tests.
Copilot, Agentforce and your in-house agents run the same step suites through the Agent Gateway. One that fails a compliance suite can only read on that step.
Your QA team signs, not Ontelis.
Each version you adopt gets a validation evidence pack for your QA team to review and sign. Ontelis staff can propose, never approve.
“The diagram used to be the thing we argued about. Now it shows us where every value came from, and it flags what our own templates got wrong.”
Senior Solution Architect, Campaign OperationsOne of the top 10 global pharma companies
What Evaluations does not do
Certify compliance or validated state.Your QA team reviews and signs the evidence.
Treat a clean pass as proof of a zero miss rate.A weekly human audit measures the real rate in production.
Mark an outside agent governed for a step before it passes that step’s suites.A failed compliance suite limits it to read actions on that step.
Test agents that do not come through the Agent Gateway.They are labelled observed and get no Ontelis checks.
Where Evaluations fits
Evaluations decides whether a version may run. Oversight shows whether it keeps working.
Ontelis Evaluations tests each AI agent version and rule on every step it covers, before release and after any change. Each step gets five kinds of test, including replays of your past approved decisions, planted compliance cases and writes read back from a sandbox copy. A failure blocks the release for that step only, and your QA team reviews and signs the evidence. Outside agents are tested only if they come through the Agent Gateway. Once a version is live, Oversight shows whether it keeps working.
No. Ontelis Evaluations gives your own validation process the evidence, and your QA team decides. For each agent version you adopt you get an evidence pack listing the tests run, the results, the pinned versions, agreement rates and known limits, which QA reviews and signs. Ontelis staff can propose a release and never approve one. Where a record falls under 21 CFR Part 11, Ontelis provides the signature mechanics and you validate them.
Yes, if they come through the Agent Gateway. Evaluations puts Copilot, Agentforce or an agent your team built through the same tests as an Ontelis agent on that step, on synthetic data, before it's cleared for the step. One that fails a compliance test can only read on that step until it passes. Vendors can change models without notice, so the tests rerun on a schedule your process owner sets. An agent that doesn't come through the Gateway isn't tested.
Ontelis Evaluations plants compliance cases on every step, and the agent has to route every one correctly. It buries an adverse event in a call note, or plants an off-label request, a withdrawn consent, expired content or a speaker over the cap. Each case comes paraphrased, mid-paragraph and in other languages, so "patient developed a rash" has to be caught in Spanish too. One miss blocks the release for that step. The agent only detects and routes; your safety and medical information staff assess.
An Ontelis agent stays on its current model until your team adopts the new one. Versions are pinned, you choose the upgrade window, and a process you've validated can pin its agent version. A model change is a release: Evaluations reruns the tests on every step it touches, runs the new model in shadow on live traffic before cutover, and gives QA a new evidence pack. A step that fails stays on the current model. Salesforce and Veeva Vault releases trigger the same reruns, on the preview environment first.
A miss in production is handled as an incident, and the case joins that step's tests, so every later version has to pass it. Ontelis Evaluations doesn't treat a clean pass as proof of zero misses. The tests are sized to the miss rate you want to be able to claim, and a weekly human audit of a random sample, spread across risk tiers, measures the real rate in production.
Evaluations needs your past approved decisions and some of your experts' time. It replays decisions your people already approved, such as past MLR pre-reviews, to see whether the agent flags what they flagged. Two of your qualified people decide the same sample independently, and how often they agree is the ceiling an agent is measured against on that step. Whole end-to-end runs use synthetic HCPs, territories, contracts and vendors, with no real data. Your QA team reviews and signs each evidence pack.
The customer’s QA team reviews and signs each validation evidence pack.
Vendor documents
Ontelis provides: a SOC 2 Type II report, an ISO 27001 certificate, a HIPAA business associate agreement wherever Ontelis processes PHI, a data processing agreement with transfer clauses, and a sub-processor list.
Models
Ontelis builds no foundation model. No model is trained on customer data, except an opt-in adapter that never leaves the customer’s data plane.
Your controls
Ontelis’s posture as a vendor is kept separate from Govern, which holds your own controls.