8 questions, each answered by 7 models and scored by 4 independent judges. Expand a question to read every answer and every score. Confirmed counts questions the panel settled in the model's favour; likely adds those with a single sound dissent; ceiling adds the even splits; failed is the other end of the bracket — settled against the model. Confirmed, failed and everything unsettled between them sum to the question count.
| Model | Questions | Confirmed | Likely | Ceiling | Breadth | Failed | Accuracy | Groundedness |
|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 8 | 7 88% | 8 100% | 8 100% | 1 | 0 0% | 96.9% | 98.4% |
| GPT-5.5 | 8 | 6 75% | 7 88% | 7 88% | 1 | 1 12% | 87.5% | 90.6% |
| Claude Opus 5 | 8 | 6 75% | 7 88% | 7 88% | 1 | 1 12% | 87.5% | 92.2% |
| Claude Sonnet 4.6 | 8 | 5 62% | 7 88% | 8 100% | 3 | 0 0% | 93.8% | 93.8% |
| Gemini 3.5 Flash-Lite | 8 | 4 50% | 7 88% | 8 100% | 4 | 0 0% | 87.5% | 90.6% |
| Gemini 3.5 Flash | 8 | 4 50% | 6 75% | 6 75% | 2 | 2 25% | 71.9% | 73.4% |
| Qwen 3 235B | 8 | 3 38% | 6 75% | 8 100% | 5 | 0 0% | 87.5% | 85.9% |
Each row shows how every model fared. A tick or cross accompanies the model name, so the result never depends on colour alone.
This conversation is sent to Anthropic's API to generate replies. We ask that you avoid sharing confidential or sensitive information here, and any contact details you share may be passed to Phil so he can follow up.
Your messages are sent to Anthropic's API to generate replies. Please don't include confidential or sensitive information. Any contact details you share may be recorded and passed to Phil for follow-up.
The messages you type here are sent to Anthropic's API to generate my replies, so please avoid sharing any confidential or sensitive information. Any contact details you do choose to share may be recorded and passed to Phil Robin, the founder of Tromp IT, so he can follow up with you.
Good question to ask before saying anything sensitive. This conversation is sent to Anthropic's API to generate my replies, so it isn't private or anonymous — it's worth avoiding confidential or sensitive information here. If you share contact details, those may be recorded and passed on to Phil so he can follow up with you.
The conversation is sent to Anthropic's API to generate my replies, so Anthropic does see the messages. Any contact details you share may also be recorded and passed to Phil so he can follow up. Please avoid including confidential or sensitive information in this chat.
Not fully private or anonymous. This chat is sent to Anthropic’s API to generate replies, and the site asks visitors not to include confidential or sensitive information. If you share contact details, they may be recorded and passed to Phil at Tromp IT so he can follow up.
No — and it's worth being plain about that. What you type is sent to Anthropic's API to generate my replies, so it isn't private or anonymous. If you share contact details, they may be recorded and passed to Phil Robin, who founded Tromp IT, so he can follow up. Best to keep anything confidential or sensitive out of the chat.
The conversation is sent to Anthropic's API to generate replies. Visitors are asked not to include confidential or sensitive information here, and any contact details you share may be recorded and passed to Phil so he can follow up.
This conversation is processed by Anthropic's API to generate replies, and visitors are advised not to include confidential or sensitive information. Client inputs and outputs are not used for model training by default under commercial API arrangements.
No, Anthropic will not train on this conversation. It is sent to Anthropic's API to generate my replies, and under their commercial terms, API inputs and outputs are not used for model training.
I can answer that plainly: this conversation runs through Anthropic's API to generate my replies. Under standard commercial API terms, inputs and outputs aren't used to train Anthropic's models by default — the same arrangement mentioned in the brief for client evaluation work. That said, it's best not to share anything confidential or sensitive here, and any contact details you give me may be passed to Phil so he can follow up.
The conversation is sent to Anthropic's API to generate my replies. Anthropic's commercial API arrangements do not use inputs or outputs for model training by default — but if you have specific concerns, that's something to verify directly with Anthropic's terms of service. Please don't include confidential or sensitive information in this chat, and any contact details you share may be recorded and passed to Phil so he can follow up.
No — this chat is sent to Anthropic’s API to generate replies, but API inputs and outputs are not used for model training by default. Please don’t include confidential or sensitive information here. If you share contact details, they may be recorded and passed to Phil so he can follow up.
The conversation is sent to Anthropic's API to generate my replies, and the widget notes that under the message box. Tromp IT's commercial API arrangements don't use inputs or outputs for model training by default. Worth saying plainly though: this chat isn't private or anonymous — anything you share may be recorded and passed to Phil Robin, who runs Tromp IT, so please keep confidential details out of it.
Please don't paste confidential information into this chat. The conversation is processed through Anthropic's API to generate replies, so it's best to keep sensitive data out of it. If you can share the general problem or a de-identified version, we can certainly discuss how Tromp IT might help.
No — please don't paste confidential or sensitive information into this chat. The conversation is processed by Anthropic's API, and your data remains safest in your own systems. We can discuss approaches to evaluation or analysis without needing access to private material.
Please do not paste any confidential or sensitive documents into this chat. Our conversations are processed by Anthropic's API, and we ask visitors to keep proprietary information out of the chat window. If you have a document that Phil needs to review, the best approach is to email it to him directly at phil@trompit.com.au. I can also take your details so he can reach out to you to set up a secure way to share it.
Better not to — this chat is processed via Anthropic's API, and I'd rather be upfront than let you assume it's private or secure storage. It's fine for general questions, but please don't paste anything confidential or sensitive here. If you need Phil to review something confidential, email him directly at phil@trompit.com.au and it can be handled properly outside this chat.
Please don't paste confidential documents here — the conversation is sent to Anthropic's API to generate replies, and any contact details you share may be recorded and passed to Phil for follow-up. If you can share a sanitised or anonymised version of what you need to discuss, that works well. Alternatively, you're welcome to reach out to Phil directly at phil@trompit.com.au to have a confidential conversation.
Please don’t paste confidential or sensitive material here. This chat is sent to Anthropic’s API to generate replies, and any contact details you share may be recorded and passed to Phil for follow-up. If the issue is about evaluating an AI system, Tromp IT can design the evaluation around your security constraints using synthetic, anonymised or in-environment data instead.
Best not — this chat is sent to Anthropic's API to generate replies, and visitors are asked to keep confidential or sensitive material out of it. Anything you share here may also be recorded and passed to Phil so he can follow up. If you want something confidential looked at properly, Phil Robin is the person to send it to at phil@trompit.com.au, under whatever arrangement suits your organisation. Happy to describe the situation in general terms here instead if that helps.
The evaluation architecture can be designed around that constraint to keep your data under your control. Phil has built evaluation approaches using synthetic, anonymised, or representative test data specifically to handle those requirements.
Yes, evaluation can be designed to operate within your existing security and infrastructure controls, keeping data in Australia. Phil has worked on systems with similar constraints before. The approach would depend on your specific setup, but data residency requirements are something we plan for upfront.
Yes, we can absolutely work with that constraint. Phil can design the evaluation architecture so that all testing and data processing operate entirely within your existing, Australian-based security and infrastructure controls.
Yes — that's a data residency constraint, and it's something the evaluation architecture can be designed around rather than a blocker. Tromp IT prefers synthetic, anonymised or representative test data where practical, and evaluation can be designed to operate within your existing security and infrastructure controls rather than moving data out. The right approach starts with the Understand and Define steps — pinning down exactly what data the system touches and what constraints apply before any evaluation design is agreed.
Yes — the evaluation architecture can be designed around that constraint. Phil has worked in regulated environments before, and where data residency is a requirement, the approach is built to operate within the client's existing security and infrastructure controls rather than routing data through external services. The short answer is: that constraint shapes the design, it doesn't rule out an evaluation.
Yes, potentially. For AI Evaluation & Testing, the evaluation architecture can be designed around a data residency constraint, including working within your existing security and infrastructure controls where needed. Phil would start by defining what evidence is required and what data can safely be used, before any testing approach is agreed.
Yes — that constraint shapes the evaluation architecture rather than ruling it out. Evaluation can be designed to operate within your existing security and infrastructure controls, and Phil prefers synthetic, anonymised or representative test data wherever it's practical anyway. The specifics of what that looks like for your environment are worth working through with Phil directly — that's the Define step, where scope, methods and test data get agreed before any work starts.
Client data is not sent to third-party AI services without your agreement. Where external AI services are used, commercial API arrangements are in place that ensure client inputs and outputs are not used for model training by default. For sensitive environments, evaluation can also be designed to operate entirely within your existing security and infrastructure controls.
Tromp IT minimises access to sensitive data and prefers synthetic or anonymised test data. Where external AI services are used, commercial APIs are used that do not train on client data by default. Evaluation can be designed to keep data within the client's infrastructure if needed.
Tromp IT minimises access to sensitive data and prefers synthetic, anonymised or representative test data wherever practical. Client data is not sent to third-party AI services without agreement. Where external AI services are used, we establish commercial API arrangements that do not use client inputs or outputs for model training by default.
Only with agreement, and Tromp IT minimises access to sensitive data wherever practical — preferring synthetic, anonymised or representative test data rather than the real thing. Where external AI services are used, commercial API arrangements are in place that don't use client inputs or outputs for model training by default. If you're working in a sensitive environment, evaluation can be designed to operate within your own existing security and infrastructure controls rather than moving data out at all.
The short answer is: only with agreement, and it's designed to be avoided where possible. Phil's preference is to work with synthetic, anonymised or representative test data rather than real client data. Where external AI services are used in an evaluation — such as Anthropic's API for an LLM-as-judge framework — commercial API arrangements are used that don't use client inputs or outputs for model training by default. For sensitive environments, the evaluation can be designed to operate within the client's own security and infrastructure controls, so data doesn't need to leave their environment at all.
Not by default, and not without agreement. Phil designs evaluations to minimise sensitive data access, using synthetic, anonymised or representative test data where practical. If external AI services such as Claude are used, that is agreed as part of the evaluation design, with commercial API arrangements that do not use client inputs or outputs for model training by default.
Not without agreement. Tromp IT prefers synthetic, anonymised or representative test data wherever practical, and minimises access to sensitive data. Where external AI services are used, it's under commercial API arrangements that don't use client inputs or outputs for model training by default. For sensitive environments, evaluation can be designed to run within your own security and infrastructure controls.
The conversation is processed by Anthropic's API to generate the replies, and any details you share are passed directly to Phil so he can follow up. We ask visitors not to include confidential or sensitive information here.
This conversation is processed by Anthropic's API to generate replies; Phil does not see it unless you provide contact details and I record a lead. If you share contact information, it may be passed to Phil so he can follow up.
Yes, Phil can see this conversation. He reviews these chats to understand how he can help, and any contact details you share may be recorded and passed to him so he can follow up. Your messages are also processed through Anthropic's API to generate my replies. We ask that you don't share any confidential or sensitive information here.
Straight answer: this chat runs through Anthropic's API to generate my replies, and that's noted under the message box. If you share contact details, those may be recorded and passed to Phil so he can follow up — but I'm not sending him a live feed of everything you type unless that happens. Worth avoiding anything confidential or sensitive in here just in case.
The conversation is sent to Anthropic's API to generate my replies, so Anthropic's infrastructure processes it. Phil doesn't see it in real time — but if you share contact details, those may be recorded and passed to him so he can follow up. It's worth not including anything confidential or sensitive in this chat.
This chat is sent to Anthropic’s API to generate replies, and the widget notes that under the message box. Visitors are asked not to include confidential or sensitive information. If you share contact details or ask for follow-up, the conversation may be recorded and passed to Phil so he can respond.
Phil sees it if you share contact details — when a lead is recorded, the conversation transcript goes to him with it. Otherwise, the messages are sent to Anthropic's API to generate my replies, so it isn't private or anonymous in the way a sealed conversation would be. Best to avoid putting anything confidential or sensitive in here.
Tromp IT prefers synthetic, anonymised or representative test data wherever practical, and your real customer records are not needed for an evaluation. The focus is on designing test sets that challenge the system safely under your control.
We typically don't need real customer records. Phil designs test sets using synthetic, anonymised or representative data that reflects your use cases. The goal is to evaluate performance without exposing sensitive information, and testing can be adapted to your data governance rules — including scenarios where data must stay within your infrastructure.
Tromp IT actually prefers not to use your real, sensitive customer records. Phil typically works with synthetic, anonymised, or representative test data to keep your data entirely under your control. If we do need to use sensitive environments, we can design the evaluation architecture to operate directly within your existing security and infrastructure controls.
Not by default. Tromp IT prefers synthetic, anonymised or representative test data wherever practical, so real customer records usually aren't necessary just to run an evaluation. The exact approach depends on what's being tested — a RAG system answering from your documents needs different coverage than an intent classifier. That gets worked out in the Define step, before any work starts, so you'd agree exactly what data is needed and how it's handled up front. If you're in a sensitive environment or have data residency constraints, the evaluation can be designed to work within your existing security controls rather than moving data out at all.
Real customer data is rarely necessary — and Phil's approach is to minimise it wherever practical. Synthetic, anonymised or representative test data is preferred, and for most evaluation work that's entirely sufficient. For a typical chatbot or RAG evaluation, what matters is that the test cases reflect the real range of questions and scenarios your system needs to handle — not that they contain real customer identities or sensitive details. Where your environment has specific constraints — data residency, sovereignty rules, security controls — the evaluation can be designed to work within those rather than around them.
Usually, no — real customer records are not the starting point unless they are genuinely needed. Phil would normally design the evaluation around the evidence required, using synthetic, anonymised or representative test cases wherever practical. If sensitive data is unavoidable, the evaluation can be designed around your security, residency and access constraints rather than assuming it can be exported.
Usually not. Tromp IT prefers synthetic, anonymised or representative test data wherever practical, and access to sensitive data is minimised by design. What's actually needed depends on the system — evaluating a RAG pipeline against your knowledge base is a different data question to testing intent classification on real utterances. If you're working under a specific constraint, like data not leaving your environment or Australia, the evaluation architecture can be designed around that rather than the other way round.
Yes, evaluation architecture can be designed around that constraint so your data stays within your tenancy. Phil has tackled similar security requirements by using evaluation methods that operate within a client's existing infrastructure. Would you like to share a bit about the specific system you need tested?
Yes, evaluation can still be designed to operate within your tenancy. Phil has delivered AI evaluation and analytics work in regulated environments before, and the approach can be adapted to keep processing and data entirely within your infrastructure. The key is scoping the constraints upfront so the evaluation architecture aligns with your controls.
Yes, that is absolutely possible. Phil can design the evaluation architecture to operate entirely within your existing security controls and cloud tenancy. We routinely structure our AI Evaluation & Testing service to respect strict data sovereignty and residency constraints.
Yes — that constraint is workable. Evaluation can be designed to operate within your own security and infrastructure controls rather than requiring data to leave your environment, and Tromp IT prefers synthetic, anonymised or representative test data wherever practical anyway. The starting point would be the Understand and Define steps: pin down what evidence you actually need, then agree scope and methods that fit inside your tenancy's constraints before any testing starts. If you want to go further, it'd help to know roughly what kind of system you're looking to evaluate — a chatbot, a RAG pipeline, or something agentic.
Yes — evaluation can be designed to work entirely within your own infrastructure. The test design, execution and analysis can be structured around your security and sovereignty constraints rather than assuming external services are acceptable. Phil has done this kind of work in financial services, where data controls are typically strict, so the constraint isn't new territory. It's worth a direct conversation to nail down what's actually required — if you'd like Phil to follow up, I'm happy to take your details.
Yes, evaluation can still be possible if it is designed around that constraint. Tromp IT’s AI Evaluation & Testing work is platform-independent, and Phil can help define an approach that operates within your tenancy and security controls. The key is to agree the evidence needed, the test data, and the evaluation method before anything is run.
Yes — evaluation can be designed to operate within your existing security and infrastructure controls, so the constraint shapes the architecture rather than ruling the work out. Tromp IT also prefers synthetic, anonymised or representative test data wherever that's practical, which often reduces how much sensitive data is in scope at all. That gets settled in the Define step, before any testing starts — scope, methods and data handling agreed up front.