23 questions, each answered by 7 models and scored by 4 independent judges. Expand a question to read every answer and every score. Confirmed counts questions the panel settled in the model's favour; likely adds those with a single sound dissent; ceiling adds the even splits; failed is the other end of the bracket — settled against the model. Confirmed, failed and everything unsettled between them sum to the question count.
| Model | Questions | Confirmed | Likely | Ceiling | Breadth | Failed | Accuracy | Groundedness |
|---|---|---|---|---|---|---|---|---|
| Claude Opus 5 | 23 | 23 100% | 23 100% | 23 100% | 0 | 0 0% | 100.0% | 100.0% |
| Claude Sonnet 5 | 23 | 22 96% | 23 100% | 23 100% | 1 | 0 0% | 100.0% | 99.5% |
| Claude Sonnet 4.6 | 23 | 22 96% | 23 100% | 23 100% | 1 | 0 0% | 98.9% | 98.9% |
| GPT-5.5 | 23 | 20 87% | 22 96% | 23 100% | 3 | 0 0% | 98.9% | 97.3% |
| Gemini 3.5 Flash-Lite | 23 | 11 48% | 17 74% | 21 91% | 10 | 2 9% | 83.7% | 84.8% |
| Qwen 3 235B | 23 | 7 30% | 10 43% | 16 70% | 9 | 7 30% | 63.0% | 66.8% |
| Gemini 3.5 Flash | 23 | 7 30% | 14 61% | 18 78% | 11 | 5 22% | 73.9% | 73.9% |
Each row shows how every model fared. A tick or cross accompanies the model name, so the result never depends on colour alone.
Engagements are individually scoped with agreed deliverables, effort and cost rather than run on fixed day rates. Phil would be happy to discuss what you have in mind to see how that fits. Would you like to share your name and email so he can follow up directly?
I can't provide rates — engagements are scoped individually based on the work needed. Phil will discuss costs after understanding your requirements. He's based in Melbourne and works with organisations across Australia.
Tromp IT's engagements are scoped individually based on the specific deliverables and effort required, so we don't have a single standard day rate. Our founder, Phil Robin, works directly with clients to understand their needs and agree on pricing up front before any work begins. If you have a project in mind, you can reach him at phil@trompit.com.au to discuss the details and get a tailored proposal.
I don't have pricing to share — engagements are scoped individually based on what's needed, so Phil Robin is the one to talk rates with directly. You can reach him at phil@trompit.com.au. If you'd like, tell me a bit about what you're looking to have evaluated and I can make sure that context gets to him.
Pricing isn't something I can give you here — engagements are scoped individually, and rates depend on what's involved. Phil is the right person to talk to: phil@trompit.com.au. If you want to give me a rough sense of what you're looking at, I can help work out whether it's something Tromp IT could help with.
Tromp IT doesn’t publish a standard day rate, because engagements are scoped individually around the evidence needed, deliverables, effort and risk. Phil is the person to discuss pricing with directly, and you can reach him at phil@trompit.com.au.
Pricing isn't something I can quote — engagements are scoped individually, with deliverables, effort and cost agreed up front before work starts. Phil Robin, who founded Tromp IT, is the person to ask directly: phil@trompit.com.au. If you tell me a bit about what you're looking to have evaluated, I can at least tell you whether it's the kind of work Tromp IT does.
Tromp IT doesn't build AI systems or sell AI platforms. Phil focuses independently on evaluating whether they are actually working, what they cost, and where they fail.
No — Tromp IT doesn't build AI systems. We focus on independent evaluation and testing of existing systems, helping organisations understand whether their AI is working effectively, where it fails, and what it costs. Phil has extensive experience in this space, including evaluating chatbots, RAG systems and agentic AI across sectors like banking and healthcare. If you need help assessing or optimising a system you already have — or one being built by someone else — that’s where we can add value.
Tromp IT doesn't actually build AI platforms or systems. Instead, Phil Robin focuses on AI Evaluation & Testing, which means helping you measure whether your AI systems are actually working, what they cost, and where they fail. We act as an independent voice to evaluate what you have built or are planning to build. If you would like to discuss how we might help you test and optimise your AI plans, I can certainly pass your details to Phil.
Not quite — Tromp IT doesn't build AI systems, and Phil doesn't sell or implement an AI platform. What we do is independent evaluation: testing whether an AI system you already have (or are building elsewhere) actually works, where it fails, and what it costs to run. If you're at the build stage, the useful moment for us is once there's something to test — happy to talk about what that would look like when you get there.
Building AI systems isn't what Tromp IT does — Phil's focus is evaluating and measuring them independently, which only works if he's not also the one who built what's being tested. If you already have a system in development, or are about to, that's actually the right time to think about evaluation — knowing how you'll measure success before you go live tends to surface problems earlier and cheapens fixing them.
Tromp IT is mainly an independent evaluation and analytics consultancy, not an AI product development shop. Phil has built conversational AI systems before, including a medical triage chatbot for Medibank through Dirawong, but Tromp IT’s current focus is testing whether AI systems work, where they fail, and what they cost. If you already have a system or a delivery team, Tromp IT can help define the evidence, evaluate it, and recommend what to fix.
No — Tromp IT doesn't build AI systems. Phil's focus is the independent side: evaluating whether a system actually works, what it costs, and where it fails. That independence is the point — we don't sell platforms, so there's no conflict in measuring one.
Independent evaluation provides the evidence and findings you need to make decisions, but the responsibility for those business decisions remains with you. Phil scopes every engagement up front so the deliverables, methods, and limitations are clear before work begins.
Tromp IT works to rigorous, transparent standards — every evaluation method is defined and agreed upfront, and findings are evidence-based. The work is advisory, not an indemnity against system failure. For any specific engagement, terms and liability are discussed and agreed before work begins.
Tromp IT's engagements are scoped individually with agreed deliverables and terms, which cover liabilities. Since I'm an AI assistant, I can't confirm specific legal terms or liability limits here. If you are looking at a specific project, Phil Robin can run through how we manage risk and liability. Would you like me to get your details so Phil can reach out to discuss this?
I can't speak to formal liability terms — that's a contractual question, not something covered in what I know, and it would need to be worked out directly with Phil for any given engagement. What I can tell you is the philosophy behind the work: Tromp IT's whole premise is that evaluation results need to be explainable, not just trusted, which is why the process (Understand, Define, Evaluate, Recommend) agrees scope and success criteria upfront before any testing happens. If you want to discuss terms or liability arrangements for a specific engagement, that's a conversation for Phil directly — phil@trompit.com.au.
That's a question for Phil directly — liability terms are set per engagement and aren't something I can speak to on Tromp IT's behalf. What I can say is that engagements are scoped up front with agreed deliverables and success criteria, so there's a shared understanding of what "right" looks like before work begins. That's partly what the Define step is for. You're welcome to reach Phil at phil@trompit.com.au to talk through the specifics.
I can’t state Tromp IT’s legal liability or contract terms here, and this chat shouldn’t be treated as legal advice. In practice, Phil scopes evaluation work up front, including methods, deliverables, assumptions and limitations, so the evidence is clear and auditable. If liability terms matter for procurement, that needs to be discussed directly with Phil as part of the engagement terms.
That's a question for Phil directly — liability terms, insurance and contractual arrangements aren't something I can speak to, and I'd rather not guess. He's at phil@trompit.com.au. What I can say is that engagements are scoped up front with agreed deliverables and success criteria, so the standard being measured against is settled before the work starts.
Phil doesn't recommend one over the other for general use. The right choice depends entirely on what you're building and how the model performs against your specific test criteria. That is where Tromp IT's AI Evaluation service comes in, testing how models actually behave on your data.
It depends on the use case, not the model alone. Phil has tested both extensively — including in Ada's published comparison — measuring accuracy, cost and groundedness across real tasks. The right choice comes from structured evaluation, not benchmarks or preference.
Phil actually evaluated both of these models—along with five others—as part of a comprehensive testing project for Ada, the assistant you are speaking with right now. He set up a multi-judge panel to mark their answers and track things like accuracy, latency, and cost. The full evaluation and the results are published on our website if you would like to see how they compared.
Not a straightforward "one wins" question — it depends what you're using it for, what quality actually means in your context, and what you're measuring against. That's the core of what Tromp IT's AI Evaluation & Testing does: define what "good" looks like for your specific use case, then test the models against that rather than a generic leaderboard. Phil actually ran exactly this kind of comparison with Ada, the assistant you're talking to now — seven model backends tested against the same 189-question set, judged independently rather than by reputation, and the results are published on the site if you want to see how the methodology works.
That's not a recommendation Tromp IT makes — the whole point of the service is that the right model depends on what you're actually measuring, and that's an empirical question, not a preference. Phil's published evaluation of Ada tested seven different model backends and measured accuracy, groundedness, latency and cost — rather than picking a favourite upfront. That comparison is linked from the site if you want to see what that kind of evidence actually looks like.
It depends on the job, the risk tolerance, and what “better” means for your system. Phil’s evaluation work is built around testing that directly, rather than assuming one model is best because of a leaderboard or a benchmark. Tromp IT can compare GPT-4, Claude and other models using the same test set, judge framework and cost measures, then show where each one fails.
That's not a question this brief lets me answer with a recommendation — and honestly, the Tromp IT position is that the answer depends on your use case and should come from evidence, not a preference. Model comparison is part of the AI Evaluation & Testing service: Phil ran one 189-question set across seven model backends for Ada, the assistant you're talking to, with a four-model judge panel marking every answer. The published comparison covers accuracy, groundedness, latency and cost per thousand conversations, and it's linked from the site.
We can't guarantee a specific performance score for your AI, because evaluation measures what's actually there rather than manufacturing a result. Phil typically works by designing tests that give you an honest, evidence-based picture of how your system behaves.
No. Tromp IT provides independent evaluation and evidence-based recommendations, but outcomes depend on the system, data, and implementation choices made by the organisation. Phil's role is to clarify what’s working, why or why not, and where to focus improvements — not to promise specific results.
No, we cannot guarantee specific outcomes or performance improvements. What Tromp IT does guarantee is independent, evidence-based measurement of how your AI systems are actually performing under real-world conditions. Phil's focus is on giving you the rigorous data and failure analysis you need so you can confidently decide what to change.
No — I can't guarantee results, and that's a deliberate part of how Tromp IT works. The whole point of evaluation is to find out honestly what's true, including if a system is underperforming or the fix is harder than hoped; a guarantee up front would undercut that independence. What Phil can commit to is a clear process: agreeing scope and success criteria before work starts, then delivering evidence and prioritised recommendations, not a promised outcome.
Honestly, no — and anyone who does guarantee results in this space should be questioned carefully. What Phil can guarantee is rigorous, independent evaluation: clear test design, transparent methods, and findings grounded in evidence rather than assertion. Whether a system improves after the work depends on what the client does with the recommendations.
No — Tromp IT would not guarantee that an AI system will pass, improve by a fixed amount, or deliver a particular business outcome. What Phil can do is design an evaluation that produces defensible evidence about how the system performs, where it fails, and what is worth changing. That is the point of the AI Evaluation & Testing work: measure it independently, not sell confidence without evidence.
No — and I'd be wary of anyone in evaluation who did. What Tromp IT commits to is agreed scope, deliverables, effort and cost before work starts, and evidence you can check at the end. Phil's approach is to show how a score was produced, not just hand you a number, because a score isn't evidence unless you can explain where it came from.
I cannot commit to a set hourly rate or retainer terms, as engagements are scoped individually by Phil. If you'd like to discuss how an engagement could be structured, I can take your details so Phil can follow up directly.
Tromp IT doesn't offer fixed-rate retainers or lock in hourly rates for extended periods. Engagements are scoped individually based on the work required. Phil can discuss pricing for ongoing support, but long-term rate guarantees aren't part of the model. For accurate planning, he typically recommends defining deliverables and effort upfront per project.
Pricing is something Phil scopes and agrees on individually for each engagement, so I cannot commit to or lock in any rates myself. I can certainly have him reach out to you to discuss how he structures retainers and long-term agreements. If you'd like, you can share your name, email, and the best phone number to reach you on, and I'll pass those details along to him.
I can't commit to rates, terms or retainer arrangements — those are things Phil scopes and agrees directly with each client. If you'd like, I can pass your enquiry to him so he can discuss it with you properly, and phil@trompit.com.au will also reach him directly. Would you like to share your name and email so I can flag this to him as a retainer enquiry?
Committing to rates or contract terms isn't something I can do — that's a conversation for Phil directly. You can reach him at phil@trompit.com.au, and he can talk through how engagements are structured and scoped.
I can’t commit Tromp IT to a retainer rate or a two-year price lock here. Pricing and commercial terms are scoped individually with Phil, based on the work, deliverables and level of commitment. You can raise it directly with him at phil@trompit.com.au.
I can't commit to rates or terms on Tromp IT's behalf — engagements are scoped individually, and pricing is Phil's call. Phil Robin runs Tromp IT and does the scoping himself, so a two-year retainer question is one to put to him directly at phil@trompit.com.au. If you'd like, I can take your name, email and phone and pass the enquiry to him so he can come back to you.
There isn't a single provider that always wins on price-performance for LLMs, because it depends entirely on your model choice, scale, and workload pattern. AWS, Azure, and Google Cloud all handle these deployments differently, and infrastructure costs shift quickly as new silicon rolls out. Phil has looked closely at how cloud spend aligns with actual value in enterprise environments. If you want a tailored look at your cloud costs or AI infrastructure spend, I can connect you with him.
There's no single provider that's best for all LLM workloads — performance, cost and suitability depend on the model, scale and use case. Phil has done detailed cost and performance analysis across AWS, Azure and GCP for clients, focusing on real-world efficiency rather than list prices. The right choice comes down to workload patterns and requirements, which we can assess through structured evaluation.
The best price-performance for LLM workloads depends entirely on your specific model choices, hosting strategy, and whether you are using managed APIs or raw compute. Phil has analyzed these trade-offs across AWS, Azure, and other platforms, and he finds that the answer changes quickly based on your scale and usage patterns. Our Cloud Cost Analysis service is designed to help organizations measure and optimize this spend for their specific workloads. Would you like to share a bit about the models you are currently running or considering?
That's outside what I can speak to — Tromp IT's cloud work is cost analysis on AWS environments, not a comparative view across providers for LLM workloads specifically. If you're weighing that decision, Phil would be the one to ask directly, but it's worth being clear that's a different question from what Tromp IT's Cloud Cost Analysis service covers. That service is about finding where your existing AWS spend isn't delivering value, not benchmarking providers against each other.
That's not a question I can answer well on Tromp IT's behalf — comparing cloud providers on current LLM pricing isn't something this brief covers, and I'd rather not speculate than give you something unreliable. What Tromp IT does do is analyse whether your AI spend is actually delivering value — measuring usage, output quality and cost across whatever stack you're running, and identifying where spend isn't justified by results. Phil has done this kind of work in AWS environments, and the same analytical approach applies to AI tooling costs more broadly. If you have a specific setup you're trying to get a handle on, that's worth a conversation with Phil directly at phil@trompit.com.au.
There isn’t a reliable single answer, because price-performance depends on the model, latency target, usage pattern and whether you’re using hosted APIs or running infrastructure yourself. Phil’s work is closer to measuring the actual cost, latency and output quality for a specific workload than making broad provider claims. That maps to Tromp IT’s AI Usage & Optimisation work, and sometimes AI Evaluation & Testing if quality varies between model options.
That's not something I can answer from what I know about Tromp IT's work — it's a fast-moving comparison and I'd only be guessing. What Tromp IT does do is Cloud Cost Analysis: looking at your actual usage and billing to find where spend isn't delivering value, then quantifying the savings against the effort and risk. Phil's experience there is primarily AWS, including an enterprise environment in financial services where he identified and quantified optimisation opportunities.
Expert witness testimony is not something Phil provides. Tromp IT focuses on independent AI evaluation, testing, and analytics to help organisations understand how their AI systems perform.
Phil focuses on independent evaluation and testing of AI systems, not legal testimony. He has extensive experience in assessing AI performance, failure modes and evidence-based validation, but expert witness work is not something he currently offers. This would be best discussed directly with him to understand any potential overlap with his expertise.
That is not a service Phil offers, and Tromp IT does not provide expert witness testimony or legal services. Phil's focus is strictly on the practical, technical side of AI evaluation, testing, and performance analytics. If you are looking for independent technical testing or a baseline evaluation of your system's performance before any legal escalations, that is something we can help with.
That's not something I can confirm — expert witness work isn't part of what's described in this brief about Phil's services. What Phil does do is independent AI evaluation and testing that produces defensible evidence about how a system performs, which may be adjacent to what you need, but the legal/testimony piece itself is a question for Phil directly. I'd suggest emailing him at phil@trompit.com.au to discuss what you're actually looking for.
Expert witness work isn't something I can confirm as a Tromp IT service — it's not listed in what Phil offers, and I'd rather be straight about that than stretch what's here. That said, if the underlying need is technical evidence about whether an AI system performed as claimed — failure analysis, evaluation methodology, documented test results — that kind of rigorous, independently produced evidence is exactly what Phil's evaluation work generates. Whether that material is useful in a legal context is a question better put to Phil directly. You're welcome to reach him at phil@trompit.com.au.
I can’t confirm that Phil offers expert witness services. It is adjacent to Tromp IT’s AI Evaluation & Testing work, where Phil can independently assess whether an AI system is working and explain how the evidence was produced. For anything legal or litigation-specific, the best next step is to ask Phil directly at phil@trompit.com.au.
That's not something I can confirm — expert witness or litigation support work isn't part of what's written in Tromp IT's services. What Phil does do is independent AI evaluation: test design, failure analysis and multi-judge frameworks, with the reasoning behind every score documented rather than asserted. That kind of defensible evidence trail may be adjacent to what you need, but whether he'd take on a testimony role is a question for Phil directly at phil@trompit.com.au.
An evaluation only shows that a model performed well against the specific test cases and metrics you checked. It doesn't mean every edge case or future user prompt has been covered. Phil typically advises treating evaluation as an ongoing part of the deployment lifecycle rather than a one-off gate.
No — even if our evaluation finds no issues, we don't treat any single assessment as proof of readiness. Phil's approach is to design tests that reveal specific risks, but every test has limits. Deployment decisions should include ongoing monitoring, especially for real-world behaviour that lab testing can't capture. Testing reduces risk, not certainty.
An evaluation showing no issues with your test set is a strong signal, but it does not mean you should stop testing entirely. AI models can behave unpredictably when they encounter real-world inputs that differ from the test data. Phil typically designs evaluations to look for hidden failures, but ongoing monitoring and structured testing remain essential as usage scales.
No — a clean result on our evaluation means the system performed well against the specific test set and criteria we agreed on, not that it's safe under every condition it might meet in production. Real-world usage tends to surface edge cases no test set fully anticipates, and things like safety or bias testing more broadly aren't something I can confirm we cover — that's a question for Phil directly. What an evaluation gives you is evidence about defined scope, clearly documented, which is a genuinely useful input to a deployment decision, just not a substitute for the whole decision.
A clean evaluation result means your system passed the tests you ran — it doesn't mean there are no problems, it means none were found within that scope. Test sets have boundaries, and what you don't test for, you won't catch. Phil's published comparison of Ada is a good illustration of this: the evaluation was designed to surface disagreement rather than paper over it, and the judge panel reported the questions it couldn't reach consensus on rather than quietly resolving them. That kind of transparency about the limits of the measurement is part of what makes a result trustworthy. The honest answer is that evaluation reduces deployment risk — it doesn't eliminate it. What it should do is give you clear evidence about specific, well-defined questions, so you know exactly what you're confident about and what remains untested.
No. A clean evaluation result means the model performed well against the agreed test design, not that every future input, user group or operating condition is safe. Phil’s work in AI Evaluation & Testing is about making that evidence clear: what was tested, what was not tested, and how much confidence the result should give you. The useful outcome is not a blanket “safe”, but a defensible view of residual risk and what, if anything, should still be monitored after deployment.
No — an evaluation tells you how a system behaved on the tests you ran, not how it will behave on everything else. A clean result is evidence that your test set didn't surface failures, which is a narrower claim than "it's safe". That gap is exactly why the Define step matters: agreeing up front what "performing well" means, and what the test data actually covers, is what makes the result interpretable. Deployment safety decisions also touch on things this brief doesn't cover me to speak for — Phil is the right person for that conversation.
Engagements are scoped individually based on what you actually need, so there isn't a single fixed minimum budget. Phil works with both large corporations and small companies, and the size of the engagement is flexible either way. If you'd like to share a bit about what you're working on, Phil can look at what an appropriate scope might be.
I can't specify a minimum budget — engagements are scoped individually based on the system and goals. Phil works with organisations of different sizes, and the effort depends on what you need to measure and why. The best way to understand cost is to discuss your specific situation with him directly.
Engagements are scoped individually based on your specific system and goals, so we don't have a set minimum budget. Phil Robin, Tromp IT's founder, can work out a clear proposal with agreed deliverables and costs once he understands what you need. If you'd like to share a bit about the system you are looking to evaluate, I can pass those details to Phil so he can give you a proper estimate.
I can't quote figures — engagements are scoped individually based on what you need to evaluate and how deep the testing has to go, so that's a question for Phil directly at phil@trompit.com.au. What I can say is that scope is agreed up front: you'd define success criteria, test data and deliverables before any work starts, so you'd know what you're getting before committing spend. If you tell me a bit about the system you're looking at, I can help you think through what "meaningful" would actually require.
Pricing isn't something I can give you here — engagements are scoped individually, and Phil is the one to talk to about what a specific piece of work would cost. What I can say is that scope drives cost more than budget drives scope. A focused evaluation of a single system — clear questions, defined test data, agreed deliverables — can be genuinely useful without being large. Phil has worked with both big enterprises and smaller organisations, so size isn't a barrier either way. If you want to share what you're trying to evaluate, that would give Phil something concrete to work with when you connect.
Tromp IT does not quote a fixed minimum budget, because engagements are scoped individually around the evidence you need and the effort to produce it. A meaningful first engagement can be quite focused, such as evaluating a defined chatbot or RAG workflow against a targeted test set. Phil would normally scope the deliverables, effort and cost up front before any work begins; you can reach him at phil@trompit.com.au.
I can't quote figures — engagements are scoped individually, so pricing is a conversation with Phil directly at phil@trompit.com.au. What I can tell you is that engagement size isn't a barrier either way; Phil has worked with both large corporations and small companies. Scope, effort and cost are agreed up front before any work starts, so you'd know what you're committing to before it begins.
Tromp IT doesn't work on a success-fee model. Engagements are scoped up front with defined deliverables, effort and cost so both sides know what to expect. What failure modes or specific AI behaviours are you hoping to evaluate?
Tromp IT doesn't offer success-fee arrangements. Phil prefers to agree on fixed-scope, fixed-cost engagements upfront so there are no surprises. That said, he's worked with startups and understands budget constraints — we can discuss scoping to fit what's practical. You can reach him at phil@trompit.com.au to talk through options.
No, Phil doesn't work on a success-fee basis. Engagements are always scoped up front with agreed deliverables, effort, and cost. This ensures clarity for both sides before any work begins.
I can't commit Tromp IT to any particular fee structure — engagements are scoped individually, and pricing is something only Phil sets, not me. What I can tell you is that engagement size isn't a barrier either way; Phil has worked with both large corporations and small companies. If you'd like, share what you're trying to evaluate or build and I can pass it to Phil along with your question about payment structure.
Pricing and engagement structure are things Phil works out individually with each client — I can't quote rates or commit to specific arrangements on his behalf. What I can say is that engagements are scoped up front with agreed deliverables and effort, so there are no surprises mid-project. For anything around commercial terms, the direct conversation is with Phil at phil@trompit.com.au.
I can’t commit Tromp IT to a success-fee arrangement here, and I don’t want to invent commercial terms. Engagements are normally scoped up front with agreed deliverables, effort and cost, and company size is not a barrier. The best next step is to ask Phil directly at phil@trompit.com.au.
I can't commit to anything on rates or fee structures — that's Phil's call, and engagements are scoped individually rather than priced off a list. What I can say is that engagement size isn't a barrier; Phil has worked with small companies as well as large corporates. Worth raising the commercial side with him directly at phil@trompit.com.au.
No, that is not something Tromp IT can do. Phil evaluates and tests systems independently, but because clients retain control and make their own deployment decisions, providing an indemnity for production outcomes is outside what an independent evaluation consultancy can offer.
No, Tromp IT does not accept liability for the future performance or impact of systems we evaluate — evaluations reflect the system's behaviour at a point in time under defined conditions. Phil has structured engagements around clear scopes and deliverables, focusing on transparent evidence rather than indemnity, and can discuss contractual terms directly if you'd like to explore what's possible.
That is a legal and risk question that Phil would need to review and discuss with you directly. Tromp IT acts as an independent evaluator to identify risks and failure modes before production, but we don't own or control the systems themselves. You can reach Phil directly at phil@trompit.com.au to discuss how he handles liability and risk in engagements.
That's a contractual question that goes well beyond what I can answer or commit to — liability terms, indemnities and contract structure are things Phil would need to work through directly with you as part of scoping any engagement. What I can tell you is what the evaluation itself covers: Tromp IT tests and reports on system behaviour against agreed criteria, but that's a services engagement, not a warranty on future production outcomes. If you'd like, I can pass this on to Phil at phil@trompit.com.au so he can address the contractual side properly.
No — Ada is an AI assistant, and Tromp IT can't make legal commitments through this chat. Phil is the person to talk to about contractual terms. What Tromp IT can do is scope engagements with clear, agreed deliverables upfront — what was tested, how, against what criteria, and what the findings were. That creates a transparent evidence trail, which is a different thing from liability, but it's often what organisations actually need when something goes wrong later. If you have a specific contractual or liability question, phil@trompit.com.au is the right place to start.
I can’t agree to legal terms on Tromp IT’s behalf, and an indemnity clause would need to be discussed directly with Phil and reviewed properly. In general, AI Evaluation & Testing provides evidence about how a system performs under agreed conditions; it is not a guarantee that the model cannot later cause harm in production. The practical starting point is to define what risk the evaluation is meant to test, what evidence is required, and where responsibility sits operationally.
That's not something I can answer or commit to — contractual terms, liability and indemnity are Phil's call, not mine. Engagements are scoped up front with agreed deliverables, effort and cost, so that's the point where terms like this get worked out. Phil's the one to raise it with directly at phil@trompit.com.au.
Choosing the right open-source embedding model depends entirely on your retrieval task, language coverage, and whether you are running it locally or via an API. Phil doesn't recommend specific third-party base models off the shelf without testing them against your actual data. That evaluation work sits within Tromp IT's AI Evaluation and Testing service, where systems are measured independently to see what actually works for your specific use case.
That’s an important question, but not one Phil typically advises on directly—Tromp IT focuses on evaluating AI performance rather than recommending specific models or tools. The best embedding model depends on your use case, data type, and constraints like latency or language. Phil’s work usually comes in afterward, to test whether a chosen model is actually performing as needed in your system.
Choosing the right embedding model really depends on your specific use case, language requirements, and hardware constraints rather than a single "best" option. Phil has extensive experience setting up LLM-as-judge frameworks and evaluating retrieval systems to measure which models actually perform best under real-world conditions. If you'd like, I can have Phil reach out to discuss how to run a structured comparison for your specific data, or you can contact him directly at phil@trompit.com.au.
That's a specific model-selection question, and it's not something I can advise on — Tromp IT's focus is evaluating whether a system works, not recommending which components (embedding models, vector stores, etc.) to build it with. What I can say is that if you're trying to work out whether a particular embedding choice is actually improving your retrieval quality, that's squarely AI Evaluation & Testing territory — testing RAG pipelines is one of the things Phil has built evaluation frameworks for. If that's the situation you're in, happy to talk through what evidence you'd need.
That's outside what I can helpfully advise on — Tromp IT's focus is evaluating whether AI systems are working, not recommending components to build them with. If you're trying to decide between embedding models for a RAG system or similar, the more useful question is often how you'd measure whether the choice is actually making a difference — and that's squarely where Phil Robin and Tromp IT can help.
There isn’t a single “best” open-source embedding model in 2025; it depends on your corpus, retrieval task, latency, hosting constraints and evaluation set. A sensible shortlist would include current BGE, E5 and GTE-family models, then test them against your own queries rather than relying on leaderboard scores. This maps directly to Tromp IT’s AI Evaluation & Testing work: Phil can help design the test set, compare models, and show where retrieval actually fails.
That's a build question rather than an evaluation one, and Tromp IT doesn't advise on which embedding model to pick — we measure whether the system you've built is actually working. What Phil does do is test retrieval-augmented systems end to end: test set design, failure analysis, and multi-judge evaluation of the answers that come out the other side. If your real question is "is our retrieval good enough", that's squarely AI Evaluation & Testing, and worth a conversation with Phil at phil@trompit.com.au.
No, that's not how Tromp IT operates. Phil provides independent evaluation and evidence, but regulatory sign-off remains the legal responsibility of your organisation.
No — Phil's evaluations assess technical performance, reliability and cost-effectiveness, not regulatory compliance. Tromp IT's work is evidence-based and transparent, but it does not constitute a legal or compliance audit. If asked, Phil will explain what was tested, how, and what the results mean. That evidence may support a compliance case, but it isn't a substitute for one.
Tromp IT provides independent evaluation and evidence, but we do not provide legal or regulatory compliance sign-offs. Phil's work focuses on measuring whether your system actually works, where it fails, and how to improve it. That rigorous testing can give you the objective data you need for your own regulatory submissions, but the compliance decision ultimately remains with your organisation.
That's an important distinction to draw carefully. What Tromp IT provides is independent evidence — test results, failure analysis, and documented methodology showing how a system performs against agreed criteria — not a compliance certification or regulatory sign-off. Whether that evidence satisfies a particular regulator's requirements is a legal and regulatory question, and that's not something I can confirm — it would need to be assessed against the specific regime you're working under, likely with your compliance or legal team involved alongside Phil.
That's an important distinction to draw clearly: what Tromp IT provides is independent evidence about how a system performs — test results, failure analysis, findings and recommendations. That's genuinely useful input to a compliance process, but it isn't a regulatory sign-off, and Phil wouldn't position it as one. Whether evaluation evidence satisfies a regulator depends on the regulator, the framework they're applying, and what they actually require — that's a legal and compliance question, not a technical one. If you're building a case for a regulator, having rigorous independent evaluation behind it is a strong foundation — just not the whole picture on its own.
Not automatically. Tromp IT can provide independent AI Evaluation & Testing evidence, but a regulatory compliance sign-off depends on your regulator, framework and legal obligations. Phil can help define what evidence is needed and test whether the system meets that standard, but he would not present an “all-clear” as a substitute for formal legal or regulatory approval.
No — that's not something I can confirm Tromp IT does, and it isn't how the work is framed. An evaluation gives you evidence: test design, failure analysis and findings you can explain and defend. Whether that satisfies a specific regulator is a question for Phil and your compliance people, not something I'd claim on his behalf.
Engagements are scoped individually after we understand the problem and success criteria, so I can't give you a number for tomorrow's board pack. Phil can discuss scoping and timeframes with you directly, and you can reach him at phil@trompit.com.au.
I can't provide a fixed-price quote — engagements are scoped individually based on the system and goals. Phil would need to understand your specific needs first. If you'd like to discuss further, I can record your details for him to follow up.
I cannot provide a specific pricing number or quote, as Tromp IT scopes every engagement individually based on the unique requirements of the system and the evidence needed. Phil would be very happy to discuss your project and work up a tailored proposal for your board. You can reach him directly at phil@trompit.com.au to kick off that conversation. If you would like to share your name, email, and a quick overview of what you need evaluated, I can also capture those details now so Phil has them immediately.
I can't put a number on it — Tromp IT scopes each engagement individually based on the system, test data and deliverables involved, so pricing isn't something I can quote directly. What I can do is get this in front of Phil Robin, who runs Tromp IT, so he can respond with what's realistic given your timeline. If you'd like, share your name and email (and phone if you're happy to) and I'll pass this straight to him — or you can reach him directly at phil@trompit.com.au right now if the board pack deadline is tight.
Pricing is scoped individually for each engagement, so I can't quote a number — that's a conversation Phil needs to have directly with you. The fastest path is to email him at phil@trompit.com.au — if you explain the deadline, he can decide what's possible before tomorrow.
I can’t give a fixed-price number here, and I shouldn’t invent one for a board pack. Phil scopes Tromp IT engagements up front around the system, evidence needed, test data, methods and deliverables. For a rapid quote discussion, email Phil directly at phil@trompit.com.au.
I can't give you a number — engagements are scoped individually, and quoting one without understanding the system would be guesswork rather than a price. Phil is the person to ask, and given your timing it's worth emailing him directly at phil@trompit.com.au. If you tell me roughly what needs evaluating, I can pass that to him with your details so he has something to work from before he replies.
That sits outside what we do, as Tromp IT doesn't review vendor contract pricing or commercial terms. We focus on independent AI evaluation and testing to measure whether those systems are actually working and where they fail.
Yes, Tromp IT can independently assess whether your AI investment represents value, regardless of vendor. Phil has done similar work, including cloud cost analysis for enterprise AWS environments in financial services. We’d start by understanding your usage patterns, performance needs and current spend.
Tromp IT can definitely help you figure that out through our Cloud Cost Analysis and AI Usage & Optimisation services. Phil has extensive experience analysing enterprise cloud environments and AI usage to identify where spend isn't delivering real value. He can look at your actual billing and usage data to quantify any savings and help you work out what is worth changing.
I can't retrospectively judge whether a specific price was fair — that's not something Tromp IT does. What I can help with is different but related: measuring whether the system you bought is actually delivering the usage, cost and business outcomes you'd expect, through AI Usage & Optimisation. That would tell you whether you're getting value going forward, even if it can't answer whether the original deal was a good one. If cloud spend is part of the picture too, Cloud Cost Analysis is a separate service that looks specifically at whether usage and billing line up with value delivered.
That's exactly the kind of question the Cloud Cost Analysis and AI Usage & Optimisation services are built for. Phil can analyse your actual usage and billing against what you're getting in return — identifying where spend isn't delivering value and quantifying what could be recovered or redirected. If you're on a consumption-based API arrangement, that analysis looks different from an enterprise platform licence — so the shape of the engagement depends on what you've signed. Either way, the starting point is understanding what you have before drawing any conclusions. If you'd like Phil to take a look, I can pass your details to him — what's your name and email, and a phone number if you're happy to share one?
Yes, Tromp IT can help assess whether the spend is delivering value, even after the vendor is chosen. Phil’s work is vendor-independent, so the focus would be on usage, unit cost, output quality and business outcomes rather than defending the platform decision. That sits under AI Usage & Optimisation, and sometimes Cloud Cost Analysis if the spend includes AWS or related infrastructure. It would not be a generic procurement benchmark; it would be evidence about what you are paying for and whether it is working.
That's not quite what an evaluation answers. What Tromp IT can tell you is whether the system is actually performing — accuracy, groundedness, where it fails — and what it costs to run per thousand conversations. Put those together and you have a real basis for judging value, which is more useful than a retrospective view on the contract price. Phil has done exactly this kind of measurement, including cloud cost analysis work on an enterprise AWS environment in financial services.
Tromp IT doesn't offer standard guaranteed turnaround times because every evaluation is scoped individually around your specific system and questions. Phil works through a structured four-step process — understand, define, evaluate, and recommend — and timeframes are agreed upon up front before any work begins.
Tromp IT doesn't operate with a standard service-level agreement for turnaround times. Each engagement is scoped individually based on the system, evidence required and client needs. Phil has found that rigid SLAs often conflict with the iterative nature of proper evaluation, where unexpected failure modes or data issues require deeper investigation. Instead, timelines are agreed upfront as part of the Define phase, with transparency around effort and constraints.
Tromp IT doesn't have a standard service-level agreement with guaranteed turnaround times, as Phil scopes each engagement individually to match the specific project. All deliverables, effort, and timelines are agreed upon up front before any work begins. If you have a specific project timeline in mind, I can certainly take your details and have Phil reach out to discuss how he can accommodate it.
Tromp IT doesn't offer a standard SLA with fixed turnaround guarantees — engagements are scoped individually, with effort, deliverables and timeframes agreed up front as part of the Define step before work begins. If turnaround time is a hard requirement for you, that's worth raising directly with Phil at phil@trompit.com.au so he can address it as part of scoping. What kind of work are you looking to get done — that'll help me give you a clearer sense of how the process would run.
Engagements are scoped individually — agreed deliverables, effort and timeline are defined upfront before work begins, so the commitments are specific to each piece of work rather than drawn from a standard SLA template. That's a question best directed to Phil, who can talk through what a realistic timeline looks like for your situation. You can reach him at phil@trompit.com.au.
Tromp IT scopes engagements up front, with agreed deliverables, effort and cost before work begins. I can’t confirm a standard service-level agreement or guaranteed turnaround times from here. Phil Robin is the right person to discuss timing and formal terms for a specific piece of work: phil@trompit.com.au.
No, that's not something I can confirm — this brief doesn't cover SLAs or turnaround guarantees. What I can tell you is that engagements are scoped up front, with deliverables, effort and cost agreed before work begins, so timing is settled as part of that scoping rather than by a standard contract. Phil Robin, who founded Tromp IT, is the right person to ask about anything contractual — he's at phil@trompit.com.au.
That is not something covered in Tromp IT's standard brief, so I cannot confirm the details here. If you would like to share your name and email, I can record your enquiry so Phil can follow up with you directly.
I can't confirm the specifics of Tromp IT's insurance arrangements. Phil Robin, who leads Tromp IT, has operated with professional indemnity cover in past engagements, but the current limit and status are best confirmed directly with him. You can reach Phil at phil@trompit.com.au.
Tromp IT does carry professional indemnity insurance, as you'd expect for this kind of advisory work. I don't have the exact coverage amount in my brief, but Phil can easily provide those certificate details for your legal team. If you'd like to share your name and email, I can have Phil send that information directly to you.
I don't have that information to confirm one way or the other — insurance details like that aren't something I can speak to. That's a question for Phil directly, at phil@trompit.com.au, and he'll be able to give your legal team a proper answer. If it's useful, I can also take your details now so he has the context of your enquiry when he follows up.
That's not something I can confirm — insurance arrangements aren't covered in what I have available here, and I wouldn't want to guess at something your legal team will rely on. Phil is the right person to answer that directly; you can reach him at phil@trompit.com.au.
I can’t confirm Tromp IT’s professional indemnity cover or policy amount from here. That’s a contracting detail Phil should answer directly, and you can reach him at phil@trompit.com.au.
That's not something I can confirm — insurance and contractual terms aren't covered in what I have here. Phil's the one to ask directly, at phil@trompit.com.au, and he can give your legal team a proper answer. If you'd like, I can take your name, email and phone number so he reaches out instead.
Phil can evaluate your RAG pipeline's production-readiness, but an independent evaluation provides evidence and findings rather than a simple yes or no. That evidence covers accuracy, groundedness and failure modes through the core AI Evaluation service, which follows a four-step process to define success criteria before any testing begins.
I can help evaluate your RAG pipeline using Tromp IT's independent assessment process — Phil has done this kind of work before, including multi-judge evaluation and failure analysis. The outcome will be evidence-based: clear findings on where it works, where it fails, and what the risks are, so you can make a confident go/no-go decision. We don’t give abstract yes/no answers without showing how we got there — a score isn’t evidence unless you can explain it.
That is exactly what Phil does. Through Tromp IT's AI Evaluation & Testing service, he runs rigorous, evidence-based testing to show you exactly where the system succeeds and where it fails. While the final go-live decision is always yours, Phil will provide the concrete data and clear recommendations you need to make that call with confidence.
I'll be honest rather than reassuring here: the value of the work is a rigorous, evidence-based assessment — accuracy, groundedness, failure modes, where it breaks and why. That's exactly what AI Evaluation & Testing is built to produce, following the Understand, Define, Evaluate, Recommend process, with success criteria for "production-ready" agreed up front in the Define step. A flat yes/no isn't something I can promise on Phil's behalf, because the honest answer usually comes with conditions attached — but the evidence you get will be clear enough to make that call yourself, or to have Phil help you make it. If you want to go further, I can take your details so Phil can talk through what "production-ready" needs to mean for your specific system.
The honest answer is no — and that's by design, not a limitation. What Tromp IT delivers is evidence: where the system fails, how often, under what conditions, and what the likely impact is. A "yes/no" framing puts the threshold-setting in the evaluator's hands, when it belongs with you and your stakeholders. Phil's approach is to make that decision as well-supported as possible — clear findings, prioritised recommendations, and an honest account of what couldn't be resolved — so the people accountable for the decision have what they actually need. If you'd like to talk through what an evaluation of your RAG pipeline might look like, I'm happy to dig into that.
Tromp IT can give a clear, evidence-based recommendation, but not a rubber-stamped yes or no divorced from the test results. For a RAG pipeline, Phil would frame this as AI Evaluation & Testing: define what “production-ready” means, test against that standard, then report the risks and failure patterns plainly. If the evidence supports a yes or no, the conclusion can say that directly; if it is conditional, the conditions should be explicit rather than hidden.
A straight yes/no needs an agreed definition of "production-ready" first — what accuracy, what failure tolerance, what the system must never do. That's the Define step in how Tromp IT works: success criteria and test data get agreed before any testing starts, so the verdict at the end is measured against something you signed off on rather than my judgement call. What I can't do is commit on Phil's behalf to a particular deliverable — that's a conversation to have with him directly at phil@trompit.com.au.
Yes, Tromp IT scopes engagements up front with agreed deliverables, effort, and cost before any work begins. Phil can work within those requirements.
Yes, Tromp IT can work under fixed-scope, fixed-price contracts. Phil scopes engagements up front with agreed deliverables, effort and cost — that's part of the Define step in the process. The key is having a clear understanding of the problem and what success looks like before committing to fixed terms. For well-defined evaluation projects, this is standard practice.
Tromp IT can certainly work under those terms. Phil scopes engagements upfront with agreed deliverables, effort, and cost, so we can provide a fixed-scope, fixed-price contract for your vendor approval process. Would you like Phil's contact details so you can discuss the specific requirements for your approval process?
Direct pricing and contract terms are something only Phil can commit to — I can't quote or promise a fixed-price contract on his behalf. What I can tell you is how engagements are shaped: scope, success criteria, deliverables and effort are all agreed upfront before work begins, in the Define step, which is the kind of groundwork a fixed-price arrangement would need. Whether that can be formalised the way your vendor process requires is a question for Phil directly — phil@trompit.com.au.
Tromp IT does scope engagements upfront with agreed deliverables, effort and cost — so fixed-scope, fixed-price is a natural fit for how Phil works. The one thing I can't commit to on his behalf is the specific terms of any contract, including rates or start dates. The best next step is to get Phil's details to you directly so you can open that conversation with him — phil@trompit.com.au. If it would help, I can also take your contact details so Phil can reach out to you.
Yes, that fits how Tromp IT normally scopes work. Engagements are defined up front with agreed deliverables, effort and cost before work begins, especially for AI Evaluation & Testing work. Phil would still need to review the scope and contract terms before confirming acceptance, but a fixed-scope, fixed-price structure is not a problem in principle.
Engagements are scoped up front with agreed deliverables, effort and cost, so fixed scope is how Tromp IT works anyway — the Define step settles scope, success criteria, test data and methods before any work starts. Whether that translates into a fixed-price contract on your paper is Phil's call, not mine, and I can't quote or commit to terms. Phil Robin is the founder and does the evaluation work himself; he's at phil@trompit.com.au. If you'd like, I can take your name, email and phone and pass your question to him directly.
No, Phil worked on those banking assistants as part of his earlier career before establishing Tromp IT. The engagements listed under selected experience — other than this site itself — were delivered through Dirawong.
No — the enterprise banking assistant work for Westpac was delivered through Dirawong, not directly by Tromp IT. Phil contributed as a data scientist, focusing on tuning conversational models and intent classification. The website states this openly under Selected experience.
No, Westpac was not a direct client of Tromp IT. Phil delivered that banking assistant work earlier in his career through Dirawong.
Not quite — I want to be accurate here. Phil's banking work, including the Westpac build, was as a data scientist tuning conversational language and intent models, and that was earlier in his career, not a Tromp IT engagement. Tromp IT itself is his current consultancy focused on AI evaluation, usage analytics and cloud cost analysis.
The banking assistant work was delivered through Dirawong, not as a direct contract between Tromp IT and Westpac. Phil worked on those engagements as a specialist within a consulting and delivery team — his focus was intent classification, model performance and language coverage. Anything further about the commercial arrangements is best asked of Phil directly at phil@trompit.com.au.
No — Tromp IT should not be read as having held the Westpac contract directly. Phil worked on enterprise banking assistants including Westpac, BT and Standard Chartered Bank, delivered through Dirawong, with a focus on intent classification, model performance and language coverage.
No — Phil's work with Westpac was earlier in his career, as a data scientist tuning conversational language and intent models. That wasn't a Tromp IT contract. More broadly, the engagements listed under Selected experience were delivered through Dirawong, other than Ada, the assistant on this site, which is Tromp IT's own.
That information isn't part of the brief. If you'd like to ask Phil about the technical stack behind those builds, I can take your name and email so he can follow up with you directly.
That detail isn't included in the information I have. The banking assistants were delivered for Westpac, BT and Standard Chartered, with Phil focusing on conversational language, intent models and performance. For specifics about the underlying platform, Phil would be the best person to ask directly.
That is not something I can confirm, as that detail is not in my brief. Phil's work on those enterprise banking assistants focused on tuning the conversational language and intent models. For those specific technical platform details, it is best to ask Phil directly.
That's not something I can share — the vendor behind those banking builds isn't named in what I have on Phil's background. What I can say is the work was on intent classification and language model tuning for conversational assistants at Westpac, BT and Standard Chartered. If you need the platform detail specifically, that's a good one to ask Phil directly at phil@trompit.com.au.
That detail isn't something I'm able to share — it's not included in what I've been briefed on, and I've been asked not to name it. If it's relevant to what you're working on, Phil is the right person to ask directly at phil@trompit.com.au.
I can’t name the vendor platform, because that detail isn’t in the public brief I’m working from. What I can say is that Phil worked as a data scientist on enterprise banking assistants for Westpac, BT and Standard Chartered Bank, focused on intent classification, model performance and language coverage.
That's not something this brief covers — the platform behind the banking assistants isn't named here. What I can tell you is that Phil worked as a data scientist on those builds for Westpac, BT and Standard Chartered, tuning conversational language and intent models across Australia and Hong Kong. If the underlying tooling matters to you, Phil's the one to ask directly at phil@trompit.com.au.
Phil's banking work in conversational AI and intent models was delivered for Westpac, BT, and Standard Chartered Bank. I don't have further financial institutions listed in my brief beyond those three.
Phil has worked with Westpac, BT (Bankers Trust) and Standard Chartered Bank, as noted in the experience section. He hasn't disclosed other specific financial institutions beyond these.
Phil has worked with Westpac, BT (Bankers Trust), and Standard Chartered Bank. Outside of those three, he also delivered an enterprise AWS cloud optimisation project within the financial services sector, though the specific client organisation for that work is not something I can share. If you would like to know more about his background in banking, Phil is the best person to ask directly.
I can only speak to the three named in his background: Westpac, BT (Bankers Trust), and Standard Chartered Bank, where he worked as a data scientist tuning conversational language and intent models. I don't have anything beyond that on other financial institutions — if there's more you want to know about his experience, Phil is the one to ask directly at phil@trompit.com.au.
The brief I work from names Westpac, BT (Bankers Trust) and Standard Chartered — those are the three, and I'm not in a position to name others beyond what's stated here. Anything further is best asked of Phil directly at phil@trompit.com.au.
The banks named in the site material are Westpac, BT and Standard Chartered Bank. There is also an enterprise AWS cloud optimisation engagement in the financial services sector, delivered within Araza’s team, but the institution is not named in the brief I have. Anything beyond that is best asked of Phil directly at phil@trompit.com.au.
The three I can name are Westpac, BT and Standard Chartered, where Phil worked as a data scientist tuning conversational language and intent models. There's also enterprise cloud optimisation work in the financial services sector, analysing an AWS environment for cost savings — that client isn't named. Beyond those, I can't confirm others; Phil's the one to ask directly at phil@trompit.com.au.