30 questions, each answered by 7 models and scored by 4 independent judges. Expand a question to read every answer and every score. Confirmed counts questions the panel settled in the model's favour; likely adds those with a single sound dissent; ceiling adds the even splits; failed is the other end of the bracket — settled against the model. Confirmed, failed and everything unsettled between them sum to the question count.
| Model | Questions | Confirmed | Likely | Ceiling | Breadth | Failed | Accuracy | Groundedness |
|---|---|---|---|---|---|---|---|---|
| Claude Opus 5 | 30 | 27 90% | 29 97% | 30 100% | 3 | 0 0% | 98.3% | 97.9% |
| GPT-5.5 | 30 | 22 73% | 30 100% | 30 100% | 8 | 0 0% | 95.0% | 94.2% |
| Claude Sonnet 5 | 30 | 21 70% | 28 93% | 30 100% | 9 | 0 0% | 95.0% | 92.9% |
| Claude Sonnet 4.6 | 30 | 17 57% | 26 87% | 30 100% | 13 | 0 0% | 91.7% | 89.2% |
| Gemini 3.5 Flash-Lite | 30 | 11 37% | 19 63% | 26 87% | 15 | 4 13% | 78.3% | 79.2% |
| Qwen 3 235B | 30 | 9 30% | 19 63% | 26 87% | 17 | 4 13% | 75.8% | 75.8% |
| Gemini 3.5 Flash | 30 | 3 10% | 22 73% | 28 93% | 25 | 2 7% | 75.0% | 75.4% |
Each row shows how every model fared. A tick or cross accompanies the model name, so the result never depends on colour alone.
Tromp IT is an independent AI evaluation and analytics consultancy founded by Phil Robin. He helps organisations measure whether their AI systems actually work, what they cost, and where they fail. The core service is independent AI evaluation and testing for chatbots, RAG systems and agentic AI.
Tromp IT, founded by Phil Robin, provides independent AI evaluation and analytics, focusing on whether AI systems like chatbots and RAG applications actually work as intended. We assess performance, cost, and failure points using methods like test set design, multi-judge evaluation, and LLM-as-judge frameworks—platform-agnostically, without selling any tools. Phil brings over 30 years of experience in technology and analytics, with recent work including the evaluation of Ada (the assistant you're using) across multiple models.
Tromp IT is an independent consultancy based in Melbourne that specializes in AI evaluation, helping organisations measure whether their AI systems are actually working and where they fail. Our founder, Phil Robin, has over 30 years of experience and also helps clients with AI usage and optimisation, data analytics, and cloud cost analysis. We are completely vendor-independent, meaning we don't sell AI platforms—we just measure them. How can I help you today?
Tromp IT is an independent AI evaluation and analytics consultancy — we don't sell AI platforms, we measure whether they actually work. That covers evaluating chatbots, RAG systems and agentic AI, tracking AI usage and cost, making sense of complex operational data, and finding where cloud spend isn't delivering value. The founder, Phil Robin, has 30+ years in tech and analytics, and has built things like a medical triage chatbot for Medibank and conversational testing platforms. If you've got a specific system or problem in mind, I can tell you more about how we'd approach it.
Tromp IT is an independent AI evaluation and analytics consultancy based in Melbourne, founded by Phil Robin, who brings more than 30 years of technology and analytics experience. The core focus is helping organisations understand whether their AI systems are actually working — measuring quality, cost and failure modes rather than just accepting a score at face value. Phil also works on broader analytics, conversational AI and cloud cost analysis. Is there a particular problem you're trying to solve?
Tromp IT helps organisations work out whether their AI systems and analytics are actually delivering value. Phil Robin, who founded Tromp IT, focuses on independent AI evaluation, conversational AI and analytics, especially testing chatbots, RAG systems and agentic AI. Tromp IT is vendor-independent, so it does not sell platforms; it measures how well systems work, where they fail, and what they cost.
Tromp IT is an independent AI evaluation and analytics consultancy in Melbourne, founded by Phil Robin. The core work is evaluating whether AI systems — chatbots, RAG pipelines, agentic systems — actually perform as claimed, using test set design, failure analysis and multi-judge evaluation frameworks. Alongside that, Phil does AI usage and cost optimisation, analytics and decision support, and cloud cost analysis. The positioning matters: Tromp IT sells no AI platforms, so the measurement is genuinely independent.
AI evaluation is the independent testing of systems like chatbots, RAG pipelines, and agents to check whether they actually work, how much they cost, and where they fail. Phil Robin focuses on this work at Tromp IT, using methods like test set design, failure analysis, and LLM-as-judge frameworks. It starts with understanding your specific problem, then defining the right test data and success criteria before running the evaluation.
AI evaluation measures whether AI systems like chatbots or RAG pipelines actually work as intended. It involves testing accuracy, reliability, cost and business impact using structured methods. Phil at Tromp IT focuses on independent evaluation — assessing performance regardless of platform, with transparent, explainable results. This follows a clear process: Understand, Define, Evaluate, Recommend.
AI evaluation is the independent testing of AI systems—like chatbots or search pipelines—to see if they actually work as intended. It involves measuring things like accuracy, cost, and how often the AI hallucinates or fails. Phil Robin, the founder of Tromp IT, specializes in designing these testing frameworks so organizations can understand their AI's real-world performance.
AI evaluation is testing whether an AI system actually does what it's supposed to — accurately, reliably, and at a cost that makes sense — rather than just assuming it works because it launched. It covers things like designing test sets, analysing where the system fails, and using structured methods (including LLM-as-judge frameworks) to score responses at scale. This is the core of what Tromp IT does. Phil Robin runs it through four steps — Understand, Define, Evaluate, Recommend — starting with what "working" actually means for your system before any testing begins.
AI evaluation is the practice of systematically testing whether an AI system actually does what it's supposed to do — not just in the lab, but in ways that reflect real use. That means designing test sets, analysing failures, and producing evidence you can act on rather than a score you have to take on trust. Phil Robin, who founded Tromp IT, has done this kind of work across chatbots, RAG systems and conversational AI — including a full published evaluation of Ada, the assistant you're talking to right now, where seven model backends were tested against 189 questions and scored by a four-model judge panel. If you want to see what rigorous evaluation actually looks like in practice, that comparison is linked from the site.
AI evaluation is the structured testing of an AI system to see whether it is doing the job it is meant to do, where it fails, and what those failures cost or imply. For chatbots, RAG systems and agents, that can mean designing test sets, checking answers against evidence, analysing failure patterns, and comparing models. Phil Robin’s current work at Tromp IT focuses on this kind of independent AI Evaluation & Testing, rather than selling the platforms being tested.
AI evaluation is the practice of testing whether an AI system actually does what you need it to — measuring accuracy, groundedness, failure modes and cost, rather than trusting a vendor's demo or a single headline score. It applies to chatbots, RAG pipelines and agentic systems, and the hard part is usually designing test sets and judging criteria that reflect what "working" means for your particular use case. That's Tromp IT's core service. Phil Robin, who founded it, has built evaluation frameworks including multi-judge and LLM-as-judge approaches, and published a full comparison run using Ada — the assistant you're talking to now — across seven model backends. It's linked from the site if you'd like to see what the output looks like.
Yes, Tromp IT's AI evaluation and testing work is platform-independent, so it can assess systems built on Azure, AWS, OpenAI, Anthropic, or any other platform. Phil Robin handles this through independent evaluation and failure analysis rather than tied vendor tooling.
Yes, Tromp IT evaluates AI systems regardless of platform, including those built on Azure, AWS, OpenAI, Anthropic, or others. Phil has extensive experience working across cloud environments, with a strong focus on independent assessment of performance, cost, and reliability. The evaluation approach is vendor-agnostic — the goal is to measure what matters, not promote any tool.
Yes, Tromp IT is completely platform-independent and can evaluate AI applications built on Azure. Phil Robin, our founder, regularly designs evaluation frameworks that assess system performance regardless of the underlying cloud provider. Are you currently running a chatbot or a RAG system on Azure that you need tested?
Yes — AI Evaluation & Testing at Tromp IT is platform-independent, so systems built on Azure are squarely in scope, alongside AWS, OpenAI or Anthropic-based builds. What matters is designing the right test set and evaluation method for your system, not which cloud it sits on. If you've got an Azure-based chatbot or RAG pipeline in mind, happy to talk through what evaluating it would look like.
Yes — AI Evaluation & Testing at Tromp IT is platform-independent, so Azure-hosted systems are fair game. Phil can assess chatbots, RAG pipelines or agentic AI regardless of whether they're built on Azure, AWS, OpenAI, Anthropic or another platform. What kind of system are you looking to evaluate?
Yes. Tromp IT’s AI evaluation work is platform-independent, so Phil can assess systems built on Azure as well as AWS, OpenAI, Anthropic or other platforms. The focus is not selling or implementing the platform, but testing whether the AI system is working and where it fails.
Yes — AI evaluation is platform-independent, so we can assess applications built on Azure, AWS, OpenAI, Anthropic or anything else. What matters is the system's behaviour and outputs, not the stack underneath it. One caveat worth being straight about: for cloud cost analysis specifically, Phil's experience is primarily AWS.
Yes, independent evaluation of agentic AI is one of Phil's core services, covering multi-step reasoning, tool use and autonomous workflows. That falls under AI Evaluation and Testing, where the focus is on whether these systems actually work in practice. We follow a four-step process of understanding the problem, defining test criteria, evaluating performance, and recommending improvements.
Yes, Tromp IT evaluates agentic AI systems as part of our independent AI Evaluation & Testing service. We assess how well these systems perform, where they fail, and what they cost to run. Phil has worked on evaluation frameworks for agentic systems through his role in shaping Dirawong's Hoot platform, and applies that expertise directly with clients.
Yes, we do evaluate agentic AI systems. Our core service is AI Evaluation & Testing, where Phil Robin designs independent frameworks to measure how these complex agents perform, where they fail, and what they actually cost. Since agentic systems can be unpredictable, we focus on rigorous, platform-independent testing to see how they handle real-world scenarios. What kind of agentic system are you currently building or running?
Yes — agentic AI is explicitly one of the system types Tromp IT evaluates, alongside chatbots and RAG pipelines. That work follows the same four-step process: understand what "working" means for your specific agent, define scope and success criteria upfront, evaluate against test cases including failure analysis, then recommend fixes. Phil's own published comparison — using Ada, the assistant you're talking to now — is a live example of this kind of evaluation, testing seven model backends with a multi-judge panel and code-level checks on tool calls, which is directly relevant if your agentic system also makes tool calls that need verifying against what actually happened.
Yes — agentic AI is explicitly part of what Tromp IT evaluates. The work covers not just whether the system gives correct answers, but whether tool calls and multi-step actions actually do what the runtime says they do. Phil built that approach into the Ada evaluation — every tool call was checked in code against what the system actually did, not just what the reply claimed. That kind of verification matters more for agentic systems, where a wrong action can have real consequences, not just a wrong word.
Yes — agentic AI evaluation is part of Tromp IT’s AI Evaluation & Testing work. Phil Robin has worked on multi-judge evaluation and LLM-as-judge approaches for LLM-powered chatbots, RAG pipelines and agentic systems, including through Dirawong’s Hoot platform. The useful starting point is defining what the agent is allowed to do, what “good” looks like, and how failures should be detected.
Yes — agentic AI is explicitly part of Tromp IT's AI Evaluation & Testing work, alongside chatbots and RAG pipelines. The interesting part with agents is usually that the reply and the actual behaviour diverge, so checking tool calls against what the runtime really did matters more than judging the text. Phil did exactly that when building Ada, the assistant you're talking to now — every tool call was verified in code, not taken from the model's word for it.
Yes, data visualisation is part of our Analytics & Decision Support service. Phil helps make sense of complex operational and conversational data through dashboards and visualisations. If you have a specific visualisation challenge in mind, feel free to share a few details.
Yes, Tromp IT provides data visualisation as part of its Analytics & Decision Support service. Phil designs dashboards and visualisations to help organisations understand complex operational and conversational data. This includes everything from data cleansing and pipeline design to final visual output.
Yes, we do. Phil helps organisations make sense of complex operational and conversational data through custom dashboards and visualisation as part of our Analytics & Decision Support service. Is there a particular dataset or system you are looking to build visualisations for?
Yes — that falls under Analytics & Decision Support, one of Tromp IT's four services. It covers everything from ETL pipeline design and data cleansing through to dashboards and visualisation, with the aim of making complex operational or conversational data understandable enough to act on. If you've got a specific dataset or reporting problem in mind, I can give you a better sense of fit.
Analytics and visualisation is part of what Tromp IT offers — Phil's Analytics & Decision Support service covers everything from ETL pipeline design and data cleansing through to dashboards and visualisation. The focus is on making sense of complex operational or conversational data so people can see what's happening and act on it. If you want to tell me more about what you're working with, I can give you a better sense of whether it's a good fit.
Yes. Tromp IT does analytics and decision support work, including dashboards and visualisation for operational and conversational data. Phil Robin has a long analytics background, and the usual aim is to make complex data clear enough for people to decide what to do next.
Yes — that sits under Analytics & Decision Support. It covers the whole path from ETL pipeline design and data cleansing through to dashboards and visualisation, the idea being that people can actually see what's happening and decide what to do about it. Phil has done this kind of work for years, including building the data models that measured how Telstra's voice recognition systems performed with real callers in production.
We evaluate RAG systems in production just as readily as pre-deployment builds. Phil's work involves checking how they actually perform when real users are querying them, using independent test sets and failure analysis.
Yes, Tromp IT can evaluate RAG pipelines in production as well as pre-deployment. Phil has assessed live systems using methods like test set design, failure analysis and multi-judge evaluation to measure accuracy, groundedness and cost. The approach adapts to whether the system is staged or live, with data handled according to your security and residency requirements.
Yes, Tromp IT evaluates systems that are already in production as well as those in development. Phil Robin, our founder, has designed evaluations for live systems to measure how they perform with real production data rather than just lab results. This work falls under our AI Evaluation & Testing service, where we look at what is actually happening when users interact with your RAG pipeline.
Yes — production RAG pipelines are exactly what AI Evaluation & Testing covers, not just pre-deployment ones. The process starts with Understand and Define, agreeing what "working" means for your system and what test data and success criteria make sense, before any evaluation runs against it. Phil's built comparable evaluation work using multi-judge frameworks and failure analysis — Ada, the assistant you're talking to now, is a published example of that same approach applied to a live system.
Production RAG evaluation is actually where it tends to matter most — by then you have real traffic, real failures, and real cost data to work with. Phil's AI Evaluation & Testing service covers both pre-deployment and live systems, and the approach adapts to whichever stage you're at. For a production system, the evaluation would typically draw on existing conversation logs alongside a designed test set, looking at retrieval quality, answer groundedness, and where the system fails under real conditions — not just controlled ones. If you want to share a bit about your setup, I can give you a better sense of what that engagement might look like.
Yes — Tromp IT can evaluate RAG systems that are already in production, not just pre-deployment builds. Phil Robin’s work is about measuring what systems actually do, so production behaviour, failure patterns and cost are often the evidence that matters most. This fits Tromp IT’s AI Evaluation & Testing service, with scope agreed up front around the access, data and constraints of the live environment.
Both — evaluating a live production system is normal work, not a special case. That's AI Evaluation & Testing, and with a RAG pipeline it usually starts with what "correct" means for your retrieval and your answers, then building a test set that reflects real queries rather than tidy ones. Phil did exactly this kind of measurement at Telstra, building the data models that showed how voice recognition performed with real callers in production rather than in the lab.
Running standard benchmarks is only a small part of the picture, because public tests rarely reflect what your own users will actually do. True evaluation means designing tests around your specific system, your prompts, and the failures that matter to your business. Phil focuses heavily on that custom test design through Tromp IT's AI Evaluation and Testing service, including failure analysis and LLM-as-judge frameworks.
AI evaluation goes well beyond simple benchmarks — it’s about designing meaningful tests that reflect real-world usage, then analysing where and why systems fail. Phil has led this kind of rigorous evaluation for organisations like Medibank and Telstra, focusing on actual performance, cost, and business impact. The process starts by defining what success looks like for your specific system, not applying generic scores.
There is much more to it than just running standard benchmarks. Real-world AI evaluation requires designing custom test sets, analyzing specific failure modes, and using multi-judge frameworks to see how your system performs in production. Phil Robin, the founder of Tromp IT, specializes in setting up this kind of independent testing so you can understand exactly where your system fails and what it costs. We offer this through our core AI Evaluation & Testing service, which is entirely platform-independent.
There's a lot more to it. Public benchmarks tell you how a model performs on generic tasks, not how your specific chatbot or RAG system behaves on your real questions, your data, your edge cases — that requires custom test set design, failure analysis, and often a multi-judge framework rather than a single score you have to trust blindly. That's the core of what Tromp IT does under AI Evaluation & Testing: starting with what "working" actually means for your system (Understand), agreeing scope and test data up front (Define), then running the evaluation and digging into why things fail rather than just flagging that they did. Phil built a good example of this — a published comparison running one 189-question set against seven model backends, judged independently by a four-model panel, with every tool call checked against what actually happened rather than what the reply claimed.
Benchmarks tell you how a system performs on a fixed, public dataset — which is useful, but it's a long way from knowing whether your specific system works for your specific users. The harder questions are about failure modes that benchmarks weren't designed to find: where does the system confabulate, where does retrieval go wrong, where does it behave differently under paraphrase? Phil's approach starts with what "performing well" actually means for the system under test, then designs the test set and evaluation methods around that — not the other way around. The published evaluation of Ada (the assistant you're talking to right now) is a worked example: 189 questions, seven model backends, a four-model judge panel, and tool-call verification done in code rather than taken from the reply text.
Benchmarks can be useful, but they are only one part of AI evaluation. For a real chatbot, RAG system or agent, the harder question is whether it behaves correctly in the situations that matter to your organisation. Phil Robin’s work is focused on that evidence layer: designing test sets, analysing failures, comparing models and explaining how the score was produced.
Benchmarks tell you how a model scores in general — they don't tell you whether your system works for your users. Real evaluation starts with what "working" means for your use case, then builds test sets around it, analyses where it fails, and checks whether the judging itself holds up. That's the core of what Phil does at Tromp IT: independent evaluation of chatbots, RAG and agentic systems, with the reasoning behind each score made visible rather than taken on trust.
Tromp IT's evaluation work is platform-independent, meaning we can test and assess AI systems regardless of whether they run on AWS, Azure, Google Cloud, or direct APIs. If you'd like to talk about your specific setup, you can reach Phil directly at phil@trompit.com.au.
Tromp IT works platform-independently, so Google Cloud is not a barrier. Phil has evaluated AI systems across AWS, Azure, OpenAI, Anthropic and others, and focuses on whether the AI is working effectively, regardless of the underlying infrastructure.
Yes, absolutely. Phil can evaluate AI applications regardless of whether they are built on Google Cloud, AWS, Azure, or another platform. AI evaluation is completely platform-independent.
Yes — AI evaluation at Tromp IT is platform-independent, so a system built on Google Cloud is fine to assess. The one caveat is on the cloud cost side: that experience is primarily with AWS, so if it's GCP billing and spend optimisation you're after, that's less certain territory. For evaluating the AI system itself — chatbot, RAG, agentic — the platform underneath doesn't change how the work is scoped.
Tromp IT's documented cloud cost analysis experience is primarily AWS, so Phil would be the right person to confirm whether a Google Cloud engagement is something he'd take on — that's not something I can confirm from here. If you'd like to describe what you're trying to understand about your GCP spend, I can at least tell you whether the problem sounds like a fit for the kind of analysis Tromp IT does.
Yes, for AI evaluation work Tromp IT is platform-independent, so Google Cloud would not be a barrier. Phil’s focus is on measuring whether the AI system is working, what evidence supports that, and where it fails — not on selling or favouring a particular platform. For cloud cost analysis, the strongest stated experience is primarily AWS, so Google Cloud cost work would be one to discuss with Phil directly.
Yes for evaluation — AI Evaluation & Testing is platform-independent, so it doesn't matter whether the system runs on Google Cloud, AWS, Azure, OpenAI or anything else. Phil's evaluation work has covered chatbots, RAG pipelines and agentic systems built on a range of stacks. The one caveat is cloud cost analysis, which is primarily AWS experience — if that's what you're after, Phil is the person to ask.
Prompt engineering as a standalone service sits outside what Tromp IT does, but testing how prompts perform in production is right in our wheelhouse. Phil focuses on independently evaluating whether an AI system actually works through rigorous testing and failure analysis rather than tuning individual prompts. If you have an existing system and need to measure its outputs and where it fails, that falls under AI Evaluation and Testing.
Prompt engineering isn't a core service at Tromp IT. Phil focuses on evaluating and testing AI systems — including how well they perform with existing prompts — rather than crafting prompts themselves. If you're looking for evaluation of prompt effectiveness, that fits within AI Evaluation & Testing.
Tromp IT's main focus is on AI Evaluation & Testing rather than prompt engineering as a standalone service. However, Phil does work with prompt design and LLM-as-judge frameworks when building evaluations to measure how systems perform. If you are looking for help with a specific system, what kind of AI application are you currently working on?
Prompt engineering itself — writing or tuning prompts — isn't something I can confirm Tromp IT does. What Tromp IT does is independent evaluation: testing whether a chatbot, RAG system or agent is actually working, where it fails, and what that costs, regardless of who wrote the prompts or built the system. So if you've got prompts or a system in place and want to know how well it's actually performing, that's squarely AI Evaluation & Testing — Phil would start by defining what "working" means for your case before any testing happens. If it's really the prompt-writing itself you need help with, that's better directed elsewhere or straight to Phil to ask.
Prompt engineering isn't a service Tromp IT offers directly — it's not listed, and I won't stretch the scope to fit it in. Where there's a natural overlap is in evaluation: if you've built a system and want to know whether your prompts are actually producing reliable, accurate outputs, that's exactly what Phil's AI Evaluation & Testing work covers. Measuring whether a system works is a different discipline from building it, and that's where Tromp IT focuses. If you're trying to solve a specific problem, I'm happy to hear more — there may be a genuine fit on the measurement side.
Yes, but usually through an evaluation lens rather than as standalone “prompt writing.” Phil’s work is about testing whether an AI system is producing reliable, useful outputs, then identifying where prompts, retrieval, model choice or workflow design are contributing to failures. That fits Tromp IT’s AI Evaluation & Testing service, especially for chatbots, RAG systems and agentic AI.
Prompt engineering isn't something I can confirm as a Tromp IT service — it's not in what's written for me. It sits adjacent to the work, though: Phil's core focus is independent AI evaluation and testing, which tells you where a system is failing and why, and that often points straight at the prompt. Phil's the one to ask if you want a straight answer on scope — he's at phil@trompit.com.au.
We certainly do. Phil provides independent AI evaluation and testing for custom chatbots, RAG pipelines, and agentic systems, regardless of whether they run on OpenAI, Anthropic, AWS, or Azure. We start by understanding your problem and the evidence you need, then design a structured evaluation to measure how the model performs.
Yes, Tromp IT can independently evaluate chatbots built on third-party LLM APIs, regardless of the provider. Phil has done this kind of work before, including testing and failure analysis for enterprise systems. The evaluation focuses on accuracy, groundedness, cost, and where the system fails — all while respecting your data controls.
record_lead
Yes, we can evaluate exactly that kind of system. Our core service is AI Evaluation & Testing, which is completely platform-independent and covers API-based setups. Phil has extensive experience assessing chatbot accuracy and failure modes, including building and testing the very assistant you are talking to right now.
Yes — that's squarely what AI Evaluation & Testing covers. Whether it's built on OpenAI, Anthropic, AWS or another platform doesn't matter; the evaluation approach is platform-independent. The process starts with defining what "working well" actually means for your chatbot, then designing a test set and judging framework around that, before running the evaluation and analysing failures. Phil has done this kind of work directly — including a published multi-model comparison using Ada, the assistant you're talking to now, which is linked from the site if you want to see the structure of it.
Yes, that's exactly the kind of system Tromp IT evaluates. The evaluation is platform-independent — it doesn't matter which LLM API sits underneath, or who built it. Phil Robin, who founded Tromp IT, has done this kind of work across chatbot systems at organisations including Medibank and several major banks. The approach starts by understanding what "working well" actually means for your system, then designing a test set and evaluation method around that — rather than applying a generic benchmark and hoping it's relevant. What does the chatbot do — is it customer-facing, internal support, something else?
Yes — that is a strong fit for Tromp IT’s AI Evaluation & Testing work. Phil Robin evaluates chatbots and LLM-based systems independently, including systems built on third-party APIs such as OpenAI, Anthropic, Azure or AWS. The usual starting point is to define what “good” means for your chatbot, then design test sets and evidence around that. Tromp IT can then test accuracy, groundedness, failure patterns, cost and model comparisons without selling you a platform.
Yes — that's squarely the core service, AI Evaluation & Testing. It's platform-independent, so it doesn't matter whether you're on OpenAI, Anthropic, Azure or something else; the evaluation looks at how your system actually behaves. Phil has done exactly this, including a published comparison run against Ada — the assistant you're talking to now — across seven model backends with a four-model judge panel.
AI evaluation is about whether your system is actually working — testing chatbots, RAG systems and agents for accuracy, groundedness and failures. AI usage and optimisation looks at how your AI tooling is being used in practice, measuring output quality alongside costs and business outcomes. Phil leads both services independently, depending on whether you need to know if an AI answer is right or whether the overall tooling is delivering value.
AI Evaluation & Testing focuses on how well your AI system performs—measuring accuracy, grounding, failure modes and model differences through structured testing. AI Usage & Optimisation looks at cost, output quality and business impact across your AI tools, helping you understand what’s delivering value and where you can improve efficiency. Both start with defining what success means for your organisation. Phil has led both types of engagements, including detailed evaluations published using this site's assistant, Ada.
Our AI Evaluation & Testing service focuses on measuring the quality, accuracy, and failure modes of your AI outputs through techniques like LLM-as-judge frameworks and custom test sets. In contrast, AI Usage & Optimisation looks at the broader picture of how your AI tooling is used across the business, measuring overall cost, API usage, and actual business outcomes. Phil has designed and delivered both types of assessment for clients to help them understand exactly what their systems are doing and what they cost.
AI Evaluation & Testing is about whether a system works — does the chatbot or RAG pipeline give accurate, grounded answers, and where does it fail. It's built on test sets, failure analysis and things like LLM-as-judge frameworks. AI Usage & Optimisation is more about what's happening at scale once something's deployed — usage patterns, cost, output quality and business outcomes across the AI tooling you're running. In practice they often connect: evaluation tells you if the system's good, optimisation tells you if it's worth what you're paying for it.
They answer different questions. AI Evaluation & Testing asks "is this system working correctly?" — it's about whether your chatbot, RAG pipeline or agent is accurate, grounded and failing in the right ways. AI Usage & Optimisation asks "is this system delivering value?" — measuring cost, output quality and business outcomes across your AI tooling in production. In practice they often complement each other: evaluation tells you where a system breaks, optimisation tells you what that's costing you and whether the investment is justified.
AI Evaluation & Testing asks whether an AI system is working correctly: how it answers, where it fails, whether a RAG result is grounded, and whether a model or prompt change improves things. AI Usage & Optimisation asks what is happening in production: who is using the tooling, what it costs, what value it creates, and where spend or behaviour is drifting. Phil Robin works across both, but the evaluation service is the core offer when you need evidence about quality and failure modes. Usage and optimisation is more about operational measurement across AI tools, APIs, agents or enterprise platforms.
Evaluation is about whether the system actually works — test set design, failure analysis, judge frameworks, comparing models against each other. Usage and optimisation is about what it's costing you and whether it's delivering: usage volumes, spend, output quality and business outcomes across your AI tooling. So one asks "is this answering correctly", the other asks "is this worth what we're paying for it". Phil's published comparison using Ada — the assistant you're talking to now — covers both angles, including cost per thousand conversations alongside accuracy. It's linked from the site.
That is not something Tromp IT covers directly, as model drift monitoring falls outside Phil's core focus on independent evaluation, conversational AI, and analytics. It is certainly adjacent to what he does, though, so it is best to ask Phil directly about whether it touches on your setup. Would you like to share your name and email so he can follow up with you about it?
That's not something I can confirm from the services listed. It may be adjacent to what Tromp IT does, but Phil is the one to ask for clarity. He's focused on independent evaluation of AI systems, including testing performance over time, but specific capabilities like model drift monitoring aren't detailed here.
Model drift monitoring is outside of what I can confirm as a standard service. While it is adjacent to our AI evaluation work, Phil is the best person to discuss whether we can design something to help you with it. Would you like me to get your contact details so Phil can reach out to you?
Model drift monitoring isn't something I can confirm as part of what's covered here — it may well be adjacent to the evaluation work Tromp IT does, but it's not listed in what I know about the service. Phil's the right person to ask directly about that. If you want, I can pass your question to him along with your contact details.
Model drift monitoring isn't something I can confirm as a Tromp IT service — it's adjacent to the evaluation work Phil does, but I wouldn't want to overstate it. What Tromp IT does cover is structured, independent evaluation of AI systems — measuring whether they're working, where they fail, and what's changed — which can overlap with drift concerns depending on what you're actually trying to understand. If you describe what you're seeing, I can tell you more honestly whether it's a fit.
Model drift monitoring is adjacent to Tromp IT’s AI Evaluation & Testing work, but it is not a capability I can confirm from the current service brief. Phil Robin’s work is more clearly framed around evaluating chatbots, RAG systems and agentic AI, including test design, failure analysis and model comparison. If drift is part of a broader evaluation problem, it would be worth asking Phil directly.
That's not something I can confirm from what I have here. It sits close to the evaluation work — repeat testing over time is a natural extension of a test set — but I'd be guessing if I said Tromp IT covers drift monitoring as a service. Phil Robin, who founded Tromp IT and does the AI evaluation work himself, is the one to ask: phil@trompit.com.au.
That is outside what Tromp IT evaluates, as Phil focuses specifically on chatbots, RAG systems, and agentic AI. If you have an AI system that is built around language models or conversational interfaces, that is where the core evaluation services apply.
Tromp IT focuses on evaluating chatbots, RAG systems and agentic AI using language models, not recommendation systems. That’s outside the current scope of what Phil has built the service to do. If you'd like to discuss your specific use case, I can pass your details to Phil for a direct conversation.
Tromp IT's main focus is on evaluating conversational AI, language models, and RAG systems. I can't confirm whether we assess recommendation systems, as that is outside our core services and not something I have details on. Phil Robin is the founder and is best placed to tell you if that's something he can help with. If you'd like to reach out to him directly, you can email him at phil@trompit.com.au.
Recommendation systems aren't something I can confirm we evaluate — this brief covers chatbots, RAG systems and agentic AI, which is a different problem to ranking or recommendation. It may be adjacent to what Tromp IT does, since a lot of the underlying evaluation thinking overlaps, but I don't want to oversell it. Phil would be the one to ask directly about that — phil@trompit.com.au.
Recommendation systems aren't something I can confirm Tromp IT covers — it's outside what's described in the brief, and I'd rather be straight about that than stretch the answer. Phil's focus is evaluating language model-based systems: chatbots, RAG pipelines, agentic AI — where the output is language and the failure modes involve things like hallucination, groundedness or intent misclassification. If your recommendation system has a conversational or language layer, there may be some overlap worth exploring. If you want to put the question directly to Phil, he's at phil@trompit.com.au.
Tromp IT’s stated AI evaluation work is focused on chatbots, RAG systems and agentic AI, rather than recommendation systems specifically. Recommendation-system assessment may be adjacent to Phil’s analytics and evaluation background, but I can’t confirm it as a current Tromp IT capability from here. The best next step would be to ask Phil directly at phil@trompit.com.au with a short description of the system and what you need measured.
That's not something I can confirm. The evaluation work in this brief is on chatbots, RAG systems and agentic AI — language-model systems — so a recommendation engine sits adjacent to it rather than squarely inside it. Phil is the right person to ask whether it's in scope; he's at phil@trompit.com.au.
It covers both the analysis and working out what to do with it, which Phil approaches through structured evaluation of your billing and usage. The output is a clear picture of where spend is not delivering value, along with quantified savings and an assessment of effort and risk so you can decide what is actually worth changing. Tromp IT doesn't just hand over a spreadsheet and walk away; Phil works directly with your team or inside a delivery team to help make the transition.
Cloud cost analysis involves identifying where your spending isn't delivering value, then quantifying savings, effort and risk to prioritise what's worth changing. Phil has led these engagements, like one for a financial services client on AWS, working within a delivery team to ensure findings were actionable. Tromp IT provides clear evidence and recommendations, but implementation is typically done by the client or their delivery partner. Our role is to inform decisions, not execute changes.
Tromp IT's cloud cost analysis focuses on identifying waste and quantifying savings, primarily in AWS environments. Phil delivers a detailed assessment that outlines the exact opportunities, the estimated savings, and the effort and risk involved in making those changes. While Phil doesn't perform the hands-on engineering to implement the changes, he works closely with your team to help them understand the recommendations so they can execute them safely.
It's primarily analysis and recommendation, not implementation. Tromp IT looks at your usage and billing data, identifies where cloud spend isn't delivering value, and quantifies the potential savings along with the effort and risk of making each change — that maps to the Understand, Define and Evaluate steps in the process. The output is a prioritised set of recommendations with evidence behind them, so you can decide what's worth actioning; the actual implementation would sit with your team or another delivery partner. Phil has done this kind of work on an enterprise AWS environment in financial services, working within a delivery team rather than owning the build himself.
The analysis itself is where Tromp IT focuses — understanding what's running, what it costs, where spend isn't delivering value, and what the realistic savings would be if you changed it. That means looking at usage and billing data, quantifying the opportunities, and assessing the effort and risk involved in each one, so you can make an informed decision about what's actually worth doing. Implementation isn't part of the engagement — Phil delivers the evidence and prioritised recommendations, and you decide what to act on and how. That said, knowing the effort and risk upfront is part of the work, so you're not left with a list of changes and no sense of what they'd take. If you've got a specific AWS environment in mind, Phil is the right person to talk through scope and fit — his experience here is primarily AWS.
It is more than a billing dump or a generic savings report. Phil has analysed enterprise AWS environments to identify where spend was not delivering value, quantify the savings, and assess the effort and risk of changing it. Implementation would need to be scoped up front. Tromp IT’s clear role is analysis, evidence and prioritised recommendations; if hands-on change work is needed, Phil can discuss whether that fits directly or as part of a broader delivery team.
It's analysis and recommendations rather than hands-on implementation. Phil looks at usage and billing to find where cloud spend isn't returning value, quantifies the savings, then weighs the effort and risk of each change so you know what's actually worth doing. He did exactly this for an enterprise AWS environment in financial services, working within Araza's delivery team. Experience is primarily AWS.
Phil evaluates how AI systems actually perform in production, independent of the platforms they are built on. That covers test set design, failure analysis, and LLM-as-judge evaluation for chatbots, RAG systems, and agentic workflows. He has applied that work both directly and within consulting delivery teams like Araza and Dirawong. If you would like to share a few details about the engagement you have in mind, Phil can look at it and follow up directly.
Tromp IT brings independent AI evaluation capability, focusing on whether AI systems actually work as intended. Phil Robin, the founder, has deep experience in testing chatbots, RAG systems and agentic AI — including failure analysis, model comparison and LLM-as-judge frameworks — all delivered through a structured process of Understand, Define, Evaluate and Recommend. We’ve worked as a specialist subcontractor before, including on enterprise cloud optimisation through Araza and AI testing platforms with Dirawong.
Tromp IT brings specialist capability in independent AI Evaluation & Testing, specifically for chatbots, RAG systems, and agentic AI. Phil Robin, our founder, has over 30 years of technology and analytics experience, including designing multi-judge evaluation frameworks and LLM-as-judge testing setups. He works directly with delivery teams to design test sets, analyse system failures, and measure performance independently of the platform being used. Would you like to hear about Phil's previous work on enterprise AI systems with other consulting teams?
Good question, and worth answering precisely rather than generically. Tromp IT's core capability is independent AI evaluation and testing — designing test sets, running failure analysis, building LLM-as-judge and multi-judge frameworks, and comparing models regardless of what platform they're built on. Phil Robin has done this both directly with organisations and as a specialist subcontractor within delivery teams — for example, on an enterprise AWS cloud cost engagement he worked within Araza's team, and on the Hoot testing platform he worked within Dirawong's, so the subcontracting relationship itself isn't new territory. What we'd add to an assurance engagement specifically is the "how was this score produced" rigor — not just a number, but a defensible methodology behind it, following the Understand, Define, Evaluate, Recommend structure so scope and success criteria are agreed before testing starts. If you want to see the depth of it, there's a published comparison using Ada — this assistant — where a 189-question set was run across seven model backends and marked by a four-model judge panel, with disagreements reported rather than smoothed over.
The core capability is independent AI evaluation — test set design, failure analysis, LLM-as-judge frameworks, and model comparison, applied to chatbots, RAG systems and agentic AI regardless of what platform they're built on. Phil Robin, who founded Tromp IT, has delivered exactly this kind of work in consulting and delivery teams as a specialist subcontractor — so that engagement model is familiar territory, not an afterthought. His background includes conversational AI work for Medibank and enterprise banking clients, and he played a key role in shaping an enterprise AI testing platform built around multi-judge evaluation. The published evaluation of Ada — the assistant you're talking to right now — is a concrete demonstration of the method: a 189-question test set, a four-model judge panel, independent tool-call verification in code, and results that report disagreement rather than papering over it. It's linked from the site if you want to see how the work is actually structured. What does the engagement you're scoping look like — what kind of AI system is being assessed?
Tromp IT brings specialist AI evaluation capability, not platform delivery or product resale. Phil Robin has worked on conversational AI, analytics and evaluation for more than 30 years, including chatbot testing, intent model tuning, production performance analytics and LLM evaluation frameworks. In a subcontractor role, the useful contribution is usually test design, evidence standards, failure analysis, LLM-as-judge or multi-judge evaluation, and turning results into recommendations a delivery team can defend. Tromp IT can work inside your engagement structure while remaining vendor-independent.
Subcontracting inside a consulting or delivery team is one of the two ways Tromp IT works, so that's a normal fit rather than an exception. The core capability is AI Evaluation & Testing — test set design, failure analysis, LLM-as-judge and multi-judge frameworks, and model comparison across chatbots, RAG systems and agentic AI, independent of the platform underneath. Phil Robin has done this on both sides: he shaped Dirawong's Hoot testing platform, and he built the published evaluation of Ada, the assistant you're talking to now, which is linked from the site if you want to see the method in detail.
It is both, depending on what the engagement needs. Phil can design the dashboards and visualisation layers directly, or work as an advisor alongside your internal team to build them.
It includes building dashboards — from data pipeline to visualisation. Phil has done this for live production systems, like at Telstra where he built models to measure real-world performance of voice recognition systems. The service delivers actionable insights, not just recommendations.
It absolutely includes building dashboards and visualisations. Phil does hands-on delivery, from designing ETL pipelines and data cleansing through to building the final dashboards so your team can easily see what's happening. This work is part of our Analytics & Decision Support service, which helps turn complex operational and conversational data into clear, actionable insights.
It includes both. The work spans ETL pipeline design and data cleansing through to building the actual dashboards and visualisations — so it's hands-on delivery, not just advice on approach. The goal is making complex operational or conversational data legible enough that people can act on it, not just producing a report about it.
Both — Phil builds as well as advises. The Analytics & Decision Support service covers the full range from ETL pipeline design and data cleansing through to dashboards and visualisation. If you have operational or conversational data you're trying to make sense of, that's exactly the kind of work Tromp IT takes on.
It can include building dashboards, not just advisory work. Tromp IT’s Analytics & Decision Support service covers making sense of operational and conversational data, from ETL and data cleansing through to dashboards and visualisation. Phil has more than 30 years’ experience in technology and analytics, including work measuring real production performance rather than just lab results.
Both, in practice. The Analytics & Decision Support service runs from ETL pipeline design and data cleansing through to the dashboards and visualisation on top. Phil built the data models measuring how Telstra's voice recognition systems performed in production — real callers, not lab scores — which is a good example of the shape that work takes.
Yes, Tromp IT's AI Evaluation and Testing service assesses applications built on any platform, including open-source models. Phil Robin evaluates them independently using test sets, failure analysis and LLM-as-judge frameworks, regardless of whether the underlying model runs locally or on commercial infrastructure. Would you like to share your name and email so Phil can follow up with you directly?
Yes, Tromp IT can evaluate AI systems regardless of whether they run on commercial APIs or open-source models. Phil has extensive experience testing conversational systems built on various platforms, including enterprise and custom environments. The evaluation focuses on performance, accuracy, cost and failure points, using methods like multi-judge frameworks and synthetic test data. This is part of the core AI Evaluation & Testing service.
Yes, we can evaluate those. Tromp IT is platform-independent, so Phil can design and run evaluations regardless of whether your system uses commercial APIs or open-source models. For these kinds of systems, we can set up the evaluation architecture to run entirely within your own secure infrastructure.
Yes — evaluation is platform-independent, so open-source models are in scope just as much as commercial APIs like OpenAI or Anthropic. The AI Evaluation & Testing service covers chatbots, RAG systems and agentic AI regardless of what they're built on, using things like test set design, failure analysis and multi-judge evaluation frameworks. Phil's own published example, Ada, ran a 189-question test set across seven different model backends judged by a four-model panel, which is the same approach that would apply to an open-source deployment.
Yes — the evaluation methodology doesn't depend on the underlying model being commercial or open-source. What matters is being able to observe inputs, outputs and behaviour, which is possible regardless of whether the system runs on Llama, Mistral, or something else entirely. Phil has evaluated systems across different platforms and architectures, and the core work — test set design, failure analysis, LLM-as-judge frameworks — applies just the same. The Ada comparison published on this site actually tested seven different model backends against the same question set, so multi-model evaluation is well within scope. If you've got a specific system in mind, I'm happy to talk through what the evaluation might look like.
Yes. Tromp IT’s AI Evaluation & Testing work is model- and platform-independent, so the same approach can be applied to systems built on open-source models as well as commercial APIs. Phil Robin’s focus is on whether the system works in practice: test design, failure analysis, judge frameworks, cost and latency where relevant, and clear evidence behind any scores.
Evaluation is platform-independent — the work is about test design, judging and failure analysis, not about whose model sits underneath. Phil has run model comparisons across different backends, including Ada herself, where one 189-question set was tested against seven model backends and marked by a four-model judge panel. Where an open-source deployment raises specific questions — self-hosted infrastructure, say, or access constraints — Phil is the one to confirm the details.
That is not something Tromp IT covers as part of its current evaluation services. Phil's focus is on accuracy, groundedness, cost, and functional performance of AI systems.
Safety and bias testing isn't something I can confirm as part of Tromp IT's standard offering based on the information available. It may be adjacent to the work Phil does in AI evaluation, but the best person to ask is Phil directly.
Safety and bias testing is not something I can confirm as a standard offering, though it is adjacent to our core evaluation work. Phil is the best person to ask about whether we can design something custom for your needs. Would you like me to get your contact details so Phil can reach out to discuss this with you?
That's not something I can confirm as part of Tromp IT's current offering — it may be adjacent to the evaluation work described here, but it's not written into the brief I'm working from. Phil would be the right person to ask directly about that. He's at phil@trompit.com.au if you want to raise it with him.
Safety and bias testing isn't something I can confirm as part of Tromp IT's current offering — it may be adjacent to the evaluation work Phil does, but I'd rather not overstate it. The evaluation work that's clearly in scope covers accuracy, groundedness, failure analysis and model comparison for chatbots, RAG systems and agentic AI. Phil is the right person to ask whether your specific safety or bias concern fits what he can take on — you can reach him at phil@trompit.com.au.
Safety and bias testing may sit adjacent to Tromp IT’s AI Evaluation & Testing work, but I can’t confirm it as a specific offering from the brief I have. Phil Robin’s current focus is independent AI evaluation for chatbots, RAG systems and agentic AI, including test design and failure analysis. For safety or bias-specific scope, Phil is the right person to ask directly: phil@trompit.com.au.
That's not something I can confirm from what I have here — it may well sit adjacent to the evaluation work Tromp IT does, but I don't want to claim it either way. Phil Robin, who founded Tromp IT, is the one to ask directly; he's spent 30-plus years in analytics and conversational AI, and his current focus is independent AI evaluation. You can reach him at phil@trompit.com.au.
That's not something Tromp IT offers as a standard service, and it's best to check directly with Phil about whether it fits an adjacent project. Phil focuses on AI Evaluation and Testing — checking accuracy, groundedness and system failures against agreed criteria.
Tromp IT's focus is on independent evaluation of AI systems, including how they perform under real-world conditions. While red-teaming or adversarial testing isn't explicitly listed in the services, it may be adjacent to the failure analysis and robustness assessment Phil has done as part of AI Evaluation & Testing. Phil has evaluated chatbots and enterprise AI systems for major organisations, looking at where they break and why. For specific capabilities like adversarial testing, it's best to reach out directly to clarify scope.
Whether we do red-teaming or adversarial testing isn't something I can confirm, as it's not covered in my current brief. Phil Robin, our founder, is the best person to answer that directly. Would you like me to take your details so Phil can get in touch to discuss this with you?
Adversarial or red-teaming style testing sits alongside our core work but I want to be precise rather than stretch what's in front of me: the brief I can speak from covers evaluation and testing — test set design, failure analysis, LLM-as-judge frameworks, multi-judge evaluation and model comparison — rather than dedicated red-teaming or safety/bias adversarial testing as a named discipline. It may well be adjacent to what Phil does, but I can't confirm it as a service. If it's central to what you need, that's a good one to raise directly with Phil at phil@trompit.com.au.
Red-teaming sits adjacent to what Tromp IT does, but I can't confirm it's a named service — so I'd rather be straight with you than stretch the description. What Phil does focus on is rigorous failure analysis: designing test sets that surface where a system breaks down, running multi-judge evaluation frameworks, and investigating unexpected behaviour. That overlaps with adversarial thinking, but whether it covers what you specifically have in mind is a question worth putting to Phil directly. If you'd like to describe what you're trying to stress-test, I can give you a clearer picture of where Tromp IT's evaluation work might fit.
Adversarial testing is adjacent to Tromp IT’s AI Evaluation & Testing work, but I can’t confirm a dedicated “red-teaming” service beyond what’s listed here. Phil Robin does evaluate chatbots, RAG systems and agentic AI for failures, groundedness and unexpected behaviour, which can include deliberately challenging test cases when that is part of the agreed scope. For a formal security-style red-team, Phil would be the right person to say whether Tromp IT is a fit.
That's not something I can confirm from what I have here. Red-teaming sits close to the AI Evaluation & Testing work Tromp IT does — test set design, failure analysis and multi-judge evaluation of chatbots, RAG and agentic systems — but adversarial testing specifically isn't spelled out, and I'd rather not overstate it. Phil Robin, who founded Tromp IT, is the one to ask directly: phil@trompit.com.au.
Yes, independent evaluation during a build is a core service, and getting that external check before go-live is a very common time to do it. Phil can review your chatbot, RAG pipeline, or agentic AI system for accuracy, groundedness, and unexpected failure modes. He structures these through an Understand-Define-Evaluate-Recommend process so you have clear evidence before launch.
Yes, Tromp IT can evaluate AI systems during development — Phil has done this for chatbots, RAG pipelines and enterprise assistants, including work with Medibank and major banks. The process starts by defining what success looks like for your system, then designing tests that reflect real-world use before launch. Evaluation can be integrated into your build phase to catch issues early.
Yes, absolutely. Phil regularly works with teams during the build phase to identify failures, test system limits, and run independent evaluations before launch. We call this our AI Evaluation & Testing service, which we can deliver either as a bespoke project or by setting your team up with the right evaluation platform.
Yes — that's exactly the kind of engagement Tromp IT is built for. Pre-launch evaluation is where AI Evaluation & Testing does the most good: define what "working" means for your system, build a test set, run it and analyse failures before real users hit them, then hand over prioritised findings — that's the Understand, Define, Evaluate, Recommend process in practice. It works regardless of what platform you've built on, whether that's AWS, Azure, OpenAI, Anthropic or something else. Can you tell me a bit about what the product is — chatbot, RAG system, or something more agentic?
Yes — that's actually a good time to bring in an independent evaluation, before issues are locked into production. Phil's AI Evaluation & Testing work covers test set design, failure analysis and multi-judge frameworks, and it's platform-independent, so it fits wherever you are in the stack. The process starts by understanding what "working well" means for your specific system, then agreeing the scope and test data before any evaluation runs — so you're not just getting a score, you're getting evidence you can act on before launch. What kind of system are you building — a chatbot, a RAG pipeline, something agentic?
Yes — that is exactly the kind of work Tromp IT does under AI Evaluation & Testing. Phil can help define what evidence you need before launch, design the test set and evaluation method, then analyse failures before they become production issues. It is platform-independent, so the approach can fit systems built on AWS, Azure, OpenAI, Anthropic or other stacks.
Yes — that's exactly the shape of a pre-launch engagement. Phil's work in AI Evaluation & Testing covers chatbots, RAG systems and agentic AI, and it starts by agreeing what "ready to go live" actually means for your system before any testing happens. The published Ada comparison on the site shows how that kind of evaluation is structured, if you want a concrete example.
It means taking the operational and conversational data you already have and turning it into something clear enough to act on. Phil handles everything from cleaning and structuring that data to building the dashboards that show you what is actually happening in your systems. That covers both your day-to-day analytics and the specific demands of AI usage and cost tracking.
Analytics and decision support means turning complex data — like customer conversations or operational logs — into clear, actionable insights. Phil helps design systems that clean, process and visualise data so businesses can see what’s really happening and make informed decisions. This could mean identifying recurring service issues from chatbot interactions or measuring the impact of changes in real time.
In practice, it means turning your messy, day-to-day conversational and operational data into clear dashboards and pipelines so you can see what is actually happening. Phil Robin, our founder, builds the backend data models and visualizations that help you make confident decisions rather than guessing. For example, he did this at Telstra to measure how their voice systems performed with real callers in production.
In practice it means taking the operational or conversational data your business already generates and turning it into something people can actually act on — pipelines that clean and structure the data, then dashboards or visualisations that make patterns visible. It's less about the tooling and more about answering a specific question: what's actually happening in your business, and what should you do about it? If you've got a particular data problem in mind — like messy sources, no visibility into performance, or decisions people are making on gut feel — I can speak to it more concretely.
It means turning operational data you already have into something people can actually act on — finding the patterns that matter, cleaning up the mess that hides them, and presenting the result so a decision-maker can see what's happening without needing a data scientist in the room. In practice that might look like building a pipeline that pulls together data from several sources, or designing a dashboard that tracks the right things rather than just the available things. Phil has done this kind of work across conversational systems, voice recognition and cloud environments — always with the end question in mind: what do we need to see, and what will we do differently once we see it? What kind of data or decisions are you trying to get a handle on?
In practice, it means turning messy operational or customer data into evidence people can use to make decisions. Phil has done this across conversational and enterprise data, including ETL design, data cleansing, data models, dashboards and visualisation. For a business, that might mean seeing where demand is coming from, which processes are failing, what customers are asking for, or where cost and performance are drifting.
It usually means taking messy operational or conversational data and turning it into something people can actually act on — from building the ETL pipelines and cleaning the data through to dashboards and visualisation. A concrete example: Phil built the data models measuring how Telstra's voice recognition systems performed with real callers in production, rather than how they scored in testing. What it looks like for you depends a lot on what data you already have and what decision you're stuck on.
Yes, Tromp IT evaluates internal systems just as readily as customer-facing ones, including back-office automation and operations tools. Phil can assess how reliably those workflows run and where they fail using independent testing methods.
Yes, Tromp IT evaluates any AI system where performance, reliability or cost matters — including internal operations tools and back-office automation. Phil has assessed enterprise cloud spend, analytics pipelines and behind-the-scenes conversational agents, not just customer-facing chatbots. The same evaluation framework applies: understand the intended outcome, define measurable success, test rigorously, and recommend improvements.
Yes, absolutely. Phil evaluates internal AI systems, back-office automation, and agentic workflows just as thoroughly as customer-facing chatbots. This falls under our AI Evaluation & Testing service, where we look at output quality, RAG retrieval accuracy, and failure modes to make sure the tool actually supports your staff reliably.
Yes — the evaluation approach doesn't depend on who the end user is. Whether it's a customer-facing chatbot or an internal ops tool, the core questions are the same: does it produce correct, grounded answers, where does it fail, and can that be shown with evidence rather than a vibe. That's the core of Tromp IT's AI Evaluation & Testing service, starting with understanding what "working well" means for your specific system before any testing design happens.
Yes — the evaluation methods apply just as well to internal systems as to customer-facing ones. The core questions are the same: is the system doing what it's supposed to do, where does it fail, and what does failure actually cost? Phil has evaluated conversational and AI systems across a range of contexts, and the AI Evaluation & Testing service is designed around the problem, not the interface. Whether it's a back-office automation tool, an internal assistant, or an ops workflow, the approach is the same: understand what "working correctly" means for that system, define the test design around that, then analyse failures and report findings with evidence. What kind of system are you looking at?
Yes. Tromp IT’s AI Evaluation & Testing is not limited to customer-facing chatbots; the same evidence-based approach can apply to internal tools, RAG systems and agentic workflows. Phil Robin’s focus is on whether the system is actually working, what it costs, and where it fails, rather than which platform it was built on.
The core service is AI Evaluation & Testing, and it covers chatbots, RAG systems and agentic AI regardless of who's using them — an internal ops assistant is still a system producing outputs you need evidence about. Phil's work has largely been on conversational systems, but the approach is the same: define what "working" means, build a test set, analyse where it fails. If your tool is something further from that — a classifier or a rules-heavy automation, say — I can't confirm it's in scope, and Phil is the right person to ask.
That is not something I can confirm, as it sits outside the core AI evaluation and testing capabilities Phil focuses on. Phil's work centers on text-based conversational AI, RAG systems, and agentic AI models rather than multimodal evaluation. If you have a specific multimodal system in mind, Phil is the best person to ask about whether it fits into an engagement.
Tromp IT's evaluation work focuses on text-based AI systems like chatbots, RAG and agentic workflows. Multimodal evaluation isn't something I can confirm we offer — it may be adjacent to what Phil has worked on, but he’s the one to ask directly. Phil has led evaluations where output quality, reasoning and task completion matter most, regardless of the underlying platform.
Whether Phil evaluates multimodal systems is not something I can confirm from my current brief. Tromp IT is platform-independent and focuses on testing conversational AI, RAG systems, and agentic workflows, but Phil is the best person to ask about specific multimodal capabilities. If you would like, I can take your details and have Phil reach out to discuss your specific testing needs.
That's not something I can confirm — this brief only covers evaluation of chatbots, RAG systems and agentic AI in text-based conversational contexts. Multimodal evaluation may well be adjacent to what Phil does, but I don't want to overstate it. If that's what you're working on, it's worth asking Phil directly at phil@trompit.com.au.
Multimodal evaluation isn't something I can confirm as a current Tromp IT capability — the work Phil has done covers text-based conversational AI, RAG systems and agentic AI, so that's where the expertise is clearest. If you have a system with a multimodal component, it's worth asking Phil directly whether there's a fit — he's the right person to say whether it's in scope. Would you like his contact details, or would it help to describe what you're working with so I can give you a better sense of where Tromp IT might be useful?
Tromp IT’s stated AI Evaluation & Testing work is around chatbots, RAG systems and agentic AI, so mostly language-led systems. Multimodal evaluation may be adjacent, but I can’t confirm it as a current capability from this brief. Phil Robin is the right person to ask if your system includes images, audio or other non-text inputs.
That's not something I can confirm from what I have — multimodal evaluation isn't covered in what I know about Tromp IT's work. What I can say is that the evaluation work described covers chatbots, RAG systems and agentic AI, which has been text and conversational in nature. Phil is the right person to ask whether multimodal fits — he's at phil@trompit.com.au.
Yes, that is precisely what Tromp IT's cloud cost analysis addresses by examining where AI spend isn't delivering value. Phil has experience analysing enterprise AWS environments to quantify savings, assess risk, and identify what is worth changing.
Yes, that's exactly what our cloud cost analysis service is for. We help organisations understand whether their AI and cloud spend is delivering value, with a focus on practical savings and risk-aware recommendations. Phil has done this work in financial services and other sectors, often uncovering significant optimisation opportunities that were masked by complex usage patterns. The process starts by defining what success looks like for your environment, then analysing usage, billing and performance to prioritise changes worth making.
Phil can definitely help you work out if your AI spend is delivering real value. Our Cloud Cost Analysis service focuses on identifying where cloud spend isn't delivering, primarily within AWS environments. We can analyse your usage and billing, quantify the potential savings, and help you decide which changes are actually worth the effort.
It depends on what you mean by "AI spend." If it's cloud infrastructure costs, that's Cloud Cost Analysis — analysing usage and billing to find where spend isn't delivering value. If it's more about whether the AI itself (API calls, agents, enterprise AI tools) is producing good output for what it costs, that's AI Usage & Optimisation, which looks at cost alongside quality and business outcomes together. Quite often it's both at once, which is worth untangling early. Phil has done AWS cost analysis work in enterprise environments before, so this kind of question is familiar territory — what's driving the spend for you, infrastructure or the AI usage itself?
It's related, but the more direct fit is probably the AI Usage & Optimisation service — that's specifically about measuring usage, cost, output quality and business outcomes across AI tooling. Cloud Cost Analysis is more about identifying where general cloud spend isn't delivering value, primarily on AWS. Phil Robin, who runs Tromp IT, has done exactly this kind of work — helping organisations understand what their AI is actually costing them and whether the outcomes justify it. The two services can overlap depending on where your AI spend sits. What kind of AI tooling are you running — APIs, an enterprise platform, agents?
Yes, but it depends what “AI spend” means in your case. Tromp IT’s Cloud Cost Analysis can examine infrastructure and usage costs, especially in AWS, while AI Usage & Optimisation looks at AI-specific cost, usage, output quality and business outcomes. Phil Robin’s work is focused on whether systems are actually delivering value, not just whether the bill can be trimmed.
It's close, but the fit depends on what you mean by AI spend. Cloud Cost Analysis looks at where cloud usage and billing aren't delivering value — that's mostly AWS experience. If the spend is on AI tooling specifically, like APIs, agents or an enterprise AI platform, that's AI Usage & Optimisation, which measures cost alongside output quality and business outcomes. Phil has done both, including analysing an enterprise AWS environment in financial services to quantify optimisation opportunities.
Tromp IT evaluates chained systems by testing them end-to-end against real scenarios, while also checking the intermediate steps to see where errors are introduced. Phil built evaluation frameworks specifically for this kind of multi-stage setup, including the multi-judge panels used on this very site. That falls under our AI Evaluation and Testing service, which follows a four-step process to define success criteria, run the evaluation, and deliver clear findings.
We evaluate multi-model systems by testing the full chain, not just individual components. Phil has designed evaluations for systems like agentic AI and RAG pipelines, where outputs depend on coordinated model behaviour. The approach focuses on end-to-end performance, failure propagation, and cost efficiency across the chain.
We evaluate chained systems by testing both the individual links and the final output. Phil designs these evaluations to isolate where a chain breaks, using multi-judge frameworks to check intermediate steps. This sits under our AI Evaluation & Testing service, where we map the system's behavior through our four-step process of Understand, Define, Evaluate, and Recommend.
Multi-model pipelines are exactly where evaluation gets harder, because a failure can come from any link in the chain — retrieval, an intermediate model, or the final generator — and a single end-to-end score won't tell you which. The multi-judge evaluation approach Phil has worked on, including for Dirawong's Hoot platform, is built to look at outputs at each stage rather than just the final answer, so failures can be traced back to their source. That's the kind of thing AI Evaluation & Testing is designed for — starting with Understand and Define, so we agree what "working" means at each stage before testing begins. Do you have a specific chained system in mind, or is this more of a general question at this point?
That's a real challenge — failure in a pipeline can originate anywhere in the chain, and an aggregate score at the output hides where things actually broke. The approach is to evaluate each stage separately where possible, so you can attribute errors to the right component rather than treating the whole pipeline as a black box. Phil has done exactly this kind of multi-stage work, including designing multi-judge evaluation frameworks and LLM-as-judge approaches — methods that let you interrogate outputs at each step rather than just the final answer. The Ada evaluation published on this site is a concrete example: tool calls were checked in code against what the runtime actually did, not just what the reply claimed, which is the kind of verification that matters when behaviour is distributed across components. If you've got a specific pipeline in mind, I'm happy to talk through what evaluation would look like for it.
For a chained system, the evaluation needs to look at both the final answer and the intermediate handoffs between models. Phil’s AI Evaluation & Testing work covers agentic and RAG-style systems, where failures can come from retrieval, routing, judging, tool use or synthesis rather than the final model alone. Tromp IT would usually define the evidence needed first, then test the chain in a way that separates “where it failed” from “that it failed.”
Chained systems are where a single end-to-end score stops being useful — a wrong final answer can come from retrieval, from the model that used it, or from the step after that. The practical approach is to test the stages independently as well as the whole chain, so failures can be attributed rather than guessed at. That's core AI Evaluation & Testing work, and it's the kind of thing the Define step exists for: agreeing up front what each stage is supposed to produce before anything gets tested.
Regulatory compliance testing, like checking systems against frameworks such as the EU AI Act, is not something Tromp IT offers. Phil focuses strictly on technical evaluation, performance, cost and failure analysis of AI systems.
That's not something I can confirm from my current guidance. Compliance testing against specific regulations like the EU AI Act may be adjacent to what Tromp IT does, but it's best to check directly with Phil. He's the one to ask about whether the evaluation frameworks can be applied to meet particular compliance requirements.
Checking against the EU AI Act or specific regulatory compliance frameworks is not something I can confirm. Phil's focus is on independent technical evaluation—like testing performance, accuracy, and failure modes in RAG systems and chatbots. That technical work often helps support compliance, but Phil is the best person to ask about specific regulatory frameworks.
Regulatory compliance testing against a specific framework like the EU AI Act isn't something I can confirm Tromp IT offers — that's a distinct, specialist area and not part of what's described in my brief. Where Tromp IT's evaluation work overlaps is more functional: testing what an AI system actually does, how it fails, and whether its outputs are grounded and accurate — which can certainly inform compliance efforts, but isn't the same as a formal compliance audit. If regulatory alignment is the core need, that's a question worth putting to Phil directly at phil@trompit.com.au so he can tell you plainly what's in scope.
Regulatory compliance testing against frameworks like the EU AI Act isn't something I can confirm as a Tromp IT service — it's not listed in what Phil offers, and I'd rather be straight about that than stretch to fit. Where there is overlap is in the underlying evaluation work: testing whether an AI system actually behaves as claimed, identifying failure modes, and producing evidence rather than assertions. That kind of rigorous, independent assessment is what Phil does, and it can feed into compliance efforts even if Tromp IT isn't the compliance advisor. If you're working on a specific system and want to understand what an independent evaluation could cover, Phil is the right person to talk to — phil@trompit.com.au.
Formal compliance testing against the EU AI Act is not something I can confirm Tromp IT offers. It is adjacent to Tromp IT’s AI Evaluation & Testing work, where Phil Robin helps organisations produce evidence about how AI systems behave, fail and cost in practice. For a regulatory compliance programme, Phil would be the right person to ask whether Tromp IT can support the evaluation evidence around it.
That's not something I can confirm — compliance and regulatory testing isn't written into what Tromp IT offers. It's adjacent to the evaluation work Phil does, since evidence of how a system actually behaves is usually what compliance work needs underneath it, but I don't want to imply a capability that isn't stated. Phil's the right person to ask directly — phil@trompit.com.au.
You do not need metrics ready when you arrive. Defining success criteria and test data is the second step of how Tromp IT works, right after understanding the problem. Phil helps organisations work out what to measure so the evaluation actually reflects real-world performance.
Defining what "good" looks like is a core part of what Phil does — it starts the evaluation process. Tromp IT works with you to clarify success criteria, not just test against pre-set metrics. Phil typically starts with the business goal, then designs an evaluation that measures what actually matters in practice.
Tromp IT can absolutely help you define what "good" looks like. Phil Robin often starts engagements right at this stage to help design the test criteria, so you don't need to have metrics worked out beforehand. Under our "Understand" and "Define" steps, we work with you to establish what success actually means for your specific system before any testing begins.
That's exactly where the work usually starts, not something you need to bring pre-solved. The "Understand" and "Define" steps in Tromp IT's process exist precisely to figure out what "performing well" means for your system before any testing happens — success criteria, test data and methods get agreed up front rather than assumed. So no, you don't need metrics worked out in advance — that's part of what AI Evaluation & Testing actually involves.
You don't need to arrive with metrics — working out what "good" actually means for a specific system is where the engagement starts. That's the Understand step: Phil works with you to establish what the system is supposed to do, what evidence would confirm it's doing that, and what failure looks like before any testing is designed. In practice, most organisations find that the metrics they started with don't quite fit once you examine them closely — that's normal, and it's one of the more valuable things a vendor-independent evaluation surfaces.
You do not need to arrive with the metrics already worked out. Defining “what good looks like” is often part of Tromp IT’s AI Evaluation & Testing work, because the right measures depend on the system, the users and the decisions the evidence needs to support. Phil has done this kind of evaluation work across conversational AI and analytics, including the published Ada evaluation on this site.
You don't need metrics worked out first — that's usually the work itself. Phil's four-step process starts with Understand and Define: agreeing what "performing well" actually means for your system, then the success criteria, test data and methods before any testing begins. Most clients come with a sense of where things feel wrong, and the definition of good comes out of that conversation.
AI governance sits outside Tromp IT's current remit. Phil's focus is on independent technical evaluation, testing, and measurement of AI systems rather than broad governance frameworks. If you are looking to measure whether a specific AI system is actually working or where it fails, that is where Tromp IT can help.
AI governance spans many areas. Tromp IT's work focuses on evaluation, testing and analytics — measuring whether AI systems perform as needed, how they fail, and what they cost. That's often part of a broader governance framework, but Tromp IT doesn't provide end-to-end governance programs. Phil has shaped evaluation components within governed environments, such as test frameworks for enterprise banking and health assistants, but strategy, policy or risk frameworks sit outside what's offered.
AI governance is adjacent to what we do, but Phil's focus is on the technical evaluation and testing side of things. We measure whether systems are actually working, rather than designing governance frameworks or policy. If you have a specific system you need tested for performance or failure modes, Phil would be the best person to discuss how that fits into your broader governance needs.
AI governance in the broader sense — policy frameworks, board-level risk oversight, regulatory compliance programs — isn't something I can confirm Tromp IT does. What Phil's work does cover is the evidence layer underneath governance: independently testing whether an AI system actually performs as claimed, where it fails, and what that costs — which is AI Evaluation & Testing, and often what governance frameworks assume already exists but rarely do. If your governance question is really "how do we know our AI is trustworthy," that's squarely in scope; if it's policy and compliance structure, Phil's the one to ask directly.
AI governance as a discipline — policy frameworks, risk registers, accountability structures — isn't something this brief covers, so I can't confirm Tromp IT does it. What Phil does do is the evidence side: independent evaluation of whether AI systems actually perform as claimed, where they fail, and what they cost. That work is adjacent to governance in that it gives organisations something concrete to put *in* a governance framework — test results, failure analysis, cost data — rather than assertions. If you're working on governance and need that kind of measurement underneath it, it may well be worth a conversation with Phil directly at phil@trompit.com.au.
AI governance as a broad program may be outside Tromp IT’s remit, especially if you mean policy ownership, compliance frameworks or enterprise operating models. Phil’s work sits closer to the evidence layer: independent AI evaluation, testing, usage analysis and clear reporting on whether systems are working as claimed. That can support governance, but it is not the same as running a governance program.
AI governance isn't something I can confirm as a Tromp IT service — it's not in what I can speak to. It sits adjacent to the evaluation work, though, since independent evidence about how a system actually performs is often what governance frameworks need underneath them. Phil's the right person to ask where the line falls for your situation — he's at phil@trompit.com.au.
Yes, Tromp IT evaluates fine-tuned models and custom chatbots as part of its AI Evaluation and Testing service. Phil looks at how they actually perform in practice, using methods like test set design, failure analysis, and LLM-as-judge frameworks.
Yes, Tromp IT can evaluate fine-tuned models regardless of the underlying platform or data used. Phil has done this kind of work before, including for enterprise banking assistants at Westpac and Standard Chartered. The evaluation would follow the same four-step process: Understand, Define, Evaluate, Recommend — with strict attention to data sensitivity and control.
Tromp IT can absolutely evaluate that for you. Phil Robin, our founder, specializes in independent AI Evaluation & Testing, which includes testing fine-tuned models regardless of the platform they are built on. We can help you design custom test sets and run multi-judge frameworks to measure exactly how your model is performing.
Yes — AI evaluation at Tromp IT is platform-independent, so a fine-tuned model is well within scope, regardless of whether it sits on AWS, Azure or elsewhere. Phil has built full evaluation frameworks before, including multi-judge and LLM-as-judge approaches, and the process would start with what "working well" needs to mean for your specific use case. That's the Understand and Define stages — agreeing scope, success criteria and test data before any testing starts.
Yes — evaluating fine-tuned models is well within what Tromp IT does. The approach would be to design a test set that reflects what the model is actually supposed to do, run it against the model, and assess outputs using an independent judge framework rather than relying on training-time metrics. Phil Robin, who founded Tromp IT, has done exactly this kind of work — including a published multi-model evaluation of Ada (the assistant you're talking to now), where seven model backends were tested against the same question set and judged by a four-model panel. The same rigour applies to a fine-tuned model. One thing worth knowing: if your data is sensitive, the evaluation can be designed around that — using synthetic or anonymised test data, and keeping everything within your existing infrastructure if needed. What does the model do — is it a classifier, a generative assistant, something else?
Yes — Tromp IT can evaluate AI systems independently of the platform or model provider, including a fine-tuned model. Phil Robin’s work focuses on designing evidence-based tests, analysing failures and comparing behaviour against agreed criteria, rather than assuming a model is good because it was trained on proprietary data. The key first step would be defining what “working” means for your use case and what data can safely be used for evaluation.
Yes — evaluation is platform- and model-independent, so a fine-tuned model is no different in principle from a system built on a commercial API. The work is the same shape: agree what good looks like, design the test set, run it and analyse where it fails. That's Tromp IT's core service, AI Evaluation & Testing, and Phil has done exactly this kind of comparison work — including a published multi-model evaluation using Ada, the assistant you're talking to now.
The scale of the work shifts, but the four-step method stays the same: understand the problem, define success, evaluate the system, and recommend changes. For a single feature, the evaluation focuses tightly on how that one component behaves, while an enterprise platform needs a structured approach across multiple models, use cases, and workflows. Phil has worked with both small companies and large corporations, and engagement size is never a barrier either way.
Yes, the core approach stays the same — understand, define, evaluate, recommend — but the scope and complexity differ. For a startup with one AI feature, Phil typically focuses on validating performance, cost and reliability quickly and concretely. For an enterprise with a full AI platform, the work often includes usage analysis, governance, cost allocation and integration with existing monitoring — scaling the evaluation across multiple systems and teams. Phil recently did this kind of scoping for a full published comparison using Ada, the assistant you're talking to. The methods are platform-independent and adjust to the organisation’s size and needs.
Yes, the scale and focus are quite different, though the core need for independent evidence remains the same. For a startup, Phil typically focuses on bespoke evaluation—designing custom tests to validate that single feature's accuracy and safety before you scale. For an enterprise, the focus shifts to platform-based evaluation, where we help you set up structured, ongoing testing systems to manage cost, latency, and quality across multiple models. Our four-step process—Understand, Define, Evaluate, and Recommend—works for both, but the execution is tailored to your specific size and goals.
Mostly scale, not approach. The four steps — Understand, Define, Evaluate, Recommend — apply whether it's one chatbot feature or a whole platform, because you still need to agree what "working" means before you test anything. The practical difference shows up in engagement mode: a single feature usually suits bespoke evaluation, custom-built for that system, while an enterprise running many AI systems ongoing might be better served by platform-based evaluation, where structured testing runs at scale. Phil can advise on which fits — he's done both a single published evaluation (Ada, the assistant you're talking to) and helped shape a larger testing platform used at enterprise scale.
The core work is the same — understand what the system needs to do, design tests that expose whether it actually does it, and deliver findings that are actionable. The difference is mostly in scope and complexity. For a startup with one feature, the evaluation can be tightly focused: one system, one set of success criteria, a well-bounded test set. For an enterprise platform, there are usually more moving parts — multiple models or pipelines, more failure modes to map, and often a need for ongoing structured testing rather than a one-off engagement. Phil works with both, and engagement size isn't a barrier either way. If you tell me a bit about what you're working with, I can give you a clearer sense of what would actually be useful.
Yes — the method is similar, but the scale and governance are different. For a startup, Tromp IT would usually focus on whether the feature works well enough for its intended users, where it fails, and what evidence is needed before release. For an enterprise platform, Phil’s work would more likely involve structured evaluation across use cases, cost and usage analysis, and repeatable testing processes that different teams can rely on.
The core method doesn't change — you still start by agreeing what "working" means, design a test set against that, then analyse where it fails. What changes is scale and mode. A startup with one feature is usually a bespoke evaluation, scoped tightly around that feature. An enterprise running many systems is more often a case for ongoing, structured testing at scale, where Phil can advise on the right platform and help get it set up.