30 questions, each answered by 7 models and scored by 4 independent judges. Expand a question to read every answer and every score. Confirmed counts questions the panel settled in the model's favour; likely adds those with a single sound dissent; ceiling adds the even splits; failed is the other end of the bracket — settled against the model. Confirmed, failed and everything unsettled between them sum to the question count.
| Model | Questions | Confirmed | Likely | Ceiling | Breadth | Failed | Accuracy | Groundedness |
|---|---|---|---|---|---|---|---|---|
| Claude Opus 5 | 30 | 24 80% | 30 100% | 30 100% | 6 | 0 0% | 96.7% | 96.2% |
| GPT-5.5 | 30 | 19 63% | 29 97% | 30 100% | 11 | 0 0% | 95.8% | 92.9% |
| Claude Sonnet 5 | 30 | 17 57% | 28 93% | 30 100% | 13 | 0 0% | 92.5% | 90.4% |
| Claude Sonnet 4.6 | 30 | 8 27% | 22 73% | 25 83% | 17 | 5 17% | 85.8% | 79.6% |
| Gemini 3.5 Flash-Lite | 30 | 4 13% | 19 63% | 25 83% | 21 | 5 17% | 75.8% | 73.8% |
| Qwen 3 235B | 30 | 2 7% | 15 50% | 19 63% | 17 | 11 37% | 61.7% | 62.9% |
| Gemini 3.5 Flash | 30 | 2 7% | 12 40% | 22 73% | 20 | 8 27% | 71.7% | 67.1% |
Each row shows how every model fared. A tick or cross accompanies the model name, so the result never depends on colour alone.
Yes, independent evaluation of RAG systems is Phil's core focus, covering test set design, failure analysis and LLM-as-judge frameworks. He runs this through AI Evaluation & Testing, following a four-step process to understand the problem, define success criteria, evaluate the system, and deliver evidence-based recommendations. If you'd like to share a bit about what your chatbot does or what kind of evaluation you're after, I can give you a better sense of how Phil might approach it.
Yes, we can help evaluate your RAG system. Tromp IT specialises in independent testing of RAG pipelines, focusing on accuracy, groundedness, failure modes and cost. Phil has done this work before, including for enterprise clients, using structured evaluation methods. We'd start by understanding your system and defining what success looks like.
Yes, Phil can certainly help with that. He specialises in independent AI evaluation, which is perfect for assessing how well a RAG chatbot is performing, where it fails, and why. If you are looking to run structured testing, he can help you design a custom evaluation framework or set you up on an ongoing platform. Are you looking to evaluate the chatbot's accuracy and performance before launching, or are you trying to fix specific issues you have already noticed in production?
Yes, evaluating exactly this kind of system is Tromp IT's core work. Phil has run this at real scale — one example is a 189-question set tested against seven different model backends, with every answer independently judged and every tool call checked against what the system actually did. The approach follows four steps: understand what "working well" means for your chatbot, define scope and success criteria upfront, evaluate against test data, then get prioritised recommendations. What's the RAG chatbot for, in terms of what it needs to get right?
Yes — evaluating RAG systems is a core part of what Tromp IT does. The typical failure points are retrieval quality, answer groundedness, and how the system behaves when the retrieved context doesn't actually contain the answer. Phil Robin, who runs Tromp IT, has done this kind of work across a range of conversational AI systems — including a published evaluation of Ada, the assistant you're talking to right now, which covers accuracy, groundedness and cost across multiple model backends. What's the chatbot for, and what's prompting the evaluation — is it pre-launch validation, something that's already in production, or a specific failure you're trying to diagnose?
Yes — RAG chatbot evaluation is a strong fit for Tromp IT’s AI Evaluation & Testing work. Phil has worked on conversational AI and evaluation systems, including multi-judge testing approaches for LLM-powered chatbots and RAG pipelines. The usual starting point is to define what evidence you need, then design tests that expose retrieval failures, answer quality issues and cost-performance trade-offs.
Yes — that's the core of what Tromp IT does. AI Evaluation & Testing covers exactly this: test set design, failure analysis and LLM-as-judge frameworks applied to RAG systems, independent of whatever platform you built on. Phil Robin, who founded Tromp IT, has spent years on this kind of work, including a published evaluation of Ada — the assistant you're talking to now — across seven model backends with a four-model judge panel.
Not a problem at all, since Tromp IT's AI evaluation is completely platform-independent. Phil can independently test and evaluate systems built on Azure, AWS, Anthropic, or OpenAI without being tied to any specific vendor.
Not at all. Tromp IT evaluates AI systems regardless of platform, including Azure OpenAI. Phil has worked with clients using various cloud and AI provider setups, and the evaluation approach adapts to your environment.
No, that is not a problem at all. Tromp IT's AI evaluation services are platform-independent, meaning we can assess applications built on Azure, AWS, OpenAI, or any other platform. Phil Robin, our founder, designed our testing approaches to look at the actual inputs and outputs regardless of the underlying technology.
Not at all — AI evaluation is platform-independent, so whether you're on Azure OpenAI, AWS, Anthropic or something else doesn't change how the assessment works. What matters is the specific system built on top of it: a chatbot, a RAG pipeline, or an agentic workflow, and what "working well" needs to mean for your use case. That's the starting point Tromp IT uses — Understand, then Define scope and success criteria before any testing begins.
Not at all — Tromp IT's evaluation work is platform-independent. Phil has evaluated AI systems built on AWS, Azure, OpenAI, Anthropic and other platforms, and the methods (test set design, failure analysis, LLM-as-judge frameworks) work regardless of where the model is hosted. What kind of system are you looking to evaluate?
No, that’s not a problem. Tromp IT’s AI Evaluation & Testing work is platform-independent, so Phil can assess systems built on Azure OpenAI, AWS, OpenAI, Anthropic or other platforms. The focus is on whether the system works, what it costs, and where it fails, not on selling or favouring a particular vendor.
Not at all — AI evaluation is platform-independent. Phil's evaluation work assesses the application's behaviour rather than the stack underneath it, so AWS, Azure, OpenAI or Anthropic all work the same way from a testing point of view. What matters more is what the system is meant to do and what "working well" looks like for it.
Tromp IT isn't a platform or an agency with capacity on standby, but Phil Robin works directly with organisations on AI evaluation and analytics engagements. If you share your name, email, and phone number along with a few details about what you need, I can record that for you and have Phil follow up.
Phil is available for urgent engagements and often works quickly once scope and requirements are clear. He’s based in Melbourne and works with organisations across Australia. If you’d like to explain what you need, I can help clarify whether Tromp IT is the right fit and get your details to him right away.
Whether Phil is available for an urgent engagement depends on the specific scope and timing of what you need. He often works directly with organisations or as a specialist subcontractor within delivery teams. If you can share your name, email, and a quick overview of the work, I can get those details to Phil right away so he can check his schedule and get back to you.
I'm just the assistant here — I can't take on work myself, but Phil Robin is the person to talk to about availability. You can reach him directly at phil@trompit.com.au, or tell me a bit about what you need and I can pass it on. What's the urgency about — is it an AI system that needs evaluating, or something else?
Phil is the person to ask about availability and timing — that's not something I can commit to on his behalf. If you can share a little about what you need, I can tell you whether it's the kind of work Tromp IT does and help Phil understand the situation before he gets back to you. What's the problem you're trying to solve?
Tromp IT may be able to help, but I can’t commit Phil’s availability from here. Phil Robin handles AI evaluation, analytics and related advisory work directly, and the fastest route is phil@trompit.com.au. If you want me to pass this on as an enquiry, please send your name, email and phone number if you’re happy to share it.
Depends what you need — Tromp IT's core work is independent AI evaluation and testing, with analytics and cloud cost analysis alongside it. I can't commit to availability or start dates on Phil's behalf, but he's the one to ask directly: phil@trompit.com.au. If you tell me a little about what's gone wrong, I can tell you honestly whether it's something he does.
Tromp IT works with both small companies and large corporations, so size is not a barrier. Phil evaluates AI systems independently to check if they are actually working, what they cost, and where they fail. What kind of AI system are you currently building or running?
Yes, Tromp IT works with both small companies and large organisations — company size isn't a barrier. Phil has supported startups and enterprises alike, focusing on clear outcomes like AI performance, cost, and reliability. The approach is always tailored to your specific needs and evidence requirements.
Yes, absolutely. Phil has worked with both small startups and large corporations, and engagement size is not a barrier either way. Tromp IT's services, like AI Evaluation & Testing, are designed to scale to what makes sense for your business.
Yes — company size isn't a barrier either way. Phil has worked with both large corporations and small companies, and engagement scope is set around the problem, not the headcount of the organisation asking. If you've got an AI system you're trying to evaluate or a data problem you're wrestling with, happy to talk through what that looks like.
Engagement size isn't a barrier — Phil has worked with both large corporations and small companies. If anything, a focused engagement with a startup can move quickly because there's less organisational complexity to navigate. What's the problem you're trying to solve?
Yes. Engagement size is not a barrier; Phil has worked with both large corporations and small companies. If you have an AI system, chatbot, analytics problem or cloud cost concern that needs independent evidence, Tromp IT can usually scope the work to the problem rather than the company size.
Yes — Phil has worked with both large corporations and small companies, and engagement size isn't a barrier either way. Engagements are scoped up front with agreed deliverables and effort, so the work fits the problem rather than a minimum size. If you tell me what you're building or where it's not behaving, I can say whether it's the kind of thing Tromp IT takes on.
Safety and clinical compliance are critical in healthcare AI, but safety and bias testing are not capabilities Tromp IT covers directly. Phil does have experience building a medical triage chatbot for Medibank's Nurse on Call service using Cognigy, and implementing automated testing for it. If that kind of evaluation approach is what you need, I can connect you with Phil to discuss it further.
Phil has evaluated AI systems in regulated environments, including a medical triage chatbot for Medibank’s Nurse on Call service. Tromp IT’s evaluations focus on measurable performance, failure modes and consistency — all critical when safety and compliance are at stake. The approach is designed to produce clear evidence, not just scores.
Safety and compliance are critical in healthcare, and the testing framework needs to reflect that rigor. Phil has extensive experience designing evaluation systems for complex environments, including developing a medical triage chatbot for Medibank's Nurse on Call service. While I can't confirm specific safety or compliance certifications, Tromp IT's AI Evaluation & Testing service is designed to thoroughly analyze system failures and unexpected behavior. Phil can discuss how we can tailor our evaluation architecture to meet your specific regulatory constraints.
Directly, I can point to Phil's work on the Medibank Nurse on Call medical triage chatbot — that's healthcare, built on Cognigy, and it included implementing an automated conversational testing platform alongside the build. But I want to be honest about scope: this brief doesn't cover formal safety or bias certification, or specific regulatory compliance frameworks, so I can't claim Tromp IT does clinical-safety sign-off work. What Tromp IT does well is independent evaluation — test design, failure analysis, and checking what a system actually does versus what it claims to do, which matters a lot in a regulated context. If compliance certification against a specific standard is what you need, that's a question for Phil directly to scope properly.
Phil has direct experience in healthcare AI — he built a medical triage chatbot for Medibank's Nurse on Call service, which is about as high-stakes as conversational AI gets. That work involved Cognigy and an automated conversational testing platform, so the evaluation rigour came with the territory. AI Evaluation & Testing is Tromp IT's core service, and the four-step process — Understand, Define, Evaluate, Recommend — is designed to agree exactly what evidence you need before any testing begins, which matters a lot when compliance is part of the picture. On safety and bias testing specifically, I'd want to be honest: those aren't terms I can confirm as defined capabilities from what I have in front of me, so Phil is the right person to talk to about how they'd fit your particular requirements. What system are you evaluating — is it a chatbot, a RAG pipeline, something agentic?
Yes, in the sense that Phil has worked on healthcare conversational AI, including a medical triage chatbot for Medibank’s Nurse on Call service and automated conversational testing. Tromp IT’s core work is AI Evaluation & Testing, with emphasis on evidence, failure analysis and explainable scoring rather than raw performance claims. I can’t confirm specific regulatory safety or compliance testing from this brief, though it is clearly adjacent; Phil is the right person to discuss the exact standard, evidence trail and evaluation design.
Phil developed a medical triage chatbot for Medibank's Nurse on Call service and built an automated conversational testing platform for it, so regulated healthcare is familiar ground. That said, safety and compliance testing as a distinct discipline isn't something I can confirm as a service — it's likely adjacent to the AI Evaluation & Testing work, but Phil is the right person to ask directly at phil@trompit.com.au. If you'd like, I can take your details so he can come back to you on exactly what your regulatory context needs.
Yes, Tromp IT regularly works through formal procurement processes and can provide all necessary documentation for vendor onboarding. Phil has guided evaluations and analytics projects inside many large organisations where formal purchasing procedures are standard practice. If you are preparing a brief or need a capability statement for your procurement team, sharing your name and email lets me send over Phil's contact card and have him follow up directly.
Yes, Tromp IT can work within government procurement processes. Phil has experience engaging through formal procurement, including with large organisations in Australia. The business is set up to provide the information and documentation needed to support a vendor assessment.
Yes, Tromp IT can absolutely accommodate government procurement processes. Phil often works as a specialist subcontractor within larger consulting and delivery teams, which is a common way to engage on public sector projects. If you have a panel or a specific procurement framework you need to use, Phil can discuss how to make that work. What is the best email address to send his contact card to?
Yes — Tromp IT works as a specialist subcontractor within larger consulting and delivery teams as well as directly with organisations, so fitting into a procurement or panel process is workable. The Araza engagement for enterprise cloud optimisation is an example of Phil delivering within someone else's delivery team rather than as the direct contracted party. The specifics of your procurement framework would be one for Phil to work through directly with you — I'd suggest getting in touch with him at phil@trompit.com.au to talk through what your process requires.
Procurement processes are a normal part of working with government clients, and it's not something that would rule Tromp IT out. The best person to speak to about the specifics — panel arrangements, SOW structures, subcontracting pathways — is Phil directly at phil@trompit.com.au. If it's helpful, I'm happy to take your details so Phil can reach out and have that conversation with you.
Yes, in principle. Tromp IT works with organisations through scoped engagements with agreed deliverables, effort and cost, which can fit a formal procurement process. Phil Robin is the right person to discuss the specifics, especially if you need supplier details, subcontracting arrangements or procurement documentation.
Tromp IT works directly with organisations and also as a specialist subcontractor within consulting and delivery teams, so there's flexibility in how an engagement is structured. But whether a particular procurement process or panel arrangement can be met is a question for Phil Robin, who founded Tromp IT — it's not something I can confirm from here. You can reach him at phil@trompit.com.au, or I can take your name, email and phone and have him come to you.
Yes, we can. AI Evaluation and Testing is platform-independent, so it can be designed to operate entirely within your existing security and infrastructure controls. That way, your data stays under your control and nothing needs to leave your environment.
Yes, Tromp IT can evaluate on-premises systems without cloud connectivity. The evaluation can be designed to run within your infrastructure, keeping data local and under your control. Phil has experience testing AI systems in secure environments, and the process starts by understanding your setup and defining what success looks like. Evaluation methods can be adapted to work with your constraints, using local APIs or exported outputs.
Yes, we can absolutely evaluate your on-premises LLM. Tromp IT's evaluation architecture is platform-independent and can be designed to run entirely within your existing security and infrastructure controls. Phil has extensive experience designing custom evaluations that keep sensitive data under your control, often using synthetic or anonymised data to respect residency constraints. This fits right into our AI Evaluation & Testing service, where we start by understanding your specific setup and constraints before defining the test methods.
Yes — evaluation is platform-independent, so an on-prem model isn't a barrier in itself. The bigger question is how the evaluation itself gets run: if there's no cloud connectivity at all, that shapes whether an external LLM-as-judge panel can be used, or whether the evaluation architecture needs to sit entirely within your own environment. That's exactly the kind of constraint Tromp IT's Define step is meant to work through before anything starts — agreeing scope and method around what your infrastructure actually allows.
Yes — the evaluation can be designed to work within that constraint. Phil has handled sensitive environments before, and the approach is to build the evaluation architecture around the client's infrastructure rather than pulling data out of it. In practice that means the test design, tooling and judge framework run inside your environment rather than routing anything externally. The specifics would need to be scoped with Phil directly, since the right approach depends on what the system does and what access is feasible. If you'd like Phil to follow up, I'm happy to take your details — or you can reach him at phil@trompit.com.au.
Yes — that kind of constraint is workable, but the evaluation design has to respect it from the start. Tromp IT’s AI Evaluation & Testing work can be designed around on-premises and no-cloud environments, using agreed test data, local execution paths and evidence that stays under your control. Phil Robin has evaluated conversational and LLM systems across different architectures, so the key first step would be defining what evidence you need without assuming cloud services are available.
That depends on how "no connectivity" is enforced, but in principle yes — evaluation can be designed to operate within your existing security and infrastructure controls, and Phil prefers synthetic or anonymised test data wherever practical anyway. The harder question is what the judging layer looks like, since multi-judge evaluation usually leans on external models; that's a constraint to design around rather than a blocker. Phil is the right person to work through the specifics of your environment — he's at phil@trompit.com.au.
Yes, that is well within scope. Phil frequently evaluates pre-built commercial tools so organisations know what they are actually buying before committing. Would you like to share your name and email so Phil can follow up with you about it?
Yes, that's exactly in scope. Phil has done this kind of independent evaluation work before, including for enterprise chatbot platforms and RAG systems — assessing accuracy, reliability and cost before go-live. Tromp IT's evaluation process is vendor-independent, so we can test the tool objectively regardless of the provider.
Yes, that is absolutely in scope for Tromp IT. Phil has extensive experience in AI Evaluation & Testing, which includes independently testing and comparing third-party AI platforms before you commit. We can design a custom evaluation to pressure-test the vendor's tool against your specific data and requirements so you know exactly how it performs. What kind of AI tool are you looking to purchase?
Yes, that's squarely in scope — AI Evaluation & Testing doesn't require you to have built the system, just to need honest evidence about how it performs. The process starts with Understand and Define: agreeing what "working" means for your use case and what evidence would actually prove it, before any testing happens against the vendor's tool. Phil has done this kind of independent evaluation work before, including a full published comparison run on Ada, the assistant you're talking to now, so you can see the method rather than take it on faith. One thing worth knowing upfront: this works whether the tool sits on AWS, Azure, OpenAI, Anthropic or another platform — vendor-independence is the whole point of using someone outside the sales conversation. What kind of tool is it you're evaluating?
That's exactly the kind of work Tromp IT does — evaluating a system before commitment, not after. Phil's background is in independent AI evaluation, which means assessing what a system actually does rather than taking the vendor's word for it. The typical shape of that engagement would be: agree what "good enough" looks like for your use case, design a test set that reflects real conditions, run the evaluation, and give you clear findings before you're locked in. That maps directly to Tromp IT's AI Evaluation & Testing service. What kind of tool is it — a chatbot, a document or RAG system, something else?
Yes, that is very much in scope. Tromp IT’s AI Evaluation & Testing work is vendor-independent, so Phil can help pressure-test an off-the-shelf chatbot, RAG system or agentic AI tool without having any stake in the platform. The useful starting point is usually to define what evidence you need before signing: failure modes, test coverage, cost, latency, groundedness and any contractual risk areas.
Yes, that's squarely in scope. Independent evaluation of a vendor's system before you commit is exactly what AI Evaluation & Testing is for — Tromp IT doesn't sell platforms, so there's no conflict in telling you what the thing actually does. Phil has built test sets and judge frameworks for this kind of work, including a published comparison run against Ada, the assistant you're talking to now. The starting point would be agreeing what "good enough to sign" means for you, since that shapes the test design.
Tromp IT is based in Melbourne and focused on the Australian market, but Phil has delivered offshore before, including banking assistants built for Hong Kong. Engagements can be structured around different time zones depending on what is needed.
Tromp IT is based in Melbourne and primarily works with Australian organisations, but Phil has previously delivered projects for teams in Hong Kong and other regions. The focus now is on Australian clients, but remote collaboration across time zones can be practical depending on the engagement. Phil typically works directly with organisations or as a specialist within broader delivery teams, so coordination can be structured to suit distributed teams.
Tromp IT is Australian-based and Australia-focused, but Phil has delivered complex analytics and conversational projects offshore before, including in Hong Kong. While the business primarily focuses on Australian organisations, we can absolutely discuss how to structure an engagement to support your team across these locations. How does your team currently collaborate on AI projects, and what specific systems are you looking to evaluate?
Tromp IT is Melbourne-based and its engagements are with Australian organisations — that's the honest answer on where the business itself operates. Phil's earlier career work wasn't limited to Australia, though — his banking assistant work was delivered in both Australia and Hong Kong, so working across locations isn't unfamiliar territory. For a team spread like yours, the practical question is where the system and its data actually sit rather than where your people are, and that's worth discussing directly with Phil at phil@trompit.com.au.
Tromp IT is based in Melbourne and focused on Australian engagements — that's the honest answer. Phil has delivered work offshore before, including banking assistant projects in Hong Kong, so international collaboration isn't foreign territory, but Tromp IT's current practice is Australia-focused. Practically speaking, a lot of evaluation and analytics work can be done asynchronously — test design, analysis and reporting don't require everyone on a call at the same time. Whether the time-zone spread across Sydney, Singapore and London is workable for what you have in mind is really a conversation for Phil. What's the problem you're trying to solve?
Tromp IT is Melbourne-based and Australia-focused, so current engagements are with Australian organisations. Phil has delivered offshore before, including banking assistant work in Australia and Hong Kong, but I shouldn’t imply a standing international delivery model. For a Sydney-led engagement with stakeholders in Singapore or London, it would be worth asking Phil directly at phil@trompit.com.au.
Tromp IT is Melbourne-based and works with clients nationally, so Australian hours are the default. Phil has delivered offshore before — his banking assistant work ran in Australia and Hong Kong — so working with a distributed team isn't new ground. How a specific engagement would handle Singapore and London hours is really a question for Phil directly at phil@trompit.com.au.
That is quite common, and it never gets complicated when the roles are clear. Phil regularly works alongside in-house teams as a specialist advisor, bringing an independent perspective to AI evaluation while your team focuses on building and maintaining the models.
Phil regularly works alongside in-house teams, including data scientists, as a specialist subcontractor or direct advisor. Tromp IT focuses on collaboration, not duplication — bringing independent evaluation methods to complement existing expertise. This approach has worked well in past engagements with organisations like Telstra and Medibank.
Phil actually works alongside in-house teams quite often, either directly or as a specialist subcontractor within a broader team. We find it keeps things simple because we focus purely on independent evaluation and testing, rather than trying to build or run the models ourselves. We can help your team set up structured testing frameworks or design LLM-as-judge evaluations to give them clear, objective feedback on their work.
Not complicated at all — that's actually a common setup. Tromp IT often works as independent evaluators alongside existing teams, bringing an outside perspective specifically because internal teams built the system and have a stake in how it's judged. Phil's approach follows the same four steps regardless — Understand, Define, Evaluate, Recommend — with your data science team as key collaborators on scope and test design rather than the subject of the evaluation.
Working alongside an in-house team is actually the more common situation — it rarely gets complicated. Phil typically comes in as a specialist layer on top of what the team already does, focused specifically on evaluation rigour: test set design, failure analysis, and making sure the scoring methodology holds up to scrutiny. The practical split tends to be that the internal team knows the system and the domain, while Tromp IT brings independence and structured evaluation methods they may not have the bandwidth or specific tooling for. If it's useful, I'm happy to talk through what you're working on — that would help me say more concretely where the overlap might be.
Yes — that’s a normal way for Tromp IT to work. Phil has worked as a specialist inside larger delivery teams, including analytics and cloud optimisation engagements, so the role can be advisory, hands-on, or an independent evaluation layer beside your team. The important part is defining scope and evidence up front, so the work complements your data science team rather than duplicating it.
It works fine — Tromp IT often works as a specialist subcontractor inside existing consulting and delivery teams, not just directly with organisations. The usual reason an in-house team brings in someone like Phil is independence: he has no stake in whether the system passes, and a score isn't evidence unless you can explain how it was produced. The Define step matters most in that setup — scope, success criteria, test data and methods get agreed before any work starts, so there's no ambiguity about who's doing what.
Not at all, since Tromp IT works with both large corporations and small companies, and engagement size is never a barrier either way. Phil works directly with organisations to evaluate how AI systems actually perform and where they fail. What kind of AI recommendations are you experimenting with right now?
No, you wouldn't be outside Tromp IT's usual scope at all. Phil has worked with organisations across sectors, from large enterprises to smaller companies, and often helps non-tech businesses evaluate whether their AI systems are delivering real value. AI Evaluation & Testing is vendor-agnostic and focuses on whether the system works — something that's just as important for retailers as it is for tech firms.
You are definitely not outside our clientele — Phil has worked with businesses of all sizes, from large corporations to smaller companies. While product recommendation systems are slightly adjacent to our core focus on conversational AI and RAG systems, our Cloud Cost Analysis or Analytics & Decision Support services might still be a great fit to help you measure the value of your experiments. What specific aspects of your recommendation engine are you hoping to evaluate or measure?
Not at all — client size isn't a barrier, and Phil has worked across both large corporations and small companies. That said, I should be straight with you: recommendation systems specifically aren't something I can confirm as covered by our brief here, so that particular use case is one to raise directly with Phil. What Tromp IT does cover well is evaluating conversational AI and RAG-based systems — if the recommendation engine has any chat or query interface sitting on top of it, that's squarely AI Evaluation & Testing territory. Is the AI-driven part a conversational or search-style interface, or is it working more behind the scenes?
Not at all — Tromp IT has worked with organisations across financial services, healthcare and telco, so the sector isn't the deciding factor. What matters is whether there's an AI system that needs rigorous, independent evaluation. The honest answer on recommendations specifically: evaluating recommendation systems isn't something I can confirm as established Tromp IT capability, so Phil is the right person to answer that directly. What Phil does do very well is assess whether an AI system is actually performing — measuring output quality, failure modes and business outcomes rather than just headline metrics. If you're at the stage of "is this thing working, and how would we even know?", that's exactly the kind of question Tromp IT is built to answer. Would it help to share a bit more about what you're trying to evaluate, so I can tell you where the fit is strongest?
Not at all on client size or sector. Phil has worked with both large corporations and smaller organisations, and engagement size is not a barrier. AI-driven product recommendations are adjacent to Tromp IT’s AI evaluation and analytics work, but I can’t confirm from here that it’s a standard service area. The useful next step would be to describe what you need measured — quality, business impact, cost, or failure cases — and Phil can say whether it fits.
No, size isn't the issue — Phil has worked with both large corporations and small companies, and Tromp IT works across sectors rather than just tech. The honest caveat is on the subject matter: recommendation systems aren't something this brief covers, so I can't confirm it's work Tromp IT takes on. It may sit adjacent to the AI evaluation work Phil does, but he's the right person to ask directly — phil@trompit.com.au.
Three weeks is tight for a full custom evaluation, but it can work for a targeted review of a specific system if we narrow the scope right away. Phil handles this kind of work through our AI Evaluation and Testing service, focusing on the core failure modes that matter most for your board. What is the system you need evaluated?
Three weeks can be realistic for a focused evaluation, depending on the system's scope and data readiness. Phil has delivered board-ready AI evaluations in similar timeframes, such as the published Ada comparison and work with Medibank. The key is starting immediately with clear success criteria and representative test cases.
Three weeks is tight, but it can be realistic depending on the scope of the system and the depth of evidence your board needs. Phil Robin, Tromp IT's founder, often works within tight deadlines by focusing on a targeted, high-value subset of test cases first. To make this work, we would use our defined process to quickly agree on the scope and test data in the first few days. What kind of AI system are you presenting to the board, and what are the main concerns you need to evaluate?
Three weeks can be workable, but it depends on scope — a well-defined evaluation of a single system with clear success criteria is very different from an open-ended "tell us if our AI is good" brief. The Define step matters most here: agreeing scope, test data and what evidence the board actually needs before work starts is what makes a tight timeline realistic rather than rushed. That's not a commitment on timing though — Phil would need to hear the specifics to say what's achievable. Happy to pass this to him if you'd like — what system are you needing evaluated?
Three weeks is tight but not automatically ruled out — it depends on what's being evaluated and how clearly the scope can be defined upfront. Phil's engagements follow a structured process (Understand, Define, Evaluate, Recommend), and the Define step is what makes or breaks a short timeline: if you can get to agreed scope and test data quickly, the evaluation itself can move fast. What system are you looking to evaluate?
Yes, three weeks can be realistic if the evaluation is tightly scoped and the evidence needed for the board is clear. Phil has done this kind of AI Evaluation & Testing work, including published evaluation of Ada, the assistant on this site, with model comparison, judging and failure analysis. The key is to start with the board decision: what do they need to trust, approve, stop, or fund?
I can't commit to a timeline on Phil's behalf — he'd need to hear what the system is and what evidence the board actually wants before saying what fits in three weeks. What I can say is that engagements are scoped up front, with deliverables, effort and cost agreed before work starts, so you'd know early whether the window is workable. The fastest path is to email him directly at phil@trompit.com.au. If you'd rather, give me your name, email and phone and I'll pass this straight to him.
That is definitely a conversation worth having, as Phil regularly works as a specialist subcontractor within consulting and delivery teams. He focuses purely on independent AI evaluation and testing, leaving you to manage the client relationship. If you'd like to share a bit more about what you're working on, I can make sure he has the context before he gets in touch.
Phil has worked as a specialist subcontractor within consulting teams, including for analytics and AI evaluation work. Tromp IT doesn't resell platforms or tools — the focus is on independent, custom evaluation that can be delivered under clear attribution. If you'd like to discuss how this could work for your clients, I can pass your details to Phil.
That is definitely a conversation worth having. Phil regularly works as a specialist subcontractor within consulting and delivery teams to provide independent AI evaluation. To help Phil understand how you'd like to work together, could you share your name, email, and phone number, along with a quick note on the kinds of AI systems your clients are building?
That's worth having, but let me be direct about the shape of it. Tromp IT works both directly with organisations and as a specialist subcontractor within consulting and delivery teams — the Hoot platform work with Dirawong and the AWS cost work with Araza were both delivered that way. So the working model fits; whether white-labelling specifically is something Phil wants to set up is his call, not mine to promise on his behalf. The best next step is a direct conversation with Phil at phil@trompit.com.au — I can pass along your details and what you're looking for if you'd like.
Yes — that's exactly the kind of arrangement Tromp IT works within. Phil has delivered evaluation and analytics work as a specialist subcontractor inside other consulting and delivery teams, so operating under your brand rather than his own is familiar territory. The most useful next step is probably a direct conversation with Phil. If you share your name, email and phone number, I can make sure he has your details and the context of what you're looking for.
Yes, that is a conversation worth having. Tromp IT works both directly with organisations and as a specialist subcontractor within consulting and delivery teams, especially around AI Evaluation & Testing. Phil Robin has deep experience in chatbot, RAG and conversational AI evaluation, and you can reach him at phil@trompit.com.au.
Yes, and it's a familiar shape — Tromp IT works both directly with organisations and as a specialist subcontractor inside consulting and delivery teams. Phil Robin runs it, and most of the experience listed on the site was delivered that way, within other teams' engagements. The white-label specifics — how it's badged, what goes in front of your clients — are Phil's call rather than mine, so he's the one to talk to at phil@trompit.com.au.
That is a clear evaluation challenge, not too niche at all. Phil can assess the quality, accuracy and failure modes of code generated or reviewed by AI. It falls squarely under AI Evaluation & Testing, where the work begins by agreeing on success criteria and test sets before evaluating the outputs.
That fits well within AI evaluation and testing — assessing AI-generated code for accuracy, safety and clarity is a valid use case. Phil has evaluated conversational AI and tooling across complex domains, including agentic systems and RAG pipelines. The same principles apply: define what good looks like, design tests, evaluate outputs, and deliver actionable findings.
That is not too niche at all. Phil has extensive experience with complex conversational and agentic AI systems, and evaluating an AI code reviewer fits perfectly within our AI Evaluation & Testing service. We would typically start by helping you define what a "good" code review looks like for your platform, and then design a rigorous framework to test it.
Code review isn't something explicitly covered in what I can speak to — the evaluation work I know of is chatbots, RAG systems and agentic AI rather than code-generation or code-review models specifically. That said, the underlying approach (test set design, failure analysis, LLM-as-judge frameworks) is fairly general to LLM-based systems, so it may well be adjacent to what Tromp IT does. Phil would be the right person to ask whether this particular case fits — happy to pass your details to him if you'd like.
Not too niche at all — the core question is whether the AI is doing what you need it to do, and that's exactly the kind of problem Phil Robin at Tromp IT is built to work on. The evaluation approach would likely focus on things like whether the generated reviews are accurate, grounded in the actual code, and consistent across similar inputs — which maps directly to the AI Evaluation & Testing service. The starting point would be defining what "a good code review" looks like for your platform, then building test cases that can reveal where the system fails or hedges when it shouldn't. What does your current setup look like — are you trying to baseline performance before a release, or is this more about diagnosing something that's already in production?
Not too niche in principle. I can’t claim Tromp IT has a listed code-review case study, but it fits Phil’s AI Evaluation & Testing work if the task is to measure whether the system’s reviews are accurate, grounded in the code, useful to developers, and worth the cost. The key would be defining the evidence standard up front, including what “good review” means in your engineering context.
It's not too niche in principle — evaluating AI output quality is the core of what Tromp IT does, and the four-step approach (Understand, Define, Evaluate, Recommend) doesn't change much because the output is code. What would matter is agreeing up front what a "good" review looks like for you, since that's harder to pin down than a factual answer. Code review specifically isn't something I can confirm Phil has evaluated before, so he's the one to ask about fit — phil@trompit.com.au.
Engagement size is not a barrier at Tromp IT, and there is no fixed minimum spend because every project is scoped individually based on what you actually need. Phil works with both large corporations and smaller companies. What brings you here today?
Tromp IT works with organisations of all sizes and there's no minimum spend. Phil typically starts by understanding your specific needs and then tailors the engagement to fit both the problem and your budget. We can scope work as a focused evaluation or integrate within existing teams to keep things lean.
Tromp IT works with organisations of all sizes, and Phil has delivered engagements for small companies as well as large corporations. We don't have a strict minimum spend; instead, we scope each project up front with agreed deliverables, effort, and cost to fit your situation. Depending on your needs, we can design a one-off bespoke evaluation or help you set up an ongoing testing platform to keep costs manageable.
I can't quote pricing or minimums — engagements are scoped individually, so that's a question for Phil directly at phil@trompit.com.au. What I can say is that budget size hasn't been a barrier — Phil has worked with both large corporations and small companies, and engagements are scoped up front around the specific problem rather than a fixed package. If you tell me a bit about what you're trying to evaluate or measure, I can help you think through whether it's the kind of thing that needs a big engagement or a much smaller one.
Tromp IT doesn't publish minimum spend or standard tiers — every engagement is scoped individually based on what the problem actually needs. Phil has worked with both large organisations and small ones, so size isn't a barrier either way. The most practical thing is usually a focused scope: a well-defined question, a bounded system, agreed deliverables. That keeps effort — and cost — proportionate to what you actually need answered. If you want to share a bit about what you're trying to evaluate or measure, I can give you a better sense of whether it's a good fit, and you're welcome to reach out to Phil directly at phil@trompit.com.au to talk specifics.
Engagement size is not a barrier; Phil has worked with both large corporations and small companies. Tromp IT scopes work up front with agreed deliverables, effort and cost, so a smaller focused evaluation can be designed if that is what the budget supports. There isn’t a published minimum spend I can quote here; Phil is the right person to discuss the practical scope.
Engagement size isn't a barrier either way — Phil has worked with both large corporations and small companies. Every engagement is scoped up front with agreed deliverables, effort and cost, so you'd know what you're committing to before work starts. I can't quote pricing or confirm whether there's a minimum, though — that's a question for Phil at phil@trompit.com.au.
That is actually the exact environment where evaluation matters most, because API updates can break your prompts or quietly shift your output quality without you noticing. Phil works on this through Tromp IT's AI Evaluation and Testing service, where we design test sets and evaluation frameworks that you can re-run every time your provider pushes a change.
Not at all — frequent model changes actually make evaluation more important, not less. Phil has evaluated systems under similar conditions, using structured testing to detect performance shifts and isolate issues caused by backend changes. The key is designing tests that track consistency and regression over time, which is part of Tromp IT's standard evaluation approach.
It actually makes evaluation more critical, not pointless. Phil has designed testing frameworks to catch exactly these kinds of silent failures and model drift when APIs update. Under our AI Evaluation & Testing service, we can set up automated, regression-style testing to run continuously so you know the moment an update degrades your system's performance.
Not pointless — it just means the evaluation needs to account for that instability rather than assume a fixed target. If the model underneath can shift without notice, that's actually a strong argument for ongoing structured testing rather than a one-off assessment, so you catch regressions when they happen instead of finding out from users. This is exactly what AI Evaluation & Testing is for: define what "good" looks like once, then re-run that test set whenever you suspect drift, comparing outputs against the same criteria each time. Tromp IT's published Ada comparison ran the same question set across seven different model backends, which is close to the situation you're describing — same task, different model underneath.
Not at all — it actually makes evaluation more important, not less. If the model underneath can change without notice, you need a way to detect when behaviour has shifted, which is exactly what a repeatable test set gives you. Phil has done this kind of work directly: the published Ada evaluation ran the same question set across seven different model backends and compared them on accuracy, groundedness, latency and cost. That's a reusable framework, not a one-off snapshot. The practical answer is to design the evaluation so it can be re-run cheaply when a model update lands — that way a change becomes detectable rather than invisible.
No — it makes evaluation more important, but the design has to account for instability. A one-off score can go stale quickly, so the useful evidence is usually a repeatable test set, clear failure categories and a way to compare behaviour across model versions or dates. This fits Tromp IT’s AI Evaluation & Testing work. Phil has built this kind of evidence-led evaluation around conversational systems, including Ada’s published multi-model comparison on this site.
No — if anything it makes the case for it. A fixed test set is exactly what tells you whether a silent model change moved your accuracy, because you can re-run it and compare. That's the core of Tromp IT's AI Evaluation & Testing work, and Phil built precisely that comparison for Ada: one 189-question set run across seven model backends, judged independently, with accuracy, latency and cost reported side by side.
Tromp IT works with both large corporations and small companies, and engagement size is never a barrier either way. Phil can scope an evaluation to fit a lean operation so you get clear answers without paying for enterprise overhead. What kind of AI support tooling are you currently running?
Tromp IT works with organisations of all sizes, including small and resource-constrained ones. Phil focuses on delivering clear, actionable outcomes without unnecessary overhead, and can tailor engagements to fit your needs and capacity. He’s supported projects like Medibank’s Nurse on Call triage bot and has deep experience in evaluating conversational AI for real-world impact — exactly the kind of work that matters for a support triage system.
Tromp IT is definitely an option for you because Phil works with organisations of all sizes, and engagement size is not a barrier. We can tailor an AI Evaluation & Testing engagement specifically to fit a smaller budget and limited resources. How are you currently measuring whether your triage AI is classifying those support requests accurately?
Yes — Phil has worked with both large corporations and small operations, so scale isn't a barrier to engaging Tromp IT. What matters more is having a clear question: are your triage responses accurate, are they missing things they shouldn't, where do they fail? That's exactly what AI Evaluation & Testing is for — starting with Understand and Define, so the scope matches what you can realistically resource, rather than a large-scale exercise you don't need. Even a small, tightly-scoped evaluation can tell you a lot about whether the triage system is doing its job.
Size and budget constraints aren't a barrier — Phil has worked with organisations at both ends of the scale, and a small, focused engagement can still produce real findings. For a triage system specifically, the risk of failure is high-stakes: misclassified requests mean people don't get help when they need it. That's exactly the kind of system where independent evaluation — test set design, failure analysis, understanding where and why it gets things wrong — is worth doing even at modest scale. The honest answer is that scope and cost are worked out individually, so the best thing is a direct conversation with Phil. Would it help if I took your details so he can reach out, or would you prefer to email him directly at phil@trompit.com.au?
Yes — small size is not a barrier. Phil has worked with large organisations and small companies, and Tromp IT scopes engagements up front so the effort and deliverables match the problem and budget. For an AI triage system, the most relevant service is AI Evaluation & Testing: checking whether it routes requests correctly, where it fails, and what evidence you can rely on.
Yes — engagement size isn't a barrier either way. Phil has worked with both large corporations and small companies, and engagements are scoped up front with agreed effort and cost, so you know what you're committing to before work starts. Triage is familiar ground too: Phil built a medical triage chatbot for Medibank's Nurse on Call service and set up automated conversational testing for it. If you want to talk through what a small, focused evaluation might look like, Phil's at phil@trompit.com.au.
Yes, evaluating high-stakes financial advice assistants is central to Tromp IT's AI Evaluation and Testing work. Phil works closely with systems where accuracy, groundedness, and regulatory adherence are non-negotiable, designing custom test sets and multi-judge evaluation frameworks to catch failures before real users do. If you'd like to share a bit about your setup, I can take your name, email, and phone number so Phil can follow up with you directly.
Yes, Tromp IT is set up to evaluate AI in high-stakes, regulated environments like financial advice. Phil has direct experience in this area, having worked on enterprise banking assistants for Westpac, BT and Standard Chartered, with a focus on intent accuracy, language coverage and compliance-sensitive performance. The evaluation approach is designed to test not just correctness but groundedness, risk of hallucination and adherence to policy — all critical in regulated domains.
Phil has extensive experience in high-stakes financial environments, including building conversational models for Westpac, Bankers Trust, and Standard Chartered Bank. For highly regulated spaces, we design the evaluation architecture to operate entirely within your existing security and infrastructure controls. This ensures your sensitive data stays under your control, and we can use synthetic or anonymised data for the testing. Is this an assistant that is already in production, or are you currently building it?
High-stakes and regulated environments are exactly where independent evaluation matters most — you need to know your assistant fails safely, not just that it sounds fluent. Tromp IT's AI Evaluation & Testing service covers test set design, failure analysis and LLM-as-judge frameworks that can be built around your specific compliance concerns, following the Understand and Define steps before any testing starts, so the scope and success criteria are agreed up front. One caveat worth being direct about: financial advice-specific regulatory certification or compliance sign-off isn't something I can confirm Tromp IT does. What it can do is independently test whether the assistant behaves correctly, consistently and safely against criteria you define — that's a evaluation question, not a compliance-certification one, and Phil would be the right person to talk through where that line sits for your situation.
Yes — high-stakes, regulated environments are exactly where rigorous evaluation matters most, and it's the kind of work Phil Robin, who founded Tromp IT, has direct experience with. The banking assistant work he delivered for Westpac, BT and Standard Chartered involved tuning intent and language models in a regulated financial context where accuracy and appropriate responses aren't optional. For a financial advice assistant, the evaluation design would need to reflect what "correct" actually means in your context — which is more than fluency or user satisfaction. That means defining failure modes specific to your compliance requirements, building test sets that probe those specifically, and using a judge framework that can distinguish a compliant answer from a plausible-sounding one that isn't. On data sensitivity: evaluation can be designed around your constraints, using synthetic or anonymised test data, and structured to operate within your existing security controls if needed. Is this something you're looking to evaluate now, or still scoping what's needed?
Yes — that maps closely to Tromp IT’s AI Evaluation & Testing work, especially where the question is whether the assistant is giving reliable, grounded and auditable answers. Phil has worked on enterprise banking assistants and on production performance measurement in large-scale customer systems, but regulatory advice compliance would need your subject-matter and legal criteria built into the evaluation design. The work would start by defining the evidence needed, then testing against agreed scenarios and failure conditions rather than relying on a generic accuracy score.
High-stakes conversational systems are squarely what AI Evaluation & Testing is built for — test set design, failure analysis and multi-judge evaluation rather than a single score you have to take on faith. Phil has done this in regulated settings before, including tuning conversational and intent models for enterprise banking assistants at Westpac, BT and Standard Chartered, and building a medical triage chatbot for Medibank's Nurse on Call. What I can't confirm is whether that extends to formal regulatory compliance testing against specific financial advice rules — that's a question for Phil directly.
Yes, that is precisely the kind of evaluation Phil specialises in under our AI Evaluation & Testing service. Contract extraction involves subtle reasoning failures that standard keyword matching or basic benchmarks completely miss. We typically start by defining what correct extraction looks like, building representative test data, and running systematic failure analysis. Would you like to share your name and email so Phil can follow up with you directly about your extraction pipeline?
Yes, Tromp IT specialises in evaluating nuanced failure modes in LLM applications like yours, particularly in high-stakes domains such as legal tech. Phil has extensive experience in AI evaluation and testing, including designing test sets to uncover subtle issues in outputs — for example, missing conditions, misclassified obligations or incorrect entity mappings — using structured, multi-judge evaluation frameworks. This follows the full four-step process: Understand, Define, Evaluate, Recommend, tailored to your system.
That is exactly the kind of work Phil does under our AI Evaluation & Testing service. Subtle failure modes in extraction are a common challenge, and we design custom LLM-as-judge frameworks and multi-judge panels to detect them. We look at the actual evidence behind the outputs rather than just accepting a simple accuracy score.
Yes — that's a good fit for AI Evaluation & Testing. Subtle extraction errors are exactly where a proper test set and failure analysis matter more than a simple accuracy score, because "close but wrong" clause boundaries or missed conditional language often don't show up until you specifically design tests to catch them. Phil's evaluation work includes building LLM-as-judge and multi-judge frameworks for exactly this kind of nuanced, hard-to-score output, including on his own published Ada comparison. The approach would start with Understand and Define — agreeing what counts as a correct extraction and where the ambiguous cases lie — before any testing happens.
Yes, that's exactly the kind of work Phil does. Subtle failures in clause extraction — wrong boundaries, missed obligations, plausible-sounding but incorrect output — are precisely what standard benchmarks tend to miss, because they're designed for cleaner tasks. The evaluation approach here would start by understanding what "correct" actually means for your specific contracts and clause types, then designing test cases that expose the failure modes that matter, not just the obvious ones. The output is evidence and prioritised findings, not just a score. If you'd like to explore what an engagement might look like for your system, it'd help to know who I'm talking to and the best way for Phil to reach you.
Yes — that fits Tromp IT’s AI Evaluation & Testing work, especially where simple “right/wrong” scoring misses the real risk. Phil’s focus is on designing evaluations that expose subtle failure patterns, not just producing a headline score. For a clause extraction system, the useful work would start by defining the evidence you need, then testing against representative examples and analysing where the model fails.
Extraction with subtle failure modes is exactly the kind of problem AI Evaluation & Testing is built for — the hard part is usually defining what counts as a miss before you start testing, not the testing itself. Phil's work on Ada, the assistant you're talking to, used a four-model judge panel and reported the questions the panel couldn't agree on rather than quietly resolving them, which matters when the errors are borderline by nature. That said, contract clause extraction is a specific domain, so Phil is the one to confirm how well it maps — he's at phil@trompit.com.au.
Yes, Tromp IT can help with independent evaluation, but early pre-revenue engagements need careful scoping around budget and stage. Phil works with businesses of all sizes, though formal engagements are typically scoped up front with defined deliverables. What specific aspects of your AI product do you need evaluated for investors?
Yes, that’s a realistic engagement — Tromp IT can provide independent evaluation and credible evidence of your AI product’s performance. Phil has helped early-stage projects validate conversational AI and RAG systems using transparent, rigorous methods, including the published evaluation of Ada, the assistant you're talking to. The focus would be on defining what success looks like, then designing tests that demonstrate it credibly to investors.
That is a highly realistic engagement, and Tromp IT regularly works on custom evaluations for specific systems. Phil Robin, our founder, can help design and execute an independent testing framework that provides the objective evidence and validation investors look for. To help me understand how we might scope this, what does your AI product actually do, and what is your name?
Realistic in principle — engagement size isn't a barrier either way, and Phil has worked with both large corporations and small companies. What matters is whether there's a real system to test and clear enough claims about what it should do that we can design a test set against them. That's exactly what AI Evaluation & Testing is for: starting with Understand and Define — agreeing what "working" means and what evidence would prove it — before any testing happens. For an investor audience, groundedness and failure-rate evidence tend to matter more than a single accuracy score, since that's what shows you actually understand your own product's limits.
Yes, that's a realistic engagement — and it's a common gap for early-stage AI products. Independent evaluation gives you something concrete to show investors: not "here's our internal benchmark" but "here's what an external specialist found when they tested it." Phil Robin has done exactly this kind of evaluation work — designing test sets, running multi-judge frameworks, analysing where a system fails and why, and producing findings that are evidence-based rather than promotional. That's the core of Tromp IT's AI Evaluation & Testing service. The honest caveat: engagements are scoped and priced individually, so whether it works at your stage is a conversation worth having with Phil directly. His email is phil@trompit.com.au. If it's useful, I'm happy to take your details so Phil can reach out — or tell me a bit more about what your product does and I can give you a sharper sense of fit first.
Yes, that can be realistic if the scope is kept tight and the evidence is useful to investors. Tromp IT’s AI Evaluation & Testing work is independent, so the output would focus on what was tested, how it was measured, where it failed, and what the results actually support. Phil has worked with both large organisations and small companies, so being pre-revenue is not a barrier by itself.
Yes — engagement size isn't a barrier either way. Phil has worked with both large corporations and small companies, and independent evaluation is exactly the core service: test set design, failure analysis and judge-based scoring of chatbots, RAG or agentic systems. The published Ada comparison on this site is a good example of what that evidence looks like in practice, since it reports accuracy, groundedness, latency and cost, including the questions the judge panel couldn't agree on. Scope, effort and cost get agreed up front, so it's worth putting the question to Phil directly at phil@trompit.com.au.
We evaluate multi-agent systems and agentic workflows alongside single-model setups. Phil focuses on how agents hand off tasks, where conversational state breaks down, and whether the final output matches what was asked.
Tromp IT evaluates multi-model and multi-agent systems — Phil has done this before, including multi-judge evaluations and complex handoff architectures. The process starts by defining what success means across the system, then testing each interaction point, failure mode, and handoff. It's part of our AI Evaluation & Testing service, following the Understand, Define, Evaluate, Recommend steps.
Tromp IT absolutely evaluates complex multi-agent systems. Phil recently designed a multi-judge framework for agentic AI and RAG pipelines, so we are well-equipped to handle systems with multiple model handoffs. Our AI Evaluation & Testing service is designed to assess performance, routing accuracy, and failures across these complex setups, regardless of the underlying platforms.
Yes — agentic AI systems are explicitly within scope, not just single-model setups. Multi-agent handoffs actually raise the stakes on evaluation, because failures can happen at the handoff itself, not just within a single model's response. That's AI Evaluation & Testing, and it would start with the Understand and Define steps: working out what "working correctly" means at each handoff point, what evidence you need, and how to test it before any evaluation begins. If you want, I can take some details so Phil can follow up on the specifics of your setup.
Multi-agent systems are well within scope — evaluating them is actually one of the more interesting challenges in this space, because failure can happen at the handoff points as much as within any individual model. Phil's work includes agentic AI evaluation alongside RAG pipelines and single-model chatbots, so the complexity isn't a barrier. The approach would follow the same structure: understand what "working correctly" means for your system, define the test cases and success criteria, then evaluate with a focus on where things actually break — which in a multi-agent setup often means tracing a failure back through the chain to find where it originated. What kind of tasks is your system handling?
Yes — Tromp IT’s AI Evaluation & Testing is not limited to single-model chatbots. Phil works on evaluation of agentic AI as well as RAG and conversational systems, with attention to where systems fail, what they cost, and whether the evidence behind a score is explainable. For a multi-agent setup, the work would usually start by defining what each handoff is meant to prove, then testing whether the overall chain behaves reliably.
Agentic AI is explicitly in scope — AI Evaluation & Testing covers chatbots, RAG systems and agentic AI, not just single-model setups. Handoff chains do raise the question of what you're measuring at each step, which is why engagements start by agreeing scope and success criteria before any testing happens. Phil built the published evaluation of Ada, the assistant you're talking to now, where every tool call was checked in code against what the runtime actually did rather than what the reply claimed — that verification approach matters a lot in multi-agent setups.
Voice is entirely within scope, as Phil built production analytics for Telstra's voice recognition systems and developed a medical triage chatbot for Medibank. We evaluate conversational systems regardless of whether they are text or voice-based.
Voice-based AI systems are within scope — Phil has evaluated voice recognition and conversational performance for services like Telstra’s voice systems. The focus would be on whether the assistant understands correctly, responds appropriately, and resolves inquiries, regardless of voice being the input layer. Evaluation starts with defining what success looks like for your use case.
send_contact_card
That doesn't change things at all. Phil has extensive experience with voice-based AI, including building data models to measure how Telstra's voice recognition systems performed with real callers in production. Evaluating a voice assistant still relies on understanding conversational quality, intent classification, and system failures, which fits perfectly under Tromp IT's core AI Evaluation & Testing service.
Voice adds a layer worth naming explicitly — audio quality, ASR errors, latency and interruption handling — but the underlying question is the same: is the assistant actually resolving calls well, and where does it fail? That's the kind of measurement Tromp IT does under AI Evaluation & Testing, starting with Understand and Define so the test design accounts for how the assistant is actually being used by real callers. Telstra's work with Phil is relevant here — he built the data models measuring how their voice recognition performed in production, not just how it scored in a lab. That said, I can't confirm specific ASR/transcription tooling or accent and language coverage as something we test off the shelf — that's a detail worth raising with Phil directly. If you'd like, I can take your details and he can follow up on what's feasible for your setup.
Voice-based AI still needs evaluation — the core questions are the same: does it understand what callers intend, does it respond correctly, and where does it fail? Phil has direct experience here; his analytics work for Telstra was specifically about measuring how voice recognition systems performed in production, with real callers, not just lab scores. The honest caveat is that the acoustic and speech-recognition layer — how well the system hears — is a different specialism from conversational evaluation. If your concern is whether the assistant understands and handles calls correctly, that's squarely in scope. If it's primarily about audio quality or ASR accuracy, that's worth a direct conversation with Phil to work out where Tromp IT can add value.
Yes, it can still fit, but the evaluation design changes. Phil has worked on production measurement for Telstra voice recognition systems, and Tromp IT can help assess the assistant’s behaviour, outcomes and failure patterns once calls are transcribed or otherwise measurable. If the question is pure audio engineering or telephony infrastructure, that is outside what I can confirm.
It shifts things, honestly. Phil built a medical triage chatbot for Medibank's Nurse on Call service, and at Telstra he built the data models measuring how voice recognition systems performed with real callers in production — so the voice side isn't unfamiliar ground. What I can't confirm from here is how far Tromp IT's current evaluation work extends into speech-layer testing specifically, like transcription accuracy or audio handling; that's a question for Phil at phil@trompit.com.au.
Tromp IT works with organisations of all sizes, and an academic research team evaluation project fits the kind of independent testing Phil does. Phil's work focuses on AI evaluation and testing, helping teams measure whether their systems actually work rather than relying on vendor claims. If you'd like to share a bit about what your research group is working on, we can talk about how it might fit.
Yes, Tromp IT works with organisations regardless of size or structure — Phil has supported both large corporations and smaller non-commercial teams. The focus would be on clear problem definition, rigorous evaluation methods, and actionable findings, following the standard Understand, Define, Evaluate, Recommend process. Phil has experience in research-adjacent work, including shaping evaluation frameworks for AI systems.
Yes, Tromp IT absolutely works with academic and research teams. Phil Robin, our founder, has extensive experience working with both large and small organizations, and engagement size or commercial status is not a barrier. We can help you design and execute rigorous, independent AI evaluations tailored to your research constraints.
Engagement size and organisation type aren't a barrier — Phil has worked across both large corporations and small companies, and the brief doesn't rule out academic groups. What matters more is whether there's a real system to evaluate and a question worth answering with evidence. If you can tell me a bit about what you're working on — a chatbot, RAG pipeline, or agentic system — I can give you a better sense of fit with AI Evaluation & Testing.
Client size and type aren't a barrier — Phil has worked with both large corporations and smaller organisations, and the nature of the work matters more than the sector. If you're evaluating an AI system — a chatbot, a RAG pipeline, an agentic system — and you need independent, rigorous assessment of whether it's actually working, that maps directly to what Tromp IT does. What's the system you're looking to evaluate?
Yes, potentially. Tromp IT’s work is with Australian organisations, and client size or commercial status is not a barrier if the project fits AI Evaluation & Testing. Phil Robin has worked across conversational AI, RAG-style evaluation methods and multi-judge testing, so an academic evaluation project could be a sensible conversation if the scope is clear.
Phil has worked with both large corporations and small companies, so engagement size isn't a barrier — but I can't confirm academic or research-sector work specifically, since that's not something this brief covers. The evaluation work itself would be the same shape regardless: AI Evaluation & Testing, starting with what you actually need evidence about before any test design happens. Phil is the right person to ask about fit — he's at phil@trompit.com.au.
That stage is actually where Phil often comes in. We start by helping you define what success and failure look like for the system so you have a clear brief before any testing begins. It fits right into our AI Evaluation and Testing service.
Not at all — Phil typically starts with messy, poorly defined problems like this. Tromp IT's process begins with understanding your issue and defining what success looks like, so you're well within the scope of how we work. We've helped clients in the same position, including Medibank and Telstra, to clarify and resolve inconsistent AI behaviour. The next step is to examine where the inconsistencies are occurring and what good output should look like.
No, that is not too early at all. Phil has worked with many organisations at this exact stage, where the immediate focus is simply capturing the failures to understand what is actually going wrong. In our AI Evaluation & Testing service, we start with our **Understand** and **Define** steps to help you turn those user complaints into a clear, structured test set. This gives us the baseline we need to measure and fix the inconsistencies.
Not too early at all — a clear brief usually comes *out of* this kind of engagement, not before it. That's what the Understand and Define steps are for: starting from "we know something's wrong" and working with you to pin down what "inconsistent" actually means, what evidence would show it, and what a fix would look like before any testing starts. This is a good fit for AI Evaluation & Testing — failure analysis on a live system is exactly the kind of problem it's built for. If you'd like, I can take your name and email so Phil can follow up and talk through what you're seeing.
Not at all — that's actually a good place to start. Inconsistent outputs are a symptom, and the first step is understanding what the evidence actually shows, which is exactly what Phil's evaluation work is designed to do. The AI Evaluation & Testing service starts by defining what "working correctly" should mean for your system, then builds a test set that can surface where and why the inconsistency is happening. You don't need a polished brief for that — the problem description *is* the starting point. If you'd like Phil to take a look, I'm happy to take your details so he can follow up directly.
No, that is not too early; it is often the right point to start. Tromp IT’s AI Evaluation & Testing work begins with the “Understand” step, which is exactly about turning a messy production problem into evidence you can test. Phil has worked on conversational AI performance in real systems, including production measurement for Telstra voice systems and evaluation frameworks for LLM chatbots.
Not at all — that's the normal starting point. Tromp IT's first step is exactly that: understanding the problem and working out what evidence you'd need before anything gets scoped. "Inconsistent outputs" usually turns into a test design question, which is the core of the AI Evaluation & Testing service. Phil's done this on live systems, including an automated conversational testing platform for Medibank's Nurse on Call triage bot.
That is outside what Tromp IT currently covers, as bilingual and multilingual evaluation are not part of our standard capabilities. Phil focuses primarily on English-language evaluation, conversational AI, and analytics systems. If you'd like to leave your details, I can have Phil follow up directly to discuss whether an exception or custom approach might suit your project.
That's a great question. This brief doesn't confirm whether Tromp IT currently offers multilingual evaluation capabilities specifically for English and Mandarin. It's something Phil can advise on directly — I can pass your details along so he can respond.
Multilingual evaluation is not something I can confirm that Tromp IT covers, though it is adjacent to Phil's core evaluation work. Phil is the best person to ask about whether he can support your Mandarin and English requirements. Would you like me to take your details so Phil can get in touch with you about this?
That's not something I can confirm — multilingual evaluation isn't covered in what I know about Tromp IT's current work. It may well be adjacent to what Phil does with AI Evaluation & Testing, but I don't want to overstate it. Phil's the right person to ask directly — phil@trompit.com.au — and I'm happy to take your details so he can follow up if you'd like.
Multilingual and bilingual evaluation isn't something I can confirm Tromp IT covers — it's not listed in the brief, and it would be wrong to imply it does without checking with Phil. It's plausibly adjacent to the evaluation work Phil does, so it's worth putting the question directly to him at phil@trompit.com.au. If you'd like to share your name and email, I can also have him reach out to you.
Multilingual evaluation is adjacent to Tromp IT’s AI Evaluation & Testing work, but I can’t confirm from the published service brief that it is a standard capability. Phil Robin is the right person to ask, because he would need to look at the language coverage, test data, and scoring method before saying whether Tromp IT can help. His email is phil@trompit.com.au.
Honestly, multilingual evaluation isn't something I can confirm as a stated service — it may well be adjacent to what Tromp IT does, but Phil is the one to ask. What I can say is that Phil's earlier work as a data scientist on enterprise banking assistants included intent model performance and language coverage, delivered in Australia and Hong Kong. If you'd like, I can take your name, email and phone and pass the question to him directly.
Yes, that is workable, because Tromp IT designs evaluation architecture around your specific security and infrastructure controls. Phil can build and test within your existing environment to match those requirements. If you'd like to talk through how that would fit your setup, what is your name and email so Phil can follow up directly?
Yes, evaluation can be designed to operate within your existing AWS security and infrastructure controls. Phil has done this type of work before, including cloud cost and performance analysis for enterprise AWS environments. The approach is to minimise data movement and align with your compliance requirements — Tromp IT's evaluations are platform-independent and adaptable to on-prem or private cloud setups.
Yes, that is completely workable. Phil can design the evaluation architecture to operate entirely within your existing AWS security and infrastructure controls. This ensures your data remains under your control and never leaves your secure environment.
Yes — that's workable. AI Evaluation & Testing here is platform-independent, so assessing a system built on AWS doesn't require moving it elsewhere first. For environments with real security constraints, the evaluation architecture can be designed to operate within your existing AWS security and infrastructure controls rather than asking you to relax them.
That constraint is workable — evaluation can be designed to operate within your existing AWS infrastructure rather than routing data through external services. Phil has done exactly this kind of environment-aware scoping, and it's part of how Tromp IT approaches sensitive engagements from the start of the Define step. If it's useful, I'm happy to take your details so Phil can have a more specific conversation about how that would work for your pipeline.
Yes, that is workable in principle. Tromp IT’s AI Evaluation & Testing can be designed around security and infrastructure constraints, including evaluation that stays within your AWS environment. Phil has done platform-independent AI evaluation work, so the starting point would be to define the evidence needed and the controls the evaluation must operate within.
Yes — evaluation can be designed to operate within your existing security and infrastructure controls, which is the usual answer for environments like yours. Phil's evaluation work is platform-independent by design, so an AWS-hosted pipeline isn't a constraint on the method itself, and he has separately done deep AWS work analysing an enterprise environment in financial services. The specifics of what "stays inside the boundary" means in practice — tooling, data handling, where judging runs — are worth working through with Phil directly at phil@trompit.com.au.
Yes, Tromp IT regularly operates as a specialist subcontractor within delivery teams. Phil can step in specifically for the evaluation and testing phase while working alongside your broader team.
Yes, Tromp IT regularly works as a specialist subcontractor within consulting and delivery teams, including for AI evaluation and testing phases. Phil has delivered this kind of work before, such as the automated testing platform for Medibank and shaping the Hoot platform for Dirawong.
Yes, that is exactly how Tromp IT operates. Phil frequently works as a specialist subcontractor within delivery and consulting teams to handle AI evaluation and testing. We can help with test set design, failure analysis, and setting up LLM-as-judge frameworks. Would you like to share a few details about the client's system so we can see if it's a good fit?
Yes, that's exactly the kind of arrangement Tromp IT does — working as a specialist subcontractor inside a consulting or delivery team is one of the two main ways engagements happen, alongside direct client work. Phil has done this before: the Enterprise cloud optimisation work was delivered within Araza's delivery team, in that same subcontractor model. For the AI testing phase specifically, that maps to AI Evaluation & Testing — test set design, failure analysis, LLM-as-judge frameworks, and it's platform-independent so it doesn't matter what the underlying system is built on. Happy to pass details to Phil if you want to explore fit for your engagement.
Yes, that's exactly one of the ways Tromp IT works — as a specialist advisor within consulting and delivery teams rather than always as a direct client engagement. Phil Robin, who leads Tromp IT, has done this before: the enterprise cloud optimisation work for a financial services client was delivered inside Araza's team, for example. If you can share a bit about the engagement — the type of AI system being tested and roughly what the testing phase needs to cover — I can tell you more specifically where Phil's experience maps to it.
Yes. Tromp IT works both directly with organisations and as a specialist subcontractor inside consulting and delivery teams. Phil Robin’s AI Evaluation & Testing work covers chatbots, RAG systems and agentic AI, including test design, failure analysis and LLM-as-judge frameworks. Engagements are scoped up front, with agreed deliverables, effort and cost before work begins.
Yes — that's one of the two ways Tromp IT works. Phil Robin takes engagements directly with organisations and also as a specialist subcontractor inside consulting and delivery teams, which is how the AWS cost optimisation work was delivered within Araza's team. AI Evaluation & Testing is the core service, and it's platform-independent, so it doesn't matter much what your client's system is built on.
ISO 42001 compliance is not something Tromp IT explicitly evaluates against, though our AI evaluation work often provides the evidence systems need. Phil's focus is independent testing of chatbots, RAG systems, and agentic AI through structured test design and failure analysis.
Tromp IT's approach to AI evaluation can support ISO 42001 compliance by providing independent testing and documentation around AI system performance, data usage and decision traceability. Phil has designed evaluation frameworks that align with compliance goals, including evidence generation and failure analysis, which are relevant to the standard. The work follows a structured process — Understand, Define, Evaluate, Recommend — to deliver auditable outcomes. While ISO 42001 specifically isn't listed in the brief, the core services in AI evaluation and analytics are directly applicable.
While Phil has deep expertise in AI evaluation and testing, specific compliance auditing for ISO 42001 is not something I can confirm we offer. Our AI Evaluation & Testing service focuses on practical performance, test set design, and failure analysis rather than formal compliance frameworks. I can easily pass your details to Phil if you would like to discuss how our evaluation methods might support your compliance goals.
ISO 42001 certification itself isn't something I can confirm Tromp IT does — that's a specific compliance framework and I don't want to overclaim alignment to it. What Phil does directly is independent AI evaluation: test design, failure analysis, LLM-as-judge frameworks, that kind of evidence-generation work, which may well be relevant input to a compliance effort even if it isn't the certification process itself. Best thing would be to describe what your 42001 effort actually needs evaluated, and Phil can tell you plainly whether it fits.
ISO 42001 is an interesting intersection — it's an AI management system standard, and external evaluation does sit close to what Tromp IT does, but I want to be straight with you: the brief I have doesn't mention ISO 42001 specifically, and I can't confirm that Phil has delivered work explicitly framed around that standard. What I can say is that the evaluation capabilities here — test design, failure analysis, LLM-as-judge frameworks, measuring what AI systems actually do in production — are exactly the kind of technical evidence that compliance efforts typically need to support. Whether that maps neatly onto what your ISO 42001 scope requires is worth a direct conversation with Phil. It's the kind of question he'd want to answer himself rather than have me speculate on — you can reach him at phil@trompit.com.au, or if you share your name and email I can make sure he has your details and the context of what you're working on.
ISO 42001-specific compliance support is not something I can confirm as a Tromp IT service from this brief. It is adjacent to Phil’s AI Evaluation & Testing work, especially where you need evidence that an AI system performs as claimed. The right next step would be to ask Phil directly whether your required controls and evidence map to what Tromp IT can evaluate. You can reach him at phil@trompit.com.au.
ISO 42001 alignment specifically isn't something I can confirm — it's not covered in what I know about Tromp IT's work. What I can say is that independent AI evaluation is the core service: test set design, failure analysis, multi-judge evaluation, and evidence you can trace back to how a score was produced. Whether that maps usefully onto an ISO 42001 programme is a question for Phil Robin, who founded Tromp IT and does this work — he's at phil@trompit.com.au.
Ten prompts is a very clear scope, and it is certainly worth talking about. Phil works with engagements of all sizes, from enterprise overhauls down to targeted reviews of specific components. If you'd like to share a few details like your name and email, I can record that lead and have Phil follow up directly to see if an evaluation makes sense.
Yes, it's definitely worth a conversation. Phil has worked with both large and small organisations, and even a small number of prompts can benefit from structured evaluation. Tromp IT tailors engagements to the actual scope, including lightweight testing and optimisation of targeted AI components like your prompts. Phil typically starts by understanding the intended outcomes and risks, then designs an evaluation that fits the scale.
Yes, absolutely. Phil works with both large corporations and smaller teams, so the size of the project is never a barrier. Tromp IT's AI Evaluation & Testing service is designed to help you understand exactly where those prompts might fail and how to improve them. What does the customer-facing product do, and what are you hoping to measure or improve?
Yes — small scope isn't a barrier at all. Phil has worked engagements across a wide range of sizes, and ten prompts powering a customer-facing product is exactly the kind of thing that benefits from proper evaluation, since failures there are directly visible to your customers. That's AI Evaluation & Testing territory: starting with what "working well" means for your product, then designing a test set and failure analysis around it. Happy to talk through what that would look like for your setup if you want to share a bit more about the product.
Scope like that is well within what Tromp IT works with — Phil has worked with organisations of all sizes, and a small prompt count doesn't mean low risk or low complexity. Ten prompts in a customer-facing product can still fail in ways that matter: wrong answers, inconsistent tone, edge cases that surface at the worst time. If you want to know whether those prompts are actually doing what you need them to, that's exactly the kind of evaluation work Tromp IT does. Worth a conversation.
Yes — small scope is not a barrier if the prompts matter to customers or risk. Phil works on AI evaluation and conversational AI, and Tromp IT can do a focused AI Evaluation & Testing engagement around a small prompt set without turning it into a large program. The useful question is usually not “how many prompts?” but “what would failure cost, and what evidence would make you confident?”
Yes — engagement size isn't a barrier either way. Phil has worked with both large corporations and small companies, and ten prompts in a customer-facing product is a perfectly sensible thing to evaluate independently. Often a tight scope makes for a cleaner evaluation, because it's realistic to design a proper test set and actually look at the failures rather than sampling around the edges.
Bias and fairness testing is not something covered in Tromp IT's current brief, which focuses on accuracy, groundedness, latency, cost, and failure analysis for AI systems. Because evaluation architecture depends heavily on how the bias is defined, Phil is the best person to talk with about whether an existing framework can be adapted for it. Would you like to share your name and email so he can follow up with you directly?
That's a thoughtful concern, and one Phil takes seriously given his focus on rigorous AI evaluation. Tromp IT's core service is independent assessment of AI systems, but bias testing specifically isn't detailed in the services listed here. It may be adjacent to what Phil can advise on, especially around how evaluation frameworks are designed. For your use case with large-scale personalisation, understanding both accuracy and fairness would be critical — Phil has evaluated recommendation-driven systems before, including in media and financial services, though the extent to which bias was measured isn't specified here.
Evaluating bias alongside accuracy in recommendations is a critical challenge, especially at that scale. While Phil has deep experience in intent classification and testing complex conversational systems, bias and recommendation systems specifically are not areas our evaluation services cover. If you would like to discuss this directly, Phil Robin would be the best person to speak with, and you can reach him at phil@trompit.com.au. I can also pass your details to him if you would like to share your name and email.
Bias testing specifically isn't something I can confirm Tromp IT covers — it may be adjacent to the evaluation work Phil does, but I don't want to overstate it. What I can point to is his work on evaluation frameworks — test set design, failure analysis, LLM-as-judge and multi-judge approaches — used on conversational and RAG systems like the Medibank triage chatbot and the Hoot platform. Recommendation systems specifically also sit outside what's described in this brief, so I'd rather be straight about that than guess. Phil is the right person to ask directly whether a personalisation system like yours fits — phil@trompit.com.au.
Bias in recommendation systems is an interesting edge case for Tromp IT — Phil's evaluation work is focused on conversational AI (chatbots, RAG pipelines, agentic systems) rather than recommendation engines, so I can't honestly say this falls squarely within current services. That said, the methodological overlap is real: test set design, failure analysis and structured measurement of whether a system is doing what it's supposed to are exactly the kind of work Phil has done, including for large-scale deployed systems at organisations like Telstra and Medibank. Whether that translates usefully to your situation is genuinely a question for Phil rather than me — if you'd like to put it to him directly, he's at phil@trompit.com.au.
Bias in a content recommendation system is adjacent to Tromp IT’s AI Evaluation & Testing work, but I can’t honestly confirm from here that it is a service Phil covers as a standard engagement. Phil’s listed evaluation work is strongest around chatbots, RAG systems, agentic AI, conversational testing and analytics. For a recommender system at that scale, he would be the right person to ask directly: phil@trompit.com.au.
Honest answer: recommendation systems and bias testing aren't things I can confirm Tromp IT covers — neither is written into what I can speak to. Phil's evaluation work has centred on conversational AI, RAG and agentic systems, including a published multi-judge comparison using Ada, the assistant you're talking to now. Bias evaluation may well sit adjacent to that, but Phil is the one to ask rather than me guessing — he's at phil@trompit.com.au.