Simulate

Simulate is Arato's platform for validating AI agents before they meet real users — it generates realistic simulated user scenarios, runs them at scale against a company's AI system, and surfaces where it breaks: hallucinations, tone drift, accuracy gaps, compliance risk. I designed and shipped it from zero as the sole designer.

Early framing treated this as a narrower, prompt-tuning problem. Through the design process, I reframed it — not a tool for tuning prompts, but a platform for understanding product quality and clarity at scale. That shift in framing changed how the product was built and who it resonated with.

Watching people react to early versions, a pattern held: the moment someone realized they could simulate many users at once and iterate — that's when it landed, because it hit the real pain point — no clear visibility into their own product's quality. That insight became the core of how I framed the product.

Teams building AI agents were shipping mostly blind — testing prompts by hand, with no structured way to see how their system would actually behave once real users, with all their variation, got their hands on it. The tools that existed treated this as a prompt-tuning problem. It isn't — it's a product quality problem, and nobody had built for that yet.

Before there was an app to design, there was a problem to understand: how do you gather context on a customer's AI system when nothing exists to gather it in? We used tools we already had — Notion, Google Docs — as makeshift UX test surfaces, watching which interactions actually worked before committing to how the real context-gathering feature should behave inside the product. That informal testing became the spec for a real feature.

Alongside that, since there was no live app yet, I built a wizard-style flow purely to explain — in demos — how the product was supposed to work. It gave the team something concrete to walk through with prospective customers before the real product existed.

The hardest part isn't UI — it's that customers don't have an existing mental model for what this is. It's not "analytics, but for AI." The nearest reference points people reach for are analytics/BI platforms, user-testing tools, and security pentesting — and the product has to borrow just enough familiarity from each without fully being any of them, while still being legible in a 60-second demo.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.

The platform's structure wasn't the obvious starting point. A flatter version — generate, run, and see results in one pass — was simpler to build, but it skipped what customers actually needed:

• Real context about their own AI system before anything else made sense
• Granularity in how scenarios got built, so they reflected real user variation
• Granularity in the analysis itself, so a failure could be traced back to why, not just that it happened

Each step exists because skipping it broke trust in the result.

The interface is built around three things: a navigation structure I designed early to keep the system legible, a report view that every flow leads back to, and guided, self-serve steps layered on top so a user (or a demo) could go from zero to a finished result without hand-holding.

I didn't wait for a design-review cycle to hand something back to engineering — for a feature already mid-flight, I opened the pull request it lived in and made the interface changes there myself. The gap between "engineer ships the mechanism" and "customer sees something usable" closed in one pass instead of a handoff loop.

Simulate takes a customer from "I don't know how my AI will hold up" to a clear, drillable answer. Users start by giving context on their AI system — no deep integration required. Simulate generates realistic, varied user scenarios against that context, runs them at scale as multi-turn conversations, and returns an analysis a user can zoom through at three levels: the whole system, a single conversation, or a single turn. Where a failure shows up, it's traceable back to why — and, increasingly, actionable directly from within the report.

Simulate is now in active use with early customers, and the shift in framing — from a prompt-tuning tool to a platform for product quality and clarity at scale — visibly changed how people responded to it. That's the clearest signal I have that the reframing worked, not just as a pitch, but as a product.