mh studio Book a free call
← All work
AI agents · Ops · Concept project, fictional client

relay.

Mission control for AI agents. Watch them work, approve risky actions, undo mistakes and replay any run. A web app, a mobile app and a website, all clickable.

Role
Lead product designer, end to end
Team
Solo concept project
Deliverables
Web app, mobile app, website
Benchmarks
Zapier, Datadog, Linear, LangSmith
▶00:00·frame 01

The brief

Relay helps ops teams run AI agents for refunds, invoices and ticket triage. The brief from the (fictional) client: make agents visible, put a human in the loop exactly where it matters, and make every decision auditable.

Follow one refund through the whole case: €480 for Ava’s broken lamp. It starts as a live demo on the website, waits in the web app’s Approvals, and gets approved on Omar’s phone.

Assumptions I designed against

Not research findings
assumption 01

Trust, not capability, stops ops leads from letting agents touch money or customers.

How I’d validate: Interview 6 ops and finance leads: what would they need to see before an agent issues a refund alone?

assumption 02

Investigating one bad agent action takes days, because logs are written for engineers.

How I’d validate: Time a real investigation end to end in 2 teams, from complaint to root cause.

assumption 03

Tracing tools such as LangSmith and Langfuse serve the engineer who builds the agent, not the ops lead who signs off on a refund.

How I’d validate: Ask ops leads to answer “why did it do that?” in an existing trace view.

Constraints

Money moves are hard to undo once they settle
Approvers are often away from a laptop
Every decision must be explainable to an auditor
The model can be wrong and sound sure of itself
▶00:40·frame 02

What I owned

I owned

  • Problem framing and the assumptions above
  • Benchmark teardown and research plan
  • Information architecture for all three surfaces
  • All UI: web app, mobile app, website
  • Design system and status language
  • Interactive prototypes, built in code
  • Product copy

I’d share in a real team

  • Success metrics and guardrail defaults, with ops and finance
  • What the agent can explain, with ML engineers
  • Approval rules, with compliance

Not mine

  • Backend, integrations and model choice
  • Legal and compliance sign-off
  • Pricing (the site’s prices are placeholders)

Honest constraints of a concept project

No real client, no users to interview, no live data and no engineers. So the flows rest on desk research, public benchmarks and agent-ops patterns, and every number on this page is an assumption or a target, never a result. Research plan with real access: 6 interviews with ops and finance leads, shadow 2 approval queues for a day, and review 50 past refund tickets for the cases agents get wrong.

Planned timeline for this concept

About 7 weeks, split into five phases. This is how I would plan the work as a real engagement, not a record of time spent.

  1. Week 1
    Discover
    Zapier, Datadog, Linear and LangSmith teardown, personas
  2. Weeks 2 to 3
    Design
    Mission control layout, approvals and the status language
  3. Weeks 4 to 5
    Prototype
    Working web app, mobile app and website
  4. Week 6
    Build
    Design system, failure and replay states, developer handoff
  5. Week 7
    Test and launch
    Approval flow tests with ops leads and guardrail defaults
▶01:20·frame 03

Who it’s for

NO
Nadia Okafor
Head of support ops · owns the agents
Persona voice

“I’ll let agents work as long as I can stop them.”

  • See every agent at a glance
  • Investigate a bad action in minutes, not days
  • Explain decisions to auditors
OM
Omar Malik
Finance lead · approves money moves
Persona voice

“I approve payments from airports and taxis.”

  • Decide from his phone
  • Trust the agent’s evidence, not just its numbers
  • Pause anything that looks off

Personas are assumptions built from desk research, not interview subjects.

▶02:00·frame 04

Benchmarks

Desk research: a teardown of four public products that ops and engineering teams already trust. What I kept and what I refused from each:

Zapier

Steps as plain-language blocks anyone can read.

Kept: Builder steps read as sentences: “Look up order”, “Ask for approval”.
Refused: Its “set it and forget it” promise. Agents need watching.
Datadog

Status colours that mean the same thing everywhere.

Kept: One traffic-light set across web, mobile and site.
Refused: Dense dashboards of charts. Ops leads want answers, not graphs.
Linear

Calm, quiet UI so the one urgent thing stands out.

Kept: Warm neutral surfaces with a single coral accent for “needs a human”.
Refused: Keyboard-first flows as the default. Approvers are often on a phone.
LangSmith

Full traces of every model call and tool call.

Kept: The raw call is always one click away in Replay.
Refused: Leading with JSON. Replay leads with Saw, Thought and Did.
▶02:40·frame 05

Mission control

Mission control in four views: Live (a flow map of the selected agent and a feed of everything happening), Approvals, Replay and Builder. The icon rail on the left is the main menu. Try it below.

app.relay.ai/live · prototype, sample data

Mission control

4 agents · 1,572 actions today
1 agent paused itself
Agents running
3 / 41 paused automatically
Actions today
1,572+12% vs Mon
Waiting for you
3oldest 18 min
Hours saved (est.)
12.5this week, team of 5
Refund agentHandles refund tickets in Zendesk
🎫New ticketZendesk
📖Read ticketreason, order
🔎Check orderShopify
🧠Deciderefund or reply
✋Approvalover the limit
💸Issue refundStripe
run #1840·Waiting for a new ticketActiveDoneNeeds approval
Live feed
10:45Refund agentRefunded €24 to order #40238, damaged item
10:44Triage agentRouted ticket #8812 to Billing
10:43Invoice checkerMatched INV-1042 to PO-771
10:42Refund agentAsked for approval: €480 to order #40211
10:41Churn watcherFlagged Northwind, usage down 42%
  • ▶ Approve or edit the €480 refund
  • ▶ See who owns each approval and when it escalates
  • ▶ Open the undone run Refund #40244 in Replay
  • ▶ Check the guardrails in Builder
▶03:20·frame 06

Approve on the go

Good morning, Omar
3 things need you
4/4healthy
1,572
actions today
12.5 h
saved this week (est.)
99.1%
success rate
0
you approved
Needs you3 open

Approvals can’t wait for someone to open a laptop. The phone app is built around one job for Omar: decide quickly and safely, wherever he is.

Every request shows who owns it and when it moves to the backup approver, so nothing waits in silence.

  • ▶ Wait for the push and tap it
  • ▶ Hold to approve the €480 refund
  • ▶ Or tap “Can’t hold?” to approve in two taps
  • ▶ Pause an agent in Agents
  • ▶ Filter Activity to problems
▶04:00·frame 07

When the agent is wrong

Agent ops is mostly about failure, so I designed the bad day on purpose. This is run Refund #40244, which you can scrub through in the web app’s Replay view.

  1. 01Confidently wrong

    Ava asks to swap one lamp. The agent reads it as a refund, with high confidence, and refunds €190. It is under the €200 limit, so no human is asked.

  2. 02Guardrail trips

    Ava replies. The agent tries to refund the second lamp too. A rule outside the model blocks it: two refunds on one order within an hour.

  3. 03Auto-pause and escalation

    The refund agent pauses itself and pings Nadia, the owner, with the run attached. If she doesn’t answer in 30 minutes, it goes to Omar.

  4. 04Rollback

    Refunds under the limit are held for 10 minutes before they reach Stripe. Nadia undoes the held refund in one click and sends a swap instead.

  5. 05Learn

    From the run, she adds a rule: messages that mention “swap” go to a human. Replay keeps the whole story for the audit.

Rules live outside the model
A clever prompt or a confused agent can’t talk its way past them.
Every money move has an undo window
Small refunds skip approval but still wait 10 minutes before they settle.
Every approval has an owner, a backup and a deadline
An action never waits in silence, and a failure always reaches a person.
▶04:40·frame 08

The website

The site proves the promise instead of describing it: the hero is a live agent run that stops and asks the visitor to approve. The rest scrolls like a story, with a sticky panel that changes with each control. Logos, prices and figures on it are labelled as placeholders.

Let AI agents do the work. Keep the final say.
Refund agent · live demo
Refund €480 to Ava?
ApproveReject
Open the full website ↗
▶05:20·frame 09

Key decisions

decision 01

A map, not a log

Ops leads think in workflows. The live view draws each agent as a flow with the active step glowing, so you can tell where something is stuck at a glance.

Endless log lines
Live flow map
Rejected: A filterable log table as the home screen. It stays in Replay’s full log, where detail matters.
Trade-off: A map only reads well for short flows. Agents with more than 12 steps would need grouped steps.
decision 02

Explain before asking

Every approval leads with evidence: what the agent checked, the policy it matched and what the team decided in similar cases. Confidence shows as a band (High, Medium, Low), not a raw percentage. Below Medium, the agent always asks.

“Approve refund?”
Evidence, then a confidence band
Rejected: A precise “86% sure”. Models are rarely that well calibrated, so a raw number invites false trust.
Trade-off: Cards get taller. The queue only shows the headline and risk; the evidence opens on select.
decision 03

Replay like a video

Audits and debugging both start with “what happened?”. A scrubber with Saw, Thought and Did turns a run into a story anyone can follow.

JSON trace
Saw · Thought · Did timeline
Rejected: An engineer-style trace tree. Right for builders, unreadable for auditors.
Trade-off: Summaries can hide detail, so the raw tool call is always shown next to them.
decision 04

Hold to approve on mobile

A single tap is too easy for a €480 decision on a phone. Holding for a second adds just enough friction without a second screen.

Tap
Hold 1 second, or two taps
Rejected: Face ID on every approval (too heavy for a €24 refund) and swipe to approve (too easy to trigger in a pocket).
Trade-off: Holding slows power users and excludes some people, so “approve in two taps” is always one tap away.
decision 05

Every approval has an owner and a deadline

Each request names one owner, a backup and a time limit. At 30 minutes it moves to the backup, and payments over €1,000 need two approvers.

Waits forever
Owner, backup, deadline
Rejected: Posting to a shared channel for “whoever is free”. When everyone owns it, nobody does.
Trade-off: Teams must name a backup before an agent can go live, which adds a setup step.
decision 06

Trust is earned, not toggled

After 50 approvals in a row with no edits, Relay suggests raising the refund limit from €200 to €300. A person always confirms, and one click rolls it back.

Autonomy slider on day one
Limits that grow with evidence
Rejected: Letting teams set full autonomy at setup, before they have seen the agent work.
Trade-off: Teams reach full automation more slowly. That is the point.
▶06:00·frame 10

Accessibility

Hold has a two-tap fallback

Holding a button is hard with some motor impairments, screen readers and switch control. “Can’t hold? Approve in two taps” sits right under it, and the second tap changes colour and wording so it can’t be a slip.

Status is never colour alone

Every traffic light has a word next to it: Running, Needs approval, Problem, Paused. The four colours also differ in lightness, not just hue.

Coral is for fills, not small text

Coral (#FF6B4A) is under 3:1 against white, so small coral text uses Deep coral or Ink. Known gap: white labels on coral buttons are 2.9:1. v2 moves those labels to Ink.

Calm motion

The flowing edges and pulses stop when the system asks for reduced motion. The live feed doesn’t announce every event; toasts use a polite status region.

Big targets on mobile

Every action on the phone is at least 44 px tall; approve is 62 px, full width, at thumb height.

▶06:40·frame 11

Design system

Calm mission control: warm paper instead of black screens, one coral accent for anything that needs a human, and traffic-light status colours used the same way everywhere. Each colour has one job.

Ink
Text, primary actions
#1D1B19
Coral
Needs a human
#FF6B4A
Deep coral
Coral text, errors
#E2452A
Amber
Waiting
#FFB547
Signal green
Running, done
#1FB57A
Paper
Surface
#F6F4F0
Night
Device and dark panels
#1E1C1A
RunningNeeds approvalProblemPaused
Aa
Plus Jakarta Sans · interface
run #1840
IBM Plex Mono · runs, logs and tool calls
▶07:20·frame 12

Success targets

These are targets I’d set with the team for a 4-week pilot with 3 ops teams, not results. No pilot was run. Speed metrics are paired with safety metrics, so faster approvals can’t hide rubber-stamping.

MetricBaselineTargetHow measured
Median time to approveBaseline: Unknown; assumed hours (approvals by email and Slack)Target: Under 5 min in working hoursMeasured: Request created to decision, median per week
Approvals later reversedBaseline: Not tracked todayTarget: Under 2%Measured: Approved actions undone or refunded back within 7 days. The counter-metric: fast must not mean careless
Agent decisions overruledBaseline: Not tracked todayTarget: Falling week on week, under 10% by week 4Measured: Rejected or edited approvals ÷ all approvals
Approvals per person per dayBaseline: Not tracked todayTarget: Under 15Measured: Watches for approval fatigue; above it, low-risk items get batched
Time to investigate a bad actionBaseline: Assumed 1 to 3 daysTarget: Under 15 minMeasured: Complaint received to root cause found in Replay
Hours saved per ops team (team of 5)Baseline: Not tracked todayTarget: 10+ hours a weekMeasured: Tickets fully handled by agents × average handling time
▶08:00·frame 13

What I left out of v1

Multi-agent hand-offs

One agent passing work to another needs a graph view and new failure modes. No ops lead in the brief asked for it yet.

Cost and token dashboard

Matters to the engineer paying the bill, not to the ops lead’s daily job. v2 adds monthly spend per agent.

Full eval suites and prompt versioning

v1 Builder saves edits as a draft and runs one test ticket. Proper evals before publishing come next.

Audit tab with CSV export

Replay answers the auditor’s question in v1. Export of who approved what, when and from which device is first in line for v2.

Batching low-risk approvals

The fix for approval fatigue, but only worth building once we see real approval volumes.

▶08:40·frame 14

Reflection

What I’d change

I drew the happy path first and the failure story later. Designing the bad day first would have shaped the approval card sooner: the undo window and the owner-and-deadline rule changed it more than any visual decision.

What I’d test first

Whether approvers read the evidence or approve on the headline alone. If they skip it, the strongest piece of evidence has to move into the headline.

The biggest risk

Approval fatigue. A team that approves dozens of requests a day starts rubber-stamping, which is why “approvals later reversed” sits next to “time to approve” in the targets.

Next project
Carely

Doctor appointments, booked in under a minute.

View case study →