Snapshot
- Role
- Solo product design
- Focus
- Three interface concepts for agent transparency, tested with 60 participants
- Status
- OZCHI '26, accepted as a long paper, publishing November 2026
- Domain
- Airline customer service
The problem
LLM agents are moving from passive assistants to systems that act on someone's behalf. They are non-deterministic and prone to silent failures, errors that look correct and arrive without warning.
Transparency is the standard proposed remedy, resting on an assumption nobody had tested in an agent context: that showing more reasoning helps people catch more mistakes.
I expected it to hold. I thought full transparency would win, and I had not considered what volume of explanation would do to someone's willingness to read it.
Three designs
Rather than test transparency as a dial, I designed three distinct answers to how much should an agent show?, each drawing the line in a different place.
Opaque
The decision, and nothing else. What most deployed agents do today.
Partial
The factors considered and their values, without the policy that combined them. You can see what it looked at; you can't see the rule.
Transparent
Full reasoning, factors, values, and explicit policy logic. Everything.
Three distinct answers to how much an agent should show.
The interesting boundary is between the second and third. Partial shows inputs; transparent shows inputs and the rule. That distinction turned out to matter more than the total quantity of text.
Airline customer service was chosen because it is one of the two domains in τ-bench, a standard benchmark for evaluating agents on multi-turn tool use under domain policy constraints, so the scenarios sit in a context the field already uses for this kind of evaluation. Three scenarios: ticket upgrade, booking change, compensation.
What We Found
Transparency improves perceived understanding, but not error detection
Complete explanations made wrong answers more convincing. Participants accepted incorrect decisions at nearly the same rate whether the agent explained everything or explained nothing. Partial transparency was the only condition that improved error detection. Fluency created an impression of reliability that survived the errors it was supposed to expose.
Error detection differed across conditions, χ²(2) = 9.23, p = .009. Partial outperformed opaque; full did not.
Full transparency reduces perceived negotiability
Explaining the rule made the agent feel non-negotiable. As transparency rose, users increasingly read the agent as fixed rather than open to challenge. Showing the policy behind a decision communicated that the policy was settled.
Perceived steerability declined monotonically with transparency, p = .004.
Transparency redirects, rather than eliminates, strategic interaction
Users didn't stop pushing, they pushed differently. Overall rates of contestation barely moved, but its composition shifted: more acceptance, less direct challenge, more probing of the framework itself.
Strategy distribution differed across conditions, χ²(4) = 11.23, p = .024.
Design rules that follow
Show factors, not policy. Disclosing what an agent considered helps people evaluate a decision. Disclosing the rule that produced it tells them the decision is closed.
Design for auditing, not explanation. The goal is not for a user to understand the agent's reasoning. It is for them to be able to check it. Those need different interfaces.
Treat fluency as a risk. A well-written explanation is more persuasive regardless of correctness. Anywhere an agent can be wrong, the explanation UI should preserve friction rather than remove it.
Leave room to argue. If an interface makes a decision look final, users stop trying to change it, even when it's wrong and even when it could have been changed.
What I'd build next
Concepts, not shipped work.
Progressive disclosure with a floor. Start at partial. Let a user request the policy explicitly rather than receiving it by default, the request itself signals they're auditing rather than skimming.
What-if probes. Instead of explaining the rule, let users test it: change one input and see whether the decision changes. Auditing by manipulation rather than by reading.
Uncertainty without a confidence score. My RAG study found self-reported confidence doesn't move trust. An agent needs a way to signal I am less sure here that shows evidence rather than asserting a number.
Three concepts, drawn
These are concepts rather than shipped work. Each one takes a finding from the study and turns it into an interface decision.
Progressive disclosure with a floor. The agent opens at partial transparency, factors and values, no policy. A single control reveals the rule, and the act of asking for it is the signal that someone is auditing rather than skimming. The floor matters as much as the ceiling: below partial, error detection got worse.
A what-if probe. Rather than explaining the rule, the interface lets someone test it. Change one input, fare class, notice period, loyalty tier, and watch whether the decision moves. Auditing by manipulation rather than by reading, which is the difference between understanding a decision and being able to check it.
Uncertainty without a score. A confidence percentage is a claim, not evidence, my RAG study found it doesn't move trust. The alternative is to show the agent's own working: which precedent it matched, how closely, and what it couldn't find. Less sure looks like thinner evidence, not a smaller number.
Three concepts, drawn. Each takes a finding from the study and turns it into an interface decision.
More transparency made people feel they understood the agent better, and made them less likely to challenge it. Which of those matters more in the systems you're building?
Where this applies
Any system where a person is meant to supervise rather than comply: code review agents, clinical decision support, content moderation tooling, financial advice. Anywhere an agent acts and a human is accountable for the outcome.
Full method and statistics
Within-subjects design, 60 participants with prior chatbot experience, recruited through professional networks. Three airline scenarios under three transparency conditions, order counterbalanced by Latin square.
Responses were scripted rather than live-generated, giving precise control over correctness and explanation quality at the cost of some ecological validity.
Error detection: χ²(2) = 9.23, p = .009. Partial vs. opaque Z = 110.50, p = .002, r = 0.43. Full vs. opaque p = .385.
Steerability: χ²(2) = 7.46, p = .024. Opaque M = 3.89, transparent M = 3.26. Z = 142.50, p = .004, r = 0.40.
Strategy: χ²(4) = 11.23, p = .024, Cramér's V = 0.24. Adversarial rates p = .281. Acceptance +15%.
Divya Ravi. 2026. Seeing Inside AI Agents: How Transparency Shapes Human-Agent Interactions. OZCHI '26.
This page shows partial detail by default and full detail on request, which is the finding it reports.