All work

        Interpres Security·Explainable AI decision support·Flagship
      

Designing a governed, explainable AI assistant for threat & exposure management

Security analysts spent ~45 minutes per incident synthesizing exposure data across fragmented dashboards. I led the 01 design of Interpres's first AI assistant, which cut time-to-insight to 27 minutes and raised analyst trust from 62% to 81%.

case.md

$ cat outcome

time_to_insight: 45 → 27 min

analyst_trust: 62% → 81%

Interpres Assistant chat interface with a structured response card — Assistant chat with a structured response card

Why it mattered

Every minute spent stitching dashboards together was a minute an exposure stayed unaddressed.

60-second read

Problem

Analysts spent about 45 minutes per incident synthesizing exposure data across fragmented dashboards.

System decision

Replace open-ended prose with a deterministic reasoning chain that is evidence-linked and confidence-scored.

My contribution

Led the 0 to 1 design end to end: interaction model, evidence layer, safety and uncertainty states, design system.

Impact

Time to first insight 45 to 27 minutes. Analyst trust 62% to 81%.

01 · Shipped

70%

of analysts opened it daily

45 27 min

time to first insight

2

customer wins enabled during Beta

A working 01 prototype, designed and validated in two weeks, that moved into Beta with real analysts, not a concept deck. It became the entry point to their workday, and a factor in closing new business.

The Interpres assistant dock open beside the exposure workspace, the surface analysts opened at the start of every shift — The surface 70% of analysts opened daily

02 · The problem

Security teams work across fragmented tools: alerts in one, exposure graphs in another, control coverage elsewhere. Even where the underlying data was rich, synthesis was entirely manual. Across 12 analyst interviews and 3 SOC manager sessions, the gap wasn't intelligence. It was structured, contextual reasoning.

“I don't trust automated suggestions unless I see the source.”

“Switching tools breaks my thinking.”

03 · What failed

First pass

Open-ended prose responses. In a security context that was unacceptable: hallucination risk, slow to scan under time pressure, and analysts finished reading unsure what to actually do next.

Illustrative, not the shipped copy

“Based on the available data, this could indicate a potential control gap related to access management, though further investigation may be needed to confirm the specific technique and assess the overall exposure level across affected systems...”

What replaced it

A deterministic reasoning structure, rendered as structured response cards instead of prose, with a “Why this matters” block on every answer.

              EXPOSURE
              
              CONTROL GAP
              
              MITRE TECHNIQUE
              
              RECOMMENDED ACTION

04 · Where it lives

The assistant docks into the existing investigation rather than becoming a place you go.

Exposure dashboards, asset details, and MITRE views stay visible while the assistant works with whatever is selected. The dock inherits the active exposure, asset, or technique, so a question never starts from nothing.

Evidence links return focus to the source view instead of opening a separate AI destination. That single decision is what keeps it an assistant rather than a second product.

The Interpres campaign view for Operation CuckooBees, showing the description, a relationship graph linking techniques, software, threat groups, vulnerabilities and detections, and a properties panel — The investigation the analyst is already in. Campaign, relationship graph, defense surface, and associated threat elements.↗ open full size

The sequence it supports

It starts in place. The selected surface supplies the starting context for the question.

01

Clarify the score

Follow-up questions keep the current object and explain product language.

An assistant response explaining how the Priority Score is calculated, breaking it into its Visibility and Detections inputs — Product language explained in place. The analyst asks how a score is calculated and gets its two inputs, rather than a definition page in another tab.↗ open full size

02

Inspect why it matters

Evidence and impact stay inspectable before the analyst chooses an action.

The Why This Matters panel showing the MITRE ATT&CK vector Access Token Manipulation T1134 with an 85 confidence ring, an explanation of the exposure, a Learn more link and an urgent action callout, beside a role selector offering Analyst, Responder and CISO framings — Evidence, impact, and confidence in one panel, plus the role selector. The same finding changes depth and language for an analyst, a responder, or a CISO.↗ open full size

03

Recover safely

Low confidence offers refinement or an alternate query rather than guessing.

The AI uncertain state warning that the answer is based on limited data, offering two recommended recovery paths: refine your question by narrowing scope, or try an alternative query with worked examples — The refusal, with a way forward. Two recovery paths, each with a worked example, instead of a guessed answer the analyst might act on.↗ open full size

The patterns in place

Confirmation dialog warning that an action will modify firewall policy, with Confirm and Cancel — Human-in-the-loop. Any action touching a live control stops for explicit confirmation before it executes.

AI Uncertain state offering recovery paths: refine your question, or try an alternative query — The “AI Uncertain” pattern. Low confidence surfaces recovery paths instead of a guessed answer.

Still working on it fallback state with options to continue waiting or stop — Fallback for slow queries. The system stays legible instead of stalling silently.

05 · Live recreation

Not a screenshot. A working recreation of the reasoning structure and the evidence layer, with the two branches that matter most: uncertain and requires confirmation. Send the query, take either follow-up, then try the sensitive action below.

              Interpres Assistant · recreation
              Interactive
            

Recreation · not live data

The second branch: a sensitive action

Every action that touches a live control stops here. One action, one confirmation, every time, and the dialog states the blast radius in the analyst's terms rather than the system's.

Try it. Confirm, or cancel and keep the existing rule.

Human-in-the-loop

Recommended action: enable attachment sandboxing for Finance OU.

06 · Confidence people
can use

A percentage helps only when someone knows what it means, can verify its basis, and understands the next action.

Precision without meaning creates false assurance, which in a security tool is worse than no number at all. So I treated the confidence score as three separate design problems rather than one label.

1

Score definition

Name what it represents

A number is ambiguous until you say which thing it measures: model certainty, data quality, or the likelihood the recommendation is correct. Then explain its limits and the uncertainty that remains, in plain language.

2

Evidence and action

Show the basis, match the workflow

Reveal contributing signals, source lineage, freshness, and a short explanation. High confidence streamlines the path while keeping evidence available. Low confidence invites review, comparison, or escalation. Either way the analyst can correct, give feedback, or override.

3

Calibration and behavior

Measure use, not display

Across comparable cases, an 80% score should prove correct about eight times out of ten. So the thing worth instrumenting is behavior: acceptance and overrides, ignored scores, errors and recovery. Watch for blind acceptance and habitual dismissal, because both mean the number is not doing its job.

The Why This Matters panel naming the technique, the confidence ring, and the recommended action together in one view — Score, evidence, and consequence in one panel

The confidence ring and the named technique in the sequence above are this framework in one panel: a score, the thing it measures, the evidence behind it, and the consequence if it is right.

Design goal

Calibrated trust, not maximum trust

An analyst who trusts the assistant exactly as much as it deserves is the target. Analyst trust rose from 62% to 81%, which is the number I would defend, because it moved without the accuracy claim changing.

07 · Built with engineering

A confidence score is not a design decision or an engineering decision. It is both, or it is a number nobody should trust.

The percentage had to combine measurable signals from the response pipeline and then map that uncertainty onto product behavior. Neither discipline could specify it alone, so we ran five stages with a design and an engineering track through each one.

01

Frame

Design

Define what analysts need to judge

Engineering

Identify measurable system signals

02

Instrument

Design

Specify evidence and state requirements

Engineering

Expose freshness, sources, and conflicts

03

Contract

Design

Map confidence to interface behavior

Engineering

Build the API and fallback logic

04

Validate

Design

Test interpretation and recovery

Engineering

Run calibration against reviewed cases

05

Monitor

Design

Study acceptance and overrides

Engineering

Track drift, latency, and failures

Decisions we made together

Four questions where a design answer without an engineering answer would have been fiction, and the reverse would have shipped a number with no meaning.

Score definition

What the percentage represents

Thresholds

When the experience should slow down

Human control

Which actions always require confirmation

Release criteria

When calibration supports production use

The artifacts were joint too: a confidence definition, a state model, an API contract, an evaluation plan, and a monitoring dashboard.

The contract, in practice

Design defined the experience contract. Engineering confirmed what the system could return reliably. This is the shape we settled on.

confidence-response.json Illustrative

{
  "confidence": 0.89,
  "level": "high",
  "factors": [
    "MITRE mapping verified",
    "Three current sources"
  ],
  "limitations": [
    "Business criticality unavailable"
  ],
  "behavior": "review_and_confirm",
  "evidence_ids": [
    "asset-420", "mitre-T1134"
  ]
}

confidence Displays the calibrated percentage

factors Explains what supports the score

limitations Makes missing context visible

behavior Selects review or recovery state

evidence_ids Returns focus to source content

Illustrative shape, written for discussion. It does not reproduce proprietary production code.

08 · What I cut

Multi-step autonomous execution. The assistant could have chained patch, policy update, and notify with no human between them. I scoped it to one action, one confirmation, every time, trading speed for trust, on purpose.

09 · Across the stack

I owned the interaction model end-to-end. Not a chat surface bolted onto dashboards, but how the AI reasons, grounds itself in evidence, and earns trust at each layer.

Analyst sees this layer

D

Tool / Agent

Proposes actions. A human confirms before anything runs.

C

Orchestration

Turns a question into cards, evidence and next steps.

B

Data

Grounds every claim in a tagged, linked source.

A

Model

Reasons deterministically instead of improvising.

Each layer builds on the one below

E

Performance

F

Safety

Four stacked layers, with performance and safety running through all of them

A · Model

Deterministic reasoning enforced over open-ended generation.

Lower hallucination risk, faster scanning

B · Data

Every response evidence-linked, source-tagged, schema-bound.

78% evidence validation rate

C · Orchestration

Chat blended with structured cards and suggested follow-ups.

Question visual workflow

D · Tool / Agent

AI could filter, search, and map exposures, with every action traceable.

8% “AI Uncertain” rate

E · Performance

Streaming layouts and status messaging under real latency.

Responsive on large datasets

F · Safety

Confidence indicators, recovery paths, role-based gating.

Trust by explicit calibration

10 · Response

62 81%

Analyst trust

78%

Evidence validation

4 - 6

Follow-ups per chain

8%

AI uncertain rate

“Feels like a junior analyst who already knows our environment.”
Senior SOC Analyst

“For the first time, I can brief leadership using language they understand.”
CISO

Source: Mixpanel and OneStory analytics, measured during Beta.

In cybersecurity, ambiguity is risk. Design must reduce it.

The design system