Research synthesis: days → 15 minutes
Synthesis cost so much that research only happened once it was already too late.
Role
Design Lead
Company
Mula
Timeline
December 2025 - February 2026
01 / The Problem
Synthesis was too expensive to run often
I was the sole designer at Mula, a B2B SaaS company, after the team contracted. I ran the interviews myself, usually without an observer and sometimes alone, so nothing was captured live. Every round meant rewatching each recording in full and rebuilding the picture from scratch.
So research only ran when it was unavoidable. Problems surfaced only after they were already built into the product, where they cost the most to fix.
02 / The Diagnosis
The obvious fix made the wrong call early
The temptation was to drop the transcript into an LLM and ask for pain points. The output looked clean. It was wrong in the ways that mattered most: the hesitations, the things users said once and didn't repeat, the observations that only mean something in context.
When one prompt is asked to extract, judge, and synthesise at once, it makes confident decisions about what matters before it has seen everything. The most important observations get filtered out before synthesis even begins.
03 / The Architecture
Separation of concerns
Each node does one job, so judgement never contaminates extraction.
The pipeline
01
Extractor
Pulls every observation and verbatim quote from the raw transcript. No analysis. No judgment. No filtering.
02
Normaliser
Processes each interview independently. Assigns theme, severity grade, and issue type to each observation.
03
Synthesiser
Runs once all interviews have cleared the first two nodes. Detects patterns, formulates problems clearly, surfaces job stories.
The synthesiser only ever sees clean, labelled observations. Extraction is the one node that can't be wrong: get it wrong and everything downstream inherits the error, so that first node does nothing else.
04 / The Work
The build
Getting the architecture right took longer than building it.
Iteration 1 · The two-node mistake
My first version ran the LLM work in two stages: extractor and synthesiser. But the extractor was pulling observations and judging them in one pass, so the data was already contaminated before it reached the synthesiser.
Iteration 2 · The guardrail spiral
Getting the prompts stable took longer. After two days of accumulating guardrails, each one added to fix the last problem and each creating new failure modes, I used a second model to diagnose the first. The structural diagnosis: too many simultaneous instructions, rules overwriting each other, token limits forcing the model to drop observations. Reformulated on that basis, both nodes stabilised within four hours.
Iteration 3 · The labelling problem
The synthesiser tallies were grouping by feature section rather than by individual problem. Every section showed 6 out of 6, useless, because every user had touched every section. The issue wasn't the synthesiser. It was the normaliser's labels.
05 / Validation
As rigorous as the manual pass
The pipeline isn't a replacement for research judgement, it's a structured debrief aid. As soon as a transcript is ready, I run it through the extractor and normaliser while the interview is still fresh, then review and correct the structured output in minutes instead of rewatching recordings. The synthesiser runs only once every interview has been reviewed.
- Issue detection
- Matched the manual pass
- Synthesis
- Formulated problems, drafted job stories
- Time per round
- 3–4 days per user test → 15 min
On issue detection the pipeline matched the manual pass. On synthesis it went further by design: the synthesiser prompt carries the anatomy of a job story, so it clusters the detected issues, formulates each problem in a sentence or two, and drafts job stories for the main ones.
The drafts weren't final and some needed real reworking, but the right formulation was usually already visible. Starting from a drafted formulation rather than a blank page is the whole difference.
06 / What's next
What's next: portability
Themes are hardcoded into the normaliser prompt today, so each new research context means reconfiguring it by hand. The version worth building lets the researcher define their hypotheses up front, with the themes living outside the prompt so the pipeline adapts to any project without a rebuild.
The point isn't only convenience. A pipeline that holds its themes loosely can also surface the emergent ones, the patterns nobody hypothesised, which is where the most interesting findings usually live. That is the next thing I would build.
When synthesis stopped being expensive, research could finally run continuously, letting us build on evidence instead of assumption, and catch problems before they're built into the product, where they cost the most to fix.