← Back to overview
CLASSLINK · ANALYTICS+ · ANALYSISSOURCE: "HOW TO MEASURE UX"

Takeaways and a simple strategy, from the talk to the reskin

Vitaly Friedman's throughline: success isn't happiness, and most UX measurement fails because teams jump into solution space before mapping the problem, then try to take credit for business metrics that are too many steps removed to prove. Below is what that means for the dataviz reskin specifically, read against the full transcript — Part A and Part B.

01Takeaways
  1. Success ≠ happiness. Ryanair example: satisfaction and happiness aren't the same as success — a chart can look polished and still fail if people can't extract the right number from it.
  2. Map reality before judging solutions. Sketches are pure solution space; without cataloguing where people actually misread today's charts first, you're "designing pixels" and results are random.
  3. Content success ≠ navigation success. In the European Parliament case study, 81% of people reached the right page but only 52% got the right answer — exactly the failure mode named in our brief: "charts show a number with no context."
  4. Measure at two levels, always. Local (did this one chart get read correctly) and product (did the page it lives on perform better) — never just the company-wide KPI, which is too far removed to credit to you.
  5. Design metrics are the missing link between product metrics and business metrics in most value architectures. Nobody tracks them, so design's contribution stays invisible until someone builds that bridge deliberately.
  6. Distrust vanity metrics. Time-on-dashboard, clicks, and NPS are ambiguous or gameable on their own — more time can mean confusion, not engagement.
  7. Correlation ≠ causation. If a metric improves the same week as an unrelated campaign or feature launch, don't claim it.
  8. You don't need huge samples. 15–18 participants per audience segment (top-task methodology) stabilizes task-success-rate and time-on-task data. Target >80% success, treat <60% as a real problem.
  9. TARS (Target → Adoption → Retention → Satisfaction) is a compact funnel for judging whether a shipped feature actually works, and it sorts work into quadrants that double as a prioritization tool.
  10. Prioritize by frequency × severity × frustration, not just impact/effort — fix what hurts the most people the most often first.
  11. Build the coalition alongside the metrics. Looping in customer support/success and naming invisible influencers makes the case far harder to reject.
02The strategy, in five steps
1
Run a comprehension audit, not a preference crit
Before converging on sketches, test the current chart with 15–18 people per audience on one real task each. Record both navigation success (right page) and content success (right answer) — this baseline turns "which direction feels right" into "which direction measurably fixes the misreads we found."
2
Score sketches against that baseline, not against taste
Re-run the same tasks on 2–3 finalist directions per category before full convergence. Ship whichever lifts content-success the most — not the one that wins the room's "I like this" vote.
3
Build a one-page value architecture for the reskin
Business goal ← product metric ← design initiative ← local metric, the way Friedman draws it on the whiteboard. This is the artifact for leadership — framed around time and support cost, not "it looks more modern."
4
Roll out with TARS per chart category as each ships
% exposed → % who engage → % still using it a month later → one-question satisfaction prompt. Use the quadrant output to decide what to reskin next.
5
Settle open color/encoding decisions with evidence, not a vote
Color is meant to carry meaning in our own spec — so resolve the open hue and colorblind questions with a comprehension test, the way Friedman runs his event-storming exercise, rather than a designer poll.