The Perception–Reality Gap
A full-stack UX audit of MetalCloud — the operating system for India’s metal foundries. Renewals looked healthy. Floor-level usage had quietly gone dark.
COMPANY
NowPurchase
ROLE
Sole Product Designer & UX Researcher
TEAM
Solo project
PLATFORM
MetalCloud Platform
The Terrain
The Question I Was Handed
How I Designed the Study
The JTBD Instrument
Into the Field
The People Behind the Data
Perception vs. Reality
The Emotional Journey
The Renewal Paradox
The Customization Myth
What It All Meant
Where I Pointed the Roadmap
What Changed
Limitations I'd Flag Myself
Closing Reflection
The Terrain
MetalCloud digitises the foundry floor — from the chemistry of a molten heat, to the discipline of the melt itself, to the quality certificate that leaves with the casting. Three modules carry that promise:
SpectroPro (Chemistry) — reads spectrometer chemistry instantly and pushes addition/dilution guidance to the melter’s WhatsApp, so the crew knows what to add without walking to the lab.
DigiMelt (Execution) — live heat tracking and melting-SOP enforcement; every step of the melt is meant to be logged in real time, giving managers a transparent, traceable record.
Assure (Quality) — automated Material Test Certificate generation and QA traceability: audit-ready proof of what a casting is made of, without the manual paperwork.
The tension: on paper, adoption looked fine — modules installed, reports flowing, renewals landing on time. But the signals coming back from the field didn’t match the dashboards. Something between bought and used wasn’t adding up.
The Question I Was Handed
I framed the entire audit around three questions — one for each module, asked the same way every time. If a module couldn’t answer them cleanly, its healthy renewal number was hiding something.
What pain does it actually solve? Not the pain on the pitch deck — the specific, real problem a melter or quality head feels on a live heat.
Is that pain burning and frequent — or nice-to-have? A tool that solves a rare, low-stakes annoyance will never earn a daily habit, however elegant.
Who truly uses it — by role — and why? Owners buy. Operators use. The audit had to separate the two, because they renew for very different reasons.
The stakes: if renewals were riding on optics rather than outcomes, the retention story was fragile — one disappointment away from churn. My job was to separate what customers believed from what they actually did, and to say so honestly, even where it contradicted the sales narrative.
How I Designed the Study
I built a two-track investigation so neither the numbers nor the narrative could go unchecked — then anchored both to hypotheses I wrote up front, so I was testing beliefs rather than fishing for a story.
Track A — Quantitative: behaviour, at scale, from the product itself. Mapped each module’s journey in Amplitude, found the drop-offs and dead ends, and analysed 121 MS Clarity session recordings to quantify what people do, not what they say.
Track B — Qualitative: the humans who live those numbers daily. Case-based interviews (not opinion surveys) with shop-floor operators and internal adoption leads, plus WhatsApp-readership validation calls to cross-check every number against a voice.
Hypothesis-first: for every module I wrote explicit Problem, Solution, and Implementation hypotheses, each with a measurable validation signal — e.g. for SpectroPro: foundries lose ≥3% per heat to slow manual dilution; a tagged spectrometer would cut alloy cost 1–2% at 95% suggestion accuracy. That turned a vague “is it working?” into something I could actually prove or disprove.
The five-step process: Hypotheses (write problem/solution beliefs per module) → Interviewees (pick roles that reveal use vs. non-use) → Scripts (build case-based JTBD questionnaires) → Fieldwork (interview, record, collate the raw data) → Synthesis (build the Perception-vs-Truth table).
The JTBD Instrument
To separate what people say from what they do, I ran every interview through a case-based Jobs-to-be-Done instrument — the same card, filled from one real event, for every role and every module.
One real case — reconstruct a specific, recent event, never a hypothetical or an average.
Show, don’t tell — pull the actual reading, message, or file; evidence over opinion.
Same card, every time — identical fields across roles and modules, so answers compare cleanly.
I filled one of these per module, per role — reconstructed from interview transcripts and call recordings, never from memory. That’s what let me later line up the same job across a melter, a quality head, and an owner, and watch the answers diverge.
SpectroPro
“Know exactly what to add to hit target chemistry — fast, without walking to the lab and back.”
JTBD Card — composite of a recurring live-heat case, SpectroPro
TRIGGER
A bath reading comes back off-spec mid-heat, furnace already hot.
DESIRED OUTCOME
The right addition or dilution decided in seconds, before the melt cools.
STRUGGLES
Waiting on the lab printout; temperature dropping; re-typing readings by hand.
WORKAROUNDS & TOOLS
Lab printout, WhatsApp, a senior’s verbal sign-off.
PEOPLE
Doer — melter · Viewer — shift supervisor · Approver — metallurgist/owner.
FREQUENCY
Several heats every shift — high and recurring.
STAKES IF LATE OR WRONG
Off-spec heat, re-melt, wasted alloy, missed pour schedule.
IMPACT & EVIDENCE
Minutes + alloy ₹ lost per heat, traced via WhatsApp readings & lab prints.
Into the Field
Opinion surveys tell you how people feel. I wanted to know what they did. So every interview walked one real, recent case end-to-end — the last heat, the last melt plan, the last certificate — asking: what triggered it, what did you open first, what did you reach for outside the product, and what would you do if this module vanished tomorrow? Facts and artefacts, not feelings.
Stats: 3 modules audited · 13 foundries covered · 9 months of usage data · 121 Clarity sessions · 40+ shop-floor calls.
Who I listened to: melters, melting heads, quality heads, and metallurgists across foundries — the people whose hands are on the heat. To triangulate, I also debriefed the internal onboarding and sales-adoption leads, then held their claims up against the raw Amplitude and Clarity behaviour.
The tell: one customer’s weekly reports “weren’t landing.” I traced it — they were being sent to a decision-maker’s number he no longer used. Nobody had checked. A small thing, but it captured the whole problem: we were measuring delivery, never absorption.
Behaviour first, interviews second: I analysed the behavioural record before any interview, so each conversation tested what the numbers suggested. The 121 Clarity recordings independently corroborated the field story — dead clicks, rage clicks, and task-blocking errors eroding trust in exactly the modules users described abandoning, while the one workflow users completed consistently (ChargeMix) was also the one they spoke about with confidence.
The People Behind the Data
The study surfaced a hard split: the person who buys MetalCloud is almost never the person who uses it — and the two renew for opposite reasons. Five archetypes carried the whole pattern, synthesised from the JTBD interviews, 40+ validation calls, and the readership deep-dives, then checked against the behavioural data.
The Buyer
The Owner (Partner/Decision-maker) — Reassured, falsely. Hires the product to prove the foundry is modern, digitised, and in control to clients and auditors. What they actually do: rarely opens the tool; sees WhatsApp summaries, assumes the floor is acting, renews.
The Floor
The Trusting Operator (Melter/Metallurgist) — Confident, engaged. Hires the product to know what to add, fast, without a lab trip. What they actually do: reads every reading, trusts it, adjusts the melt — where value truly lands.
The Overwhelmed Operator (Melter/Melting Head) — Learned helplessness. Same job, but the report is unreadable to them. What they actually do: skims, doesn’t understand, quietly falls back to old habits.
The Cautious Verifier (Quality Control) — Passive, low-trust. Hires the product to be certain before committing to a decision. What they actually do: reads, but double-checks manually and won’t act independently.
The Compliance-only QA (Quality Head) — Checkbox, guilt. Hires the product to produce the certificate when an audit or customer demands it. What they actually do: touches Assure only under external pressure, never as daily QA.
The exception worth chasing: a handful of power users read the graphs to diagnose what went wrong in a heat — proof the value is real and reachable, just locked today behind readability and friction.
Perception vs. Reality
When I laid the promise beside the ground truth, the same gap opened under all three modules. What was sold, and what actually happened on the floor, were two different products — not failing, not succeeding: renewed for visibility, not for value.
Module | What was promised | What the floor did |
|---|---|---|
SpectroPro (Chemistry) | Instant chemistry on WhatsApp — save 5–10 min per heat, decide faster. | Readings often ignored; crews still wait for the lab printout before acting. |
DigiMelt (Execution) | Live heat tracking with the melting SOP enforced step by step. | Steps back-filled after the fact or skipped — logged only when someone is watching. |
Assure (Quality) | Automated, audit-ready MTCs and hands-off QA traceability. | A checkbox tool, opened only when an audit or a customer’s MTC demand forces it. |
Every internal lead I interviewed arrived at the same verdict independently — the product solved real problems, but its adoption model was misaligned with how foundry teams actually behave. The gap wasn’t technical. It was behavioural.
The Emotional Journey
I mapped sentiment for the buyer and the operator across the same journey. The two lines start close, then split. That widening gap is the whole risk, and it’s invisible on any usage dashboard.
The buyer’s line barely moves: report delivery keeps the owner reassured from pitch to renewal — the optics never break, so the sentiment never drops.
The operator’s line falls: confusion at the first report hardens into apathy in daily use, and quiet failure at the exact moment the tool should matter most — from “samajh nahi aata” (don’t understand) mid-journey to falling back to the lab entirely.
Two people, one product, two entirely different emotional experiences — and the business was only ever measuring one of them.
(Method note to preserve: sentiment coded from interview transcripts at six shared journey moments — Pitch & demo, Install, First reading, Daily use, When it matters, Renewal — per persona. The curve is directional, a synthesis of what people said and did, not a measured instrument.)
The Renewal Paradox
If the floor barely used SpectroPro, why did owners keep paying for it? I called through a 40+ user sample to find out. The answer had almost nothing to do with usage.
A representative quote: “Kabhi padhta hoon, par utna samajh nahi aata” (Sometimes I read it, but I don’t really understand it) — a melter, on the readings that reach his phone every day.
Why it still renews — six drivers, none of them usage:
Perceived visibility — a TV screen and WhatsApp pings create a constant sense of activity, even when no one acts on the data.
Zero maintenance — once installed it runs passively; no daily effort means no friction, no complaints, no reason to cancel.
Symbol of a “digital foundry” — chemistry on a screen signals modernity to clients and auditors — a reputation asset, not a tool.
No negative signal — no operator says “this is useless”; they quietly ignore it, and silence reads as success upstairs.
Cheap to justify — against the cost of machinery, renewal is trivial — easier to keep than to reopen the decision.
Owner-visible messages — the same alerts reach owners, who assume everyone downstream is acting on them; no tracking, no gap noticed.
READS IT
TRUSTS IT
ACTS ON IT
Arun
Kharde
Gopal
Santosh
The Customization Myth
Customers had one recurring theory of their own low usage: “build my custom request, and we’ll finally use it.” DigiMelt’s own execution data said otherwise.
The number that mattered: of 17,364 heat plans created across 13 foundries over nine months, only 6,932 were actually executed — 40%. Foundries that got their custom features built didn’t climb; one high-volume foundry sat near 14% execution regardless.
The blocker was never the missing feature. It was readability — even literate melting heads struggled to interpret the reports — compounded by complexity and a missing feedback loop: nobody ever learned whether reading a report changed an outcome, so the habit never formed. The handful of power users who did read the graphs proved the value was real, just locked behind friction.
A representative quote: “Report aata hai, dekhte bhi hain — par usse samajhna padta hai” (The report comes, we even look at it — but you have to work to understand it) — a melting head.
The reframe: custom requests were an emotional signal, not an adoption lever. Building them felt like progress and delivered none. That single insight redirected the conversation from “what feature next?” to “what habit are we actually building?”
What It All Meant
Across every module, the same six forces kept surfacing. Together they describe a product with a behaviour gap, not a feature gap.
Adoption–Value Gap — usage falls off sharply after install; the value promised at purchase is never realised on the floor.
Floor-Level Disconnect — owners buy the vision; operators experience the tools as extra work, not support.
Habit Never Formed — the product fits the workflow in theory, but nothing anchors it into the operator’s daily rhythm.
No Feedback Loop — users never see whether acting on a reading helped, so the behaviour is never reinforced.
Ownership Assumption — buyers assume teams will adopt automatically; without a push, they simply don’t.
Delivery ≠ Absorption — a report on WhatsApp feels modern, but modern delivery masks near-zero comprehension.
The one line that redirected the roadmap: emotional signals were misleading the team into building features that don’t sustain adoption. There was no feature gap — there was a habit gap.
The three questions, answered:
What pain does it solve? Real ones — chemistry speed, melt discipline, audit-ready proof all map to genuine, validated jobs on the floor.
Is the pain burning and frequent? Unevenly — chemistry decisions recur several times a shift; QA traceability only burns when an audit or customer demands it, which is exactly how each module gets used.
Who truly uses it, and why? Owners renew, operators abstain. Buyers pay for the optics of a digitised foundry; a minority of floor operators extract real value; the rest never built the habit.
Where I Pointed the Roadmap
Diagnosis is only half the job. I translated the findings into a sequenced set of moves — reframing what “success” even meant, then fixing the floor before adding to the ceiling.
Redefine success: measure floor-level active usage, not installs or renewals. A renewal built on optics is a false positive — the health metric had to become “did an operator act on this today?” not “is it switched on?” Measured by: weekly operator-level actions per module, tracked against the renewal book.
Quick wins (0–1 mo): fix reliability before features. Kill the dead-click and rage-click hotspots and grade-duplication errors surfaced in Clarity; add clear loading and disabled states — trust erodes the moment the tool stalls. Measured by: dead/rage-click and task-error rates in Clarity, month over month.
Structural (1–2 mo): design for the operator, not the buyer. Make readings self-explaining, cut cognitive load in the reports, add guided navigation so users stop getting lost between modules. Measured by: unprompted comprehension in follow-up calls; fewer cross-module dead-end journeys.
Strategic (2–3 mo): ship habit mechanisms, not just customizations. Close the feedback loop — show operators that acting on a reading changed the outcome. Reinforcement, in-flow nudges, and visible wins are what turn a delivered report into a daily behaviour. Measured by: repeat-usage streaks; heat-plan execution climbing above the 40% baseline.
What Changed
An audit succeeds when it changes decisions, not slides. Here’s what moved, stated honestly, without inflating a research project into a revenue miracle.
Success was redefined. Module health shifted from installs-and-renewals to floor-level active usage — the false-positive metric the whole audit exposed was retired as the yardstick.
The roadmap flipped from features to behaviour. Stability and report readability moved ahead of new custom builds; customization requests were reframed as signals to validate, not promises to fulfil.
One belief went straight into a live test. The long-standing “change the format and they’ll use it” theory became an instrumented experiment — the requested change shipped with usage tracked before and after, instead of shipped on faith.
Leadership got a per-module truth table. The Perception-vs-Reality synthesis became the shared evidence base for invest/fix/hold conversations across the three modules.
Limitations I'd Flag Myself
Small qualitative core — depth interviews were few by design; confidence comes from triangulation across three independent data sources, not from n.
Recall bias — mitigated, not eliminated, by the case-based method: one real event, reconstructed with its artefacts, never opinions or averages.
Directional sentiment — the emotional journey is coded synthesis, not a measured instrument. It maps the shape of the gap, not its magnitude.
Uneven instrumentation — not every module was equally tracked; the audit itself surfaced event-tracking gaps, which capped how far some behavioural claims could go.
Closing Reflection
“The most dangerous metric is the one that looks healthy.”
Why it matters: a product can pass every dashboard and still be failing on the floor. This audit built the apparatus to see that, and gave the team permission to act on it.
What I took from it: the researcher’s real job is to find the gap between what people say, what they buy, and what they actually do — and to name it plainly, even when the numbers look comfortable.
