01 · Impact Measurement Framework
Use: The mental model every report, dashboard, and SROI calculation hangs off. If a metric doesn't fit one of the four tiers, we don't publish it.
The four tiers
Tier 1 OUTPUTS What we delivered
│
Tier 2 OUTCOMES What changed for participants
│
Tier 3 IMPACT What changed for the organisation / system
│
Tier 4 SROI Monetised social value, with confidence factor
Reports always show all four tiers. Skipping Tier 1 makes the report unfalsifiable; skipping Tier 4 lets us off the hook on commercial value.
Tier 1 — Outputs
Count of things we did. Hard to get wrong. Easy to over-rely on.
| Metric | Source |
|---|---|
| Cohorts delivered | cohorts table (Sprint 5 / 05) |
| Participants enrolled | participants table |
| Workshop hours delivered | derived from cohort schedule |
| 1:1 coaching hours delivered | coach_client_sessions (see src/server/coach-client-sessions.functions.ts) |
| Nudges sent | email_send_log (deduplicated by message_id) |
| Manager retrospectives held | event log |
Rule: Outputs are necessary but never sufficient. Never headline a report with an output ("we delivered 412 sessions") without an outcome alongside it.
Tier 2 — Outcomes (participant-level)
Self-reported change at participant level, captured via the pulses we already ship:
- Sprint 1 / 06 nudge feedback
- Sprint 3 / 05 day-30 pulse
- Sprint 4 / 08 financial confidence pulse
- Sprint 5 mid-engagement and exit pulses
| Metric | Instrument |
|---|---|
| Confidence delta (manager mental health) | Sprint 1 pulse, pre / post |
| Confidence delta (financial wellbeing) | Sprint 4 / 08 pulse, pre / post |
| Self-reported sleep / stress / energy change | Sprint 5 mid-engagement pulse |
| "Would recommend to a colleague" | exit pulse |
| Net behaviour change ("I have done X differently because of this") | exit pulse, free-text + coded |
Rules:
- Never report a cohort outcome with n < 5. Suppress and explain.
- Always show the distribution (e.g. % at each level), not just the mean. Means hide bimodal distributions.
- Pair every quantitative outcome with at least one anonymised qualitative excerpt (file 09).
Tier 3 — Impact (organisation-level)
Change for the client organisation. Slower-moving, harder to attribute, but the thing the board actually cares about.
| Metric | Source | Attribution rule |
|---|---|---|
| Manager-led absence reduction | Client HRIS, opt-in | Reported as "associated with," never "caused by," unless we have a control cohort. |
| Voluntary attrition delta | Client HRIS | Same. |
| Internal mobility increase | Client HRIS | Same. |
| Engagement / eNPS delta | Client engagement survey | Same. |
| Manager confidence to have a hard conversation | Sprint 1 pulse rolled to cohort | Direct attribution OK. |
| Menopause-related accommodations granted | Sprint 2 / 01 policy uptake | Direct attribution OK. |
| Financial confidence cohort delta | Sprint 4 / 08 pulse | Direct attribution OK. |
Rule on attribution: If we don't have a control or comparison cohort, we say "associated with" — not "caused" or "drove." Repeated in every report's methodology footnote.
Tier 4 — SROI
Monetised social value, calculated using the methodology in file 04 and the confidence factor in file 05.
Reported as: £X of social value per £1 of engagement fee, with a confidence factor of Y%.
Never reported as: a single point estimate without confidence, or without a sensitivity range.
What we deliberately don't measure
- Anything requiring clinical assessment (PHQ-9, GAD-7, WEMWBS for treatment-effect claims). We can use validated wellbeing scales for direction of travel at population level, but not to make per-person clinical claims. Tied to the no-clinical-services scope.
- "Time saved by HR" without HR sign-off on the saving — too easy to inflate.
- Productivity uplift in £, unless the client supplies the per-FTE figure and signs off on the assumption.
- Anything from a third-party AI sentiment classifier as a primary measure. Sentiment scoring is useful for triage, not for board-grade claims.
Cadence
| Tier | In quarterly client report? | In annual public report? | In board dashboard (monthly)? |
|---|---|---|---|
| Outputs | yes | yes | yes |
| Outcomes | yes | yes | yes |
| Impact | yes (with caveats) | yes | yes |
| SROI | yes (per engagement) | yes (rolled up) | yes (rolled up) |