Designing a Connection Evaluation You Can Actually Run

The evaluation plan you write in a funding bid and the evaluation you can complete eighteen months later are usually two different documents. The bid version has a validated loneliness measure at baseline, three and twelve months, a comparison group, and linked GP data. The completed version has 180 intake forms, 41 follow-ups, and a set of quotes.

This is a solvable problem, and the way to solve it is to design the smaller study first rather than downgrading the ambitious one in stages. The most useful published guide to doing that is the social prescribing literature, because it is the largest body of description of connection programmes that were actually evaluated rather than merely delivered.

What the evaluated schemes measured — and what that tells you

Two systematic reviews published in 2021 are worth reading side by side. The review in Perspectives in Public Health found nine studies of social prescribing initiatives with loneliness outcomes. All nine reported positive individual impacts. Three reported reductions in GP, emergency, social worker, or inpatient service use. The review in the International Journal of Environmental Research and Public Health found that increases in self-esteem and self-confidence were the outcomes most consistently reported, and was explicit that trial evidence was limited and that programmes were too heterogeneous to pool.

Nine out of nine positive should not make you confident. It should make you cautious about the design of the underlying studies, because a body of evidence in which nothing fails is a body of evidence with a filter somewhere in it — publication, selective follow-up, or outcome choice made after the fact. The American Heart Association’s August scientific statement, led by Crystal W. Cene, made the same point from the other direction: after establishing that social isolation and loneliness are associated with roughly a 30% increased risk of heart attack, stroke, or death from either, it named the absence of intervention evidence as the central research gap. We know the exposure matters. We do not know what fixes it.

That is your actual situation as a practitioner. You are operating in a field where the risk factor is well characterised and the intervention evidence is thin, which means a modest, honestly-reported evaluation of your own programme has real value — and an over-promised one has negative value, because it adds to a literature that already looks suspiciously uniform.

The four decisions, in order

1. Pick one primary outcome and one instrument you did not write.

The single most common design error is a bespoke questionnaire. It feels responsive to your programme and it produces numbers that cannot be compared to anything. The AARP Foundation’s 2018 survey of 3,020 adults aged 45 and older used the 20-item UCLA Loneliness Scale rather than a bespoke instrument specifically so its finding — one in three midlife and older adults lonely — sits alongside the academic literature. Short forms of the UCLA scale are widely used in service settings where a 20-item form is not viable. The UK’s 2018 loneliness strategy took the same approach at national scale, embedding a standard set of loneliness questions into Office for National Statistics collection so that different programmes’ results could be read against a national baseline.

Choose one. Use it unmodified. Do not add or drop items.

2. Decide the follow-up point before you enrol anyone, and staff the follow-up, not the baseline.

Baselines are easy — people are in front of you, filling in forms. Follow-up is a phone call to someone who stopped coming. Budget accordingly: if you have 40 hours of administrative time for measurement, put ten into intake and thirty into chasing follow-ups.

3. Measure something your programme can plausibly move in your timeframe.

A twelve-week group with 60 participants will not detect a change in cardiovascular events. It may detect a change in self-reported loneliness, network size, or confidence — the outcomes the social prescribing reviews found were actually shifting. Holt-Lunstad’s 2021 review in the American Journal of Lifestyle Medicine argues for treating social connection as a modifiable protective factor alongside diet, exercise, and smoking. That framing is useful for your case to a funder. It is not a licence to claim mortality outcomes from a pilot.

4. Add service-use data only if you already have the access.

Three of the nine studies in the loneliness review reported service-use reductions. Those are the findings commissioners want most, and they require a data-sharing route that takes months to establish. If you do not have it on day one, treat it as a second-phase question rather than a promise.

The loyal-forty problem

Here is the failure mode that quietly ruins more community evaluations than any other. You enrol 180 people. You get baseline data from nearly all of them. At six months you obtain follow-up data from 41 — and those 41 are the people still attending, because they are the easiest to reach and the most willing to fill in a form.

Your dataset now describes the experience of your most successful participants. The improvement you report is real for them and tells you almost nothing about your programme, because the people who left after two sessions — including everyone for whom it did not work — are absent. Call it the loyal-forty problem, and design against it explicitly: a fixed follow-up window applied to everyone enrolled regardless of attendance, a named person responsible for reaching non-attenders, and a reported figure for how many you reached. An evaluation with 60% follow-up and a smaller effect is worth more than one with 25% follow-up and a large one.

Claim you might want to make	Evidence status
Social isolation and loneliness raise cardiovascular risk by around 30%	Strong. AHA scientific statement, 2022, pooled across observational studies
Group activity programmes reduce self-reported loneliness	Weak to moderate. All nine studies in the 2021 review positive, but limited trial evidence and clear risk of selective reporting
Social prescribing raises self-esteem and confidence	Moderate for direction, weak for magnitude. Most consistently reported outcome across the 2021 wellbeing review
Programmes reduce GP and emergency service use	Preliminary. Three of nine studies; not pooled
Health systems should assess isolation routinely	Endorsed by expert consensus. National Academies, 2020, and the clinician-facing commentary in the American Journal of Geriatric Psychiatry
Your specific programme reduced loneliness	Depends entirely on your follow-up rate

What this means in practice If you can only do one thing well, do this: one validated instrument, administered to everyone at enrolment and again at a fixed date, with a named person accountable for the follow-up rate and that rate reported alongside every result. Everything else — comparison groups, service-use linkage, qualitative strands — is an upgrade on that foundation, not a substitute for it.

Use your baseline data to test reach

There is a diagnostic use of measurement that most programmes miss. The Harvard Making Caring Common survey found 36% of Americans reporting serious loneliness, rising to 61% of young adults aged 18 to 25. The National Academies put roughly a quarter of adults 65 and over in the socially isolated category. If your intake data shows baseline loneliness at or below those population figures, your recruitment is reaching people who are comparatively well connected — and no amount of outcome improvement fixes that.

Read your baseline distribution before you read your change scores.

What this does not solve

None of this gives you causal attribution. Without a comparison group you cannot separate your programme from regression to the mean, seasonal effects, or the fact that people who sign up for a connection group are people already moving toward connection.

It also does nothing about reach. Every design here measures people who arrived — self-referred, or sent by a link worker. The Harvard survey found that about half of lonely young adults said nobody had spent more than a few minutes recently asking how they were in a way that felt genuine. Those people are not in your intake data, and a better evaluation will not find them.