Digital or In-Person Groups: What to Actually Require in an Evaluation

A bid lands on your desk proposing a peer support group delivered over video call. The cost per participant is a third of the in-person alternative you also fund. The temptation is to treat this as a straightforward efficiency win: same outcome, lower cost, more reach. It is not straightforward, and the evidence you would need to settle the question does not really exist yet.

This is the decision most funders and commissioners are actually facing: not “does connection programming work” but “does this delivery format produce the same thing for less money, or a different and lesser thing that looks similar on a spreadsheet.” The second question is the one nobody’s dashboard answers.

Why the comparison is harder than it looks

Almost everything cited to justify social connection funding — the mortality meta-analyses, the isolation prevalence figures, the Surgeon General’s 2023 advisory — describes the consequences of disconnection, not the comparative effectiveness of delivery formats. None of it was designed to tell you whether a WhatsApp check-in group produces the same reduction in loneliness as a weekly walking group in a park.

The closest indirect evidence points in a specific direction, and it is worth taking seriously even though it is not a head-to-head trial. Eric Klinenberg’s account of social infrastructure argues that shared physical space does causal work that a conversation alone does not — his analysis of the 1995 Chicago heat wave found that neighbourhoods with functioning public spaces and dense local ties had markedly better survival, because those spaces created the conditions for people to actually notice and check on each other. Ray Oldenburg’s concept of the third place makes a related argument: informal public gathering spots do social work partly through incidental, low-stakes repeated contact that is hard to replicate when a session has a start time, an end time, and a mute button.

A 2022 qualitative meta-synthesis of social prescribing found something that should worry anyone assuming contact itself is the active ingredient: participants described benefit as coming from restored meaningful participation and purpose, not just from talking to someone, and structured, purposeful group activity appeared to do more than contact alone. That is a finding about structure and purpose, not about digital versus physical delivery — but it is a caution against assuming any format that gets people “in a room” (virtual or otherwise) is equivalent.

None of this proves digital delivery is worse. It proves you don’t currently have grounds to assume it’s equivalent, and a bid that treats cost-per-head as the deciding metric is quietly assuming equivalence.

What the evidence actually supports

Claim	Evidence status
Social isolation and loneliness carry serious health risk, comparable in scale to established risk factors	Strong — consistent across multiple large meta-analyses and the 2023 Surgeon General advisory
Structured, purposeful activity produces more benefit than contact alone	Reasonable — supported by qualitative synthesis, though not modality-specific
Physical shared space does distinct social work beyond facilitating contact	Reasonable — argued at length by Klinenberg and Oldenburg, largely from observational and historical evidence rather than controlled comparison
Digital delivery produces equivalent outcomes to in-person delivery at lower cost	Unsupported — no source in the current literature directly tests this
Social prescribing reduces use of GP, emergency, or inpatient services	Mixed — a minority of included studies in the relevant systematic review report this; most report only individual-level benefit
Neighbourly, low-stakes contact is protective independent of formal programming	Reasonable — AARP’s 2018 survey found lonely rates of 33% among people who talk to neighbours versus 61% among those who never do

The gap in that table is the one that matters for your decision. You are being asked to fund a bet, not a proven substitution, and the evaluation you require should be built around that fact rather than around whichever metric the format happens to make easy to collect.

The completer-count trap

Digital programmes report attendance more easily than in-person ones — a log-in is a log-in — and that ease quietly becomes a substitute for measuring whether anything happened. Call this the completer-count trap: a report that cites “240 people reached” over eight weeks, where reach means logged into at least one session, tells you nothing about whether those 240 people formed anything resembling a relationship. Cigna’s 2020 workplace data is a useful corrective here: lonely workers missed work roughly twice as often for illness and five times as often for stress, which is the kind of downstream cost that a login count will never surface and that only sustained relationship, not exposure, plausibly reduces.

The in-person equivalent failure mode exists too — a well-attended single event that nobody returns to — but digital delivery makes it easier to mistake volume for depth, because the marginal cost of adding one more participant to a video call is close to zero, which is exactly why funders like the format.

What this means in practice: Do not accept cost-per-participant-reached as a comparison metric between formats. Require cost-per-completer (someone attending at least 60–70% of sessions), and require a measure of whether contact continued after the funded programme ended — a phone check three months out, not a satisfaction survey at week eight.

What to build into the evaluation before you fund either format

1. Specify the unit of success before the bid is scored. Decide whether you are buying attendance, self-reported loneliness reduction (ideally using a validated instrument such as the UCLA scale used in the AARP survey, not a bespoke house measure), or continued contact outside the programme. Score digital and in-person bids against the same unit.
2. Require retention data broken out by week, not an aggregate. A group that loses half its members between week two and week four tells you something different from one that loses them steadily. If a provider cannot produce this breakdown, that itself is informative.
3. Ask what happens when the programme ends. Meetup’s model — recurring, member-run, free-tier groups with no defined end date — works partly because it does not have a cliff edge. A time-limited funded cohort, digital or physical, does. Ask explicitly what mechanism, if any, is meant to carry contact forward once funding stops.
4. Treat “we can scale this digitally” as a claim to be tested, not a reason to fund. Scale without retention is reach without effect. If a provider proposes scaling a digital format, ask for evidence from a smaller cohort that people who join are still in contact at month three.
5. Fund a genuine within-programme comparison where you can. Few individual providers can run a controlled comparison, but a funder holding a portfolio of both formats can require comparable outcome measures across grantees and look at the aggregate pattern across a funding cycle. This is closer to real evidence than any single evaluation report will give you.

What this does not solve

This framework tells you how to evaluate a choice you have already decided to fund; it does not tell you whether digital delivery reaches people that in-person programming structurally excludes — people with mobility limits, caring responsibilities, or no transport — or whether it merely reaches the digitally confident subset of an already-motivated population. The social prescribing literature’s central weakness, noted in its own systematic reviews, is heterogeneity and thin trial evidence across the board, not something specific to modality. Whichever format you fund, you are still funding for people who signed up. The people least likely to be found by either a video link or a community hall booking are the ones neither format is currently reaching, and no evaluation design changes that.