This is a case study of two linked papers. The first describes the development and validation of a conceptual framework for identifying ‘health-equity relevant’ RCTs that provide evidence about the distributional effects of interventions on individuals or groups along dimensions of social disadvantage and for improving the design, conduct, and reporting of RCTs so that they produce ‘health-equity relevant’ evidence. The second paper offers an assessment of the reporting of equity considerations in RCTs identified as ‘health-equity relevant’ and identifies a range of limitations. Both the authors’ conceptual framework and their assessment of published RCTs form the basis for recommendations to improve the design and reporting of randomised trials to support equity-relevant policy and practice.

When does an RCT provide evidence about health equity? (Paper 1: Jull, J., et al. “When is a randomised controlled trial health equity relevant? Development and validation of a conceptual framework.” BMJ open 7.9 (2017): e015815.)

Unable to locate pre-existing frameworks for answering this question, the authors developed a new conceptual framework detailing the criteria for classifying an RCT as ‘health-equity relevant’. This framework was arrived at through an iterative, consensus-building process involving a team of international, interdisciplinary experts, including members of the public and patient representatives.

According to this framework, an RCT is ‘health-equity relevant’, when:

i) the study includes individuals or a population that experience ill-health due to social disadvantage; AND ii) the study assesses (actually or potentially) the effects of an intervention on the health, or its determinants, of these individuals or populations. The study can either focus exclusively on individuals or populations considered socially disadvantaged or it can consist of a mixed group provided that the differential impacts of the intervention are assessed.

The authors highlight two considerations as crucial in identifying, designing, and reporting ‘health-equity’ relevant trials. The first is the inclusion of estimates of differential impacts in study design, planning, analysis, and reporting. They argue that equity considerations require a move beyond a focus on average efficacy or effectiveness to understanding the distributional effects of intervention effects: assessed either through a consideration of differences between and within subgroups or by looking at gradients of effects across socially stratifying factors. The second consideration is the importance of setting and context.

The authors sought to validate their conceptual framework by using it to screen the results of a MEDLINE database search for randomised trials using the PROGRESS-Plus search terms. Screening confirmed that the framework’s criteria for health equity relevance could be applied consistently to randomised trials.

What are the design, planning and reporting limitations of existing ‘health-equity relevant’ RCTs? (Paper 2: Petkovic, Jennifer, et al. “Reporting of health equity considerations in cluster and individually randomized trials.” Trials 21.1 (2020): 1-12.)

Using their criteria for identifying health-equity relevance, the authors screened the results of a systematic search across three research databases until 200 trials were identified (100 individually randomised (RCTs) and 100 cluster randomised controlled trials (CRTs)). These were then assessed for their reporting of health equity considerations.

In line with previous studies, the authors found that these RCTs either did not report or under-reported: i) sociodemographic details (e.g. socioeconomic status) of participants; ii) results that are disaggregated for specific populations across PROGRESS-Plus characteristics; and, iii) sub-group analyses. The lack of disaggregation or of other descriptive details makes it impossible to explore the differential effectiveness of the interventions and thus to make judgements about the relevance for action to improve health equity.

Limitations in design and planning

The authors report that the RCTs in this sample rarely considered subgroup hypotheses across PROGRESS-Plus characteristics at the design and planning phase of the trial. Only one out of the 200 trials reported formally planning for a subgroup analysis during the design of the trial by considering sample size and statistical power. The authors acknowledge the challenges involved in well-planned (and ad hoc) subgroup analyses as well as the sensitivity of PROGRESS-Plus characteristics, which may account for their absence or under-reporting in trials.

Limitations in reporting

The authors note that whilst many trials in the sample recorded equity-relevant baseline descriptors only about a third of studies reported a subgroup analysis across a population characteristic associated with disadvantage. This suggests that the relevant data were available and that therefore there was an (ultimately missed) opportunity to consider how the analyses might inform equity considerations.

Another example of a missed opportunity are the studies that used types of analysis in which population characteristics are adjusted for (as covariates) but not compared, meaning that readers are unable to draw conclusions about the intervention’s differential impacts. In the sample of RCTs, there were almost twice as many studies which adjusted for PROGRESS characteristics as studies which performed subgroup analyses. Overall, no subgroup analyses were reported for place of residence, occupation, religion, education, or social capital.

The authors identified further deficiencies in study reporting with regards to: the rationale for subgroup analyses, the statistical power for these analyses, and details about the process of recruitment and engagement with people with relevant lived experience. As the authors concluded, even though the trials in their sample were all health-equity-relevant, “only 73% described a rationale for their focus on equity socially disadvantaged population; only 25% reported engagement with communities or individuals who are socially disadvantaged, and only 20% mentioned the importance of outcomes for socially disadvantaged populations.”

Recommendations to improve the design, planning and reporting of trials

The limitations identified by these authors in this sample of potentially equity relevant RCTs led to a loss of potentially important equity-relevant information which could contribute to future hypothesis generation and testing and to future higher-powered studies e.g. meta-analyses.  In this context, the authors make the following recommendations:

– At the stage of design and planning

Wider use of the recently published equity extension of the Consolidated Reporting Standards for Randomized Trials (CONSORT-Equity) may help improve the delineation of hypotheses related to socially disadvantaged populations, and the transparency and completeness of reporting of health-equity considerations in RCTs.

The collection of data on social determinants of health (e.g. PROGRESS-Plus characteristics) even when the trial does not have health equity as a primary research objective. This will enable: the reporting of results disaggregated in terms of characteristics relevant to health equity; the exploration of post hoc subgroup analyses and interactional effects; and the assessment of whether socially disadvantaged populations are excluded, under-represented or adequately represented. Moreover, this data can be utilised in future analyses (e.g. meta-analyses) and can support new hypothesis generation.

A priori, well-evidenced (where possible) plans of analysis (e.g. of required sample sizes and statistical power) of clearly formulated, theory-informed hypotheses regarding distributional effects. When planning a trial, investigators need to: i) consider potential population differences in baseline risk of the condition or problem being studied, and the possibility of differential effectiveness of the intervention; and ii) decide whether subgroup analyses are appropriate and can be accommodated in the sample-size calculation. The authors acknowledge the potential, additional challenges of designing for and obtaining distributional estimates, including increased costs and complexity of analysis and reporting.

– At the stage of reporting

RCT reporting needs to improve by: i) reporting disaggregated data for relevant subgroups; ii) state and justify planned hypothesis testing and present the results of all such analyses; iii) describe sample-size considerations.

If additional subgroup hypotheses of interest are identified during the analysis, they should be clearly presented as exploratory and interpreted with caution. The authors advise that subgroup analyses in trials are reported even when underpowered because: a) they can provide useful information about the nature and direction of any potential effect and used for hypothesis-generation for future studies; b) they can be combined in the future with data from other studies, e.g. in meta-analyses, or other studies where greater power could be achieved.

The authors highlight the potential of sharing data in online repositories as a mechanism to facilitate the availability of disaggregated data for meta-synthesis. This would allow meta-studies to be conducted which may have greater power to detect subgroup differences. In addition, trial registries could request details about planned subgroup analyses and the availability of equity variables