Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients

It is recognised that treatment-related clustering should be allowed for in the sample size and analyses of individually-randomised parallel-group trials that evaluate therapist-delivered interventions such as psychotherapy. Here, interventions are a treatment factor, but therapists are not. If the aim of a trial is to separate effects of therapists from those of interventions, we propose that interventions and therapists should be regarded as two potentially interacting treatment factors (one fixed, one random) with a factorial structure. We consider the specific design where each therapist delivers each intervention (crossed therapist-intervention design), and the resulting therapist-intervention combinations are randomised to patients. We adopt a classical Design of Experiments (DoE) approach to propose a family of orthogonal factorial designs and their associated data analyses, which allow for therapist learning and centre too. We set out the associated data analyses using ANOVA and regression and report the results of a small simulation study conducted to explore the performance of the proposed randomisation methods in estimating the intervention effect and its standard error, the between-therapist variance and the between-therapist variance in the intervention effect. We conclude that more purposeful trial design has the potential to lead to better evidence on a range of complex interventions and outline areas for further methodological research.

💡 Research Summary

The paper addresses a methodological gap in individually randomised parallel‑group trials of therapist‑delivered interventions, where treatment‑related clustering (i.e., patients clustered within therapists) is often recognised but not fully incorporated into the design. The authors argue that when the trial aim is to disentangle the effects of the intervention from those of the therapists, both should be treated as experimental factors: the intervention as a fixed factor and the therapist as a random factor. They focus on the “crossed” therapist‑intervention design, in which every therapist delivers every intervention, and propose a family of orthogonal factorial designs that randomise the therapist‑intervention combinations to patients.

Three increasingly complex designs are presented:

1. Completely Randomised Factorial Design – suitable for a single centre with constant therapist‑intervention availability. All possible therapist‑intervention pairs are equally replicated and randomly allocated to patients in a single block.

2. Randomised Block Factorial Design – introduces a blocking factor (e.g., time batches) to capture therapist learning curves or other temporal effects. Within each block, the same set of therapist‑intervention pairs is randomly assigned, preserving balance while allowing block‑specific variation.

3. Multicentre Randomised Block Factorial Design – adds a second blocking factor (centres) to the previous design. Therapists are nested within centres, and each centre‑batch combination receives its own random permutation of therapist‑intervention pairs. This mirrors the structure of many multi‑site trials.

For each design the authors provide a clear randomisation algorithm based on systematic assignment followed by independent random permutations (i.e., permuted‑block randomisation). They illustrate the designs with theoretical examples derived from two motivating trials (a large IAPT trial and a smaller psychotherapy project).

The statistical analysis framework is developed in parallel using ANOVA and linear mixed‑effects regression. The ANOVA representation lists fixed effects (intervention, block, centre), random effects (therapist, therapist‑by‑intervention interaction, therapist‑by‑block, centre‑by‑block), and residual error. The equivalent regression model is expressed as a linear mixed model with appropriate random‑effects structures, enabling implementation in standard software (e.g., lme4 in R). The orthogonal nature of the designs guarantees that main‑effect and interaction estimates are uncorrelated, simplifying inference and improving power.

A small simulation study evaluates the performance of the proposed designs under varying numbers of therapists, interventions, blocks, and centres, as well as different variance component magnitudes. Results show that the orthogonal factorial designs produce unbiased estimates of the intervention effect, accurate standard errors, and reliable estimates of both therapist variance and therapist‑by‑intervention variance. Importantly, accounting for therapist‑related random effects reduces the required total sample size compared with conventional designs that ignore these components.

The discussion highlights practical considerations such as therapist availability, patient recruitment rates, and block size selection, and points to future methodological work on unbalanced designs, non‑continuous outcomes, and extensions to more than two interventions. The authors conclude that purposeful incorporation of therapist factors through orthogonal factorial designs can yield more efficient and informative trials of complex, therapist‑delivered interventions, ultimately strengthening the evidence base for such treatments.