Value of Information (VoI) Analysis for estimating the benefits of undertaking additional Studies Within A Trial (SWATs) on pens as a retention strategy

Background: Trial Forge Guidance 2 is a guidance for researchers to determine whether a further Study Within A Trial (SWAT), related to a specific retention or recruitment strategy, is recommended to reduce the underlying uncertainty of the evidence (Treweek et al., 2020). Due to its high internal validity, however, this guidance is subject to an increased likelihood of encouraging an over-investment into further SWATs of an existing strategy given the budget constraints. 

Methodology: We used an already established framework of Value of Information (VoI) analysis, based upon random sampling on parameters from standard meta-analysis results (Claxton et al., 2015), to address the aforementioned biasedness associated with Trial Forge Guidance 2. A case study of pens as a retention strategy was used, for which Trial Forge Guidance 2 recommends that further SWATs be undertaken according to the uncertainty of the evidence from five SWATs. The modified VoI analysis framework was gradually adopted across the five SWATs and enabled the estimation of figures related to the value of implementation and the value of additional research. 

Results: Given the existing evidence, the value of additional research is 88 and the value of implementation is 4,814. In other words, 70 additional SWATs need to be funded to observe the identical expected incremental retention rates as those obtained under the existing evidence for pens, and only one further SWAT would be enough to eliminate any uncertainty related to pens. Therefore, the VoI analysis suggests that no further SWATs of pens as a retention strategy be undertaken, and pens be instantly adopted as a retention strategy in all trials involving returns of questionnaires during follow-up. Also, the evidence from the first two SWATs of pens was sufficient for instantly adopting pens as a retention strategy given its estimated effectiveness and the underlying uncertainty.

Discussion: The study’s VoI analysis framework or making investment decisions on SWATs is strongly recommended for all recruitment and retention strategies. It is strongly encouraged that VoI analysis is combined with Trial Forge Guidance 2 to prioritise SWAT-related research, for finding a balance between internal validity and financial constraints when making investment decisions on SWATs of recruitment and retention strategies and deciding which of these strategies should be adopted in future trials’ recruitment and retention processes. 

Bibliography:

1. CLAXTON, K., GRIFFIN, S., KOFFIJBERG, H. & MCKENNA, C. 2015. How to estimate the health benefits of additional research and changing clinical practice. BMJ, 351, h5987.
2. TREWEEK, S., BEVAN, S., BOWER, P., BRIEL, M., CAMPBELL, M., CHRISTIE, J., COLLETT, C., COTTON, S., DEVANE, D., EL FEKY, A., GALVIN, S., GARDNER, H., GILLIES, K., HOOD, K., JANSEN, J., LITTLEFORD, R., PARKER, A., RAMSAY, C., RESTRUP, L., SULLIVAN, F., TORGERSON, D., TREMAIN, L., VON ELM, E., WESTMORE, M., WILLIAMS, H., WILLIAMSON, P. R. & CLARKE, M. 2020. Trial Forge Guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed. Trials, 21, 33.

Zoom details:

Join Zoom Meeting
https://york-ac-uk.zoom.us/j/95683653566?pwd=NU1YUjRra0Qwc3Q3TXZVNGxmZm5OUT09

Meeting ID: 956 8365 3566
Passcode: 866318