2018: Do the new “estimand” strategies compromise the standards of benefit assessments?

Discussion of EMA’s current methodological proposals

“Estimands” have lately been the subject of increased discussion among researchers, drug manufacturers and public authorities, particularly within the context of drug approval. An “estimand” means the effect to be estimated in a planned study, for example the difference between two drugs regarding a patient-relevant outcome under certain conditions to be chosen. These depend on, among others, the population of interest, the handling of intercurrent events and the effect measure used. The topic has become more important since the publication of a paper by the European Medicines Agency (Addendum to the EMA guideline ICH E9) in August 2017. The specifications in this paper could not only affect clinical trials and approval procedures, but also influence benefit assessments – according to critics, at the expense of the current high Standards.

Results might be unusable for benefit assessments

The topic of estimands is one of the aspects addressed by the current addendum to the EMA guideline ICH E9 on statistical principles for clinical trials. The addendum describes five strategies that lead to different estimands. However, not all estimands can be estimated using well-established methods without having to accept a high risk of bias. This is the case, for example, if not the data of all randomized study participants are analysed, but those of subgroups (which can only be defined hypothetically) with insufficient structural equality.

Hence the question arises which estimands can be used in a meaningful way in benefit assessments, and which estimands are irrelevant to HTA decisions. There is the additional risk that the strategies described are misused to justify that important data, such as adverse events, are no longer recorded completely in clinical trials.

At “IQWiG in dialogue” on 15 June 2018, experts from research, regulatory authorities and industry discussed possible uses and problems of estimands for benefit assessments.