IQWiG in dialogue
Presentations from previous "IQWiG in dialogue" Events
In our English-language event "Information Retrieval Meeting (IRM 2022): Transition into the Digital Age", there were several inspiring keynotes, presentations and workshops on current developments and practical experiences with digitization and automation in information retrieval.
Discussion of EMA’s current methodological proposals
“Estimands” have lately been the subject of increased discussion among researchers, drug manufacturers and public authorities, particularly within the context of drug approval. An “estimand” means the effect to be estimated in a planned study, for example the difference between two drugs regarding a patient-relevant outcome under certain conditions to be chosen. These depend on, among others, the population of interest, the handling of intercurrent events and the effect measure used. The topic has become more important since the publication of a paper by the European Medicines Agency (Addendum to the EMA guideline ICH E9) in August 2017. The specifications in this paper could not only affect clinical trials and approval procedures, but also influence benefit assessments – according to critics, at the expense of the current high Standards.
Results might be unusable for benefit assessments
The topic of estimands is one of the aspects addressed by the current addendum to the EMA guideline ICH E9 on statistical principles for clinical trials. The addendum describes five strategies that lead to different estimands. However, not all estimands can be estimated using well-established methods without having to accept a high risk of bias. This is the case, for example, if not the data of all randomized study participants are analysed, but those of subgroups (which can only be defined hypothetically) with insufficient structural equality.
Hence the question arises which estimands can be used in a meaningful way in benefit assessments, and which estimands are irrelevant to HTA decisions. There is the additional risk that the strategies described are misused to justify that important data, such as adverse events, are no longer recorded completely in clinical trials.
At “IQWiG in dialogue” on 15 June 2018, experts from research, regulatory authorities and industry discussed possible uses and problems of estimands for benefit assessments.
The first IQWiG methods paper (version 1.0) was published on March 1, 2005. Since then the IQWiG methods have been regularly updated. Version 5.0 is planned to be published in July 2017. A number of methodological issues have been discussed during the IQWiG in Dialogue events since 2008. The 10th anniversary event of IQWiG in Dialogue 2017 puts the IQWiG methods in an international context. Several international HTA organizations will introduce their own methods guides and important methodological issues of comparative effectiveness research can be discussed from various perspectives.
Please note: the presentations are in English.
Stricter criteria for the assessment of study results have been repeatedly called for in the current scientific literature. For instance, statisticians call for the general application of stricter significance levels in scientific studies (e.g. 0.005 or 0.001 instead of 0.05). Proposals have been made in the Cochrane Collaboration to use more rigorous methods and criteria for systematic reviews such as stricter monitoring of data quality, adjustment for multiple outcomes or the use of sequential methods in meta-analyses. We would like to discuss the necessity and practicability of these proposals with experts from industry, the Cochrane Collaboration and research.
The following question commonly arises in the process of approval decisions on new drugs and subsequent decisions on their (added) benefit determined in benefit assessments: Are the fundamental principles on the basis of which these decisions are made comparable at all?
Specifically planned confirmatory studies are required for the approval of a new drug. In contrast, following the principles of evidence-based medicine, benefit assessments use data already available. The presentations and discussions at this year’s IQWiG in Dialogue address the question as to what extent HTA decisions can be of a confirmatory nature and to what extent such a demand can be meaningful at all.
Switching treatments during a clinical study, e.g. from the control to the treatment arm with the new drug, does not only have consequences for the patient, but also for the assessment of study data by the regulatory authorities and IQWiG: Particular requirements and methodological problems for the informative value of the data result from this treatment switching.
But would it be ethically justified at all to withhold a drug from a patient by prohibiting treatment switching in clinical studies if, in the course of the study, this drug turns out to be probably effective, and to treat this patient with best supportive care instead? In many cases, particularly in oncology, investigators tend to generally allow patients to cross over from the control to the treatment arm already during the study – despite the problems this causes for the analysis of the study results.
The presentations and discussions at this year’s “IQWiG in dialogue” therefore deal with the particular requirements and methodological problems from the perspectives of clinical practice, regulatory approval, research and IQWIG.
The therapeutic indication a drug is approved for does not always correspond to the approval studies actually conducted. This can pose a problem - particularly for the early benefit assessment according to AMNOG - for IQWiG and the G-BA as well as for the drug manufacturers.
There are two main reasons when the approval population deviates from the study population: On the one hand, the approval studies themselves may have provided indications that a positive benefit-risk relation exists only for a subgroup of the patients examined. It is then only necessary to analyse subgroups of the approval studies - also in order not to provide a false negative assessment of the new drug. On the other hand, general reservations of the regulatory authority concerning certain therapeutic indications can lead to the regulation that a new drug is not to be widely used, but only in selected areas, e.g. as second- or third-line therapy. However, adequate subgroup analyses might not be possible in these cases because the drug was not tested as second- or third-line therapy in studies. This is a dilemma both for the manufacturer and the institutions conducting the assessment.
Presentations and discussions during this year's "IQWiG in dialogue" will explore the reasons for these discrepancies and ways of avoiding them or dealing with them appropriately.
Since the new Version 4.0 of the General Methods in 2011, IQWiG has been using the three terms "proof", "indication" and "hint" to formulate conclusions on the benefit or harm of medical interventions. The use of these terms depends on the certainty of results of the underlying studies. "Proof” of a benefit can only be confirmed in cases with high certainty of results. "Indication” refers to the weaker, while” hint” refers to the weakest level of certainty of results within the three categories. If, for instance, studies use surrogate endpoints or indirect comparisons, the uncertainty of study results increases.
In this year's "IQWiG in Dialogue”, for discussion with experts the Institute presents its approach to drawing conclusions on benefit in cases of increased uncertainty.
Institutions assessing the benefits and harms of medical interventions are often confronted with the fact that, due to their heterogeneity, studies forming the basis of the assessment can be pooled only with difficulty or not at all. This is the case, for example, when studies show different characteristics, e.g. regarding disease severity, dosage of medication, or study duration.
The meeting "IQWiG in Dialogue 2011” therefore addressed the statistical methods for examining heterogeneity, as well as the handling of heterogeneity in the assessment of the benefits and harms of interventions.
It is a well-known and recognized requirement that in clinical trials, in addition to evaluating (statistical) significance, the (clinical) relevance of results should also be evaluated. This year "IQWiG in dialogue” therefore deals with the approach to such an evaluation within the framework of systematic reviews and its importance to the approval of drugs, as well as to the conduct and assessment of clinical trials.
On 18 June 2010, 155 participants discussed the necessity, methods and consequences of relevance assessments with representatives from IQWiG, regulatory authorities, universities and industry.
In addition to assessing desirable effects and (additional) benefit, IQWiG is also required by law to assess the possible harm of medical interventions. However, assessing the harm of therapies is not as well defined in systematic reviews and HTA reports as assessing the benefit. Consequently in 2009 "IQWiG in dialogue" addressed various aspects concerned with assessing harmful effects.