How important is the certainty of results?
IQWiG's benefit assessments often have a substantial impact on patient care in Germany. Therefore, the studies included in the assessment should offer a high degree of certainty of results.
Whether a study offers sufficiently reliable results depends on
- whether the study is relevant to the medical question at hand (= external validity),
- whether there were weaknesses in the study design or implementation (= internal validity), and
- whether the study was large enough to deliver sufficiently precise results (= statistical precision).
The most important factors for the certainty of results are the study design and conduct. Only a fair comparison makes it possible to assess the true effect of the treatment. It is crucial that two (almost) identical patient groups are compared; this can best be achieved through randomization, i.e. random and concealed group allocation.
It is also usually important that the study was blinded: if the study participants, the medical staff and the additional study staff did not know who received which study treatment, it can be ruled out that differences in concomitant therapy and subjective expectations distorted the study results. This applies in particular to the placebo effect.
In its reports, IQWiG describes in detail the strengths and weaknesses of the studies available on a given topic. The conclusion of a benefit assessment ultimately depends heavily on the certainty of the results. IQWiG usually uses the terms "proof", "indication" or "hint" to express how certain or uncertain the results for a particular question were. If the results are very uncertain or missing, it may also happen that IQWiG is unable to answer a question.
Fundamental aspects of benefit assessments explained simply
IQWiG's methods are described in detail in its methods paper. But why does IQWiG work the way it does? What is so important about a direct comparison? And what exactly is the appropriate comparator therapy? We explain these and other aspects of our methods briefly on these pages.