Systematic identification of confounders: saving resources is possible.

Non-randomized study designs can also play a role in benefit assessments, because among other things, according to legal requirements, routine practice data collection is to be conducted without randomization. However, without randomization, the treatment groups or study arms often differ structurally from one another. For example, patients' age or the severity of their illness may influence their allocation to a particular treatment group, which can change the risk of serious, drug-related side effects.

Such variables, known as confounders, can introduce bias into study results, but this can be mitigated to some extent using statistical methods. However, this requires confounders to be recorded in the study, so they must be known from the outset. The scientific literature describes a specific approach to systematically identifying confounders (Pufulete 2022). This approach comprises three steps: a systematic literature review, interviews with clinicians to identify confounders, and a survey in which further clinicians evaluate the relative relevance of the confounders.

Depending on the research question, this approach can be very resource-intensive. The German health technology assessment agency, the Institute for Quality and Efficiency in Health Care (IQWiG), therefore evaluated this approach with the support of an external working group, using a comparison of two drug therapies for relapsing-remitting multiple sclerosis (RRMS) as an example. The three objectives were to systematically identify confounders based on the approach by Pufulete (2022), evaluate the approach's basic feasibility, and derive recommendations, particularly for saving resources. The results were published as a working paper on the IQWiG website in June 2025. An English translation is now available.

Feasible, but resource-intensive, depending on the indication

In the first step, the external working group identified and analysed numerous studies. This resulted in the identification of 125 potential confounders. In the second step, qualitative interviews with eight clinicians yielded a further 35 potential confounders. These confounders were then evaluated by clinicians in a quantitative survey.

The report by the external working group showed that this final step requires a major investment of resources due to the large number of variables identified. For this reason, IQWiG researchers summarized the potential confounders further by identifying duplicates and overlaps, without losing any relevant information. Examples of overlapping concepts include indication-specific and generic health-related quality of life. Following this more intensive summarization of content, 28 potential confounders remained.

Specific recommendation

Overall, our findings show that identifying confounders based on the approach by Pufulete 2022 is feasible and represents a fundamentally sound approach. However, in order to identify potential confounders in advance when planning a non-randomised comparison to evaluate intervention effects, IQWiG recommends a main change from the Pufulete 2022 approach. The confounders identified in the first step must be summarized intensively to markedly reduce the amount of resources required for the final step, in which clinicians evaluate the relevance of the confounders.

Further simplifications sought

In addition to the resource-saving changes, the rapid report also contains changes aimed at improving the subsequent use of confounders for the registry study.

The modified approach to systematically identifying confounders still requires a major input of human resources, e.g., from registry operators. In the next step, the IQWiG team would therefore like to explore further resource-saving options, although this must always be weighed against a possible loss of certainty of results.