Kommentare des IQWiG zum "Diskussionspapier: Nutzung von Patientenregistern für Zulassungszwecke - methodische und operative Überlegungen" (Englisch)

27. Juni 2019

Comments from:

Institute for Quality and Efficiency in Health Care (IQWiG)
Im Mediapark
50670 Cologne
Germany

Please note that these comments and the identity of the sender will be published unless a specificjustified objection is received. When completed, this form should be sent to the European Medicines Agency electronically, in Word format (not PDF).

1. General comments

IQWiG appreciates the provided opportunity to comment on the discussion paper on patient registries.

In general we are convinced that good quality patient registries can be helpful to answer clinically relevant questions (e.g. detection of rare adverse events by longterm observation of less selected populations), and are especially useful to set up registry-based studies, either observational or interventional (e.g. RCTs). Therefore, we appreciate the clarification of registries vs. registry-based studies and we particularly support the recommendations for good registry practice given by the EMA especially in Chapter 5 of the discussion paper.

Nevertheless we want to point out that patient registries and the derived observational evidence for the purpose of comparative benefit and harm assessment of health technologies face severe methodological challenges.

A second general comment refers to the objective of usefulness of patient registries for both regulatory and HTA purposes. Although the overlap of expectations and requirements probably will be huge, there will also be differences in need. The EMA discussion paper primarily focuses on a regulatory view which basically is legitimate. But because – to our best knowledge – the majority of the existing patient registries in Europe currently at best partially fulfill the recommended criteria in this paper, and therefore would have to be significantly developed and expanded in the future, we suggest creating a common list of criteria for the sake of a clear orientation for registry holders. This list could be presented in a differentiated manner, indicating which criteria / requirements are (especially) important for regulators, HTA bodies, or both. For example, there are obviously different data needs of regulators and HTA bodies. This aspect will be picked up in the following part with specific comments.

2. Specific comments on text

Page 6, Lines 15-19

Comment:

As HTA is fundamentally based on comparisons to evaluate (additional) benefits and harms of medical technologies for patients, only disease specific registries allow for comparisons of different treatments or therapeutic strategies without using further external data sources. Registries demanded to be set up by regulators (e.g. EMA) in the past were most often (65%) product registries, see:

Bouvy JC, Blake K, Slattery J, De Bruin ML, Arlett P, Kurz X. Registries in European post-marketing surveillance: a retrospective analysis of centrally approved products, 2005-2013. Pharmacoepidemiol Drug Saf. 2017.

Therefore we appreciate the clear preference for disease specific registries, which might help to avoid problems with questionable indirect comparisons and historical control groups.

Page 7 Lines 9-18; See also Page 9 Lines 4-11

Comment:

From a regulatory perspective it is understandable that any MAH initiated, managed or funded active safety data collection for his medicinal product in a disease registry has to follow the regulatory requirements for a PASS. But for the comparative analyses HTA bodies have to conduct, and in order to avoid bias, there is a need to have the same grade of detail, rigor and reliability in the collection of safety /adverse event data for the comparators. This of course also holds true for all other outcome data used in comparative effectiveness analyses.

Proposed change (if any):

Please add a sentence after page 7, line 18:

“From an HTA perspective it is important for comparative analyses that a disease registry collects safety data (and further relevant outcome data) in the same way, quality and detail for a certain medicinal product of interest and the alternative interventions suitable as potential comparators.”

Page 14 Line 19-23 and table 1

In accordance to chapter 6.1, where it is described how randomization can be incorporated in a disease registry setting, we propose to mention the option of a registry-based RCT (“RRCT) as part of the interventional registry studies.

Proposed change (if any):

Please add in line 23: … treatment is given, “or in case of a registry-based-RCT [insert following citations].”

Lauer MS, D'Agostino RB Sr: The randomized registry trial: the next disruptive technology in clinical research? N Engl J Med 2013; 369: 1579-81.

Li G, Sajobi TT, Menon BK, et al.: Registry-based randomized controlled trials: advantages, challenges and areas for future research. J Clin Epidemiol 20.08.2016 [Epub ahead of print]

Please add in Table 1, row “patient enrolment”, column “Registry study”: “In case of a registry-based RCT, allocation to treatment has to be documented in addition.”

Page 16 Line 7

The ethics and/or feasibility of randomization do not depend on the prevalence of a disease. The observation of very large (i.e. “dramatic”) effects in relevant outcomes rather precludes any (further) comparative study, not only RCTs, from an ethical point of view. Similarly, the presence of (very) strong preferences may limit the feasibility of an RCT. However, because observational studies are notoriously challenged by (selection) bias and increased heterogeneity (variance), those studies typically need the inclusion of much more patients and need much more effort to come to more or less conclusive results.

Proposed change (if any):

Please omit the example in parentheses (“(e.g. very rare diseases)”)

Page 16 Lines 23-25

Comment:

We don´t support this recommendation, because electronic healthcare databases (EHDs) and routinely collected data (like claims data) are associated with even more problems of validity, quality and bias than observational data stemming from good quality disease registries. Up to now this approach could not fulfill its huge promises for example for postmarket surveillance of drugs, see:

Moore TJ, Furberg CD. Electronic Health Data for Postmarket Surveillance: A Vision Not Realized. Drug Saf 2015; 38: 601-610

In addition, the cited EMA study of Pacurariu et al. concluded, that only “a few European databases meet minimal regulatory requirements”, and “confirmed the fragmentation, heterogeneity and lack of transparency existing in many European EHDs.”

The authors stated also in their discussion (page 7):

“First, the limited capture of inpatient prescribing poses a problem for regulators and investigators since many newly approved drugs are specialised drugs, used exclusively in secondary care. Second, some disease-specific variables (e. g, biomarkers, laboratory tests and genetic data) are only exceptionally recorded, and they are required more and more often in study protocols.”

A further critical aspect of using routinely collected data for regulatory or HTA purposes has been demonstrated by a meta-epidemiological study which analysed treatment effects for mortality comparing routinely collected data with subsequent randomised trials. The authors found that studies based on routinely collected data “showed significantly more favorable mortality estimates by 31 % than subsequent trials.”

Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ 2016; 352: i493

Proposed change (if any):

We propose to delete this sentence.

Page 16 Line 32

Comment:

As mentioned in our first general comment, we don´t believe that PAES or PASS results based on observational data from disease registries – with the exemption of RRCTs (Registry-based randomised controlled studies) – can prove safety or efficacy of health technologies. Consequently, subgroup analyses based on non-experimental data can only lead to hypotheses on effect modifying factors.

Proposed change (if any):

please reformulate: … (PAES), to generate hypotheses on varying efficacy/effectiveness in patient sub-groups...

Page 16 Line 41 to Page 17 Line 2

We do not support this view from our HTA perspective. Instead of the mentioned uses of clinical practice data from patient registries, we see a big advantage of good quality registries in setting up pragmatic registry-based RCTs which can answer relevant clinical questions, e.g. studying effects of adapted drug dosing schemes in clinical practice which deviate from those used in pivotal trials.

Examples of RRCTs are given here:

Frobert O, Lagerqvist B, Gudnason T, et al.: Thrombus Aspiration in ST-Elevation myocardial infarction in Scandinavia (TASTE trial): a multicenter, prospective, randomized, controlled clinical registry trial based on the Swedish angiography and angioplasty registry (SCAAR) platform; study design and rationale. Am Heart J 2010; 160: 1042-8.

Rao SV, Hess CN, Barham B, et al.: A registry-based randomized trial comparing radial and femoral approaches in women undergoing percutaneous coronary intervention: the SAFE-PCI for Women (Study of Access Site for Enhancement of PCI for Women) trial. JACC Cardiovasc Interv 2014; 7: 857-67.

Proposed change (if any):

We propose to add the following sentence: “In addition, good quality registries allow for the set-up of interventional studies as well, for example pragmatic registry-based RCTs.”

Page 18 - 34

Comment:

In general, we support EMA´s considerations on good registry practice and harmonisation to international standards outlined in chapter 5.

Page 27 Table 4 last column

We conclude from certain comments in the last column, that without additional funding the required data quality in the routine practice of the EBMT can currently not be guaranteed.

Page 31 Lines 22 -29

Comment:

We absolutely agree on the necessity to protect personal data. We also understand the fears of registry coordinators and scientists regarding intellectual property and publication rights. We are optimistic that these problems could be solved in contracts between the involved parties, but never should lead to a lack of transparency or completeness of the clinical data which build the ground for regulatory (or HTA) decisions. In this sense we cannot see what kind of clinical data could be “commercially confidential information” to be withhold from publication by the EMA?

Proposed change (if any):

Please add a sentence for clarification: “However, it must be guaranteed that all clinical data relevant for final regulatory decisions have to be published in the respective EMA documents.”

Page 35 Line 28-36 Line 42 -43

Comment:

We want to make the point that RRCTs (registry-based RCTs) can also play a role for the prospective evaluation of patient risks / safety issues in interventional PASS, not only in the context of PAES.

Proposed change (if any):

Please add a sentence in line 36:

“Randomisation can be incorporated in a disease registry setting, if called for by the study design best to address the research question (e.g. safety).”

Page 39 Line 10

Comment:

Here you mention the “comparison of safety between different treatments” as a possible objective of data analysis. Please see our second specific comment to page 7 and 9!

Page 42 Line 4-25

It is correctly stated that for PASS imposed by regulators legal obligations have to be followed, e.g. the final study report has to follow a specific format. PASS (as well as PAES) can be considered as an essential part of the clinical data required for an authorisation. Therefore, EMA’s publication policy on clinical data derived from clinical trials submitted by the MAH for authorisation should also apply to PASS and PAES, which means that study reports for PASS and PAES should also be published on EMA’s website “Clinical Data” (https://clinicaldata.ema.europa.eu).

Proposed change (if any):

After line 19 on page 42 add as follows:

“Full study reports on PASS and PAES will be published on EMA’s website Clinical Data (https://clinicaldata.ema.europa.eu) six months after submission of these documents to the regulatory authority.”

Page 42 Line 24-25

Comment:

Please see our specific comment to page 31, lines 22-29