Stellungnahme zum „Entwurf eines Gutachtens zu klinisch interpretierbaren Behandlungseffektmaßen auf der Grundlage von wiederkehrenden Ereignis-Endpunkten, die eine effiziente statistische Analyse ermöglichen“ (Englisch)

16. September 2019

Comments from:

IQWiG, Cologne, Germany
Please note that these comments and the identity of the sender will be published unless a specific justified objection is received.

When completed, this form should be sent to the European Medicines Agency electronically, in Word format (not PDF).

1.General comments

The Institute acknowledges the thorough investigations and discussions of the Academic Consortium regarding the value and limitations of different treatment effect measures and the corresponding statistical methods for the analysis of recurrent events. Furthermore, the Institute welcomes the effort of the Scientific Advice Working Party to provide scientific advice on the definition of clinically interpretable treatment effect measures and suitable statistical analysis methods for recurrent event data.

From our point of view, the 2 most important limitations of the draft qualification opinion are given by the following.

Firstly, methods for competing risks and methods for a joint analysis of the terminal event and the recurrent events are not considered in the scenarios with terminal event (Ghosh & Lin, Biometrics 2000; 56: 554-562 / Cook. & Lawless, Stat. Methods Med. Res. 2002; 11: 141-166 / Rogers et al., Stat. Med. 2016; 35: 2195-2205).Lately, efforts are being made in Germany to improve the statistical methodology applied to the analysis of adverse events in clinical trials. One of the points, which should be improved, is the consideration of competing risks and the application of suitable competing risks methods for time-to-first-event endpoints (Unkel et al., Pharm. Stat. 2019; 18: 166-183.). It is inconsistent to apply adequate competing risk methodology for the analysis of time-to-first-event endpoints, but to neglect this important issue in the analysis of the recurrent-event endpoints.

Secondly, it is not mentioned that a thorough analysis of recurrent events should not be provided for one selected endpoint only. It should be added that an analysis of recurrent events should be performed for all relevant endpoints of this type to enable a meaningful and fair decision making

One general formal comment: Some tables and figures copied from the papers of the Academic Consortium have very bad quality. In part, they are unreadable.

2.Specific comments on text

Lines 80-82

Comment:
The presented confidence intervals for the rates per 100 patient years for all-cause mortality, CV death and first HFH are probably based upon the assumption that the corresponding survival times are exponentially distributed. The validity of this assumption is questionable. A better option is given by the citation of the corresponding hazard ratios given in Murray (New Engl. J. Med. 2014; 371: 993-1004).

Proposed change (if any):
Replace the presented rates per 100 patient years for all-cause mortality, CV death and first HFH by the corresponding hazard ratios given in Murray (New Engl. J. Med. 2014; 371: 993-1004).

Lines 258-259

Comment:
The consideration of the frequency of HHF independently of mortality is inadequate and should not be done in practice.

Proposed change (if any):
Please add that the presented consideration is just a theoretical one used for explanation and that in practice the interpretation of HHF results independently of mortality should not be done.

Lines 262-264

Comment:
The presented “HTA-conclusion” obviously only takes costs into account and neglects the overall benefit-risk ratio for the patients. This is not in line with the general HTA view and should be rephrased.

Proposed change (if any):
Replace “HTA-conclusion” by the phrase “… the conclusion if only costs were considered …” or something like this.

Lines 267-268

Comment:
It is correct that the independent interpretation of treatment effects for recurrent events and terminal events leads to obvious problems. Therefore, such an independent interpretation should not be done in practice.

Proposed change (if any):
Please add that treatment effects for recurrent events should not be interpreted independently of terminal events in practice.

Lines 311-313
Table7a

Comment:
In Table 7a Type-1 error rates are presented for 1-sided tests. The corresponding results for the usual 2-sided tests originally presented by the Academic Consortium are preferable.

Proposed change (if any):
Replace the results for the 1-sided tests by the corresponding results for the usual 2-sided tests.

Lines 353-354
and 453-460

Comment:
It is correct that terminal events complicate the estimation of the reduction in recurrent events. We support the CHMP encouraging research for this data situation. However, methods for competing risks and methods for a joint analysis of the terminal event and the recurrent events are not considered. It is sensible to apply and extend methods for competing risks not only for time-to-first-event endpoints but also for recurrent-event endpoints considered here (Ghosh & Lin, Biometrics 2000; 56: 554-562 / Cook. & Lawless, Stat. Methods Med. Res. 2002; 11: 141-166). Another option is given by the joint analysis of the recurrent events and the terminal event to avoid a misleading interpretation of the results for recurrent events independently of the terminal event (Rogers et al., Stat. Med. 2016; 35: 2195-2205).

Proposed change (if any):
Please add that the application and extension of competing risk methodology for recurrent-event endpoints is useful and add a discussion of available methods for the the joint analysis of the recurrent events and the terminal event.

Lines 434-460

Comment:
We agree that the analysis of recurrent events is useful when the corresponding effect measures provide a better description of the patients' disease burden than the analysis of the first event only. However, for a meaningful decision making such analyses should not only be performed for one selected endpoint but for all relevant endpoints of this type.

Proposed change (if any):
Please add that an analysis of recurrent events should not only be performed for one selected endpoint but for all relevant recurrent-event endpoints to enable a meaningful and fair decision making.