Stellungnahme des IQWiG zum "Leitfaden zur Bewertung von Onkologika in Menschen – Entwurf" (EMA/CHMP/205/95 Rev.5) (Englisch)

26. August 2016

Comments from:

Institute for Quality and Efficiency in Health Care (IQWiG)
Im Mediapark
50670 Cologne
Germany

Please note that these comments and the identity of the sender will be published unless a specificjustified objection is received. When completed, this form should be sent to the European Medicines Agency electronically, in Word format (not PDF).

1. General comments

IQWiG appreciates the opportunity to comment on the draft guideline.

IQWiG strongly supports the revision of the “Guideline on the evaluation of anticancer medicinal products in man” with respect with the aim of improving the reporting of adverse events (AEs). The guideline could be further improved by adding clear-cut recommendations for valid statistical analyses of adverse events based upon adequate survival time methods to perform an appropriate benefit-risk assessment.

2. Specific comments on text

Line 95 (and all other mentions of “cross-over”)

Comment: The term “cross-over” might be misleading because it could be confused with cross-over studies. A better term than “cross-over” would be “treatment switching”.

Proposed change (if any): Replace “cross-over” by “treatment switching” or add an explanation so that confusion with cross-over studies is avoided.

Lines 121 - 132

Comment: It seems that the description of the changes is incomplete. Explanatory notes regarding the planned Appendix 3 are missing as well as comments regarding the main aim of the revision. (The aim of this revision was “to find ways on how to report AEs in order to improve the understanding of the toxicity and tolerability profiles of medicinal products.”

Proposed change (if any): Add explanatory notes regarding the planned Appendix 3 as well as comments regarding the main aim of the revision.

Lines 333 - 335

Comment: IQWiG supports inclusion of randomised control arms also in Phase II studies. The control arm should provide an appropriate standard treatment.

Proposed change (if any): Inclusion of a randomised control arm providing standard treatment is encouraged.

Lines 380 - 381

Comment: IQWiG supports the assessment of symptom control in patients with symptomatic disease at baseline. For a study to generate meaningful results concerning symptom control, a control arm and blinding of the trial wherever possible is required.

Proposed change (if any): In patients with symptomatic disease at base line, the assessment of symptom control is encouraged. This requires a randomised phase II trial, which should be blinded if possible.

Lines 786 - 790

Comment: IQWiG supports the requirement of the control group providing a best available, evidence-based therapeutic option. Whether a treatment option is evidence-based cannot be clarified by any non-systematic analysis of available studies but needs a systematic review of the available studies. Therefore, the justification for the choice of the control group should be based on a systematic review of the available evidence.

Proposed change (if any): The choice of reference regimen should be justified and normally this regimen should be selected from best available, evidence-based therapeutic options. In this context, “best available, evidence-based” should be read as a widely used, but not necessarily licensed regimen with a favourable benefit-risk convincingly documented in a systematic review of randomised trials and considered at least as good from a benefit/risk perspective as alternative, treatment options.

Lines 808 - 813

Comment: IQWiG agrees that in some cases investigator’s best choice can be the appropriate control group. However, for studies against investigator’s best choice to provide interpretable and meaningful results certain requirements concerning treatment allocation have to be fulfilled. If possible, investigator’s best choice should be allocated according to defined criteria (e.g. based on performance status or pre-treatment), reasons for choosing a certain treatment could be documented and treatment allocation should be determined before randomisation, so that stratified randomisation can be used when possible.

Proposed change (if any): Clarify the design characteristics of studies using investigator’s best choice as a control group.

Lines 814 - 818

Comment: It is unclear, why in last line therapies, studies in less advanced patients supported by salvage single arm studies would be more informative than studies versus BSC/investigator’s best choice. Clinical practise would best be informed by studies investigating the treatment options available in this situation.

Proposed change (if any):

Lines 834 - 835

Comment: Neither the guideline nor the referenced Appendix 1 contain criteria, which have to be fulfilled for the exclusion of detrimental effects on OS. A suitable criterion would be a statistically significant result of an appropriate non-inferiority test regarding OS.

Proposed change (if any): Add criteria for the exclusion of detrimental effects on OS, e.g., a statistically significant result of an appropriate non-inferiority test regarding OS.

Lines 857 - 858

Comment: It is unclear what is meant by “showing trends towards superiority”. If just a non-significant increase in OS is meant, this is insufficient to exclude relevant negative effects.

Proposed change (if any): Add appropriate criteria (based upon non-inferiority testing) for the conclusion that there are no relevant negative effects on OS.

Lines 922 - 924

Comment: The section on secondary endpoints requests surrogate outcomes like ORR and rate of tumour stabilisation, but considers patient-relevant endpoints like HRQoL and PRO (e.g. symptoms) only possibly informative in palliative settings. HRQoL and symptoms should generally be considered as endpoints in cancer trials.

Proposed change (if any): Include the requirement of including HRQoL and symptom-related endpoints as secondary endpoints.

Line 985

Comment: In case of increased toxicity, superiority at least in terms of PFS is required. There is no specification as to which effect size would be required. This seems insufficient, especially, if PFS is asymptomatic and only based on radiographic data.

Proposed change (if any): Please specify appropriate effect sizes for PFS effects (e.g. surrogate threshold effects to ensure an OS effect) required in cases of increased toxicity.

Line 1125

Comment: IQWiG agrees that interim analyses based on PFS data other than for futility should not be used. However, there are many pivotal cancer trials in which interim analyses based on PFS are used to decide to stop a study for proof of efficacy. In most cases, as a consequence treatment switching is allowed meaning that no relevant OS data (as well as adverse event data, morbidity data and HRQoL data) can be collected anymore.

Proposed change (if any): Strengthen the recommendation not to perform interim analyses based on PFS.

Lines 1306 - 1315

Comment: It is correctly stated that bias is introduced if the collection of AEs is stopped at the time of study drug discontinuation or shortly thereafter. As the stopping of the documentation of adverse events when the study medication is discontinued is current practice, it should clearly be stated that this practice should be changed.

Proposed change (if any): Add a clear statement that the documentation of adverse events should not be stopped when the study medication is discontinued to enable a fair comparison of treatment strategies.

Line 1316

Comment: Extended safety data collection is described as optional, however, safety data collection until the end of the study should be mandatory.

Proposed change (if any): Extended safety data collection until the end of the study, including off-therapy and on-new therapy, is required, even if not chosen as the primary analysis cut-off for safety outcomes.

Line 1327

Comment: Comparative studies for marketing authorisation should not only be recommended; comparative studies are mandatory.

Proposed change (if any): Replace “recommended” by “mandatory” in the sentence “Therefore, whenever possible, comparative studies are recommended for marketing authorisation.”

Lines 1365 - 1367

Comment: It is mentioned that time to event should play a role for key events. It should be more clearly stated that for time-to-event data the application of appropriate survival time methods is required, which means that it may be necessary to deal adequately with competing risks and recurrent events.

Proposed change (if any): Add a clear statement that for time-to-event data the application of appropriate survival time methods is required, which means that it may be necessary to deal adequately with competing risks and recurrent events.

Lines 1369 - 1370

Comment: It is stated that “Time-adjusted analyses for AEs, e.g. incidence by different cut-off dates or event rates per 100 patient-years, may also be indicated if properly justified by the pattern of events.” However, the justification to use event rates per 100 patient-years is given by the strong assumption that the considered endpoint follows an exponential distribution, which is frequently questionable. For descriptive purposes event rates per 100 patient-years may be useful, but the corresponding statistical inference (significance tests, confidence intervals) requires the exponential distribution, which is frequently not the case.

Proposed change (if any): Add a more clear-cut guidance on the use of event rates per 100 patient-years including a statement that the justification for statistical inference for event rates per 100 patient-years is given by the exponential distribution, which, however, is rarely valid in practice.

Lines 1455 - 1456

Comment: It is stated that “Modelling and simulations may provide complementary information where data in (parts of) the paediatric population are difficult to obtain.” It should be clearly stated that modelling and simulations are no substitute for empirical comparative data.

Proposed change (if any): Add a clear statement that adequate empirical comparative data are required to demonstrate the safety of anticancer medicinal products.