Stellungnahme zur ICH-M14-Leitlinie für Planung, Design und Analyse von pharmakoepidemiologischen Studien, die Real World Data zur Sicherheitsbewertung von Arzneimitteln nutzen
| Line number(s) of the relevant text, Section number | Comment and rationale; proposed changes |
|---|---|
| 28-32, Section 1.2 | Comment: (a) We do not understand why the HARPER guidance is beyond the scope of the guideline. The planning and the design of observational studies that utilize fit-for-purpose data for safety assessment of medicines requires a lot of details. It may be not required to repeat all these details in a guideline on general principles. However, it would be more clear, if the documents, in which these details are described, are clearly cited as relevant, rather than saying that these details are "beyond the scope". (b) Why is the FDA Sentinel Innovation Center a "non-governmental group"? (c) A reference to the PRINCIPLED framework is missing. Proposed change: Replace the sentence "In addition, frameworks for study design and conduct are being developed by non-governmental groups, such as The Sentinel Innovation Center with the PRINCIPLED framework and ISPE/ISPOR’s HARmonized Protocol Template to Enhance Reproducibility (HARPER) Initiative, which provide additional detail that is beyond the scope of this guideline [1, 5]." by a statement like this: "In addition, frameworks for study design and conduct are being developed by the FDA Sentinel Innovation Center with the PRINCIPLED framework [REF] and ISPE/ISPOR’s HARmonized Protocol Template to Enhance Reproducibility (HARPER) Initiative [1, 5]. These documents contain important additional details which should be taken into account in the planning and the design of observational studies that utilize fit-for-purpose data for safety assessment of medicines". New Reference: Desai RJ, Wang SV, Sreedhara SK, Zabotka L, Khosrow-Khavar F, Nelson JC et al. Process guide for inferential studies using healthcare data from routine clinical practice to evaluate causal effects of drugs (PRINCIPLED): Considerations from the FDA Sentinel Innovation Center. BMJ 2024; 384: e076460. |
| 46-48, Section 1.3 | Comment: We support the reference to non-regulatory guidelines. However, some references are incomplete and the list with only 4 references is very short. We propose to complete the references and to update the list. Proposed change: See the recommendations below (regarding lines 1144-1153). |
| 129-131, Section 4.1 | Comment: It is mentioned that prior to a formulation of an adequate reserach question, a literature review should be conducted This is an important issue to avoid research waste. Therefore, a systematic review of the literature should be performed. Proposed change: Add the word "systematic" before "review of the literature" in line 131 to emphasize the importance of the review. |
| 139-142, Section 4.1 | Comment: Both the target trial approach as well as the estimand framework are mentioned as examples for "a principled framework for study design". However, both concepts reflect the state-of-the-art approaches. The estimand framework, presented in the ICH E9 addendum referenced in the glossary, describes the statistical principles for conducting randomized clinical trials. On the other hand, the target trial emulation, proposed by Hernán and Robins, is a well acknowledged approach to translate causal question to non-randmized situations. Proposed change: The importance of both state-of-the-art approaches (estimand and target trial) should be highlighted by summarizing and discussing these in an additional introducing paragraph in Section 4. |
| 159-165, Section 4.2 | Comment: In the listed design elements the important issue of the start of follow-up (time zero) is missing. This should be added. Proposed change: Between lines 160 and 161 add the design element "Start of follow-up (time zero)" as second important design element. |
| 223-224, Section 4.2 | Comment: It is described that regulatory guidances provide additional information on the characteristics of an appropriate comparator. We propose to add guidelines as well as systematic reviews of clinical studies (aggregated evidence) in the present therapeutic indication as information sources. Proposed change: Replace the sentence "Regulatory guidances provide additional information on the characteristics of an appropriate comparator" by a statement like this: "In determining an appropriate comparator therapy regulatory guidances, as well as current guidelines and systematic reviews of clinical studies in the present therapeutic indication should be taken into account." |
| 262-264, Section 5.1 | Comment: In line 264 historical controls are listed as example for comparators. However, in lines 53-54 trials with external comparators are described as out of scope. This should be clarified. Proposed change: Either delete "trials with external comparators" in lines 53-54 or delete "historical controls" in line 264. |
| 296-301, Section 5.2 | Comment: It is mentioned that researchers should consider the steps need to be taken to harmonize data across institutions or data sources. We propose to move up the cross-reference "see Federated Data Networks" from line 299 to line 296 and to change the sentence order. Proposed change: Change the order of the sentences from: "In recent years, federated networks of RWD sources have been developed in various regions. When utilizing multiple data sources, either as a network or through data linkage, researchers should consider the steps taken to harmonize data across institutions or data sources (see Federated Data Networks). Some of these networks have been specifically designed to support scientific evaluations and regulatory decision-making, allowing a growing number of studies to include data from these federated networks, often from different countries." to: "In recent years, federated networks of RWD sources have been developed in various regions (see Federated Data Networks). Some of these networks have been specifically designed to support scientific evaluations and regulatory decision-making, allowing a growing number of studies to include data from these federated networks, often from different countries. When utilizing multiple data sources, either as a network or through data linkage, researchers should consider the steps taken to harmonize data across institutions or data sources [REF]." New Reference: Fortier I, Raina P, Van den Heuvel ER et al. Maelstrom Research guidelines for rigorous retrospective data harmonization. Int J Epidemiol 2017; 46(1): 103-105. (see also the last recommendation [regarding lines 1144-1153] on adding references) |
| 484-490, Section 5.2.3 | Comment: It is unclear why these lines are formatted in italic. Probably, normal formatting should be used. Proposed change: Delete the italic formatting in lines 484-490. |
| 491-503, Section 5.2.4 | Comment: No consequence is described for the case that data are not intended to be collected in the data source and therefore are not available. It should be added that if any of the specified required variables (exposure, comparator, outcomes, covariates) is completely missing, the corrsponding data source is not fit-for-purpose and cannot be used for the desired safety assessment. Proposed change: Add in Section 5.2.4 that the consequence of the second szenario has the consequence that the considered data source is not fit-for-purpose and cannot be used for the desired safety assessment if the completely missing data correspond to one of the specified required variables (exposure, comparator, outcomes, covariates). |
| 541-542, Section 5.3 | Comment: There is an incorrect cross-reference in line 542. Proposed change: Correct the cross-reference in line 542. |
| 658-662, Section 5.4.2 | Comment: Due to the high relevance of patient-important outcomes, we recommend that diagnostic criteria for defining clinical outcomes should include information about whether an outcome was symptomatic or not. In the same sense, it is highly important and common standard to classify safety outcomes according to their seriousness. This should be already mentioned here. Proposed change: Add at the end of the paragraph: "It is essential to define and to describe whether a clinical outcome was symptomatic, serious, or both." |
| 725-296, Section 5.4.3 | Comment: It is described that researchers may consider whether proxies for a missing covariate are appropriate. The simple consideration is insufficient. If a proxy variable should be used for a missing covariate a clear reasoning is required that it is appropriate to use the proxy instead of the missing covariate. The consequence should be added that the considered data source is not fit-for-purpose if a covariate is missing and no appropriate proxy is available. Proposed change: Add in line 278 after "… whether proxies for the covariate are appropriate" a statement like this: "A clear reasoning is required that it is appropriate to use the proxy instead of the missing covariate. Without such a clear reasoning the considered data source is not fit-for-purpose and cannot be used for the desired safety assessment if a covariate is missing and no appropriate proxy is available. |
| 732-734, Section 5.4.3 | Comment: It is described that covariates are typically identified and assessed during the period before the start of the exposure of interest (baseline). Instead, it should be stated that possible covariates must be systematically identified and prespecified. Otherwise, it cannot be assessed whether all relevant covariates are covered by the selected data source. Proposed change: Replace the sentence "Covariates are typically identified and assessed during the period before the start of the exposure of interest (baseline)." by a statement like this: "The relevant covariates must be systematically identified and prespecified in the necessary depth of detail. Otherwise, it cannot be assessed if all relevant covariates are covered by the considered data source. If that is not the case, the data source is not fit-for-purpose and cannot be used for the desired safety assessment." Then continue with: "Covariates are typically assessed during the period before the start of the exposure (baseline) [...]." |
| 795-807, Section 5.5.4 | Comment: It is described that it is typically impossible to capture all potential confounders that are relevant to a research question. Nevertheless, for a valid analysis all relevant confounders are required. It should be added that it is required to define clearly which confounders are indispensable and have to be included in the analysis in order to minimize bias. Again, the consequence should be added that the considered data source is not fit-for-purpose if a relevant covariate is missing. Proposed change: Add in line 796 after "… or residual confounding a statement like this: "Therefore, it is essential that it is clearly defined which confounders are indispensable and have to be included in the analysis in order to minimize bias. If an indispensable covariate is missing the considered data source is not fit-for-purpose and cannot be used for the desired safety assessment." |
| 804-805, Section 5.5.4 | Comment: DAGs are state-of-the-art in planning non-randomized studies and should be used to describe the researchers' causal assumptions (see e.g., Rodrigues et al. Int J Epidemiol, 2022). Proposed change: The setentence should be changed: "Directed acyclic graphs should be used to understand the relations between the … [REF]". New Reference: Rodrigues D, Kreif N, Lawrence-Jones A et al. Reflection on modern methods: constructing directed acyclic graphs (DAGs) with domain experts for health services research. Int J Epidemiol 2022; 51(4): 1339-1348. |
| 1033-1066, Section 11.1 | Comment: It is unclear for us why in a guideline "on general principles" the specific challenges of pregnancy studies are described. It should be considered to delete this section. Proposed change: Please consider to delete Section 11.1 on pregnancy studies. |
| 1144-1153, Section 14 | Comment: The listed non-regulatory guidelines are incomplete. Sometimes only the name of the statement is given (e.g., RECORD statement). The full references should be provided. Proposed change: Provide the complete data for the references. For example, the full reference for the RECORD statement is the following: "Benchimol EI, Smeeth L, Guttmann A et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med 2015; 12(10): e1001885." |
| 1144-1153, Section 14 | Comment: The list of non-regulatory guidelines contains only 4 references. We propose to add further important non-regulatory guidelines such as STROBE and TARGET. Proposed change: Please consider to add the following references: (1) Digitale JC, Martin JN, Glymour MM Tutorial on directed acyclic graphs. J Clin Epidemiol 2022; 142:264-267. (2) Fortier I, Raina P, Van den Heuvel ER et al. Maelstrom Research guidelines for rigorous retrospective data harmonization. Int J Epidemiol 2017; 46(1): 103-105. (3) Hansford HJ, Cashin AG, Jones MD et al. Development of the TrAnsparent ReportinG of observational studies Emulating a Target trial (TARGET) guideline. BMJ Open 2023; 13(9): e074626. (4) Hernán MA, Sauer BC, Hernandez-Diaz S, Platt R, Shrier I Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol 2016; 79:70-75. (5) Vandenbroucke JP, von Elm E, Altman DG et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. Ann Intern Med 2007; 147(8): W163-W194. (6) von Elm E, Altman DG, Egger M et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Ann Intern Med 2007; 147(8): 573-577. (7) Webster-Clark M, Stürmer T, Wang T et al. Using propensity scores to estimate effects of treatment initiation decisions: State of the science. Stat Med 2021; 40(7): 1718-1735. (8) Yao XI, Wang X, Speicher PJ et al. Reporting and guidelines in propensity score analysis: A systematic review of cancer and cancer surgical studies. J Natl Cancer Inst 2017; 109(8): djw323. |