Skip to main content
KSR Number: KSRA87245

Comparative safety of immune checkpoint inhibitors in cancer: systematic review and network meta-analysis

Risk of Bias Assessment

Overall summary: High risk of bias in the review

Bottom Line

Current evidence suggests that the immune checkpoint inhibitors, nivolumab, pembrolizumab and atezolizumab, may have the best safety profile in terms of treatment-related adverse events compared to conventional therapy. Overall, atezolizumab may provide the best safety profile in general, whilst nivolumab may provide the best safety profile for patients with lung cancer. However, these conclusions must be interpreted with caution, since several methodological issues mean relevant studies may have been missed, reviewer error and bias cannot be ruled out, and substantial heterogeneity was evident in pooled anayses. Future real-world studies are needed to validate these findings in wider cancer patient populations.

Risk of Bias Assessment

Overall summary High risk of bias in the review

High

Restrictions were reported based on publication format, language and publication date, and the search strategy appeared to be unnecessarily and overly restrictive, meaning relevant studies may have been missed. It was not clear how many authors were involved in title/abstract screening, meaning reviewer error and bias cannot be ruled out. The authors acknowledged the presence of substantial heterogeneity among the included studies. The results of sensitivity analyses were not clearly reported.

A. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4? Probably no
B. Was the relevance of identified studies to the review's research question appropriately considered? Probably yes
C. Did the reviewers avoid emphasizing results on the basis of their statistical significance? Probably yes
Risk of bias in the review High

Details of Review

Number of studies 36
Number of participants 15,370
Last search date February 2018
Review type Intervention
Objective To provide a complete toxicity profile, toxicity spectrum and a safety ranking of immune checkpoint inhibitor drugs for the treatment of cancer.
Population Patients with any type of cancer.

Trials providing a summary but no site/organ/system-level toxicity data were excluded.
Interventions Immune checkpoint inhibitors (ICI) (as a monotherapy, as a dual therapy (i.e. two ICIs combined) or in combination with conventional therapy.

Studies assessing the medication order of immune checkpoint inhibitors or other immune therapies (e.g. immune vaccines or granulocyte-macrophage-colony stimulating factor) were excluded.
Comparator One or more alternative treatments (for example, a different ICI, best supportive care or chemotherapy)
Outcome Primary outcome: treatment-related adverse events (as a surrogate for immune-related adverse events).

Studies focusing on quality of life analysis, subgroup analysis, investigation after progression or predictive factors were excluded.
Study design Head-to-head phase II and III randomised controlled trials.

High quality placebo-controlled trials and high quality single arm trials were included as a validation group for additional meta-analysis.

Lower quality placebo-controlled trials or single arm trials, phase I randomised controlled trials and studies using a non-comparative design were excluded.

Results

COMPARISONS WITH CONVENTIONAL THERAPY

In terms of grade 1-5 treatment-related adverse events (TRAEs), pooled analysis reported a reduction in the odds of grade 1-5 TRAEs for nivolumab (odds ratio [OR] 0.41, 95% confidence interval [CI] 0.27 to 0.63; 7 studies, n=3276 patients), pembrolizumab (OR 0.43, 95% CI 0.34 to 0.54; 4 studies, n=2344 patients) or atezolizumab (OR 0.29, 95% CI 0.23 to 0.35; 3 studies, n=2366 patients) compared to conventional therapy. Conversely, a single study reported that the odds of grade 1-5 TRAEs were increased for tremilimumab compared to conventional therapy (OR 2.22, 95% CI 1.12 to 4.38; 1 study, n=644 patients). Similarly, a single study reported a numerical increase in the odds of grade 1-5 TRAEs for ipilimumab compared to conventional therapy (OR 2.05, 95% CI 0.90 to 4.67; 1 study, n=102 patients) ; and pooled analysis reported a numerical increase in the odds of grade 1-5 TRAEs for one immune checkpoint inhibitor (ICI) drug plus conventional therapy compared to conventional therapy alone (OR 1.76, 95% CI 0.95 to 3.24; 6 studies, n=2653 patients).

In terms of grade 3 or 4 TRAEs, pooled analysis reported a reduction in the odds of grade 3-4 TRAEs for nivolumab (odds ratio [OR] 0.23, 95% confidence interval [CI] 0.14 to 0.38; 7 studies, n=3276 patients), pembrolizumab (OR 0.40, 95% CI 0.28 to 0.58; 4 studies, n=2344 patients) or atezolizumab (OR 0.27, 95% CI 0.22 to 0.32; 3 studies, n=2366 patients) compared to conventional therapy. Conversely, a single study reported that the odds of grade 3-4 TRAEs were increased for tremilimumab compared to conventional therapy (OR 1.84, 95% CI 1.35 to 2.52; 1 study, n=644 patients); and pooled analysis reported an increase in the odds of grade 3-4 TRAEs for one immune checkpoint inhibitor (ICI) drug plus conventional therapy compared to conventional therapy alone (OR 2.21, 95% CI 1.19 to 4.09; 6 studies, n=2653 patients). A single study reported a numerical increase in the odds ot grade 3-4 TRAEs with ipilimumab compared to conventional therapy (OR 3.03, 95% CI 0.91 to 10.04; 1 study, n=102 patients).

COMPARISONS WITH IPILIMUMAB

In terms of grade 1-5 TRAEs, pooled analysis reported a numerical reduction in the odds of grade 1-5 TRAEs for nivolumab compared to ipilimumb (odds ratio [OR] 0.51, 95% CI 0.13 to 1,98; 2 studies, n=1529 patients). Pooled analysis reported a numerical increase in the odds of grade 1-5 TRAEs for two ICI drugs compared to ipilimumab (OR 1.88, 95% CI 0.40 to 8.83; 2 studies, n=764 patients). A single study reported a numerical increase in the odds of grade 1-5 TRAEs for pembrolizumab (OR 1.18, 95% CI 0.84 to 1.66; 1 study, n=811 patients) or an ICI drug in combination with conventional therapy (OR 2.67, 95% CI 0.75 to 9.47; 1 study, n=74 patients) compared to ipilimumab.

In terms of grade 3-4 TRAEs, a single study reported a reduction in the odds of grade 3-4 TRAEs for pembrolizumab compared to ipilimumab (OR 0.53, 95% CI 0.36 to 0.80; 1 study, n=811 patients). Pooled analysis reported a numerical reduction in the odds of grade 3-4 TRAEs for nivolumab compared to ipilimumab (OR 0.37, 95% CI 0.11 to 1.31; 2 studies, n=1529 patients). Conversely, pooled analysis reported an increase in the odds of grade 3-4 TRAEs for two ICI drugs compared to ipilimumab (OR 3.74, 95% CI 2.75 to 5.09; 2 studies, n=764 patients); and a single study reported a numerical increase in the odds of grade 3-4 TRAEs for one ICI drug in combination with conventional therapy compared to ipilimumab (OR 2.01, 95% CI 0.59 to 6.87; 1 study, n=74 patients).

COMPARISONS WITH NIVOLUMAB

In terms of grade 1-5 TRAEs, a single study reported an increase in the odds of grade 1-5 TRAEs for two ICI drugs compared to nivolumab (OR 3.68, 95% CI 1.93 to 6.98; 1 study, n=626 patients).

In terms of grade 3-4 TRAEs, a single study reported an increase in the odds of grade 3-4 TRAEs for two ICI drugs compared to nivolumab (OR 5.24, 95% CI 3.68 to 7.44; 1 study, n=626 patients).

POOLED RCT AND VALIDATION GROUP DATA

Pooled analysis reported that the incidence of all adverse events in immune checkpoint inhibitor (ICI) drugs for atezolizumab, nivolumab, pembrolizumab and ipilimumab was 66.4%, 71.8%, 75.1% and 86.8%, respectively. Similarly, the pooled incidence of severe or life-threatening adverse events for atezolizumab, nivolumab, pembrolizumab and ipilimumab was reported to be 15.1%, 14.1%, 19.8% and 28.6%, respectively. Compared with conventional therapy, treatment-related adverse events for ICI drugs occurred mainly in the skin, endocrine, hepatic and pulmonary systems. Among the five ICI drugs, atezolizumab had the highest risk of hypothyroidism, nausea and vomiting. Tremelimumab had the highest risk of causing rash, diarrhoea and fatigue. The predominant treatment-related adverse events for pembrolizumab were arthralgia, pneumonitis and hepatic toxicities. The main treatment-related adverse events for ipilimumab were skin, gastrointestinal and renal toxicities. Nivolumab had a narrow and mild toxicity spectrum, mainly causing endocrine toxicities.

Overall, atezolizumab was ranked as the best treatment option in terms of treatment-related adverse events among ICI drugs for patients with cancer, followed by nivolumab, pembrolizumab and ipilimumab. Moreover, atezolizumab was ranked as the best safety profile in general, and nivolumab was ranked as the best safety profile in lung cancer.

Full Risk of Bias Assessment

The research objective was clearly stated. Eligibility criteria were well described and appeared appropriate. Restrictions were reported based on publication format (conference abstracts, posters and presentations of ongoing studies were excluded), language (only English language studies were included) and publication date (only studies published from January 2007 onwards were included). In the validation group analysis, restrictions were also applied based on sample size (only studies with a sample size of >100 patients were included).

1.1 Did the review adhere to pre-defined objectives and eligibility criteria? Probably yes
1.2 Were the eligibility criteria appropriate for the review question? Probably yes
1.3 Were eligibility criteria unambiguous? Probably yes
1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate (e.g. date, sample size, study quality, outcomes measured)? Probably no
1.5 Were any restrictions in eligibility criteria based on sources of information appropriate (e.g. publication status or format, language, availability of data)? Probably no
Concerns regarding specification of study eligibility criteria High

PubMed, EMBASE, Cochrane Library and Web of Science were searched for relevant studies. The reference lists of retrieved articles and ClinicalTrials.gov were searched for additional relevant studies. The search strategy was reported in full and appeared to be unnecessarily and overly restrictive. Searches were restricted by date (January 2007 to February 2018) and language (only English language studies were included); however, these issues were already downgraded for in ROBIS 1 and therefore were not downgraded again in ROBIS 2. It was not clear how many authors were involved in title/abstract screening, but two reviewers were independently involved in full text screeningdisagreements were resolved by discussion to achieve a consensus.

2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports? Probably yes
2.2 Were methods additional to database searching used to identify relevant reports? Probably yes
2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible? No
2.4 Were restrictions based on date, publication format, or language appropriate? Probably no
2.5 Were efforts made to minimise error in selection of studies? No information
Concerns regarding methods used to identify and/or select studies High

Two reviewers were independently involved in the data extraction process, and all disagreements were resolved by discussion to achieve a consensus. Sufficient study characteristics appear to have been extracted to allow for the interpretation of the results. Relevant study results appeared to have been extracted. The quality of the included studies was assessed using the Cochrane tool for randomised controlled trials together with a modified Jadad score. Two reviewers were independently involved in quality assessment, and disagreements were resolved by discussion to achieve a consensus.

3.1 Were efforts made to minimise error in data collection? Probably yes
3.2 Were sufficient study characteristics considered for both review authors and readers to be able to interpret the results? Probably yes
3.3 Were all relevant study results collected for use in the synthesis? Probably yes
3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria? Probably yes
3.5 Were efforts made to minimise error in risk of bias assessment? Probably yes
Concerns regarding methods used to collect data and appraise studies Low

The synthesis appeared to include all relevant studies. The method of analysis was explained and appeared appropriate. There was evidence of moderate or high statistical heterogeneity for several outcomes, which remained following subgroup analyses. Sensitivity analysis was performed to test the robustness of the findings; however, the results were not clearly reported. The quality of the individual studies was considered in the synthesis.

4.1 Did the synthesis include all studies that it should? Probably yes
4.2 Were all pre-defined analyses reported or departures explained? Probably yes
4.3 Was the synthesis appropriate given the degree of similarity in the research questions, study designs and outcomes across included studies? Probably yes
4.4 Was between-study variation minimal or addressed in the synthesis? Probably no
4.5 Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses? Probably yes
4.6 Were biases in primary studies minimal or addressed in the synthesis? Probably no
Concerns regarding synthesis and findings High

Keywords