--- title: 'Sensitivity analysis of the Markov Sick-Sicker model with simulation-time dependency - Excerise' subtitle: "One-way, two-way and probabilistic sensitivity - this work builds upon the first Sick-Sicker Markov exercise" author: "The DARTH workgroup" output: html_document: default pdf_document: default keep_tex: yes self_contained: no --- Developed by the Decision Analysis in R for Technologies in Health (DARTH) workgroup. If you use this code as a basis for your own work, please cite our papers (see suggestions below) and in your acknowledgments for both the code and publications please add: **The authors thank the Decision Analysis in R for Technologies in Health (DARTH) workgroup for providing the coding framework in R.** # Introduction This code forms the basis for the state-transition model of the tutorial: 'A Tutorial on Time-Dependent Cohort State-Transition Models in R using a Cost-Effectiveness Analysis Example' This code implements the simulation-time-dependent Sick-Sicker cSTM model to conduct a CEA of two strategies: - Standard of Care (SoC): best available care for the patients with the disease. This scenario reflects the natural history of the diseaseprogression. - Strategy AB: This strategy combines treatment A and treatment B. The disease progression is reduced, and individuals in the Sick state have an improved quality of life. # Exercise II: Probabilistic sensitivity analysis of the Sick-Sicker Markov model This exercise continues based on the cohort state transition "Sick-Sicker" model with simulation-time dependency from the previous exercise. The code from this first exercise forms the basis and is what is shown in the first 9 Chapters of the template. In this sensitivity analysis exercise, you will code several sensitivity analysis chuncks and end with a probabilistic sensitivity analysis with 1000 simulations `(n_sim)`. The Table below describes the distributions for the variables you used in the previous exercise. **Table: Input parameters for probabilistic analysis** | **Parameter** | **Distribution** | **Distribution values** | |:-----------------------------------|------------------:|----------------------------------------------:| | Annual transition probabilities conditional on surviving | | | | - Rate of becoming S1 when H | Gamma | shape = 30, rate = 170 + 300 | | - Rate of becoming H when S1 | Gamma | shape = 60, rate = 60 + 60 | | - Rate of becoming S2 when S1 | `r_S1S2` | shape = 84, rate = 716 + 84 | | Annual mortality | | | | - Hazard ratio of death in S1 vs H | Lognormal | $\mu = \text{log}(3), \ \sigma = 0.01$ | | - Hazard ratio of death in S2 vs H | Lognormal | $\mu = \text{log}(10), \ \sigma = 0.02$ | | - Hazard ratio of becoming Sicker when Sick under Strategy AB | Lognormal | $\mu = \text{log}(0.6), \ \sigma = 0.02$ | | Annual costs | | | | - Healthy individuals | Gamma | shape = 100, scale = 20 | | - Sick individuals in S1 | Gamma | shape = 177.8, scale = 22.5 | | - Sick individuals in S2 | Gamma | shape = 225, scale = 66.7 | | - Additional costs of sick individuals treated with Strategy AB in S1 or S2 | Gamma | shape = 156.3, scale = 160 | | Utility weights | | | | - Healthy individuals | Beta | $\alpha = 200, \ \beta = 3$ | | - Sick individuals in S1 | Beta | $\alpha = 130, \ \beta = 45$ | | - Sick individuals in S2 | Beta | $\alpha = 230, \ \beta = 230$ | | - Utility for individuals treated with Strategy AB in S1 | Beta | $\alpha = 300, \ \beta = 15$ | | Transition rewards | | | | - Disutility when transitioning from H to S1 | Beta | $\alpha = 11, \ \beta = 1088$ | | - Increase in cost when transitioning from H to S1 | Gamma | shape = 25, scale = 40 | | - increase in cost when dying | Gamma | shape = 100, scale = 20 | ### Exercise - Task 1 Check that you opened the project for this assignment and see in your Files section all corresponding folders. You can now navigate to Chapter "10 Deterministic Sensitivity Analysis (DSA)". Run all code before this section. \newpage ```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE, eval = TRUE) ``` Change `eval` to `TRUE` if you want to knit this document. ```{r} rm(list = ls()) # clear memory (removes all the variables from the workspace) ``` # 01 Load packages ```{r, warning = F, message = F} if (!require('pacman')) install.packages('pacman'); library(pacman) # use this package to conveniently install other packages # load (install if required) packages from CRAN p_load("dplyr", "tidyr", "reshape2", "devtools", "scales", "ellipse", "ggplot2", "ggrepel", "gridExtra", "lazyeval", "igraph", "truncnorm", "ggraph", "reshape2", "patchwork", "knitr", "stringr", "diagram", "dampack") # load (install if required) packages from GitHub # install_github("DARTH-git/darthtools", force = TRUE) #Uncomment if there is a newer version p_load_gh("DARTH-git/darthtools") ``` # 02 Load functions ```{r} # all functions are in the darthtools package ``` # 03 Model input ```{r} ## General setup cycle_length <- 1 # cycle length equal to one year (use 1/12 for monthly) n_age_init <- 25 # age at baseline n_age_max <- 100 # maximum age of follow up n_cycles <- (n_age_max - n_age_init)/cycle_length # time horizon, number of cycles # Age labels v_age_names <- paste(rep(n_age_init:(n_age_max-1), each = 1/cycle_length), 1:(1/cycle_length), sep = ".") # the 4 health states of the model: v_names_states <- c("H", # Healthy (H) "S1", # Sick (S1) "S2", # Sicker (S2) "D") # Dead (D) n_states <- length(v_names_states) # number of health states ### Discounting factors d_c <- 0.03 # annual discount rate for costs d_e <- 0.03 # annual discount rate for QALYs ### Strategies v_names_str <- c("Standard of care", # store the strategy names "Strategy AB") n_str <- length(v_names_str) # number of strategies ## Within-cycle correction (WCC) using Simpson's 1/3 rule v_wcc <- gen_wcc(n_cycles = n_cycles, method = "Simpson1/3") ### Transition rates (annual), and hazard ratios (HRs) r_HS1 <- 0.15 # constant annual rate of becoming Sick when Healthy r_S1H <- 0.5 # constant annual rate of becoming Healthy when Sick r_S1S2 <- 0.105 # constant annual rate of becoming Sicker when Sick hr_S1 <- 3 # hazard ratio of death in Sick vs Healthy hr_S2 <- 10 # hazard ratio of death in Sicker vs Healthy ### Effectiveness of treatment AB hr_S1S2_trtAB <- 0.6 # hazard ratio of becoming Sicker when Sick under treatment AB ## Age-dependent mortality rates lt_usa_2015 <- read.csv("../data/HMD_USA_Mx_2015.csv") # Extract age-specific all-cause mortality for ages in model time horizon v_r_mort_by_age <- lt_usa_2015 %>% dplyr::filter(Age >= n_age_init & Age < n_age_max) %>% dplyr::select(Total) %>% as.matrix() ### State rewards #### Costs c_H <- 2000 # annual cost of being Healthy c_S1 <- 4000 # annual cost of being Sick c_S2 <- 15000 # annual cost of being Sicker c_D <- 0 # annual cost of being dead c_trtAB <- 25000 # annual cost of receiving treatment AB #### Utilities u_H <- 1 # annual utility of being Healthy u_S1 <- 0.75 # annual utility of being Sick u_S2 <- 0.5 # annual utility of being Sicker u_D <- 0 # annual utility of being dead u_trtAB <- 0.95 # annual utility when receiving treatment AB ### Transition rewards du_HS1 <- 0.01 # disutility when transitioning from Healthy to Sick ic_HS1 <- 1000 # increase in cost when transitioning from Healthy to Sick ic_D <- 2000 # increase in cost when dying ### Discount weight for costs and effects v_dwc <- 1 / ((1 + (d_c * cycle_length)) ^ (0:n_cycles)) v_dwe <- 1 / ((1 + (d_e * cycle_length)) ^ (0:n_cycles)) # Process model inputs ## Age-specific transition rates to the Dead state for all cycles v_r_HDage <- rep(v_r_mort_by_age, each = 1/cycle_length) # Name age-specific mortality vector names(v_r_HDage) <- v_age_names # compute mortality rates v_r_S1Dage <- v_r_HDage * hr_S1 # Age-specific mortality rate in the Sick state v_r_S2Dage <- v_r_HDage * hr_S2 # Age-specific mortality rate in the Sicker state # transform rates to probabilities adjusting by cycle length p_HS1 <- rate_to_prob(r = r_HS1, t = cycle_length) # constant annual probability of becoming Sick when Healthy conditional on surviving p_S1H <- rate_to_prob(r = r_S1H, t = cycle_length) # constant annual probability of becoming Healthy when Sick conditional on surviving p_S1S2 <- rate_to_prob(r = r_S1S2, t = cycle_length) # constant annual probability of becoming Sicker when Sick conditional on surviving v_p_HDage <- rate_to_prob(v_r_HDage, t = cycle_length) # Age-specific mortality risk in the Healthy state v_p_S1Dage <- rate_to_prob(v_r_S1Dage, t = cycle_length) # Age-specific mortality risk in the Sick state v_p_S2Dage <- rate_to_prob(v_r_S2Dage, t = cycle_length) # Age-specific mortality risk in the Sicker state ## Annual transition probability of becoming Sicker when Sick for treatment AB # Apply hazard ratio to rate to obtain transition rate of becoming Sicker when Sick for treatment AB r_S1S2_trtAB <- r_S1S2 * hr_S1S2_trtAB # Transform rate to probability to become Sicker when Sick under treatment AB # adjusting by cycle length conditional on surviving p_S1S2_trtAB <- rate_to_prob(r = r_S1S2_trtAB, t = cycle_length) ``` # 04 Construct state-transition models ## 04.1 Initial state vector ```{r} # All starting healthy v_m_init <- c(H = 1, S1 = 0, S2 = 0, D = 0) # initial state vector v_m_init ``` ## 04.2 Initialize cohort traces ```{r} ### Initialize cohort trace under SoC m_M_SoC <- matrix(NA, nrow = (n_cycles + 1), ncol = n_states, dimnames = list(0:n_cycles, v_names_states)) # Store the initial state vector in the first row of the cohort trace m_M_SoC[1, ] <- v_m_init ### Initialize cohort trace for strategy AB # Structure and initial states are the same as for SoC m_M_strAB <- m_M_SoC # Strategy AB ``` ## 04.3 Create transition probability matrices ```{r} ## Create transition probability arrays for strategy SoC ### Initialize transition probability array for strategy SoC # All transitions to a non-death state are assumed to be conditional on survival a_P_SoC <- array(0, dim = c(n_states, n_states, n_cycles), dimnames = list(v_names_states, v_names_states, 0:(n_cycles - 1))) ### Fill in array ## From H a_P_SoC["H", "H", ] <- (1 - v_p_HDage) * (1 - p_HS1) a_P_SoC["H", "S1", ] <- (1 - v_p_HDage) * p_HS1 a_P_SoC["H", "D", ] <- v_p_HDage ## From S1 a_P_SoC["S1", "H", ] <- (1 - v_p_S1Dage) * p_S1H a_P_SoC["S1", "S1", ] <- (1 - v_p_S1Dage) * (1 - (p_S1H + p_S1S2)) a_P_SoC["S1", "S2", ] <- (1 - v_p_S1Dage) * p_S1S2 a_P_SoC["S1", "D", ] <- v_p_S1Dage ## From S2 a_P_SoC["S2", "S2", ] <- 1 - v_p_S2Dage a_P_SoC["S2", "D", ] <- v_p_S2Dage ## From D a_P_SoC["D", "D", ] <- 1 ### Initialize transition probability array for strategy AB a_P_strAB <- a_P_SoC # Update only transition probabilities from S1 involving p_S1S2 a_P_strAB["S1", "S1", ] <- (1 - v_p_S1Dage) * (1 - (p_S1H + p_S1S2_trtAB)) a_P_strAB["S1", "S2", ] <- (1 - v_p_S1Dage) * p_S1S2_trtAB ## Check if transition probability arrays are valid ### Check that transition probabilities are [0, 1] check_transition_probability(a_P_SoC, verbose = TRUE) check_transition_probability(a_P_strAB, verbose = TRUE) ### Check that all rows for each slice of the array sum to 1 check_sum_of_transition_array(a_P_SoC, n_states = n_states, n_cycles = n_cycles, verbose = TRUE) check_sum_of_transition_array(a_P_strAB, n_states = n_states, n_cycles = n_cycles, verbose = TRUE) ``` ## 04.4 Create transition dynamics arrays ```{r} # These arrays will capture transitions from each state to another over time ### Initialize transition dynamics array for strategy SoC a_A_SoC <- array(0, dim = c(n_states, n_states, n_cycles + 1), dimnames = list(v_names_states, v_names_states, 0:n_cycles)) # Set first slice of a_A_SoC with the initial state vector in its diagonal diag(a_A_SoC[, , 1]) <- v_m_init ### Initialize transition-dynamics array for strategy AB # Structure and initial states are the same as for SoC a_A_strAB <- a_A_SoC ``` # 05 Run Markov model ```{r} # Iterative solution of age-dependent cSTM for(t in 1:n_cycles){ ## Fill in cohort trace # For SoC m_M_SoC[t + 1, ] <- m_M_SoC[t, ] %*% a_P_SoC[, , t] # For strategy AB m_M_strAB[t + 1, ] <- m_M_strAB[t, ] %*% a_P_strAB[, , t] ## Fill in transition-dynamics array # For SoC a_A_SoC[, , t + 1] <- diag(m_M_SoC[t, ]) %*% a_P_SoC[, , t] # For strategy AB a_A_strAB[, , t + 1] <- diag(m_M_strAB[t, ]) %*% a_P_strAB[, , t] } ## Store the cohort traces in a list l_m_M <- list(SoC = m_M_SoC, AB = m_M_strAB) names(l_m_M) <- v_names_str ## Store the transition dynamics array for each strategy in a list l_a_A <- list(SoC = a_A_SoC, AB = a_A_strAB) names(l_a_A) <- v_names_str ``` # 06 Plot Outputs ## 06.1 Plot the cohort trace for strategies SoC and AB ```{r} plot_trace(m_M_SoC) plot_trace(m_M_strAB) ``` # 07 State Rewards ```{r} ## Scale by the cycle length # Vector of state utilities under strategy SoC v_u_SoC <- c(H = u_H, S1 = u_S1, S2 = u_S2, D = u_D) * cycle_length # Vector of state costs under strategy SoC v_c_SoC <- c(H = c_H, S1 = c_S1, S2 = c_S2, D = c_D) * cycle_length # Vector of state utilities under strategy AB v_u_strAB <- c(H = u_H, S1 = u_trtAB, S2 = u_S2, D = u_D) * cycle_length # Vector of state costs under strategy AB v_c_strAB <- c(H = c_H, S1 = c_S1 + c_trtAB, S2 = c_S2 + c_trtAB, D = c_D) * cycle_length ## Store state rewards # Store the vectors of state utilities for each strategy in a list l_u <- list(SoC = v_u_SoC, AB = v_u_strAB) # Store the vectors of state cost for each strategy in a list l_c <- list(SoC = v_c_SoC, AB = v_c_strAB) # assign strategy names to matching items in the lists names(l_u) <- names(l_c) <- v_names_str ``` # 08 Compute expected outcomes ```{r} # Create empty vectors to store total utilities and costs v_tot_qaly <- v_tot_cost <- vector(mode = "numeric", length = n_str) names(v_tot_qaly) <- names(v_tot_cost) <- v_names_str ## Loop through each strategy and calculate total utilities and costs for (i in 1:n_str) { # i <- 1 v_u_str <- l_u[[i]] # select the vector of state utilities for the i-th strategy v_c_str <- l_c[[i]] # select the vector of state costs for the i-th strategy a_A_str <- l_a_A[[i]] # select the transition array for the i-th strategy ## Array of state rewards # Create transition matrices of state utilities and state costs for the i-th strategy m_u_str <- matrix(v_u_str, nrow = n_states, ncol = n_states, byrow = T) m_c_str <- matrix(v_c_str, nrow = n_states, ncol = n_states, byrow = T) # Expand the transition matrix of state utilities across cycles to form a transition array of state utilities a_R_u_str <- array(m_u_str, dim = c(n_states, n_states, n_cycles + 1), dimnames = list(v_names_states, v_names_states, 0:n_cycles)) # Expand the transition matrix of state costs across cycles to form a transition array of state costs a_R_c_str <- array(m_c_str, dim = c(n_states, n_states, n_cycles + 1), dimnames = list(v_names_states, v_names_states, 0:n_cycles)) ## Apply transition rewards # Apply disutility due to transition from H to S1 a_R_u_str["H", "S1", ] <- a_R_u_str["H", "S1", ] - du_HS1 # Add transition cost per cycle due to transition from H to S1 a_R_c_str["H", "S1", ] <- a_R_c_str["H", "S1", ] + ic_HS1 # Add transition cost per cycle of dying from all non-dead states a_R_c_str[-n_states, "D", ] <- a_R_c_str[-n_states, "D", ] + ic_D ### Expected QALYs and costs for all transitions per cycle # QALYs = life years x QoL # Note: all parameters are annual in our example. In case your own case example is different make sure you correctly apply. a_Y_c_str <- a_A_str * a_R_c_str a_Y_u_str <- a_A_str * a_R_u_str ### Expected QALYs and costs per cycle ## Vector of QALYs and costs v_qaly_str <- apply(a_Y_u_str, 3, sum) # sum the proportion of the cohort across transitions v_cost_str <- apply(a_Y_c_str, 3, sum) # sum the proportion of the cohort across transitions ## Discounted total expected QALYs and Costs per strategy and apply within-cycle correction if applicable # QALYs v_tot_qaly[i] <- t(v_qaly_str) %*% (v_dwe * v_wcc) # Costs v_tot_cost[i] <- t(v_cost_str) %*% (v_dwc * v_wcc) } ``` # 09 Cost-effectiveness analysis (CEA) ```{r} ## Incremental cost-effectiveness ratios (ICERs) df_cea <- calculate_icers(cost = v_tot_cost, effect = v_tot_qaly, strategies = v_names_str) df_cea ``` ```{r} ## CEA table in proper format table_cea <- format_table_cea(df_cea) table_cea ``` ```{r} ## CEA frontier plot(df_cea, label = "all", txtsize = 16) + expand_limits(x = max(table_cea$QALYs) + 0.1) + theme(legend.position = c(0.82, 0.3)) ``` # 10 Deterministic Sensitivity Analysis (DSA) Load the `calculate_ce_out` . ```{r} ## Load model, CEA and PSA functions source('../functions/Functions_markov_sick-sicker_time.R') ``` ## 10.1 Model input for SA ```{r} ## List of input parameters l_params_all <- list( # Transition probabilities (per cycle), hazard ratios v_r_HDage = v_r_HDage, # constant rate of dying when Healthy (all-cause mortality) r_HS1 = 0.15, # constant annual rate of becoming Sick when Healthy conditional on surviving r_S1H = 0.5, # constant annual rate of becoming Healthy when Sick conditional on surviving r_S1S2 = 0.105, # constant annual rate of becoming Sicker when Sick conditional on surviving hr_S1 = 3, # hazard ratio of death in Sick vs Healthy hr_S2 = 10, # hazard ratio of death in Sicker vs Healthy # Effectiveness of treatment AB hr_S1S2_trtAB = 0.6, # hazard ratio of becoming Sicker when Sick under treatment AB ## State rewards # Costs c_H = 2000, # cost of remaining one cycle in Healthy c_S1 = 4000, # cost of remaining one cycle in Sick c_S2 = 15000, # cost of remaining one cycle in Sicker c_D = 0, # cost of being dead (per cycle) c_trtAB = 25000, # cost of treatment A # Utilities u_H = 1, # utility when Healthy u_S1 = 0.75, # utility when Sick u_S2 = 0.5, # utility when Sicker u_D = 0, # utility when Dead u_trtAB = 0.95, # utility when being treated with A ## Transition rewards du_HS1 = 0.01, # disutility when transitioning from Healthy to Sick ic_HS1 = 1000, # increase in cost when transitioning from Healthy to Sick ic_D = 2000, # increase in cost when dying # Initial and maximum ages n_age_init = 25, n_age_max = 100, # Discount rates d_c = 0.03, # annual discount rate for costs d_e = 0.03, # annual discount rate for QALYs, # Cycle length cycle_length = 1 ) ``` ### Exercise - Task 2 Conduct a one-way sensitivity analysis (OWSA) on parameters `r`_S1S2` [0.05, 0.155], `c_trtAB` [18000, 36000], `u_S1` [0.65, 0.85], `u_trtAB` [0.80, 0.98]. Use net monetary benefit as the outcome. Plot 1) OWSA results, 2) OWSA optimal strategy, and 3) a Tornado plot. The [min, max] contains the range of each parameter for the sensitivity analysis. ## 10.2 One-way sensitivity analysis (OWSA) ```{r} # Your turn ``` ### 10.2.1 Optimal strategy with OWSA ```{r} # Your turn ``` ### Exercise - Task 3 Conduct a two-way sensitivity analysis (TWSA) on parameters `c_trtAB` [18000, 36000] and `u_trtAB` [0.80, 0.98]. Use net monetary benefit as the outcome. Plot TWSA results. ## 10.3 Two-way sensitivity analysis (TWSA) ```{r} # Your turn ``` ### 10.3.1 Plot TWSA ```{r} # Your turn ``` # 11 Probabilistic Sensitivity Analysis (PSA) ## 11.1 Model input ### Exercise - Task 4 Use a function called `generate_psa_params` to sample values for the uncertain parameters using the appropriate distributions. ```{r} # Your turn ``` ### Exercise - Task 5 Create a histogram of each of the model input parameters. ```{r} ``` ### Exercise - Task 6 Run the probabilistic sensitivity analysis ## 11.2 Run PSA ```{r} # Your turn ``` # 11.3 Visualize PSA results for CEA ### Exercise - Task 7 Create a cost-effectiveness plane to present discounted costs and QALYs. ### Create PSA object ```{r} # Your turn ### Create PSA object #l_psa <- make_psa_obj(cost = df_c, # effectiveness = df_e, # parameters = df_psa_input, # strategies = v_names_str) #l_psa$strategies <- v_names_str #colnames(l_psa$effectiveness) <- v_names_str #colnames(l_psa$cost) <- v_names_str ``` ### Exercise - Task 8 Create the cost-effectiveness acceptability curves (CEAC) and frontier (CEAF) for the treatment comparison assuming WTP thresholds of $\$0$ to $\$200,000$. You can use steps of 5000. ## 11.3.1 Cost-Effectiveness Scatter plot ```{r} # Vector with willingness-to-pay (WTP) thresholds. #v_wtp <- seq(0, 200000, by = 5000) # Your turn ``` ## 11.3.2 Incremental cost-effectiveness ratios (ICERs) with probabilistic output ```{r} # Your turn ``` ## 11.3.3 Plot cost-effectiveness frontier with probabilistic output ```{r} # Your turn ``` ## 11.3.4 Cost-effectiveness acceptability curves (CEACs) and frontier (CEAF) ```{r} # Your turn ``` ### Exercise - Task 9 Create the expected loss curves (ELC) plot ## 11.3.5 Expected Loss Curves (ELCs) ```{r} # Your turn ``` ### Exercise - Task 10 Create an expected value of perfect information (EVPI) plot. ## 11.3.6 Expected value of perfect information (EVPI) ```{r} # Your turn ``` We kindly request you to add the following Acknowledgement paragraph to your further work where DARTH code formed the basis. We also like to remind you that you can add other sources of reference to this paragraph to acknowledge code you got from others. # Acknowlegdement For this work we made use of the template developed by the Decision Analysis in R for Technologies in Health (DARTH) workgroup: . The notation of our code is based on the following provided framework and coding convention: Alarid-Escudero, F., Krijkamp, E., Pechlivanoglou, P. et al. A Need for Change! A Coding Framework for Improving Transparency in Decision Modeling. PharmacoEconomics 37, 1329–1339 (2019). . - Alarid-Escudero F, Krijkamp EM, Enns EA, Yang A, Hunink MGM Pechlivanoglou P, Jalal H. An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example. Medical Decision Making, 2023; 43(1). (Epub). - Alarid-Escudero F, Krijkamp EM, Enns EA, Yang A, Hunink MGM Pechlivanoglou P, Jalal H. A Tutorial on Time-Dependent Cohort State-Transition Models in R using a Cost-Effectiveness Analysis Example. Medical Decision Making, 2023; 43(1). Other work from DARTH can be found on the website: # Copyright for assignment work Copyright 2017, THE HOSPITAL FOR SICK CHILDREN AND THE COLLABORATING INSTITUTIONS.All rights reserved in Canada, the United States and worldwide. Copyright, trademarks, trade names and any and all associated intellectual property are exclusively owned by THE HOSPITAL FOR Sick CHILDREN and the collaborating institutions. These materials may be used, reproduced, modified, distributed and adapted with proper attribution.