Summary of the different analytical methods
| Method | Core assumptions | Pros | Cons |
| Scenario 1, only data after launch in the intervention area | Only the change in the data after the launch is relevant to the evaluation | Requires little data or technical knowledge | Unable to comment on the change in the outcome of interest because of the intervention, only its trend after launch |
| Scenario 2A, first and last time point of intervention period | The two data points are fully indicative of the change | Requires little data or technical knowledge | Highly dependent on a small array of data. Risks loss of important details of data, intervention effect or trends |
| Scenario 2B, disaggregated change from starting period | Last preintervention period fully represents the counterfactual | Only requires one preintervention data point. Analytically simple | Highly dependent on a small array of control data. No consideration of trend in counterfactual |
| Scenario 3A, simple average of historical intervention area data | Simple averaging of before and after data incorporates all factors, there is no value in an assessment of the trends | Only requires a small amount of pre and post data. Analytically simple | Fails to explore trends in data |
| Scenario 3B, matched preintervention and postintervention | There is a repeating periodic fluctuation, eg, seasonality, which impacts the outcome of interest and the trend over time is informative | Simple means of adjusting for periodic fluctuations | Result varies given matching approach. Blunt means of adjusting for periodic fluctuations that can result in incorrect estimates |
| Scenario 4A, comparison of averages postintervention in control and intervention areas | The selected control area fully represents the counterfactual of the intervention area | Allows for use of control area data. Only requires postlaunch data | Fails to explore trends in data. Makes no use of historical data. Difficult to determine if the control area represents a reasonable comparator |
| Scenario 4B, matched postintervention control and intervention area | The selected control area fully represents the counterfactual of the intervention area and the trend over time is informative | Allows for use of control area data. Explores trends in data without having to define a cycle length. Only requires postlaunch data | Makes no use of historical data. Difficult to determine if the control area represents a reasonable comparator |
| Scenario 5, ITS analysis of intervention area | Regression of preintervention data fully represents post-intervention counterfactual and the trend over time is informative | Allows for use of historical control data. Explores the trends | Reliant on historical intervention area data being predictive of counterfactual |
| Scenario 6, ITS analysis of control and intervention area | Control area fully represents the counterfactual of the intervention area but the match can be tested by exploring the preintervention data. The trend over time is informative | Allows for use of control area and exploration as to the closeness of the control and intervention areas | Assumption that the control area continues to represent a good match after the intervention period |
ITS, interrupted time series.