Experimentation direction for Deputy Heads
Government of Canada December 01, 2016
This document reinforces the Government’s commitment to devote a fixed percentage of program funds to experimenting with new approaches and measuring impact to instill a culture of measurement, evaluation and innovation in program and policy design and delivery. It provides context and directions for Deputy Heads on how to implement this commitment.
For the purpose of implementing the President of the Treasury Board’s mandate letter commitment to “work with [his] colleagues to ensure that they are devoting a fixed percentage of program funds to experimenting with new approaches to existing problems and measuring the impact of their programs”, experimentation is defined as: Testing new approaches to learn what works and what does not work using a rigorous method that could feature: deliberate, thoughtful, and ethical experimental design; comparisons between interventions and base cases to capture evidence (e.g., randomized controlled trials, A/B testing, counterfactual experiments, baseline performance data, pre- and post-tests); randomized assignment to test and control groups, whenever possible; rigorous impact measurement and causality assessment; and transparent publication of positive, negative and neutral results.