Assessing Program Impact: Alternative Designs
18 important questions on Assessing Program Impact: Alternative Designs
Nonequivalent comparison design
Intention of evaluator when conducting impact assessment
Bias comes into picture when:
When the level of either of these outcomes is misrepresented in the evaluation data, the estimate of the program effect, in tun, will be smaller or larger than the actual program effect (it will be biased)
- Higher grades + faster learning
- Never study anything twice
- 100% sure, 100% understanding
How can you avoid bias in measuring the observed outcome for targets exposed to an intervention
Before-after measures almost always produce biased estimates of program effects because:
Assumption of program effect regarding outcome measures
Attrition from _____ degrades design.
3 categories of experiences and events that may produce bias in impact assessments
2) interferring events
3) Maturation
Techniques to construct control groups
2) equating groups by statistical procedures
3) reflexive controls
How do you know which characteristics are essential to match?
- identify variables that are potentially related to selection process that divide targets into program participants (if you cannot match it is important to still identify them to incorporate them into data analysis and perhaps statistically adjust for remaining selection bias)
2 types of matching procedures
aggregate matching
There is ____ that statistical control of variables would completely remove selection bias because
influential differences between the interventions and control groups might still remain
2 different types of control variables in nonequivalent comparison group designs
- control variable that relates directly to selection bias, and has to do with selection of individuals into the intervention vs control group. With this control variable you could fully account for the characteristics that caused an individual to be selected for one group or the other, statistically control on these variables and perfectly offset the selection bias.
Objective of multivariate analysis of data from nonequivalent comparison groups
Multivariate analysis tries to
Propensity score analysis
The advantage of making fewer statistical assumptions that the two-stage multiple regression approach.
Disadvantage: the ability to statistically adjust away selection bias is heavily dependent on identification of the variables related to selection into groups and inclusion of them in the statistical model that generates the propensity scores.
Often cutting-point designs have not been applied, why?
- they are not well understood
When are pre-post tests most appropriate?
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding
![Trustpilot Logo](/images/design_2020/lp/clp-trustpilot-en.png)