Evidence review
What was the programme and what did it aim to do?
This study evaluates the New Deal for Communities (NDC) programme which was implemented from 2001 in 39 of the most deprived neighbourhoods in England. In targeted neighbourhoods, which had an average size of 11,000 residents, local NDC partnerships were set up to develop projects to reduce unemployment and crime, and improve health, education and housing. The programme aimed to reduce economic disparities between the poorest neighbourhoods and rest of the country, so that within 10-20 years no one should be disadvantaged because of where they live.
What’s the evaluation challenge?
Evaluating regeneration policies is difficult because they are targeted at specific types of neighbourhood and at people with specific characteristics. In this case selection was done in two steps. First, the Index for Multiple Deprivation (IMD) was used to identify the 39 most deprived local authorities. Second, within each local authority, one district was chosen by the local authority to take part on the NDC programme. As a result of this selection, if we compare differences in outcomes for individuals in NDC areas to those in other areas, these differences may not reflect the impact of the programme. Instead, they may simply reflect differences in the types of individuals who live in these areas.
What did the evaluation do?
To address this issue the study implemented a ‘difference-in-difference’ approach. This involved selecting a control group of neighbourhoods that were similar to targeted areas based on observable characteristics (deprivation and population size) in the pre-NDC period. Using data on unemployment benefit claimants, the study compared the likelihood of transitioning off benefits for individuals in NDC areas, compared to individuals in the control areas. Since the control areas are chosen to be similar to NDC areas, it is more likely that any differences reflect an effect of the policy rather than differences between individuals living in the different areas.
How good was the evaluation?
According to our scoring guide, a difference-in-difference receives a maximum of 3 (out of 5) on the Maryland Scientific Methods Scale (Maryland SMS). This is because it does well to control for observable differences (e.g. deprivation) between supported and non-supported areas but is unable to control for unobservable differences (e.g. if ‘local discretion’ leads to neighbourhoods being chosen that were thought to have the best changes of improving independent of the policy). Implementation of a difference-in-difference requires that the control areas are as similar as possible to the treatment areas to make it highly likely that they would have followed similar trends in the absence of the treatment. This study does this by comparing similar individuals in similarly deprived areas in the regression analysis. For this reason we gave the study 3 on the SMS.
What did the evaluation find?
The study finds that unemployed individuals had a greater chance of transitioning off benefits in NDC areas compared with control areas. The effect was greater for people out of work for health reasons (claiming IB/SDA – Incapacity Benefit or Severe Disablement Allowance) than for those out of work for other reasons (claiming JSA – Job Seekers Allowance). The study also finds effects differ based on gender. For JSA-claimants, there is a positive effect for men but no significant effect for women. In contrast for IB/SDA-claimants, there is a positive and significant effect for both men and women, but the largest effect is for women.
What can we learn from this?
The findings that the NDC initiative had a positive effect on worklessness, suggests they were somewhat successful in reducing the disparities between the most deprived neighbourhoods and the rest of the country (at least, relative to what they would have been in the absence of policy). The policy was also shown to be particularly beneficial for individuals who are out of work for health reasons, with effects that differ by gender. These findings provide useful insights which could potentially guide the implementation of similar policies.
Reference
Romero, R. G., & Noble, M. (2008). Evaluating England’s ‘New Deal for Communities’ programme using the difference-in-difference method. Journal of Economic Geography, lbn022. [Study 824 from our Estate Renewal review, available here]
