How to evaluate apprenticeships – Apprenticeship Bonus in Germany (statistical approach)

What was the programme and what did it aim to do?

This study evaluates the Apprenticeship Bonus, a subsidy to employers for hiring additional apprentices introduced by the German Federal Parliament in July 2008. Between 2008 and 2010, almost 48,000 apprenticeships were subsidised. The programme targeted young people who had previously failed to find an apprenticeship position, that is, hard-to-place apprenticeship applicants. The policy intended to keep these apprentices from dropping out so they would attain an acknowledged vocational degree. The total amount of the subsidy was between €4,000, and €6,000. This amounts to roughly half the yearly gross labour costs of an apprenticeship.

What’s the evaluation challenge?

Evaluating employer subsidies for hiring ‘hard-to-place’ apprentices is difficult because only specific types of employers make use of such subsidies. In this case, it may be that the better performing firms are more likely to apply for the subsidies available to them when they hire a ‘hard-to-place’ apprentice. Better performing firms may also have lower dropout rates. As a result, if we compare dropout rates of eligible apprentices at firms which receive the subsidy to dropout rates of eligible apprentices at firms which do not, these differences may not reflect the impact of the programme. Instead they may simply reflect the differences between dropout rates for apprentices at high and low performing firms.

What did the evaluation do?

The authors used data on 800 apprentices whose employers received the subsidy, as well as 3,200 apprentices who fulfilled the eligibility criteria but whose employers did not receive the subsidy. Each apprentice whose employer received the subsidy was matched with one or more similar (based on observed characteristics) apprentices at unsubsidised employers. Then, dropout rates were compared between both groups of apprentices to infer the effect of the subsidy. The method used is called propensity score matching.

How good was the evaluation?

According to our scoring guide, propensity score matching receives a maximum of 3 (out of 5) on the Maryland Scientific Methods Scale (SMS). Matching makes subsidised and unsubsidised apprentices observationally comparable, but they might still have differed with respect to unobserved characteristics. The authors observe a wealth of individual characteristics, including some rare indicators of apprentices’ motivation and effort. Thus, the matching arguably should reduce observed differences between subsidised and unsubsidised apprentices to a greater extent than studies that have less data available. We scored the study 3 on the SMS.

What did the evaluation find?

The evaluation found that subsidised apprentices did not have lower dropout rates during the first twelve months of the apprenticeship relative to unsubsidised apprentices. Given the shortcomings of the propensity score matching method, we might expect the estimated effect to be upward biased. Therefore, the estimate of a ‘zero effect’ provides a strong indication that the Apprenticeship Bonus was indeed ineffective. The study found a slightly negative (although insignificant) effect of the Apprenticeship Bonus on dropout up to around three months after the beginning of the apprenticeship, which is when the first subsidy payment was due.

What can we learn from this?

The Apprenticeship Bonus failed to reduce dropout of hard-to-place apprentices. The precise reasons for this could not be identified, but three possibilities suggest themselves: First, employers may have too little influence on apprentices’ dropout decisions. Second, the subsidy might have been taken up largely by firms who would have hired additional apprentices anyway, and therefore were not incentivised by the subsidy to keep these apprentices from dropping out. Third, the employers might have been gaming the system (lowering efforts to keep apprentices after the first half of the subsidy is paid). A broader policy implication is that employer subsidies aimed at integrating hard-to-place youth into apprenticeships should be designed very carefully, thoroughly considering employers’ incentives and their influence on apprentices’ behaviour.

Reference

Fries, J., Göbel, C., and Maier, M. F. (2014). Do employment subsidies reduce early apprenticeship dropout?, Journal of Vocational Education & Training, Vol. 66 (4), pp. 433-461. [Study 979 from our Apprenticeships Review, available here]

Downloads

Related Content

Register for WWG Updates