Evidence review
What was the programme and what did it aim to do?
This study examines the impact of the Japanese Emergency Credit Guarantee (ECG) programme set up during the 2008 financial crisis. The ECG was one of the largest credit guarantee programmes across OECD countries, with almost £250bn of loans planned. In order to receive a guaranteed loan, small businesses filed an application with a government-backed credit guarantee corporation. If the firm received approval then banks could lend to the firm without risk (i.e. up to 100% of the loan was guaranteed). The aim of the programme was to increase credit availability and improve the performance of small firms.
What’s the evaluation challenge?
Evaluating the effect of credit guarantees is difficult because such programmes are typically targeted at firms who would otherwise face difficulties getting credit. This means that these firms are likely to have little credit availability (from elsewhere) and may be performing badly on outcomes such as sales, employment and credit rating. As a result of this selection, if we compare differences in outcomes for firms who receive guarantees to those who do not, these differences may not reflect the impact of the programme. Instead, they may simply reflect differences in the types of firms who receive support.
What did the evaluation do?
The study created a control group of firms who did not receive support, but who were similar to ECG firms based on observable characteristics (such as sales, credit score, firm age, and other financial variables) that affect the likelihood of receiving support. The technique the authors use to do this is called ‘propensity score matching’. The study then examined the change in firm outcomes such as credit availability, sales or employment for ECG firms to the change seen for the matched control group who did not receive support. Since this type of comparison involves two changes (or differences) it is known as a ‘difference-in-difference’.
How good was the evaluation?
According to our scoring guide, matching combined with difference-in-differences receives a maximum of 3 (out of 5) on the Maryland Scientific Methods Scale (Maryland SMS). This is because it does well to control for observable differences (e.g. sales) between supported and non-supported firms, but is unable to control for unobservable differences (e.g. if supported firms have a poor business strategy or management style). Since this paper uses a wide range of variables in its matching and since the difference-in-difference is based on a clear treatment date (2008) we score this study 3 on the SMS.
What did the evaluation find?
The effects of the ECG programme depend on whether or not the firm gets the loan from its ‘main bank’. In the Japanese system, a main bank involves a ‘relationship lender’ that monitors the firm’s performance and may intervene to rescue a firm from financial distress. The study finds that if ECG loans are extended by a non-main bank, then credit availability does increase but the firm’s performance remains unchanged compared with the matched control firms who did not receive an ECG. More disappointing still, if the firm goes to a main bank, then any ECG loans are offset by reductions in non-ECG lending by the same bank. Furthermore, the firm performs worse on ex-post outcomes such as sales and employment.
What can we learn from this?
The study looks at the short-term effects of the policy, a year after implementation. However, since policies of this kind are partly designed to have an immediate impact, we can still take some useful lessons from the evaluation. The study shows that if relationship lenders are involved, the policy may be ineffective in increasing credit availability, even in the short term. This suggests relationship lenders use their informational advantage to transfer credit risk of lending to weaker firms off their balance sheets and on to the programme. Furthermore, even where the policy is successful in increasing credit available to small firms, this does not necessarily lead to improvements in firm performance. This could be due to the fact that there is more incentive for firms to pay back non-guaranteed loans. That said, there may be performance benefits which materialise for firms beyond the 12-month period reminding us of the importance of evaluating the long term impact of programmes.
References
Ono, A., Uesugi, I., & Yasuda, Y. (2013). Are lending relationships beneficial or harmful for public credit guarantees? Evidence from Japan’s Emergency Credit Guarantee Program. Journal of Financial Stability, 9(2), 151-167. [Study 565 from our Access to Finance review, available here]
