Conducting a randomised control trial to test the effectiveness of policy can be a daunting prospect. But at Family Mosaic, we found that the process taught us important lessons which will be invaluable as financial pressures increase in the years ahead.
Family Mosaic is a large social housing provider operating in London and the South East, with 24,000 homes housing over 45,000 people. As an organisation, we are committed to improving the health, wealth and wellbeing of our residents. As part of this commitment we wanted to find out what works best to achieve those aims.
A significant future challenge for us is the rapidly ageing demographic of our tenant population, who will need more and more support to remain independent in their own homes. With this in mind, we decided to look at different ways we can support our older tenants. In 2013, in conjunction with Paul Cheshire and Stephen Gibbons at the LSE, and local public health partners in Hackney and Islington, we commenced a research project to trial how best to do this.
The Health Begins at Home research project involved setting-up a randomised control trial (RCT) with over 50s tenants and a final sample size of 532. It tested two types of interventions:
- Low-level ‘signposting support’ from one of our neighbourhood managers (a customer-facing officer who manages tenancy-related issues such as anti-social behaviour within a defined patch of properties).
- Intensive ‘hand-holding support’ from a specialist team of health and wellbeing support workers.
Tenants were randomly assigned to the control group or one of the two treatment groups, and were then tracked over a period of 18 months, receiving assessments at 0, 9 and 18 months. Assessments covered various issues around self-reported health and wellbeing, including NHS usage.
Our project journey has been far from straightforward. However, reflecting on and actively learning from these challenges has been one of the most beneficial and enlightening aspects of this study.
The first and least surprising hurdle was the control group. As a housing and social care provider we come from an environment where we are used to providing support to anyone who needs it. The idea of a control group was a completely foreign concept and went against many of our values. It took time to persuade staff that our methodology was essential to making valid statements around the impact of our services and making a difference in the long-term.
Recruitment to the study was a major stumbling block. We had initially intended to recruit a sample of 600 tenants in three months. In reality it took 12 months to recruit 532 participants. This was still a huge achievement, but a stark reminder of the need to be realistic when designing a project. Recruitment takes time and effort; it requires mass advertising, targeted letters, phone calls and even door-knocking.
Data collection was also no mean feat. Our assessment relied on self-reported measures. During our review of the nine month data, we found a host of different issues with it. There were lots of outliers, as well as inconsistencies between assessments (where for example people would report completely different long-term health conditions across assessments). We had to therefore review our assessment structure and include additional qualitative questions to preserve data quality in our final assessments.
We learnt numerous lessons and gained invaluable insights. The results of the Health Begins at Home study are in the final stages of analysis and the findings so far indicate some small but positive effects on NHS usage where we offered an intensive intervention.
While an RCT can seem a daunting task to undertake, if planned well and reviewed frequently they are invaluable. Our RCT is helping us to make informed decisions about what works and allowing us to prevent resources being wasted. It has also opened doors to conversations with key public health and NHS stakeholders as we’re finally starting to speak their language. In a climate of increasing financial pressures, no organisation can afford to merely go with what looks good or feels right. We need to be producing hard evidence to show exactly what and how we should be doing things. As the Cabinet Office suggests, rather than asking what the cost of doing an RCT would be, we should be asking ourselves what the cost would be of not doing one.