Study design

The study will utilise intervention data and the eCATALyST study to create a prospective controlled quasi-experimental study—a ‘natural experiment’.20

Setting

The study will be set in Caerphilly county borough, Wales, UK. Caerphilly borough has a population of around 178 000 with a large variation in levels of deprivation.21 Communities First intervention areas account for roughly one-third of the lower super output areas (LSOAs) in Caerphilly and one-quarter of the resident population.

Participants

Data have been collected from 4426 participants aged 18–74 years on 31 May 2001 who responded to the 2001 and 2008 waves of the Caerphilly Health and Social Needs Electronic cohort study.16 The prospective controlled quasi-experimental study involves a comparison of the 1773 (40%) participants living in 47 LSOAs that received Communities First interventions between the two waves of data collection, with 2653 participants living in 63 LSOAs that did not receive any interventions (control areas; see figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

The 47 Communities First intervention lower super output areas (LSOAs) and 63 control LSOAs in Caerphilly County Borough (Wales, UK).

Use of existing record-linked datasets: the Secure Anonymous Information Linkage databank

The Secure Anonymous Information Linkage (SAIL) databank held within the Health Information Research Unit (HIRU) at Swansea University contains health, social and education data on three million residents of Wales, UK, and currently includes 13 data sets containing nearly one billion records.22 ,23 Information governance for SAIL is overseen by an Information Governance Review Panel.22 The smallest geographical area for which data are already linked and may be released from the databank, after scrutiny for small numbers, is the 2001 Census LSOA.

The Welsh Demographic Service (WDS) data set held by NHS Wales Informatics Service (NWIS), the NHS organisation in Wales mandated to hold personally identifiable data, contains addresses for all individuals who register with a general practitioner (GP). Dates for each address record update are held, thereby providing durations of residency for several different homes and the ability to link to local environment exposures at each. This data set will be used to track population migration and record length of exposure in a Communities First area. The WDS contains address information linked anonymously at the individual level (the anonymised linking field, ALF) which is the primary key variable for record-linkage. Using a split-file technique, NWIS supplies ALFs for the whole population of Wales to the SAIL databank.22 ,23

Communities first intervention data

We will extract detailed data on each regeneration activity, including a text description, geographical location, duration and investment (£) using information held by Caerphilly county borough council. We propose to classify the interventions delivered as part of Communities First into seven domains of regeneration, informed by a scheme used to organise projects funded by the New Deal for Communities.4 Examples are provided below.

1. Crime. Reducing crime and the fear of crime through installing CCTV, street lighting and alarms;

2. Education. Providing educational support through after school/breakfast/holiday clubs, early learning and Sure Start;

3. Health. Projects to improve the health of residents directly through provision of sport equipment, staffing of sports clubs, health improvement interventions such as healthy eating projects;

4. Housing and the physical environment. Housing maintenance and repairs, environmental improvement such as redevelopment of waste land, maintenance of green space, parks, building of play grounds; building and maintenance of roads and cycle paths;

5. Employment. Improving employment prospects included providing advice to businesses; projects to develop of computer skills of the unemployed;

6. Community. Increasing social contact and participation including the building of community facilities, staffing of youth projects, funding of luncheon and widows clubs and community events;

7. Infrastructure. Spending on the running of the partnerships. This entailed spending on staff, stationary, and training.

The classification scheme may be amended following extraction of all information on interventions.

The Caerphilly Health and Social Needs Electronic Cohort (eCATALyST)

The Caerphilly Health and Social Needs Electronic Cohort (eCATALyST) is a prospective cohort study residents of Caerphilly county borough, Wales, UK. The study has been described in detail elsewhere.16 Briefly, in 2001 a stratified random sample of 22 236 individuals aged 18 and above resulted in 10 892 respondents providing valid information on mental health. In 2008 the survey was repeated with 9551 participants who still resided in the borough. Of these, 4798 returned questionnaires, with 4426 providing data on their mental health at both waves. The study has detailed information on a wide range of social, demographic and economic risk factors (eg, age, gender, socioeconomic status, educational achievement, employment, household income, council tax band) health and lifestyle outcome data (eg, cardiovascular risk factors, SF-36v2 for the Mental Health Inventory (MHI) scale,24 limiting long-term illness), and perceptions of neighbourhood, including the Buckner Neighbourhood Cohesion scale.25

Data from general practice on consultations and prescribing

The SAIL databank currently contains data on consultations and prescribing data for around 40% of the Welsh population. To date, data from 9 of the 29 GP practices in Caerphilly borough are record-linked in SAIL. This data will be used to compare information on common mental disorders from the eCATALyST data set to that defined using data from general practice. We have already defined a set of Read codes used by general practice to define a case of common mental disorder.26 Although only around one-third of the survey data set respondents will have linked GP data, this provides an opportunity to compare results for common mental disorders reported in the community survey to those that present to primary care.

Primary outcome: mental health

Data on changes in mental health will be assessed using the 2001 and 2008 waves of eCATALyST. Mental health was assessed in 2001 and 2008 using the MHI (MHI-5) included in the SF-36 V.2 scale.27 ,28 The validity and reliability of the MHI-5 are well established28 and the scores reflect the continuously distributed nature of mental health status in the population.27 ,28 Respondents can achieve a total score within a range of 5–25, which can be transformed to a 0–100 scale.29 ,30 The primary outcome measure for the analysis is change in mental health score, wave 2–wave 1, so that positive values indicate an improvement in mental health.

We will repeat our analysis using a set of Read codes used by general practice to represent the common mental disorders. The specific set of codes will build on work conducted by our group using data from general practice to define a common mental disorder.26 They will include codes for symptoms, diagnosis and treatments for psychiatric illnesses such as anxiety disorders and major depression.

Secondary outcome measures

Mental health inequalities

The eCATALyST study includes the following measures of socioeconomic position at both waves16: Registrar General Social Class and the National Statistics Socioeconomic Classification (NS-SEC)32; housing tenure; council tax band of residence; full classification of employment status; and gross household income in two bands (above and below 60% of median income). Using the Welsh Index of Multiple Deprivation (WIMD) 2005 we will categorise each of the LSOAs into quintiles (based on four cut-points and equal counts of LSOAs) for aggregate analyses and use the WIMD score as a LSOA-level covariate in statistical models.

Sample size

In this prospective controlled quasi-experimental study the sample size is fixed and so we can calculate the power available to detect a clinically important difference in our primary outcome measure of mental health, the Mental Health Inventory (MHI-5) scale scores between groups. Of the 4426 eligible survey participants, 1773 reside in 47 intervention LSOAs and 2653 in 63 control LSOAs. Comparing changes in the MHI-5 score between those living in regeneration areas and those living in control areas would have a power of 87% for detecting a difference of 2 in the mean score, allowing for clustering at LSOA-level. This is high power to detect a small, but clinically important, numerical difference in scores.

Statistical analysis plan

The primary analysis will examine the association between changes in MHI-5 scores and residence in a Communities First intervention area or control area. Data on individuals nested within LSOAs will be available. Our analysis plan is:

1. Descriptive statistics for change in mental health, social cohesion and socioeconomic covariates 2001–2008, comparing residents of Communities First and control areas;

2. Multilevel linear models to examine changes in MHI-5 scores (2001–2008) and multilevel logistic models for the odds of a case of common mental disorder between residents of Communities First and control areas, adjusting for compositional characteristics including baseline age, gender and transitions in individual-level covariates recorded in eCATALyST. We will adjust for LSOA deprivation using quintiles of the 2005 Welsh Index of Multiple Deprivation (WIMD).33 We will then include interaction terms to examine whether the effect of Communities First varies according to individual and LSOA-level social cohesion. To investigate the different types of interventions, we will repeat models replacing the binary term for residence in a Communities First or control area with a categorical term for the different types of interventions.

   In order to address the secondary research questions:

3. We will fit further multilevel linear models described above to examine changes in levels of social cohesion;

4. We will assess the effect of population migration by including a term in these models for whether a participant has moved (yes/no) and whether that move was out of, or into, another Communities First LSOA. We will also compare a model, in which respondents are assumed not to have moved, with a model in which the correct migration is coded, following published methods.19

5. We will examine the effect of Communities First on mental health inequalities by modelling interactions between residence in a Communities First area and (1) baseline area deprivation Welsh Index of Multiple Deprivation (WIMD) 2005 scores33 and, (2) population subgroups (eg, gender, employment status);

6. A cost consequences analysis will be conducted and post-trial modelling employed to assess the cost consequences over longer time horizons than is possible within the study period. Changes in resources utilised over time in the Communities First areas relative to the control areas will be calculated and used in conjunction with the costs of setting up and delivering Communities First to generate the net cost of programme delivery per family; this will represent the incremental cost of providing the programme relative to usual service provision. The differences in primary, secondary and tertiary outcomes (including differences in utility scores derived from the SF-36 responses at each follow-up) will be used alongside the net cost of programme delivery to generate a set of indicators of relative cost-effectiveness within the study period, based on incremental cost and incremental outcomes.34 These will be used to provide indicators of the extent to which the programme can be viewed as representing value for money.

We plan to conduct a number of sensitivity analyses to examine: (1) duration of exposure to Communities First by repeating models with length of residence rather than a binary term for exposure; and, (2) contamination using definitions of each type of intervention according to their likelihood for contamination. We will also explore analysis using propensity scores in an attempt to promote balance across intervention and control areas. We will write these models in MLWiN,35 Stata,36 or R.37

Ethics and dissemination

The study has been approved by the Information Governance Review Panel (IGRP) at Swansea University (Ref: 0266 CF) to link the Communities First intervention data to outcome data from the eCATALyST study, general practice records, as well as the WDS within SAIL. The IGRP reviews all applications to the SAIL databank and members include senior representatives from the British Medical Association (BMA), the National Research Ethics Service (NRES), Public Health Wales, NHS Wales Informatics Service (NWIS) and Involving People. NRES does not consider that studies using only anonymised data require its approval. The eCATALyST study received ethical approval for the baseline survey 2001 from the former Gwent Local Research Ethics Committee (REF: JW/CC/00/59(a)) and for the wave 2 survey in 2008, approved by the SE Wales Research Ethics Committee Panel C (ref 08/WSE03/25).

Findings will be disseminated through standard academic pathways including peer-review publications, presentations at national and international conferences, and updates on our study website (http://medicine.cardiff.ac.uk/clinical-study/communities-first-regeneration-programme/). In addition, we will present our findings to policy partners in Caerphilly county borough, Public Health Wales, the Welsh Government, as well as the partnership boards who currently deliver Communities First.