Portrait of rural emergency departments in Quebec and utilisation of the Quebec Emergency Department Management Guide: a study protocol
=======================================================================================================================================

* Richard Fleet
* Patrick Archambault
* France Légaré
* Jean-Marc Chauny
* Jean-Frédéric Lévesque
* Mathieu Ouimet
* Gilles Dupuis
* Jeannie Haggerty
* Julien Poitras
* Alain Tanguay
* Geneviève Simard-Racine
* Josée Gauthier

## Abstract

**Introduction** Emergency departments are important safety nets for people who live in rural areas. Moreover, a serious problem in access to healthcare services has emerged in these regions. The challenges of providing access to quality rural emergency care include recruitment and retention issues, lack of advanced imagery technology, lack of specialist support and the heavy reliance on ambulance transport over great distances. The Quebec Ministry of Health and Social Services published a new version of the *Emergency Department Management Guide*, a document designed to improve the emergency department management and to humanise emergency department care and services. In particular, the *Guide* recommends solutions to problems that plague rural emergency departments. Unfortunately, no studies have evaluated the implementation of the proposed recommendations.

**Methods and analysis** To develop a comprehensive portrait of all rural emergency departments in Quebec, data will be gathered from databases at the Quebec Ministry of Health and Social Services, the Quebec Trauma Registry and from emergency departments and ambulance services managers. Statistics Canada data will be used to describe populations and rural regions. To evaluate the use of the 2006 *Emergency Department Management Guide* and the implementation of its various recommendations, an online survey and a phone interview will be administered to emergency department managers. Two online surveys will evaluate quality of work life among physicians and nurses working at rural emergency departments. Quality-of-care indicators will be collected from databases and patient medical files. Data will be analysed using statistical (descriptive and inferential) procedures.

**Ethics and dissemination** This protocol has been approved by the CSSS Alphonse–Desjardins research ethics committee (Project MP-HDL-1213-011). The results will be published in peer-reviewed scientific journals and presented at one or more scientific conferences.

*   Accident & Emergency medicine
*   Organisation of health services < HEALTH SERVICES ADMINISTRATION & MANAGEMENT
*   Quality in health care < HEALTH SERVICES ADMINISTRATION & MANAGEMENT

### Article summary

#### Article focus

*   This research protocol aims to develop a comprehensive portrait of all rural emergency departments in the province of Québec, Canada.

*   It will also allow the policy-makers to evaluate the utilisation and usefulness of the Emergency Department Management Guide, a unique tool to help standardise emergency care.

#### Key messages

*   As the first study of its kind in Canada, our results will undoubtedly be useful to policy-makers and can be used to guide the attribution of resources as well as distribution of healthcare services in rural areas.

*   The results will provide policy-makers with a greater understanding and appreciation of the unique challenges faced by rural emergency departments.

*   This project will likely contribute to improved health in rural Quebec.

#### Strengths and limitations of this study

*   This is the first study examining access to emergency services in rural Canada, in particular the province of Quebec.

*   Methodological limitation: we expect that current clinical databases may not capture all quality-of-care indicators.

## Introduction

The practice of emergency medicine in rural areas in Canada represents a significant challenge, and there is a lack of knowledge to properly understand this issue.1 ,2 The majority of research on emergency medicine is conducted in tertiary academic centres with patients from urban areas. It is important to study the particular difficulties encountered by rural emergency departments (ED), as these EDs constitute a safety net of sorts for the 20% of Canadians who live in rural areas.3

Healthcare professionals and patients in rural areas in Canada face numerous problems—reduced access to specialised care,4–8 medical imaging (tomodensitometry (TDM), ultrasound, MRI)9 ,10 and intensive care units; geographical distance from specialised centres and deficient means of transportation.11 Further challenges for rural emergency care include problems with personnel recruitment, level of training in emergency medicine and infrequent experiences with complex cases.12 Further, several provinces have centralised their medical care to reduce costs,7 ,13–15 resulting in limited access to local services and specialised care, and increased pressure on prehospital emergency care (PEC).13 ,14 ,16 Limited access in rural regions to primary care,4 ,17 mental healthcare and to long-term care and services centres (LTC) may increase the number of visits to EDs.18 Finally, risk of death subsequent to major trauma is considerably higher in rural regions and a correlation between geographical isolation and mortality3 ,10 ,19–23 has been observed. These problems impact a significant proportion of Canada's population. Given that the geographic factor is not changeable, it is imperative to identify factors that could potentially be modified to help resolve these problems.

## Emergency Department Management Guide

In 1997, the Canadian Association of Emergency Physicians (CAEP) presented its position on rural emergency medicine in Canada.12 Owing to the lack of research data on the issue at the time, the document was based on expert consensus. The position paper was created to inform the development of a framework to evaluate medical practices in rural EDs. The CAEP document did not, however, include specific recommendations about appropriate patient transportation times or access to TDM or other specialised services.

However, the publication of the *Emergency Department Management Guide*24 in 2006 by the Quebec Ministry of Health and Social Services (MSSS) raised a number of critical issues related to rural EDs. Developed by a multidisciplinary group of key policy-makers, the *Management Guide*24 is, to our knowledge, the most recent and concise available document. It specifies the services that should be accessible in the province's EDs based on the number of annual visits to the department and other variables. The guide24 also includes several recommendations for solving the problems faced by rural EDs. In brief, this forward-thinking guide24 could provide a starting point for the development of management standards in Canadian rural EDs. To date, no studies evaluating its implementation in rural areas have been conducted.

### Shortage of healthcare professionals in rural areas

The challenges related to rural emergency medicine are multiple and significant. In particular, problems related to the vulnerability of recruitment and retention of healthcare professionals must be addressed. Despite the critical nature of the problem, there is a significant lack of data about this sector and about healthcare services for the rural Canadian population.1 A systematic search of Cochrane Reviews yields a complete absence of rigorous studies that adequately evaluate efforts to recruit and retain healthcare professionals in rural areas. This problem is not on the verge of resolution; according to the National Physician Survey,25 only 1% of family medicine residents plan to eventually practice in rural regions, and rural family doctors tend to leave emergency medicine after only a few years. Anecdotal evidence suggests that such departures are prompted by high stress, difficult schedules and poor quality of life. Combined, these factors make training and recruiting emergency physicians in rural areas a considerable challenge.

Observational studies1 ,26–32 describe several factors that could improve retention of healthcare professionals in rural areas. Promising strategies include selection of rural students for professional training programmes, establishment of university departments and training units in rural regions, provision of grants for students who commit to working in rural areas and development of personal and professional support programmes for professionals working in rural zones.

The *Management Guide*24 also proposes interventions to increase recruitment and retention of healthcare professionals in rural areas—encouraging multidisciplinary training for healthcare professionals; ensuring access to necessary and appropriate material and technical resources and help from colleagues, specialists and surgeons when necessary; increased access to specialised treatment centres and access to continuing education. The guide24 also suggests reasonable work and on-call schedules, attractive salaries and adequate benefits. Finally, it proposes the following suggestions for recruiting and retaining healthcare professionals in rural areas: an environment conducive to raising and educating a family; a stimulating social and cultural environment and employment opportunities for the healthcare professional's partner and finally, the prospect of an overall high standard of living and excellent quality of life. The implementation of these recommendations in Quebec needs to be explored.

### Trauma, prehospital emergency care and inter-establishment transfers

Trauma is one of the most common reasons for a consultation in the ED; correspondingly, it is the most frequently studied subject in research on rural medicine. Trauma is also the leading cause of mortality in individuals under 40 and the fourth most common cause of mortality for people of all ages.3 ,33 For over 30 years, major investments have been made in developing traumatology networks.20 These networks have benefitted urban patients, but the results are less robust for rural patients who are often geographically isolated from designated trauma centres, necessitating complicated inter-establishment transfers.23 ,34 ,35 In fact, in some provinces, up to 80% of patients in rural regions are over an hour away from tertiary trauma centres.36

The distances between rural EDs and tertiary care or referral centres, and the elevated risk of medical trauma in rural areas are such that PEC is essential for rural residents. A recent American meta-analysis indicated that PEC response times are significantly longer in rural areas.11 This result can be attributed to greater travel distances, hazardous road conditions and to the challenge of locating and retrieving victims in rural areas. Another study demonstrated that the elevated rate of mortality subsequent to trauma in rural areas is partially attributable to lengthy transportation time in ambulances.22

One critical component of rural emergency medicine is the transfer of more complex cases to a referral centre. Each transfer indicates the failure of the local centre to meet the patient's critical needs. Every transfer involves considerable time and personnel, and exposes the patient to the risk of complications inherent to transportation by emergency vehicle. One Canadian study reported that almost 2% of all rural ED patients in Ontario had to be transferred to another establishment to receive more advanced emergency care.16 For a medium-sized rural ED in Quebec (20 000 annual visits), this can mean over 400 transfers per year. The rate of interestablishment transfers in Quebec rural EDs is a key variable to measure; an elevated transfer rate could indicate a local shortage or a problem in access to basic services.

### Quality-of-care indicators in the emergency department

Research evidence suggests that evaluation of quality indicators and the publication of data about quality indicators improve quality of care.37–43 The recent publication of the *Consensus on Evidence-Based Quality of Care Indicators for Canadian Emergency Departments*44 permits objective measurement of EDs’ performances, allowing objective comparison among departments. Published in March 2010, this consensus was created by a panel of 24 Canadian experts including managers, clinicians, emergency medicine researchers, health information specialists and government representatives. Of 48 indicators selected, consensus was reached on eight groups of indicators determined to have the highest levels of priority and validity. The selected indicators are related to interventions for eight pathologies often treated in EDs, including myocardial infarction, stroke, sepsis, asthma and several paediatric problems related to infection.

The *Management Guide*24 was published after the consensus document and is therefore not mentioned in this important publication. However, quality of care is one of the central principles of the *Management Guide.*24 Although the eight established quality indicators are keys for future studies comparing performance between EDs, methodological limitations must be acknowledged. First, data on all of the indicators are not included in current clinical databases. Second, the scientific consensus committee did not include representatives from rural EDs, and certain quality indicators relevant to rural EDs may not be included (eg, trauma care in rural areas with limited access to traumatology centres and investigative technology, interestablishment transfer needs and the impact of these issues on quality of care).

## Objectives

This project is designed to 

1.  Develop a comprehensive portrait of all rural EDs in Quebec;

2.  Evaluate the use of the 2006 *Emergency Department Management Guide*24 
    
    1.  Perceived usefulness and implementation of its various recommendations;
    
    2.  Factors that promote or impede the implementation in rural areas;
    
    3.  Relations between the implementation and performance indicators;
    
    4.  Relations between the implementation and healthcare professionals’ work-related quality of life.

## Methods and analysis

This project is a descriptive and evaluative study of rural EDs in Quebec, which offer 24/7 medical coverage, having hospitalisation beds and are located in a rural or small town, according to the definition of Statistics Canada.45

In a previously conducted pilot study, rural EDs were identified using the Health Canada Establishment Guide46 and confirmed by the MSSS and the *Direction Nationale des Urgences*. There are 26 rural EDs in Quebec.

### Phase 1: portrait of all rural emergency departments

To develop a comprehensive portrait of all rural EDs in Quebec, a questionnaire will be sent by email to the chief nurse to collect data on (1) hospital centre characteristics (eg, referral centres, availability of local intensive care unit beds, number of acute and long-term beds); (2) availability of health information technology (eg, internet and Wifi access); (3) knowledge transfer activities (eg, quality assurance, book club); (4) ED variables (eg, triage level, wait time, average hospital stay, number of transfers between facilities); (5) available diagnostic services 24/7 (eg, lab, basic radiography, TDM, MRI, ultrasound, portable ultrasound); (6) medical and paramedical staff (eg, number of emergency doctors, years of experience and level of training, percentage of locum doctors per period, availability of specialists, number and level of training of nurses, presence of other health professionals); (7) pre-emergency and post-emergency care resources in the region (eg, number of family doctors, availability of convalescence beds); (8) long-term housing and care centres and mental health facilities (eg, number of beds, waiting list). Some data will also be gathered from databases at the MSSS (eg, number of annual visits), the Quebec Trauma Registry (information on traumatic event, healthcare institution implied, emergency department, hospitalisation, patient acuity (triage level), etc), PEC centres (eg, number of ambulances deserving each rural hospital) and Statistics Canada (eg, data on population and rural regions).

For the first phase of the study, the project needs no further ethical evaluation since all of the data required is non-nominal.

### Phase 2: Emergency Department Management Guide

First, an online survey about use of the *Emergency Department Management Guide*24 will be developed and administered electronically to the management personnel of the EDs included in the study (chiefs of staff, head nurse). Research staff will contact managers to introduce the project and to explain the online questionnaire. Regular follow-ups will be conducted to obtain the most complete responses possible. The survey will be developed using all the *Management Guide*24 recommendations (n=69) pertaining to rural hospitals. Respondents will respond on a seven-point *Likert-*type scale to two questions: (1) *To what extent is the recommendation useful in my hospital?*; (2) *To what extent is the recommendation used in my hospital?* Further, a telephonic interview will be conducted with the respondents to evaluate factors that promote or impede implementation of the recommendations perceived equally useful and not used.

Second, the following indicators will be used to explore the association between use of the *Management Guide*24 and performance and quality of care: (1) the performance indicators assessed in the first phase (eg, average ED stay) and (2) the following eight high-priority quality of care indicators established by Canadian consensus44 in the following categories: ED operations (eg, length of stay), patient security (eg, unplanned/unexpected readmissions), pain management (eg, delay in administration of medication), cardiac and respiratory problems (eg, treatment delay for thrombolysis, corticosteroid administration percentage), stroke (eg, delay in administration of plasminogenic tissue activator), and sepsis/infections (eg, delay in administration of antibiotics). Information missing from the databases will be obtained from patient medical files. The number of file reviews necessary to obtain the relevant information will vary by indicator.

To evaluate their quality of life, two online surveys will be administered to all consenting nurses and doctors working at rural EDs. The exact number of professionals to complete the survey will vary between EDs, but the expected response rate is 70%. In an effort to boost response rate, we will telephone hospital spokespersons (eg, head nurse) to establish contact and explain the procedure.

The first survey refers to the quality-of-work life systemic inventory (QWLSI)47 and will be available for completion via [http://qualitedevie.ca](http://qualitedevie.ca). The measure includes 34 themes divided into 8 subgroups: remuneration, professional development, work schedule, social environment/relationships with colleagues, relationships with superiors, physical environment, factors that influence employees’ perception and enjoyment of the task and employee support. A supplementary module of six questions will be designed to capture aspects specific to ED, which are not covered by the existing 34 items. The QWLSI provides an organisational diagnosis and permits comparison with over 3000 workers who have already completed the measure. The internal validity (Cronbach's α) of the subgroups ranges from 0.60 to 0.82.48 The overall internal validity is 0.88 and the test–retest reliability is 0.85. The English language and French-language versions are equivalent (0.84). Lower scores (below the 25th percentile) indicate greater psychological distress and professional burnout. The second survey contains questions about sociodemographic variables, and factors related to recruitment and retention and will also be completed online.

### Statistical analyses

The statistical analyses will be achieved in collaboration with the biostatistics service from the Unité de recherche en santé des populations du Centre hospitalier affilié universitaire de Québec. The data collected as part of the phase 1 will be described as means, medians and percentages, according to the variables distribution to meet the objective 1. To meet the objective 2.1, the mean of six-point Likert scores measuring the use and usefulness of the guide24 will be presented for each of its recommendations. Likert scores will also be dichotomised with the intention of showing the agreement or disagreement between the utilisation and usefulness of the guide,24 which will allow to calculate the mean number of useful recommendations, the mean number of applied recommendations and the proportion of ED where at least 70% of recommendations are applied. Answers to the questions concerning the perceived usefulness and utilisation will be compared to evaluate the level of application of recommendations considered useful. The participants’ telephonic interview answers (objective 2.2) will be qualitatively analysed to show the obstacles and facilitators considered to be the most important to the implementation of the guide.24 With the aim of meeting the objective 2.3, the relation between the use of the Management Guide24 and the performance and quality-of-care indicators will be measured with Spearman correlation. The utilisation of the guide24 will be measured by the mean number of applied recommendations in ED. Finally, regarding the objective 2.4, the results of the two surveys will be presented with descriptive statistics in a first phase. The association between the QWLSI score and utilisation of the guide24 will be assessed with the aid of a generalised estimation equations model to take into consideration the correlation between the responders from a same ED. The utilisation of the guide24 will be measured by the number of recommendations applied (objective 2.1 analyses) and processed as a continuous or dichotomous variable. The data collected during phase 1 as well as information on characteristics of responders collected during the second survey of the objective 2.4 will serve as adjustment variables in the model. However, if the sample size does not allow such analyses in objective 2.4, the association between the quality-of-life scores and utilisation of the guide24 will be measured with Spearman correlation coefficients. Furthermore, some correlational analyses will allow to compare the quality-of-work life scores and some retention and recruitment factors.

## Ethics and dissemination

This rural project required ethics evaluation through a complex multicenter study mechanism described below. In the province of Québec, a study that is conducted in several centres must conform to an established ethics procedure according to the MSSS. Two preliminary steps must be undertaken. First, the project must be peer reviewed by a recognised expert committee (eg, Scientific research committee). Second, a main research centre ethics committee (main REC), which is normally the REC that belongs to the research centre where the project is initiated, must be determined. When these two conditions are fulfilled, the principal investigator sends the project to the main REC, to each local REC (if applicable, otherwise no review occurs) and to every participating study site. Once the local RECs have reviewed the project, they send their comments to the main REC, which takes into consideration their specific requests and decides to approve/reject the project within 2 weeks following the examination. When the expectations of the main REC are satisfied, the preliminary decision is sent to the principal investigator and to every participating study site. A feasibility study committee, which evaluates the practicability of the project in each institution, must submit its evaluation to the institutional director (Hospital CEO or director) before the end of the ethical procedure. Once the ethics and feasibility examinations are completed, the decision is reviewed by each local REC (or its designed authority if it does not have its proper REC). Once the local REC (or its designed authority) approves this decision, it sends it to the institution. Furthermore, the feasibility study committee forwards its decision to the general management of the institution, which will relay its decision to the main REC. Finally, the main REC sends its final decision to the principal research coordinator and to each institution and REC implied in the project.

The phase 1 of this study was exempted from ethical evaluation as no human subject was involved. The phase 2 of this study has been approved by the CSSS Alphonse–Desjardins main REC (Project MP-HDL-1213-011). Results from this study will be published in peer-reviewed scientific journals and presented at one or more scientific conferences.

## Discussion

To our knowledge, this will be the first study to evaluate EDs in rural Quebec and Canada at such a broad scale. It will provide a greater understanding of the factors that promote and impede the implementation of the recommendations in the *Management Guide.*24 The results could be used to develop one or several interventions designed to increase implementation of the *Management Guide*24 recommendations. The questionnaire could also be used to investigate the implementation of the *Management Guide*24 in EDs outside Quebec and Canada by researchers wishing to test the implementation of a management guide24 adapted to their own region and context.

Our use of performance indicators recently published by Schull and colleagues44 to measure the impact of a knowledge transfer tool (a practical guide) on EDs performance is a further innovation that could advance knowledge transfer research. We plan to identify performance indicators that are specific to rural EDs and were not included in the list of indicators published by Schull and colleagues.44 Eventually, we wish to explore the impact of the use of the *Management Guide*24 on the quality of care offered in Quebec relative to that offered in other Canadian provinces. The proposed project would allow us to establish an essential knowledge base that would serve to plan a future comparison with EDs in other provinces.

The results of this study will also allow a greater understanding of the factors associated with work-related quality of life in ED healthcare professionals, and those relevant to recruitment and retention of ED personnel. The research evidence generated by this study could also be used to develop interventions that could, in turn, be evaluated using the same questionnaires.

Finally, our results will undoubtedly be useful to policy-makers and can be used to guide the distribution of healthcare services in rural areas. The results will provide policy-makers with a greater understanding and appreciation of the unique challenges faced by rural EDs in the province. The results will contribute to the bank of available research data that can be used to develop policies about attribution of resources in rural areas. Ultimately, this project will contribute to improved health in rural Quebec.

## Acknowledgments

We would like to thank Julie Villa and Sylvain Bussières for their work in formatting the manuscript as well as editorial suggestions.

## Footnotes

*   Contributors RF was responsible for the original idea, literature review and study design. PA, FL, JMC, JFL, MO, GD, JH, JP, AT, GSR and JG have contributed to various aspects of the study design with input relating to their specific expertise in the field. All authors read and approved the final manuscript.

*   Funding This work was supported by the Fonds de recherche du Québec—Santé (FRQS) 22481.

*   Competing interests None.

*   Ethics approval CSSS Alphonse–Desjardins research ethics committee (Project MP-HDL-1213-011).

*   Provenance and peer review Not commissioned; internally peer reviewed.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: [http://creativecommons.org/licenses/by-nc/3.0/](http://creativecommons.org/licenses/by-nc/3.0/) and [http://creativecommons.org/licenses/by-nc/3.0/legalcode](http://creativecommons.org/licenses/by-nc/3.0/legalcode)

## References

1.  Grobler L, Marais B, Mabunda S, et al. Interventions for increasing the proportion of health professionals practising in rural and other underserved areas. Cochrane Database Syst Rev 2009;1:CD005314.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19160251&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

2.  Gruen R, Weeramanthri T, Knight S, et al. Specialist outreach clinics in primary care and rural hospital settings. Cochrane Database Syst Rev 2004;1:CD003798.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14974038&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

3.  Institut canadien d'information sur la santé. Comment se portent les Canadiens vivant en milieu rural? Une évaluation de leur état de santé et des déterminants de la santé. Ottawa: Agence de la Santé Publique du Canada, 2006.
    
    

4.  Haggerty J, Roberge D, Pineault R, et al. Features of primary healthcare clinics associated with patients’ utilization of emergency rooms: urban–rural differences. Healthc Policy 2007;3:72–85.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19305782&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=19305782&link_type=ISI) 

5.  Howlett K. Teen's death ignites debate over emergency room closures. *The Globe and Mail* 22 January 2010.
    
    

6.  Minister of Health. Canada Health Act—Annual Report 2009–2010. Ottawa: Health Canada, 2010.
    
    

7.  Ministries of Health Services and Health Planning. Standards of accessibility and guidelines for provision of sustainable acute care services by health authorities. Victoria: Government of British Columbia, 2002.
    
    

8.  Newman S. Nelson doctors issue emergency aid plea. *The Globe and Mail* 7 October 2008:S1–2.
    
    

9.  Ginde A, Foianini A, Renner D, et al. Availability and quality of computed tomography and magnetic resonance imaging equipment in U.S. emergency departments. Acad Emerg Med 2008;15:780–3.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1553-2712.2008.00192.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18783491&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000258469100015&link_type=ISI) 

10. Merkens B, Mowbray R, Creeden L, et al. A rural CT scanner: evaluating the effect on local health care. Can Assoc Radiol J 2006;57:224–31.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17128890&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

11. Carr B, Caplan J, Pryor J, et al. A meta-analysis of prehospital care times for trauma. Prehosp Emerg Care 2006;10:198–206.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1080/10903120500541324&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16531377&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000242396600007&link_type=ISI) 

12. Canadian Association of Emergency Physicians. Recommendations for the Management of Rural, Remote and Isolated Emergency Health Care Facilities in Canada. Ottawa: CAEP, 1997.
    
    

13. Fleet R, Plant J, Ness R, et al. An experience in patient advocacy by rural emergency physicians after major services cuts: the case of Nelson, BC. Can J Rural Med 2013;18.
    
    

14. McGregor J, Hanlon N, Emmons S, et al. If all ambulances could fly: putting provincial standards of emergency care access to the test in Northern British Columbia. Can J Rural Med 2005;10:163–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16079032&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

15. Thompson J, McNair N. Health care reform and emergency outpatient use of rural hospitals in Alberta, Canada. J Emerg Med 1995;13:415–21.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=7673640&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

16. Rourke J, Kennard M. Emergency patient transfers from rural hospitals: a regional study. CJEM 2001;3:296–301.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17610773&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=17610773&link_type=ISI) 

17. Gauthier J, Hagerty J, Lamarche P, et al. Entre adaptabilité et fragilité: les conditions d'accès aux services de santé des communautés rurales et éloignées. Québec: Institut national de santé publique du Québec, 2009.
    
    

18. Roberge D, Pineault R, Larouche D, et al. The continuing saga of emergency room overcrowding: are we aiming at the right target? Healthc Policy 2010;5:27–39.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21286266&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=21286266&link_type=ISI) 

19. Coben J, Tiesman H, Bossarte R, et al. Rural-urban differences in injury hospitalizations in the U.S., 2004. Am J Prev Med 2009;36:49–55.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.amepre.2008.10.001&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19095165&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000262090300008&link_type=ISI) 

20. Cudnik M, Newgard C, Sayre M, et al. Level I versus level II trauma centers: an outcomes-based assessment. J Trauma 2009;66:1321–6.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/TA.0b013e3181929e2b&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19430234&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000266021700012&link_type=ISI) 

21. Fatovich D, Jacobs I. The relationship between remoteness and trauma deaths in Western Australia. J Trauma 2009;67:910–14.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19088551&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

22. Grossman D, Kim A, Macdonald S, et al. Urban-rural differences in prehospital care of major trauma. J Trauma 1997;42:723–9.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9137264&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1997WW74400024&link_type=ISI) 

23. Newgard C, McConnell K, Hedges J, et al. The benefit of higher level of care transfer of injured patients from nontertiary hospital emergency departments. J Trauma 2007;63:965–71.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17993937&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

24. Ministère de la Santé et des Services sociaux du Québec. Guide de gestion de l'urgence. Québec: Gouvernement du Québec, 2006.
    
    

25. National Physician survey 2007. [http://nationalphysiciansurvey.ca/wp-content/uploads/2012/10/2004-National-Q18\_next.pdf](http://nationalphysiciansurvey.ca/wp-content/uploads/2012/10/2004-National-Q18_next.pdf)(accessed 26 Mar 2013).
    
    

26. Laven G, Wilkinson D. Rural doctors and rural backgrounds: how strong is the evidence? A systematic review. Aust J Rural Health 2003;11:277–84.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1440-1584.2003.00534.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14678410&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

27. Laven GA, Beilby JJ, Wilkinson D, et al. Factors associated with rural practice among Australian-trained general practitioners. Med J Aust 2003;179:75–9.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12864716&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000184529100004&link_type=ISI) 

28. Rabinowitz HK, Diamond JJ, Markham FW, et al. A program to increase the number of family physicians in rural and underserved areas: impact after 22 years. JAMA 1999;281:255–60.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.281.3.255&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9918481&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000078052800031&link_type=ISI) 

29. Rabinowitz HK, Diamond JJ, Markham FW, et al. Critical factors for designing programs to increase the supply and retention of rural primary care physicians. JAMA 2001;286:1041–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.286.9.1041&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11559288&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000170802500017&link_type=ISI) 

30. Rabinowitz HK, Diamond JJ, Veloski JJ, et al. The impact of multiple predictors on generalist physicians’ care of underserved populations. Am J Public Health 2000;90:1225–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10937001&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000088398700012&link_type=ISI) 

31. Woloschuk W, Tarrant M. Does a rural educational experience influence students’ likelihood of rural practice? Impact of student background and gender. Med Educ 2002;36:241–7.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2923.2002.01143.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11879514&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

32. Woloschuk W, Tarrant M. Do students from rural backgrounds engage in rural family practice more than their urban-raised peers? Med Educ 2004;38:259–61.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2923.2004.01764.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14996334&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

33. National center for injury prevention and control 1999–2006. 10 learning causes of death, All races both sexes. Atlanta: Centers for Disease Control and Prevention, 2010.
    
    

34. Clay Mann N, Mullins RJ, Hedges JR, et al. Mortality among seriously injured patients treated in remote rural trauma centers before and after implementation of a statewide trauma system. Med Care 2001;39:643–53.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/00005650-200107000-00001&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11458129&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000169626500002&link_type=ISI) 

35. Garwe T, Cowan LD, Neas BR, et al. Directness of transport of major trauma patients to a level I trauma center: a propensity-adjusted survival analysis of the impact on short-term mortality. J Trauma 2011;70:1118–27.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20693925&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

36. Schuurman N, Bell N, Hameed MS, et al. A model for identifying and ranking need for trauma service in nonmetropolitan regions based on injury risk and access to services. J Trauma 2008;65:54–62.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18580511&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 

37. Berwick D, James B, Coye M. Connections between quality measurement and improvement. Med Care 2003;41:130–8.
    
    

38. Galvin RS, McGlynn EA. Using performance measurement to drive improvement: a road map for change. Med Care 2003;41:I48–60.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12544816&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000180282200006&link_type=ISI) 

39. Marshall MN, Shekelle PG, Leatherman S, et al. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283:1866–74.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.283.14.1866&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10770149&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000086248600033&link_type=ISI) 

40. McGlynn EA. Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Med Care 2003;41:I1–7.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/00005650-200301000-00001&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12544811&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000180282200001&link_type=ISI) 

41. McGlynn EA. An evidence-based national quality measurement and reporting system. Med Care 2003;41:I8–15.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/00005650-200301000-00003&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12544812&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000180282200002&link_type=ISI) 

42. Institute of Medicine. Pathways to quality health care: performance measurement, accelerating improvement. Washington, DC: National Academies Press, 2006.
    
    

43. Relman A. Assessment and accountability: the third revolution in medical care. N Engl J Med 1988;319:1220–2.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=3173460&link_type=MED&atom=%2Fbmjopen%2F3%2F4%2Fe002961.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1988Q697500010&link_type=ISI) 

44. Schull M, Hatcher C, Guttmann A, et al. Development of a consensus on evidence-based quality of care indicators for Canadian emergency departments. Toronto: Institute for Clinical Evaluative Sciences, 2010.
    
    

45. du Plessis V, Beshiri R, Bollman RD, et al. Definitions of “rural”. Agriculture and rural working paper series working paper. Ottawa: Statistics Canada Agriculture Division, 2002.
    
    

46. Association canadienne des soins de santé. Guide des établissements de soins de santé du Canada. Ottawa: CHA, 2009–2010.
    
    

47. Martel J, Dupuis G. Quality of work life: theoritical and methodological problems, and presentation of a new model and measuring instrument. Soc Indicators Res 2006;77:333–68.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/s11205-004-5368-4&link_type=DOI) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000238278200007&link_type=ISI) 

48. Dupuis G, Martel J. Caractéristiques psychométriques de l'ISQVT: Fidélité test-retest, cohérence interne, validité de construit, validation transculturelle. XIIIe Congrès de l'Association internationale de psychologie au travail et des organisations de langue française. Bologne, Italie, 2004:1184–95.