Approach

The objective of this study was to examine the independent contribution of being licenced through various alternative routes on primary care performance. Given that licensure route has not been previously studied as a potential factor influencing performance, we were interested in isolating its effect by comparing each ALP group to TLPs on a variety of quality care indicators while adjusting for a number of covariates. We do not address the independent impact of the other variables that were adjusted for, as, practically, we could either focus on a small number of outcomes and explore the full multivariable models, or examine a broad spectrum of indicators representative of general family practice and narrow our focus to licensure route. We chose the latter, as we were interested in primary care performance as a whole rather than performance on any individual quality indicators. Additionally, the indicators do not have validated thresholds or gold standard rates at the individual level or population level, thus performance is better assessed globally as opposed to focusing on individual tests, screens or prescriptions. By focusing on the independent contribution of licensure route while adjusting for covariates, we aimed to understand if a physician’s licensure route is associated with subsequent performance, irrespective of other demographic or practice characteristics. The goal of this approach was to explore whether regulatory licensure policies independently influence the primary care provided to patients in Ontario, thus offering evaluative evidence for the impact and outcomes of these policies.

Study cohorts

The study population included all practising family physicians in Ontario who were registered with the College of Physicians and Surgeons of Ontario between 2000 and 2012 and billed the Ontario Health Insurance Plan (OHIP) in 2014. All publicly funded health services provided by physicians are submitted to OHIP. This population included TLPs and ALPs. TLPs are physicians who obtained a licence to practice by meeting the traditional criteria, namely the completion of postgraduate training in Canada and successful completion of the national qualifying and certification examinations (the Medical Council of Canada Qualifying Examinations parts 1 and 2, and either the College of Family Physicians of Canada or the Royal College of Physicians and Surgeons of Canada examinations). ALPs are those physicians who were missing one or more of the traditional requirements but met an alternative set of criteria at the time of licensure. There are many alternative licensure routes; in this study, we have focused on the three most commonly used by family physicians, described in table 1. A more comprehensive description of these routes has been described previously.15

Data sources

Ontario health administrative datasets held at the Institute for Clinical Evaluative Sciences (ICES) were used in this study. These datasets were linked using unique encoded identifiers and analysed at ICES under data security and privacy policies and procedures that are approved by the Office of the Information and Privacy Commissioner of Ontario.32 The following administrative databases were used: Canadian Institute for Health Information hospital Discharge Abstract Database (providing diagnostic information regarding hospital admissions), OHIP physician claims database (containing physician billings and diagnoses from 1991), the National Ambulatory Care Reporting System database (providing information on hospital-based and community-based ambulatory care, including emergency department (ED) visits, from 2000 and same-day surgery from 1991) and the Ontario Drug Benefit program database (containing information on all drug therapies dispensed to eligible individuals 65 years of age and older).

Variables

Physician demographic characteristics included age, sex, medical school region and the Human Development Index (HDI) associated with the physician’s country of medical school, which is a composite score based on life expectancy, education and per capita income that rank orders all countries.33 Physician practice characteristics included practice type (comprehensive or not), group type (family health team [FHT], non-FHT no group), scope of practice (percent providing any of the following: postnatal visits, obstetrical deliveries, postnatal visits, ED visits and long-term care visits) and practice location (urban and suburban/rural). Comprehensive family physicians are those who met specific criteria regarding the type and scope of services they provide.34 FHTs are group practices that include comprehensive family physicians working alongside primary providers such as nurses, social workers, pharmacists and nutritionists. A detailed description of the physician demographic and practice characteristics is included in online supplementary appendix A.

Primary care quality indicators based on health administrative data were calculated for chronic disease management, preventive paediatric care, cancer screening and hospital readmission rates. Chronic disease management indicators included measures for diabetes care (haemoglobin A1c (glycated haemoglobin) [HbA1c] testing, cholesterol testing, ophthalmology examinations, the receipt of prescriptions for ACE inhibitors and angiotensin II receptor blockers), congestive heart failure (CHF; echocardiogram testing within 12 months of diagnosis and ED visits), asthma (spirometry testing within 12 months of diagnoses and ED visits) and chronic obstructive pulmonary disease (COPD; spirometry testing within 12 months of diagnoses and ED visits). Paediatric care indicators include well-baby visits, the 18-month enhanced developmental assessment and the absence of paediatric vaccinations (defined as no billing for any immunisation in OHIP). Cancer screening indicators included cervical, breast and colorectal cancer screening. Hospital readmission rates were calculated at 30 days and 1 year for patients with a hospital admission. These primary care quality indicators are described in online supplementary appendix B.

For each family physician, patients who were either rostered (enrolled) or virtually rostered to them (attributed to the physician based on the majority of their billings) were included. All outcomes denote whether a patient received a given type of care, rather than whether the physician they were rostered to provided it. Therefore, patients who received care from a physician other than their family physician (eg, a walk-in clinic physician or another family physician in their practice) would appear in the data as having received that care, and this would be attributed to the family physician they are rostered to.

Statistical analysis

Demographic and practice characteristics are presented as proportions, means, percentage with any and mean percentages (table 2). Absolute rates for the primary care quality indicators are presented as means and mean percentages unless otherwise noted (table 3). Unadjusted rates are included for comprehensiveness, but only the adjusted rates are discussed to answer our research question. Confidence limits are presented where applicable. To help with interpretation of results, statistically significant differences less than 5% were considered small and statistically significant differences greater than 5% were considered larger.

Our multivariable analysis modelled the relationship between physician licensure cohort (ALP group or TLP) and clinical practice outcomes. Before carrying out the modelling, we tested the outcome measures for normality and found that many, such as the proportion of a physician’s patients with diabetes who received an eye exam within the previous year, were not normally distributed but became so after being log transformed. Based on this, we chose to use proc genmod in SAS to model the number with each characteristic (rather than the proportion) based on the Poisson distribution and including a log offset. Exponentiating the resulting parameter estimate gave us the relative rate for each outcome.

Each outcome was modelled individually. Covariates were entered into each model in a stepwise fashion, with only the significant variables retained in the final model. These included grouped age, sex, number of years in practice, urban–rural status, IMG status, whether the physician was in a patient enrolment model, HDI group, the proportion of their patients who were low income and the median age of their patients. The relative rates estimated by the models indicate the difference in outcome between each ALP group and the TLPs (reference group). All analyses were conducted using SAS V.9.3.

Patient and public involvement

There was no patient or public involvement in this study.

Demographic and practice characteristics

A total of 292 ALPs and 11 127 TLPs were included in the study (table 2). The largest group of ALPs were the Canadian trained (n=114), followed by the US trained (n=91) and the out of province (n=78). The majority of TLPs were men (56.6%) and were older (50.6 years) than all three groups of ALPs. TLPs had fewer IMGs (22.2%) and overwhelmingly came from countries with very high HDI (90.5%). All ALPs were slightly more likely than TLPs to be in comprehensive practice and were less likely to be working in an FHT. Patient age and income distributions were similar across all groups.

The ALP groups’ average ages ranged from 42.1 years to 50.6 years. The out-of-province ALPs had the highest proportion of men (69%) compared with the other ALP groups. In the out-of-province ALP group, the majority were IMGs (89.7%) and completed medical school in countries considered medium/low on the HDI (63.2%). Seventy per cent (70.1%) practised in urban environments, and they had the largest proportion in solo practice (32.2%). Similar to out of province, the US-trained ALPs were mostly IMGs (85.7%); however, they graduated primarily from medical schools from countries with a very high/high HDI (68.1%). They had the largest proportion practising in non-FHT groups (65.9%) and were the most urban group (78%). Contrary to the other ALPs, almost half (47.4%) of the Canadian-trained ALPs were non-IMGs and 72.9% came from countries with very high/high HDI. Seventy per cent were in comprehensive practice, and they had the lowest percentage in solo practice. They also had the lowest proportion practising in urban areas compared with all other groups (67.5%).

Primary care quality indicators

Table 3 shows the results of the unadjusted and adjusted comparisons between each ALP group and the TLPs (unadjusted mean numbers are included in online supplementary appendix C). Each ALP group had a unique profile of primary care quality indicators. Patients of the out-of-province ALPs had the most substantial statistically significant differences in the quality care indicators compared with patients of TLPs after multivariable adjustments. These family physicians’ patients with diabetes were 4% less likely to have received HbA1c testing, and their patients with COPD were 18% less likely to have received spirometry testing. Their patients with CHF, COPD or asthma were 7% more likely to visit an ED for any reason (ie, all-cause) than those of TLPs. Additionally, their female patients aged 50–69 years were 4% less likely to have received a mammogram in previous 2 years and their paediatric patients had 14% fewer well-baby visits, were 24% less likely to have had an 18-month enhanced well-baby visit and were 38% more likely to have received no immunisations. However, their patients were 3% more likely to have received spirometry testing, 4% more likely to receive colon cancer screening and their hospitalised patients were 9% less likely to be readmitted in 1 year.

In contrast, US-trained ALPs were comparable with their TLP counterparts, with some statistically significant differences. Their diabetic care and cancer screening rates were similar, although US-trained ALP patients were 8% more likely to have received HbA1c and lipids testing than TLPs’ patients and 2% more likely to have received a pap test or any colon cancer screening. Their patients with CHF, COPD and asthma were also 3% less likely to visit the ED, and their paediatric patients were 27% less likely to have not received any immunisations; however, they were 7% less likely to receive well-baby visits. Canadian-trained ALPs were also similar to their TLP counterparts across most indicators; however, some statistically significant differences were seen: their paediatric patients were 3% less likely to have received a well-baby visit but were 34% less likely to have not received any childhood immunisations; their patients with COPD were 11% less likely to have had spirometry testing within 12 months of diagnosis; and their patients with CHF, COPD or asthma were 9% more likely to visit an ED (all-cause) than those of TLPs. Minor differences were seen also seen with HbA1c testing, spriometry testing for asthma patients, and colon cancer screening, with Canadian-trained ALP’s patients being 3% more likely to have received testing or screening.

Out-of-province ALPs

Compared with other subgroups, the primary care performance of out-of-province ALPs was the most different from TLPs after adjustments. Most notably, their patients less than 2 years of age were significantly less likely to receive well baby visits, enhanced 18-month assessments or immunisations, highlighting a trend in preventive paediatric care. These differences may reflect provincial differences in guidelines and schedules for paediatric care. For example, there is significant variation in how 18-month assessments are approached globally and across Canada.37 Ontario has supported a longer and more comprehensive enhanced 18-month assessment by providing financial incentives through a unique billing code.37–39 It is possible that out-of-province ALPs were unaware of Ontario’s enhanced 18-month assessment or of the paediatric care expectations of family physicians in the province. It is also possible that these physicians provided 18-month assessments but did not bill for it using the Ontario-specific code. Previous research has shown that male IMGs who have been in practice for over 10 years are less likely to provide 18-month assessments in Ontario.40 In this study, age, gender and HDI were controlled for, suggesting these factors are not accountable for the differences, thus entering Ontario from another province through an alternative route appears to be an independent risk factor.

Similar to the differences seen in 18-month assessments, the lower childhood immunisation rates in the out-of-province ALPs may be in part due to interprovincial variation in policies. For example, childhood vaccine schedules differ across provinces,41 which may have implications for how physicians bill. Furthermore, in Ontario, immunisations for children under 2 years of age are predominantly done in physician offices,42 while they may be administered by nurses or other allied health professionals in other provinces. Thus, the norms and conventions from their prior jurisdictions may be reflected in the billing practices of these ALPs once in Ontario.

In addition to differences in preventive paediatric care, out-of-province ALPs differed from TLPs in rates of spirometry testing for patients with COPD, which is recommended to confirm a diagnosis of COPD.43–46 Previous research has found that spirometry test ordering among family physicians in Ontario is generally low,47 and our findings suggest that it is even lower among out-of-province ALPs (and Canadian-trained ALPs) compared with TLPs, highlighting potential provincial differences in utilisation. Out-of-province ALPs’ patients with CHF, COPD or asthma also had higher rates of all-cause ED visits, but their hospitalised patients were 9% less likely to have been readmitted within 1 year. Rates of all-cause ED visits and readmissions are sometimes associated with access to primary care48–51 but can be influenced by many factors51–54 and should thus be interpreted cautiously.

Overall, our findings highlight that ALPs entering Ontario from another Canadian province perform differently than TLPs in certain indicators of primary care. However, the performance differences noted in this study may be due to provincial differences in care expectations and reflect the context of their recent work environments. Such provincial differences in how physicians provide primary care have implications for the migration of physicians across provinces since physicians can typically practice anywhere in Canada once licenced. Given that provisional licences have been seen as a way for physicians to gain entry to larger provinces through smaller ones,17 the Federation of Medical Regulatory Authorities of Canada has begun to standardise provincial licensure requirements, and the Medical Council of Canada is facilitating a common approach to practice ready assessments for IMGs across the country. While these efforts will help to mitigate potential performance differences in Canadian physicians, our findings suggest that ALPs entering Ontario from another province may still benefit from information about Ontario care expectations at the time of licensure. Focused knowledge translation for family physicians migrating across provinces may help to educate physicians about province-specific expectations and support their adoption of provincially supported programmes and guidelines, reducing the potential for future performance differences.

US-trained and Canadian-trained ALPs

US-trained and Canadian-trained ALPs performed similarly to TLPs on most primary care quality indicators after adjustments, though some notable differences were seen. For both groups, patients less than 2 years of age were less likely to receive well-baby visits but more likely to receive immunisations. In contrast to the out-of-province ALPs, whose patients were more likely to have not received any early childhood immunisations, US-trained and Canadian-trained ALP patients were much more likely to receive them compared with the rest of the province: this 27%–34% difference was the largest difference seen between these groups and the TLPs. US-trained ALPs’ patients were also more likely to receive pap tests and colon cancer screenings, their patients with diabetes were more likely to receive HbA1c and lipid testing, and their patients with CHF, COPD or asthma were less likely to visit an ED. The higher rates of testing and screening may be reflective of their American training, as previous research has found American physicians tend to have lower thresholds for diagnostic and therapeutic interventions.55 Similar to out-of-province ALPs, Canadian-trained ALPs’ patients with COPD were less likely to receive spirometry testing, and their patients with CHF, COPD or asthma were more likely to visit an ED. Overall, the performance of both of these ALP groups was comparable with TLPs. This is perhaps unsurprising given that these physicians have similar postgraduate training and that postgraduate training has been found to be predictive of patient outcomes.56 Our findings provide evidence that their practice performance is in fact similar to TLPs in Ontario.

Strengths and limitations

This is the first study to examine the primary care performance of alternatively and traditionally licenced family physicians in Ontario. Our use of population-level data across multiple indicators of primary care allowed for a robust and comprehensive comparison of ALPs and TLPs, and our use of multivariable analysis enabled statistical adjustment of physician demographics, practice environments and patient factors, such as socioeconomic status, that are associated with primary care performance. While this approach contributes to our understanding of ALP performance, it is not without limitations. First, ALPs and TLPs were compared with each other, not to a gold standard. As such, our findings do not indicate whether physicians are meeting performance benchmarks, but rather whether ALP performance is comparable with TLP performance. Second, our results are based on 1 year of health administrative data that depict a point in time and also only represents elements of care that are funded by the Ontario Ministry of Health and Long-Term Care. Other important aspects of primary care such as the doctor–patient relationship or interprofessional collaboration with other primary healthcare providers are thus not accounted for. Last, billing data introduces unique interpretation challenges as these quality indicators are proxies for delivery of care; therefore, some of the variance in the indicators may be, in part, attributable to differences in billing practices or factors unrelated to the physician, such as patient preference.

Implications for future research

This study offers insight into the primary care performance of alternatively licenced physicians. Primary care is an important area of study given that approximately half of physicians in Ontario specialise in family medicine15; however, future research is needed to examine the practice performance of ALPs practising in other specialties. Performance is also multifaceted and must be studied using a variety of measures. Future studies could include other measures of performance, such as practice assessments or complaints profiles, to gain a comprehensive picture of ALPs’ practices.

This study also demonstrates that licensure route is a useful way of stratifying and comparing physicians. IMG studies typically define physicians based on their country of undergraduate medical school, whereas licensure route accounts for the influence of postgraduate training and previous practice experience on performance. Examining the impact of all of a physician’s training and experience on future practice performance allows for a more robust understanding of the predictors of performance and may enable more nuanced IMG research in the future.

Finally, this study represents a collaboration between a medical regulator and system partners. Such collaborations are important for linking performance data across the continuum of medical education and practice58 59 and for providing evaluative evidence for regulatory processes and policies, such as alternative licensure routes.12 14 Further collaborations of this nature will allow for robust examinations of the influence of each stage of a physician’s training on future practice performance.