Interprofessional online learning for primary healthcare: findings from a scoping review
========================================================================================

* Scott Reeves
* Simon Fletcher
* Clodagh McLoughlin
* Alastair Yim
* Kunal D Patel

## Abstract

**Objectives** This article presents the findings from a scoping review which explored the nature of interprofessional online learning in primary healthcare. The review was informed by the following questions: What is the nature of evidence on online postgraduate education for primary healthcare interprofessional teams? What learning approaches and study methods are used in this context? What is the range of reported outcomes for primary healthcare learners, their organisations and the care they deliver to patients/clients?

**Setting** The review explored the global literature on interprofessional online learning in primary healthcare settings.

**Results** The review found that the 23 included studies employed a range of different e-learning methods with contrasting course durations, use of theory, participant mix, approaches to accreditation and assessment of learning. Most of the included studies reported outcomes associated with learner reactions and positive changes in participant attitudes/perceptions and improvement in knowledge/skills as a result of engagement in an e-learning course. In contrast, fewer studies reported changes in participant behaviours, changes in organisational practice and improvements to patients/clients.

**Conclusions** A number of educational, methodological and outcome implications are be offered. E-learning can enhance an education experience, support development, ease time constraints, overcome geographic limitations and can offer greater flexibility. However, it can also contribute to the isolation of learners and its benefits can be negated by technical problems.

*   interprofessional collaboration
*   Interprofessional education; online learning
*   primary health care
*   scoping review

### Strengths and limitations of this study

*   The study performed a review of the literature into an increasingly significant educational approach in primary healthcare.

*   The review enabled the identification of effective e-learning  interventions and also reported where/how these interventions can be improved.

*   The search was limited to publications from 2000 onwards and included only those published in English. Any relevant publications which fell outside of these criteria were not included.

## Background

Online (e-learning) has been a growing part of health professions education for well over a decade.* Meta-analyses reporting the effects of e-learning studies have found that this type of education is effective for improving a range of professional competencies—attitudes, knowledge, skills and behaviours.1 2 The advantages of e-learning for health professionals include diminishing logistical barriers (anytime, anyplace learning for busy healthcare providers working in different environments) and individualised, tailored, point-of-care learning that meets the varied needs of professional learners from multiple practice settings.3 

It has been reported that online learning can be as effective as physical attendance in a traditional classroom; however, consideration must be given to factors such as development of clear guidelines for educators regarding roles and responsibilities, clear learner competencies, access to technology and sufficient funding.4 Applied learning approaches, such as scenarios and interactive ‘second-life’ programmes, can be engaging, although there is a need to ensure training is relevant to clinical evidence-based practice.5 Use of free web tools, such as Skype and Moodle have shown useful educational outcomes, while alleviating travel pressures and expenses for learners.6 

There is also evidence that the benefits of using online learning can result in less constrained discussion, as learners feel more able to engage in online discussions rather than verbal face-to-face conversations.7 In addition, it has been found that e-learning can enhance the quantity, quality, cost and accessibility of health professions education.8 

However, it has been indicated that online learning may be viewed by some as isolating when compared with traditional learning methods due to lack of a social connection.9 Further, it has been noted that technological difficulties can often be a key disrupting factor,10 for example, it can result in a loss of collegiality typically associated with traditional forms of face-to-face learning.6 

When used to promote interactions and relations between different professional groups, an increasing number of studies have suggested that the use of e-learning technologies can enhance interprofessional collaboration.11 12 While interprofessional e-learning can help with the logistics and costs of traditional face-to-face collaborative learning and can help overcome the isolating effects learners can feel when learning alone online, there is added complexity with managing new software, a loss of nonverbal group cues and technological glitches which can undermine its quality.13 Nevertheless, it has been found that this type of e-learning can support professionals to connect online to discuss and co-create solutions to real-life issues for patients/clients from geographically disparate locations.3 

As indicated above, while there has been a growth of both professional and interprofessional e-learning in health professions education, to date, there has been no effort undertaken to explore the use of interprofessional e-learning for primary healthcare teams.† To fill this gap in knowledge, this review attempted to provide an overview of the empirical literature in order to generate an insight into the nature of evidence of e-learning for primary healthcare teams.

## Methods

Scoping reviews are being used increasingly by researchers to explore healthcare evidence.14 They enable the clarification of complex areas of inquiry and refine subsequent research studies.14–16 The overall goal of a scoping review is to ‘examine the extent, range and nature of research activity, determine the value in undertaking a full systematic review, summarising and disseminating research findings or identify gaps in the existing literature’.15 

We adopted a scoping review methodology to specifically examine the extent, range and nature of evidence for the use of interprofessional e-learning for primary healthcare teams. For this review, Arksey and O’Malley's14 six-step framework for interpretive scoping literature reviews was used, with some small modifications15 17 (see table 1).

View this table:
[Table 1](http://bmjopen.bmj.com/content/7/8/e016872/T1)

Table 1 
Overview of the framework for conducting a scoping study

### Identifying the relevant research question

Responding to the intention to explore the literature on interprofessional e-learning within primary healthcare, our research questions could focus on the following: map existing work which addresses interprofessional e-learning in primary healthcare teams; generate an understanding of the influence of such work and the depth and breadth of ‘the field’, and identify significant knowledge gaps and areas for improvement. With these initial ideas in mind, the following research questions were generated:

*   What is the nature of evidence on online postgraduate education‡ for primary healthcare interprofessional teams?

*   What learning approaches and study methods are used in this context?

*   What is the range of reported outcomes for primary healthcare learners, their organisations and the care they deliver to patients/clients?

### Identifying relevant studies

Using the research questions as a guide, keywords were applied to a search strategy which was then preliminarily input  into two electronic databases: Medline and CINAHL. This offered an indication of the relevance of the search terms, and the subsequent feasibility of their application was based on the numerical results generated from this preliminary search. This process enabled the following search strategy to be adopted (see box 1).Box 1 
### Search terms

#1 Primary Care

#2 Care, Primary Care

#3 Health Care, Primary

#4 Care Primary

#5 General Practice

#6 #1 or #2 or #3 or #4 or #5

#7 Interprofessional or Inter-professional

#8 Interdisciplinary or Inter-disciplinary

#9 Multidisciplinary or Multi-disciplinary

#10 Team or Teamwork

#11 #7 or #8 or #9 or #10

#12 E-learning

#13 Electronic Learning

#14 Learning, Electronic

#15 Remote Learning

#16 Learning, Remote

#17 Learning, Blended

#18 Video conferencing

#19 #12 or #13 or #14 or #15 or #16 or #17 or #18

#20 #6 and #11 and #19

Following a consultation with university information scientists, in an attempt to gauge the efficacy of the strategy and identify further databases, these key terms were applied to six electronic sources. Including studies from January 2000 to October 2015, the following databases were searched:

*   Medline

*   CINAHL

*   British Educational Index

*   PubMed

*   Scopus

*   Web of Science

An additional search of online and grey literature through Google and Google Scholar, and a further hand search of the 10 journals which have published the most papers found in the searches also took place (see box 2).Box 2 
### Journals searched

British Journal of Community Nursing

BMC Public Health

BMC Medical Education

BMJ Quality and Safety

British Journal of General Practice

Education for Primary Care

Journal of Continuing Education in the Health Professions

Journal of Interprofessional Care

Medical Teacher

Trials

From an initial yield of 1568 potential sources (generated from electronic database and additional searches), which through a rigorous screening process (see below), the review yielded 23 included studies (see figure 1).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/7/8/e016872/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/7/8/e016872/F1)

Figure 1 
Searching and screening results. 

### Study selection

In order to address the research question for this review, the following inclusion criteria were employed:

*   Papers that describe evaluations of online/e-learning involving interprofessional teams based in primary healthcare/family care.

*   All research evaluation designs (eg, action research, case study, ethnographic, experimental, quasi-experimental studies).

*   Any reported outcome from the online/e-learning evaluation (see outcomes typology below).

As the searches and screening of potential sources progressed, it became apparent that there was very little literature reporting online *postgraduate* education for primary healthcare interprofessional teams. As a result, two key modifications were made to the inclusion criteria. First, the scope of review was widened to include postgraduate education and continuing education. Second, the review was widened to include team-based interprofessional online learning as well as general interprofessional e-learning (involving primary healthcare practitioners, but not necessarily based in the same interprofessional team).

### Charting the data

Key information from the included studies was abstracted by combining a categorisation of e-learning methods18 with an abstraction approach used in a previous systematic review.19 

Using this approach, the following information was elicited from each of the included studies:

*   Study aims/objectives

*   Research design, sampling, data collection/analysis

*   Location and duration of the e-learning intervention/activity

*   Professional mix of learners

*   Methods of e-learning employed

*   Technologies used to support e-learning

*   Assessment/accreditation of learning

*   All reported outcomes from the e-learning activity

### Collating, summarising and reporting the results

Given the heterogeneous nature of the included studies, a thematic approach to the analysis was employed.20 This allowed the emergence of key issues (themes) from the literature, enabling an insight into the characteristics related to online learning.

In addition, to capture the diversity of reported outcomes in the included papers,19 an extended version of Kirkpatrick's educational outcomes model, which has six differing but non-hierarchical levels, was used (see table 2).

View this table:
[Table 2](http://bmjopen.bmj.com/content/7/8/e016872/T2)

Table 2 
Key outcomes

### Methodological quality

All abstracts generated from the database searches (n=1303) and additional searches (n=265) were reviewed independently by two of the authors  to determine if they met the inclusion criteria (see above). The full text article was obtained (181 papers) if the abstract met these criteria. These articles were again screened independently by two authors  as a second check to determine if they met the inclusion criteria.

### Review limitations

There are three key limitations to this review. First, only English-language articles were considered for inclusion in the study. As such, we  did not include potentially relevant materials written in other languages and published in non-English speaking countries. Second, the review searched for materials published from 2000, so  any papers published before this date were not included. Third, only a partial range of grey literature was searched. For example, we  did not search primary healthcare conferences for possible materials. This restriction on grey literature was necessary to limit the volume of materials and maintain a focus on research studies.

## Results

### E-learning approaches

Of the 23 included studies, 12 were undertaken in the UK, 4 in North America (2 in USA and 2 in Canada) and 2 studies involved multiple countries (1 study included The Netherlands, France and the UK, the other Germany, Austria and the UK). In addition, one study was undertaken in each of the following countries: Brazil, Australia, France, Germany and Mexico. The professions represented included: medicine (14 studies), nursing (13 studies), pharmacy (3 studies), physiotherapy (2 studies), social work (2 studies, community workers (1 study), nutrition (1 study), occupational therapy (1 study), podiatry (1 study) and psychology (1 study).

Online supplementary appendix 1 provides references for all included studies and online  supplementary appendix 2 offers an overview of key content of the papers reviewed, including aim of the e-learning course, participants involved, use of e-learning methods, course accreditation and assessment of learning. As appendix 2 indicates, the included studies report on a variety of different primary healthcare e-learning courses in relation to aims, duration and use of underlying educational theory. For example, in terms of course duration, this varied from hours, to a few days to weeks and even years. Similarly, there was a wide range of different numbers of participating professions involved in the studies—from 24 participants in one study to over 30 000 participants in another much larger study. In addition, while just over half of the studies did not mention the use of an underpinning theory in the development of their e-learning course, a variety of contrasting theories were employed by other studies, including adult learning approaches, social learning theory, theory of planned behaviour and constructionist theory. This heterogeneity is also found in the mixture of differing e-learning approaches, types of interaction, course accreditation and assessment of learning (see online supplementary appendix 2).

### Supplementary appendix 1

[[SP1.pdf]](pending:yes) 

### Supplementary appendix 2

[[SP2.pdf]](pending:yes) 

### Methodological approaches

Most studies employed quasi-experimental designs (eg, preintervention/postintervention, postintervention only) and typically gathered data in the form of (unvalidated) surveys. Only one study employed an experimental design (randomised controlled trial) though this study gathered unvalidated survey data. There were also some use of mixed methods (studies that gather qualitative and quantitative data) and qualitative methods (eg, case study designs) that gathered interview and focus group data. Online supplementary appendix 3 presents an overview of information relating to key elements of the methodological approaches employed in each of the 23 studies. This appendix  also indicates a wide variety in the sample sizes reported for the included studies—ranging from 24 to over 16 000 participants. Most of the studies employed a convenience sampling technique.

### Supplementary appendix 3

[[SP3.pdf]](pending:yes) 

### Key educational issues

Based on the analysis and synthesis approach outlined above, a number of key educational issues emerged from the included studies. In total, the following eight issues were identified: realising the potential of e-learning, enhancing collaboration and communication, improving time pressures, overcoming geographic boundaries, economics, costs and effectiveness, convenience, flexibility and accessibility, learner isolation and technical challenges.

### Realising the potential of e-learning

The review found that 8  of the 23 included studies provided reports of how the use of e-learning changed the way primary healthcare professionals learnt together. Among these, one study21 stated that the use of e-learning technologies such as ‘interactive menus, on-line case studies and video-clips’ (p. 344) could improve the quality of collaborative learning for primary healthcare providers. Another study22 that explored the use of online blogs as a learning method in their evaluation of a postgraduate e-learning course found that the use of a blog platform promoted interprofessional interaction and learning which helped generate improved decision-making skills. One further study23 found that web-based learning was 'identified as offering the potential for students to engage in rich and effective construction of knowledge’ (p.  469).

### Enhancing collaboration and communication

Thirteen studies reported that the use of e-learning approaches could effectively support the collaborative efforts of participating primary healthcare professionals. For these studies, the advantage of using e-learning methods is that it can foster a sense of collaborative community for participating learners. One study24 found that ‘the opportunity to train as a whole team was valued […] allowed staff, as one manager said, ‘to be singing from the same hymn sheet’'. Another study23 reported that ‘the online environment has opened up enormous opportunities for interaction between students and tutors and between tutors, and has brought collaborative learning centre stage in distance education’ (pp. 470–471).

### Improving time pressures

The ability of e-learning methods to alleviate some of the time pressures on the clinical workloads of primary healthcare practitioners to engage in professional development activities was found to be an important issue within five of the included studies. One study25 which explored the effect of online learning to support the diagnosis of chronic fatigue syndrome (CFS) in primary healthcare found that ‘an online approach (to practitioner education) was preferred as face to face training was thought to be too time consuming’ (p. 9). These authors go on to note that the convenience associated with online methods was particularly welcomed, as a combination of heavy workloads and the additional complexity of CFS diagnosis meant that ease of access and speed of information transfer was paramount.

### Overcoming geographic boundaries

Four of included studies found that use of e-learning methods could help to overcome traditional issues of having to deliver the educational content of interprofessional courses in the same geographic location. Exploring the potential of e-learning in the safe use of insulin for general practitioners, nurses and pharmacists, one study26 reported that ‘e-learning provides a practical solution to the provision of evidence based learning across many different staff groups and geographical boundaries’ (p. 210).

### Economics, costs and effectiveness

Encouragingly, three  of studies reported a range of positive attributes linked to the cost effectiveness of interprofessional e-learning. For instance, in a study of an online learning course for improving screening of amblyopia in US-based primary healthcare practices, Marsh-Tootle et al.27 stated that they selected an online learning approach ‘as the best delivery mode to implement facets of adult-based learning relevant to physicians as well as allowing low cost, wide spread dissemination of standardised information to individuals separated by time and distance’ (p. 7161).

However, five of studies also noted other financial implications, some of which are not immediately obvious, that may impede the introduction and sustainability of online education. These studies acknowledged that there were uncertainties regarding the initial financial investment and subsequent funding of e-learning. For example, in their evaluation of online course for rural practitioners, one set of authors28 reported that ‘significant fiscal and human resource barriers were identified that included the uptake and retention of course participants’ (p. 635).

### Convenience, flexibility and accessibility

Many, if not all of the included papers, indirectly acknowledged these issues. However, four studies made explicit reference to them. One study29 that explored perceptions of interprofessional e-learning among primary healthcare workers in Canada found that ‘internet based technology has enabled a more convenient and flexible learning option to meet the needs of busy working healthcare providers’ (p. 265).

### Learner isolation

Although, as outlined above, online learning has the potential to develop practitioners’ professional and interprofessional competence, a small number (n=5) of studies found that the move from traditional approaches to delivering education—in the same space at the same time—to an online environment whereby interactions are virtual in nature can present a challenging transition for some learners. An example of this was mentioned by Halabisky et al.,29 who reported how ‘isolation of learners from each other’ (p. 266) impeded the effectiveness of their online course.

### Technical challenges

Four of the included studies revealed how technical difficulties linked to the delivery of the e-learning approaches they evaluated in their respective studies undermined the quality of the educational experience for participants. While these studies reported technical issues, these were relatively minor and ultimately resolvable. It should also be noted that the low number of studies which described such issues suggests that this has not been a major cause for concern when compared with other difficulties presented above.

### Reported outcomes

Table 3 provides an overview of studies which reported outcomes across the six-point outcomes typology (as presented in table 2).

As indicated in table 3, of the total number of outcomes (n=42) reported across the included studies, most (n=28) were associated with individual changes at levels 1, 2a and 2b. In contrast, fewer studies (n=14) reported broader changes at levels 3, 4a and 4b.

View this table:
[Table 3](http://bmjopen.bmj.com/content/7/8/e016872/T3)

Table 3 
Overview of reported outcomes

## Discussion

Considering our research questions (see above), the review indicated that the evidence in favour of e-learning is significant. Online learning approaches can both facilitate and improve interprofessional collaboration, its practicality and accessibility offer advantages which make them preferable to more ‘traditional’ educational methods. E-learning has the potential to facilitate complex and multifaceted collaborative practice in primary healthcare and beyond. Taking place on a number of levels, these improvements can range from local team-based relations to global communication between practitioners. Indeed, interprofessional e-learning can offer a variety of useful opportunities to develop a range of collaborative competencies supported by a number of different e-learning technologies (eg, online discussion forums, social media applications, message boards).

The review also identified that increasing intensity of primary healthcare practice often creates a distinctly time-sensitive environment which can be alleviated by the use of e-learning methods. Difficulties associated with heavy workloads can be diminished as e-learning is easily accessible and flexible for practitioners. E-learning can therefore contribute to the development of practitioners’ competence as they can, for example, incorporate a short online course during a busy working week with minimal disruption to their clinical schedules. As a result, e-learning can have a positive influence in the short and long term, benefitting practitioners as well as the care they deliver to patients/clients. These findings resonate with research reporting the positive effects of e-learning in the wider literature.1–4 10 

Given that e-learning approaches can be regarded as invaluable to the coherent and efficient implementation of healthcare practice, it is important to identify and attempt to respond to any shortcomings or areas for improvement. Importantly, the review has reported the isolating potential of remote, computer-based learning. The move from the traditional classroom-based approach has resulted in some learners feeling isolated and others noting a lack of support from their online educators. This is a direct consequence of diminishing face-to-face learner-to-learner and learner-to-educator interactions. The use of online learning may also affect interprofessional interactions/dynamics which were more easily identifiable in previous contexts. To help overcome such issues, the use of blended approaches offers a useful means of transition between virtual and real educational contexts.30 31 Although this can be regarded as a ‘solution’ which merely serves to negate the beneficial capacity of e-learning, the gradual transition from classroom to computer screen rather than an abrupt relocation may make these changes less emotionally impactful.

Technical challenges have also been reported in a small but notable number of studies. Although these issues were usually linked to minor failures of software and connectivity problems they still combined to cause frustration and disappointment for learners. It is important to note that coherent technological functioning is paramount to the successful delivery of e-learning.32 33 If possible, such minor faults should be prevented in the first instance to avoid disruption of the quality of the e-learning experience. This will ensure that e-learning applications and software meet quality requirements in enhancing the experience for the learner while fully realising the potential of (increasingly) sophisticated synchronous and asynchronous e-learning technologies.

## Conclusion/Implications

Overall, the scoping review identified a number of key benefits related to the use of interprofessional e-learning for primary healthcare practitioners. Its practicality was consistently reported to contribute to enhanced time management, the removal of geographic limitations and ease of access were found to help strengthen interprofessional collaboration and networking. It was also reported that economic savings could be made with the use of e-learning as reductions in travel costs, institutional overheads and so on could be realised. However, it was noted that e-learning could result in learner isolation, and some technical problems were also identified. These were, however, relatively minor in comparison to the reported benefits. Such findings resonate with the wider literature on e-learning in the health professions and interprofessional literature.3 13 34 

### Study outcomes

Collectively, the included studies indicated that the use of e-learning for primary healthcare practitioners generated a range of positive outcomes for participant reactions (level 1), helped to generate improvements to their perceptions and attitudes (level 2a) as well as improvements to their knowledge and skills (level 2b). In addition, while the review indicated that the use of e-learning resulted in gains to participants’ individual behaviour (level 3), improvements in the way their organisations practiced (level 4a) and could generate benefit for patients/clients (level 4b), there were fewer studies reporting at these levels. While it is important to gather data for outcomes at levels 1, 2a and 2b, future e-learning evaluations should also focus on developing the evidence for its effects on levels 3, 4a and 4b (including data on cost-effectiveness) to help build a more robust insight into the longer-term outcomes for this type of primary healthcare education. This focus on ‘lower-level’ outcomes is echoed in the wider interprofessional education literature, which also found a propensity for studies to report on levels 1–2b so overlooking ‘higher-level’ outcomes (levels 3–4b).35 36 

### Heterogeneity

While the included studies reported a promising number of outcomes associated with the use of e-learning, the wide range of e-learning activities/course does generate some limitations. Specifically, due to the heterogeneity of the e-learning approaches reported in the 23 studies, it is difficult to identify a set of robust implications that can identify which types of e-learning methods may be effective and which may be less so (a problem which is compounded by the use of a mixture of differing study designs and methods). The problem of heterogeneity of interventions and evaluation approaches has been reported elsewhere in the interprofessional education literature.37 Nevertheless, it is possible to note that of the included studies, those which employed a variety of approaches such as online self-directed learning, interactive web-based discussion supported by an e-facilitator were well evaluated when compared with studies that only employed a single form of e-learning method. In addition, blended approaches (using online and traditional learning methods) were also well evaluated. However, as noted above, such approaches did increase costs due to the need to pay for learning space and travel expenses.

### Self-report data

Another word of caution needs to be applied to the included studies. While the review indicated that these studies reported a range of positive outcomes related to the use of e-learning in primary healthcare, most of the 23 studies gathered data in the form of unvalidated surveys, individual interviews and focus groups. As a result, the bulk of outcomes are based on self-report data. This is a weak form of evidence as it is widely recognised that individuals are often inaccurate in assessing possible changes to their knowledge, skills and behaviours.38 As a result, such reports must be regarded as weak approaches to measuring change.

## Acknowledgments

The authors would like to acknowledge and thank iheed who funded this review. They would specifically like to acknowledge and thank Miriam O’Donoghue (Programs QA & Accreditation, iheed) and Dr Tom O’Callaghan (CEO and Founder of iheed). Their generosity in terms of finances, time and direction has been vital to the successful completion of this review.

## Footnotes

*   * E-learning is a term that relates to learning that uses electronic technologies to access educational curriculum outside of a traditional classroom. In most cases it refers to a course or program delivered on an online basis.

*    †  Primary health care team is a term that relates to a group of practitioners who work together as the first point of contact in a health care system. The source of primary care is general practice or family medicine.

*    ‡  The term postgraduate education refers to formal learning health professionals receive after they graduate (qualify) as practitioners. As such, we regard this term as including continuing education.

*   Contributors Given the ordered task-oriented nature of a scoping review, the following presents a list of the duties undertaken by the research team and the respective personnel who contributed to completion. Establishment of research question/s and development of search strategy: SR and KDP. Database search and record screening: SR, CM, SF and AY. Full text assessment: CM, SF and SR. Thematic analysis: SR and SF. Discussion construction: SR, SF and KDP.

*   Funding This work was supported by iheed.

*   Competing interests None declared.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement Extra data can be accessed via the Dryad data repository at [http://datadryad.org/](http://datadryad.org/) with the doi:10.5061/dryad.k9m17.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) 

## References

1.  1. Cook DA , Levinson AJ , Garside S , et al . Internet-based learning in the health professions: a meta-analysis. JAMA 2008;300:1181–96.[doi:10.1001/jama.300.10.1181](http://dx.doi.org/10.1001/jama.300.10.1181) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.300.10.1181&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18780847&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000259038700011&link_type=ISI) 

2.  2. Means B , Toyama Y , Murphy R , et al . A Meta-analysis and review of Online Learning Studies. Washington, DC: US Department of Education, Centre for Technology in Learning, 2010. [https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf](https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf) 
    
    

3.  3. MacNeill H , Telner D , Sparaggis-Agaliotis A , et al . All for one and one for all: understanding health professionals' experience in individual versus collaborative online learning. J Contin Educ Health Prof 2014;34:102–11.[doi:10.1002/chp.21226](http://dx.doi.org/10.1002/chp.21226) 
    
    

4.  4. Thomas AU , Fried GP , Johnson P , et al . Sharing best practices through online communities of practice: a case study. Hum Resour Health 2010;8:25.[doi:10.1186/1478-4491-8-25](http://dx.doi.org/10.1186/1478-4491-8-25) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21073733&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

5.  5. Sutton KP , Maybery D , Moore T . Creating a sustainable and effective mental health workforce for Gippsland, Victoria: solutions and directions for strategic planning. Rural Remote Health 2011;11:1585.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21280971&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

6.  6. Lillis S , Gibbons V , Lawrenson R . The experience of final year medical students undertaking a general practice run with a distance education component. Rural Remote Health 2010;10:1268.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20345189&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

7.  7. Murphy CJ . Focusing on the essentials: learning for performance. Hum Resour Health 2008;6:26–30.[doi:10.1186/1478-4491-6-26](http://dx.doi.org/10.1186/1478-4491-6-26) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19077195&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

8.  8. Maloney S , Nicklen P , Rivers G , et al . A Cost-Effectiveness Analysis of Blended Versus Face-to-Face Delivery of Evidence-Based Medicine to Medical Students. J Med Internet Res 2015;17:e182.[doi:10.2196/jmir.4346](http://dx.doi.org/10.2196/jmir.4346) 
    
    

9.  9. Roberts T . McInnerney J seven problems of online group learning (and their solutions). Education, Technology and Society 2007;10:257–68.
    
    

10. 10. Sivamalai S , Murthy SV , Gupta TS , et al . Teaching pathology via online digital microscopy: positive learning outcomes for rurally based medical students. Aust J Rural Health 2011;19:45–51.[doi:10.1111/j.1440-1584.2010.01176.x](http://dx.doi.org/10.1111/j.1440-1584.2010.01176.x) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21265925&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

11. 11.1.  
    2.   Leathard A  
    
     Reeves S . Freeth D New forms of information technology, new forms of collaboration?  Leathard A  , ed. Interprofessional Collaboration: from policy to practice in Health and Social Care. London: Routledge, 2003.
    
    

12. 12. Snowdon A , Shell J , Leitch K . Innovation takes leadership: opportunities & challenges for Canada's Health Care System: Ivey Centre for health Innovation and Leadership, 2010. [http://worldhealthinnovationnetwork.com/images/publications/whitepapers/InnovationTakesLeadership\_WhitePaper.pdf](http://worldhealthinnovationnetwork.com/images/publications/whitepapers/InnovationTakesLeadership_WhitePaper.pdf).
    
    

13. 13. Hanna E , Soren B , Telner D , et al . Flying blind: the experience of online interprofessional facilitation. J Interprof Care 2013;27:298–304.[doi:10.3109/13561820.2012.723071](http://dx.doi.org/10.3109/13561820.2012.723071) 
    
    

14. 14. Arksey H , O'Malley L . Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19–32.[doi:10.1080/1364557032000119616](http://dx.doi.org/10.1080/1364557032000119616) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1080/1364557032000119616&link_type=DOI) 

15. 15. Levac D , Colquhoun H , O'Brien KK , et al . Scoping studies: advancing the methodology. Implement Sci 2010;5:69.[doi:10.1186/1748-5908-5-69](http://dx.doi.org/10.1186/1748-5908-5-69) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1748-5908-5-69&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20854677&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

16. 16. Armstrong R , Hall BJ , Doyle J , et al . 'Scoping the scope' of a cochrane review. Journal of Public Health 2011;33:147–50.[doi:10.1093/pubmed/fdr015](http://dx.doi.org/10.1093/pubmed/fdr015) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1093/pubmed/fdr015&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21345890&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000287748800026&link_type=ISI) 

17. 17. Daudt HM , van Mossel C , Scott SJ . Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med Res Methodol 2013;13:48.[doi:10.1186/1471-2288-13-48](http://dx.doi.org/10.1186/1471-2288-13-48) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2288-13-48&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23522333&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

18. 18. Raymond M , Iliffe S , Pickett J . Checkists to evaluate an e-learning resource. Educ Prim Care 2012;23:458–9.[doi:10.1080/14739879.2012.11494162](http://dx.doi.org/10.1080/14739879.2012.11494162) 
    
    

19. 19. Barr H , Koppel I , Reeves S , et al . Effective interprofessional education: argument assumption and evidence. Oxford: Blackwell, 2005. [http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1405116544.html](http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1405116544.html).
    
    

20. 20. Mays N , Pope C , Popay J . Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005;10(S1):6–20.[doi:10.1258/1355819054308576](http://dx.doi.org/10.1258/1355819054308576) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1258/1355819054308576&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16053580&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

21. 21. Docherty A , Sandhu H . Student-perceived barriers and facilitators to e-learning in continuing professional development in primary care. Educ Prim Care 2006;17:343–53.[doi:10.1080/14739879.2006.11864084](http://dx.doi.org/10.1080/14739879.2006.11864084) 
    
    

22. 22. Jenkins MS , Bean WG , Luke K . Part-time, e-learning interprofessional pain management education for the primary and community care setting. Br J Pain 2014;8:16–26.[doi:10.1177/2049463713502944](http://dx.doi.org/10.1177/2049463713502944) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/2049463713502944&link_type=DOI) 

23. 23. Russell J , Elton L , Swinglehurst D , et al . Using the online environment in assessment for learning: a case‐study of a web‐based course in primary care. Assessment & Evaluation in Higher Education 2006;31:465–78.[doi:10.1080/02602930600679209](http://dx.doi.org/10.1080/02602930600679209) 
    
    

24. 24. Rudolf MC , Hunt C , George J , et al . HENRY: development, pilot and long-term evaluation of a programme to help practitioners work more effectively with parents of babies and pre-school children to prevent childhood obesity. Child Care Health Dev 2010;36:850–7.[doi:10.1111/j.1365-2214.2010.01116.x](http://dx.doi.org/10.1111/j.1365-2214.2010.01116.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2214.2010.01116.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20637028&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000282816600012&link_type=ISI) 

25. 25. Hannon K , Peters S , Fisher L , et al . Developing resources to support the diagnosis and management of Chronic Fatigue Syndrome/Myalgic Encephalitis (CFS/ME) in primary care: a qualitative study. BMC Fam Pract 2012;13:1–12.[doi:10.1186/1471-2296-13-93](http://dx.doi.org/10.1186/1471-2296-13-93) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2296-13-1&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22221509&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

26. 26. James J , Atkins H , Sturgess I , et al . The safe use of insulin e-learning module: successful roll out of a teaching programme for all working in diabetes. Practical Diabetes International 2011;28:209–11.[doi:10.1002/pdi.1595](http://dx.doi.org/10.1002/pdi.1595) 
    
    

27. 27. Marsh-Tootle WL , McGwin G , Kohler CL , et al . Efficacy of a web-based intervention to improve and sustain knowledge and screening for amblyopia in primary care settings. Invest Ophthalmol Vis Sci 2011;52:7160–7.[doi:10.1167/iovs.10-6566](http://dx.doi.org/10.1167/iovs.10-6566) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiaW92cyI7czo1OiJyZXNpZCI7czoxMDoiNTIvMTAvNzE2MCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzgvZTAxNjg3Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

28. 28. Robinson T , Hills D , Kelly B . The evaluation of an online orientation to rural mental health practice in Australia. J Psychiatr Ment Health Nurs 2011;18:629–36.[doi:10.1111/j.1365-2850.2011.01712.x](http://dx.doi.org/10.1111/j.1365-2850.2011.01712.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2850.2011.01712.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21848598&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

29. 29. Halabisky B , Humbert J , Stodel EJ , et al . eLearning, knowledge brokering, and nursing: strengthening collaborative practice in long-term care. Comput Inform Nurs 2010;28:264–73.[doi:10.1097/NCN.0b013e3181ec28b9](http://dx.doi.org/10.1097/NCN.0b013e3181ec28b9) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20736723&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

30. 30. Stacey E , Gerbic P . Effective Blended Learning Practices: Evidenceevidence-Basedbased Perspectivesperspectives. Hershey NY: IGI Global, 2009.
    
    

31. 31. Henderson S , Dalton M , Cartmel J . Using Interprofessional Learning for Continuing Education: Development and Evaluation of the Graduate Certificate Program in Health Professional Education for Clinicians. J Contin Educ Health Prof 2016;36:211–7.[doi:10.1097/CEH.0000000000000093](http://dx.doi.org/10.1097/CEH.0000000000000093) 
    
    

32. 32. van der Rhee B , Verma R , Plaschka GR , et al . Technology Readiness, Learning Goals, and eLearning: Searching for Synergy. Decision Sciences Journal of Innovative Education 2007;5:127–49.[doi:10.1111/j.1540-4609.2007.00130.x](http://dx.doi.org/10.1111/j.1540-4609.2007.00130.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1540-4609.2007.00130.x&link_type=DOI) 

33. 33.1.  
    2.   Byrne E , 
    3.   Donaldson L , 
    4.   Manda-Taylor L  
    
     MacDonald S , Mwapasa V , Petersen M , et al . The use of technology enhanced learning in health research capacity development: lessons from a cross country research partnership. In:  Byrne E ,  Donaldson L ,  Manda-Taylor L  , Global Health. vol 12, 2016:19.
    
    

34. 34. Curran V , Reid A , Reis P , et al . The use of information and communications technologies in the delivery of interprofessional education: A review of evaluation outcome levels. J Interprof Care 2015;29:541–50.[doi:10.3109/13561820.2015.1021002](http://dx.doi.org/10.3109/13561820.2015.1021002) 
    
    

35. 35. Pauzé E , Reeves S . Examining the effects of interprofessional education on mental health providers: Findings from an updated systematic review. J Ment Health 2010;19:258–71.[doi:10.3109/09638230903469244](http://dx.doi.org/10.3109/09638230903469244) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20441490&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 

36. 36. Reeves S , Fletcher S , Barr H , et al . A BEME systematic review of the effects of interprofessional education: BEME Guide No. 39. Med Teach 2016;38:656–68.[doi:10.3109/0142159X.2016.1173663](http://dx.doi.org/10.3109/0142159X.2016.1173663) 
    
    

37. 37. Reeves S , Perrier L , Goldman J , et al . Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database Syst Rev 2013:CD002213.[doi:10.1002/14651858.CD002213.pub3](http://dx.doi.org/10.1002/14651858.CD002213.pub3) 
    
    

38. 38. Davis DA , Mazmanian PE , Fordis M , et al . Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006;296:1094–102.[doi:10.1001/jama.296.9.1094](http://dx.doi.org/10.1001/jama.296.9.1094) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.296.9.1094&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16954489&link_type=MED&atom=%2Fbmjopen%2F7%2F8%2Fe016872.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000240298400011&link_type=ISI)