Health technology assessment in its local contexts: studies of telehealthcare

https://doi.org/10.1016/S0277-9536(02)00419-7Get rights and content

Abstract

Health technology assessment (HTA) is one of the major research enterprises of late modernity, reaching into fields of previously autonomous professional practice, and critically interrogating the organisation and delivery of health care. The ‘evaluation’ of new health technologies within the field of HTA is increasingly a normative political expectation, as discourses of ‘evidence-based’ practice run through health policy in the UK and elsewhere. Despite its importance in governing the direction of innovation in health care delivery, there are hardly any empirical studies of HTA in practice. In this paper, we draw on two ethnographic studies of telehealthcare implementation and evaluation in the UK to explore the practical conduct of HTA, and we focus specifically on the social organisation and conduct of randomised controlled trials of these new technologies. The paper examines how evaluation forms a mediating set of practices that make the embedding or normalisation of a new technology possible; and present a simple model of the social and technical contingencies within the evaluation process.

Introduction

In late modernity, the institutional superstructure of health care rests largely on the application of technologies to problems of understanding the nature and distribution of disease, its diagnosis, treatment and management, and the organisation of service provision. These technologies may take the form of machines, or systems, or of practices. Some are hardware, others are software; some are embodied in people, others in objects. Many take the form of hybrids and networks in which the boundaries between the ‘social’ and the ‘technical’ are hard to locate, and when they can be, seem often to be ambiguous or permeable. Indeed, the point of distinction between technologies is itself problematic, there is no concrete boundary between the ‘technical’ and the ‘social’, but rather there are practices of differentiation and demarcation on the basis of agreements (or disagreements) about their properties (Bloomfield & Vurdubakis, 1994).

This paper is about the social and technical transactions through which such agreements are formed in the field of Health technology assessment (HTA), and specifically the evaluation of a new domain of information and communications technology (ICT) in health care—telehealthcare—in the United Kingdom. Evaluation of new technologies, treatment modalities and models of service delivery is a crucial component of Research and Development in the national health service (NHS) in the UK, and attracts significant levels of government funding through an internal ‘tax’ on NHS spending (the NHS R&D levy), and is formalised through a variety of means—notably the NHS HTA R&D Programme. Evaluation is therefore a normative political expectation that connects with the thrust towards ‘evidence-based practice’ that increasingly characterises health care systems across the developed world, and which has become a deeply embedded feature of the discourses of health care that circulate in the UK (Harrison, 1996). Evaluation and adoption are intimately drawn together by this political linkage, in which the evidence base for a technology acts to discipline qualitative decisions about policy and public spending. Evaluation is neither a discrete asocial activity, nor is it self-evident. HTA is one of the major research enterprises of our time. It is broad in scope, and defined by its emphasis on formal—mainly quantitative—methods and its focus on clinical and cost effectiveness; it is therefore directed at the production of evidence about the efficacy and utility of techniques and technologies of health care delivery treatment modalities and ways of working (Woolf & Henshall, 2000) that meet particular criteria of adequacy. The formal proof of HTA is to be found in the outcomes of the randomised controlled trial (RCT), systematic review, and meta-analysis. The questions that inform it tend to arise directly from the thrust of health care policy (Faulkner, 1997), and the outcomes of HTA practitioners’ work are specifically intended to mediate between policy and practice.

So within the field of HTA, it is the method that is prioritised either in the production of primary outcomes data, or in the synthesis of existing knowledge. The expository literature of HTA reflects the priority given to methods, not theories, by locating them in a rhetoric of political and social neutrality, and emphasising applied investigatory technique over broader political questions (Lehoux & Blume, 2000). None of this is intended to imply that practitioners of HTA are unaware of the political implications of their work, or the wider social implications of their practice—the reverse is certainly the case. But HTA emulates the rhetorical form of biomedical science, constructing what seem to be methodologically secure facts and so has, at the outset, a defence against wider political critique (Giacomini, 1999).

Work to conceptualise HTA to date has mainly been directed at macro-level analyses of the relationship between policy formation and evidence production. More localised critiques have investigated the assumptions that underpin outcomes themselves, or the methods by which they are reached. Much less work has investigated the specifics of HTA as a field of practice, the socio-technical networks in which knowledge about efficacy is defined and generated, negotiations about criteria for its adequacy, or the procedures through which these are enacted in concrete practices. The production of evaluative knowledge about the utility and effectiveness of health technologies is socially organised, and thus what the ‘facts’ are, is the product of processes and practices of construction. This paper is concerned with precisely this disciplinary field, exemplified in the application of RCTs of telehealthcare systems.

This paper has two objectives. First, we are concerned to explore the practical conduct of evaluation and the social organisation of RCTs of new technologies. Our interest here is in the way that evaluation forms a mediating set of practices that make the embedding or normalisation of a new technology possible. Second, we present a simple model of the social and technical contingencies within the evaluation process. Our concern here is to map the contingent points on the journey between ideation (i.e., the emergence of ideas about the value of a new technology to practice) and normalisation (i.e., the point at which it becomes possible for it to be embedded in clinical practice). Understanding the process of evaluation is important, for we actually know little about the practical conduct of HTA, although there is a large and expanding body of literature that details either evaluation methods or the results of their application.

Section snippets

Telehealthcare as a case study

Telehealthcare offers an important and relevant focus for understanding the social contexts and practices involved in evaluation, since there is not only disagreement about the utility and efficacy of the technological systems involved, but also about the most appropriate means to assess them (Holle & Zahlmann, 1999). Telehealthcare systems are politically attractive, but clinically contentious technologies that promise new kinds of links between clinicians and patients separated by time and

Study group and method

This paper draws on analyses of ethnographic data drawn from two studies. In the course of these studies, we examined the development, implementation and evaluation of 10 telehealthcare interventions between 1997 and 2002. These are described in Fig. 1.

Study TM1 was a qualitative formative process evaluation of the implementation of three telemedicine services in an English Region, and focused on the professional and organisational dynamics of their implementation and evaluation. Between 1997

The contingency model

Our purpose in this paper is to understand the production of evaluative knowledge in HTA—and to do this we use the development of telehealthcare systems as a mediating example. Telehealthcare is intrinsically interesting, for the use of ICTS as intermediaries between clinicians and patients is a highly unstable field of activity, where evidence is by no means certain, and where proponents of these technologies have not found it easy to penetrate health care systems (Bashshur 1997;

Conclusion

The starting point for our analysis has been that telehealthcare offers an important and relevant focus for understanding the social contexts and practices that appear at the points of contact between evaluation and new forms of service delivery. We have emphasised its instability, for it is characterised by disagreements and contests about utility and effectiveness not simply about the combination and interaction of techniques and technologies involved, but also about the most appropriate

Acknowledgements

The study from which this paper is drawn is funded by the UK Department of Health (Grant ICT/032), and we have also drawn on work undertaken in a study funded by the NHS North West R&D Directorate (Grant RDO/12/20). This support is gratefully acknowledged. None of our work could have been done without the support and co-operation of respondents in these studies: we thank them for their time and considerable candour. Theresa Atkinson, Helen Doyle and Nikki Shaw undertook some of the fieldwork on

References (49)

  • B.P. Bloomfield et al.

    Boundary disputesNegotiating the boundary between the technical and the social in the development of IT systems

    Information Technology and People

    (1994)
  • T. Edwards

    Innovation and organizational changeDevelopments towards an interactive process perspective

    Technology Analysis & Strategic Management

    (2000)
  • A. Faulkner

    ‘Strange bedfellows’ in the laboratory of the NHS? An analysis of the new science of health technology assessment in the United Kingdom

  • D. Filiberti et al.

    A telemedicine transaction model

    Telemedicine Journal

    (1995)
  • M.K. Giacomini

    The which-huntAssembling health technologies for assessment and rationing

    Journal of Health Politics Policy and Law

    (1999)
  • B. Glaser et al.

    The discovery of grounded theory

    (1967)
  • J. Grigsby et al.

    The diffusion of telemedicine

    Telemedicine Journal and E-Health

    (2002)
  • J. Grigsby et al.

    Analytic framework for evaluation of telemedicine

    Telemedicine Journal

    (1995)
  • S. Harrison

    The politics of evidence-based medicine in the United Kingdom

    Policy and Politics

    (1996)
  • R. Harrison et al.

    Can telemedicine be used to improve communication between primary and secondary care?

    British Medical Journal

    (1998)
  • H. Heathfield et al.

    Evaluating information technology in health careBarriers and challenges

    British Medical Journal

    (1998)
  • Hersh, W.R., Helfand, M., Wallace, J., Kraemer, D., Patterson, P., Shapiro, S., & Greenlick, M. (2001). Clinical...
  • R. Holle et al.

    Evaluation of telemedical services

    IEEE Transactions on Communications

    (1999)
  • J.L. Huston et al.

    Evaluating a telemedicine delivery system

    Top Health Information Management

    (1996)
  • Cited by (116)

    • The Emerging Social Science Literature on Health Technology Assessment: A Narrative Review

      2020, Value in Health
      Citation Excerpt :

      These are the core topics that social scientists have explored extensively in other areas of health policy, such as healthcare reform,1-5 rationing/priority-setting,6-9 or pharmaceutical regulation.10-14 Social science has so far paid less attention to HTA, a relatively new phenomenon: in 2003, a paper in Social Science and Medicine announced that “Despite its importance in governing the direction of innovation in healthcare delivery, there are hardly any empirical studies of HTA in practice.”15 Since then, however, social scientists have been increasingly attentive to HTA, and, as the present themed section demonstrates, the HTA community shows considerable interest in input from other disciplines.

    • Health technology assessment frameworks for eHealth: A systematic review

      2020, International Journal of Technology Assessment in Health Care
    View all citing articles on Scopus
    View full text