Skip to main content
SpringerLink
Log in

A framework for evaluating usability of clinical monitoring technology

  • Published:
Journal of Clinical Monitoring and Computing Aims and scope Submit manuscript

Abstract

Technology design is a complex task, and acceptability is enhanced when usability is central to its design. Evaluating usability is a challenge for purchasers and developers of technology. We have developed a framework for testing the usability of clinical monitoring technology through literature review and experience designing clinical monitors. The framework can help designers meet key international usability norms. The framework includes these direct testing methods: thinking aloud, question asking, co-discovery, performance and psychophysiological measurement. Indirect testing methods include: questionnaires and interviews, observation and ethnographic studies, and self-reporting logs. Inspection, a third usability testing method, is also included. The use of these methods is described and practical examples of how they would be used in the development of an innovative monitor are given throughout. This framework is built on a range of methods to ensure harmony between users and new clinical monitoring technology, and have been selected to be practical to use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Crandall B, Klein G, Hoffman R. Working minds: a practitioner’s guide to cognitive task analysis. Cambridge, Mass.: MIT Press, 2006

  2. American National Standards Institute/Association for the Advancement of Medical Instrumentation. Human factors design process for medical devices. ANSI/AAMI HE74:2001

  3. International Electrotechnical Commission. Medical electrical equipment – Part 1–6: General requirements for basic safety and essential performance – Collateral standard: Usability. IEC 60601-1-6 Ed. 2.0 b:2006

  4. Neilson J (1993) Usability engineering. San Diego, CA: Academic Press

    Google Scholar 

  5. International Organization for Standardization. Software engineering – product quality – Part 1: Quality model. ISO/IEC 9126-1:2001

  6. International Organization for Standardization. Software engineering – product quality – Part 2: External metrics. ISO/IEC TR 9126-2:2003

  7. International Organization for Standardization. Software engineering – product quality – Part 3: Internal metrics. ISO/IEC TR 9126-3:2003

  8. International Organization for Standardization. Software engineering – product quality – Part 4: Quality in use metrics. ISO/IEC TR 9126-4:2004

  9. Kushniruk A, Patel V, Cimino J. Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces. Proceedings of the 1997 AMIA Annual Fall Symposium. Accessed online at http://www.adams.mgh.harvard.edu/PDF_Repository/D004362.PDF (February 06, 2006), 1997

  10. American National Standards Institute/International Committee for International Committee Stanards. ANSI-NCITS 354–2001 Common industry format for usability test reports, 2001

  11. Greenberg S. (1996) Teaching human computer interaction to programmers. ACM SIGCHI Bull 28(2):5–6

    Article  Google Scholar 

  12. Virzi R. (1992) Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 34:457–468

    Google Scholar 

  13. Ng JY, Man JC, Fels S, Dumont G, Ansermino JM. (2005) An evaluation of a vibro-tactile display prototype for physiological monitoring. Anesth Analg 101(6):1719–1724

    Article  PubMed  Google Scholar 

  14. Kushniruk A, Patel V. (2004) Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Informat 37:56–76

    Article  Google Scholar 

  15. Dumas J, Redish J. A practical guide to usability testing. Portland, OR: Intellect Books, 1999

  16. Rubin J (1994) Handbook of usability testing. Toronto, Canada: John Wiley & Sons, Inc

    Google Scholar 

  17. Denning S, Hoeim D, Simpson M, Sullivan K. The value of thinking-aloud protocols in industry: a case study at Microsoft Corporation. Proc. Human Factors Society 34th Annual Meeting, 1285–1289, 1990

  18. Wixon D, Wilson C. The usability engineering framework for product design and evaluation. In: Handbook of human-computer interaction, 2nd Edition. New York, NY: Elsevier, 1997

  19. Daniels J, et al. Usability evaluation of two vibrotactile displays for enhanced situation awareness in anesthesiology. Proceedings of the Information Technology and Communications in Health conference (ITCH) 2007

  20. Wright M, Taekman J, Endsley M. (2004) Objective measures of situation awareness in a simulated medical environment. Qual Safety Health Care 13:65–71

    Article  Google Scholar 

  21. Garmer K, Liljegren E, Osvalder A, Dahlman S. (2002) Application of usability testing to the development of medical equipment. Usability testing of a frequently used infusion pump and a new user interface for an infusion pump developed with a human factors approach. Intl J Indust Ergonom 29:145–159

    Article  Google Scholar 

  22. Backs R, Boucsein W (2000) Engineering psychophysiology. Mahwah, New Jersey. Lawrence Erlbaum Associates, Inc

    Google Scholar 

  23. Lim J, et al. Measurement of autonomic responses to tactile and auditory stimulation. Proceedings of the 2006 Society for Technology in Anesthesia Meeting, 2006

  24. Lingard L, et al. (2004) Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Safety Health Care. 13:330–334

    Article  CAS  Google Scholar 

  25. Root R, Draper S. Questionnaires as a software evaluation tool. Proceedings of the ACM CHI 1983 Conference, pp.␣83–87, 1983

  26. Chin J, Diehl V, Norman K. Development of an instrument measuring user satisfaction of the human–computer interface. Proceedings of the ACM CHI 1988 Conference, pp. 213–218, 1988

  27. Lewis J (1995) IBM computer usability satisfaction questionnaires. Int J Human–Comput Interact 7(1):57–78

    Article  Google Scholar 

  28. Caplan S (1990) Using focus groups methodology for ergonomic design. Ergonomics 33(5):527–533

    Article  Google Scholar 

  29. Goldman A, MacDonald S. (1987) The group depth interview: principles and practice. Englewood Cliffs, NJ: Prentice Hall

    Google Scholar 

  30. Greenbaum T. (1988) The practical handbook and guide to focus group research. Lexington, MA: DC Health and Co

    Google Scholar 

  31. von Hippel E (1988) The sources of innovation. New York, NY: Oxford University Press

    Google Scholar 

  32. Neilson J, Mack R. (1994) Usability inspection methods. New York, NY: John Wiley & Sons

    Google Scholar 

  33. Usability Professionals Association Consultant Directory. Accessed online at http://www.upassoc.org/people_pages/consultants_directory April 26, 2007

Download references

Acknowledgement

The authors would like to thank Natasha McCartney for her writing and editing suggestions and an anonymous reviewer for helpful suggestions. This work was supported by a Collaborative Health Research Project grant from the Canadian Institutes of Health Research and the Natural Sciences and Engineering Research Council of Canada and was presented in part as an abstract at the 2007 Society for Technology in Anesthesia Meeting in Orlando, Florida.

Author information

Authors and Affiliations

  1. Department of Anesthesiology, Pharmacology, and Therapeutics, The University of British Columbia, Room E414A, 4480 Oak Street, Vancouver, BC, Canada, V6H 3V4

    Jeremy Daniels BASc, Joanne Lim MASc & J. Mark Ansermino MBBCh, MSc

  2. Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada

    Sidney Fels PhD

  3. School of Health Information Science, University of Victoria, Victoria, Canada

    Andre Kushniruk PhD

Authors
  1. Jeremy Daniels BASc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sidney Fels PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andre Kushniruk PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joanne Lim MASc
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Mark Ansermino MBBCh, MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy Daniels BASc.

Additional information

Daniels J, Fels S, Kushniruk A, Lim J, Ansermino JM. A framework for evaluating usability of clinical monitoring technology.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Daniels, J., Fels, S., Kushniruk, A. et al. A framework for evaluating usability of clinical monitoring technology. J Clin Monit Comput 21, 323–330 (2007). https://doi.org/10.1007/s10877-007-9091-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10877-007-9091-y

Keywords

Search

Navigation