Team Training Evaluation in Healthcare: Increasing the Effectiveness of Neonatal Resuscitation Teams

Izhak Nadler (2011). Team Training Evaluation in Healthcare: Increasing the Effectiveness of Neonatal Resuscitation Teams PhD Thesis, School of Information Technol and Elec Engineering, The University of Queensland.

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
s4132052_phd_finalabstract.pdf Final abstract application/pdf 52.84KB 3
s4132052_phd_finalthesis.pdf Final thesis application/pdf 2.61MB 10
Author Izhak Nadler
Thesis Title Team Training Evaluation in Healthcare: Increasing the Effectiveness of Neonatal Resuscitation Teams
School, Centre or Institute School of Information Technol and Elec Engineering
Institution The University of Queensland
Publication date 2011-07
Thesis type PhD Thesis
Supervisor Prof. Penelope Sanderson
Dr. Helen Liley
Total pages 115
Total colour pages 15
Total black and white pages 100
Language eng
Subjects 11 Medical and Health Sciences
Abstract/Summary Patient safety is a key goal of healthcare practice. Training should be aimed at improving practice in support of patient safety. For training to be effective it must be properly evaluated. In many emergency clinical situations, teams of clinicians must provide the required care because the number of activities to be performed in a short time frame is beyond the capability of a single person to perform. Despite the importance of teamwork in emergency situations, the evaluation of team training effectiveness in healthcare is currently limited. Studies do not test all aspects of training and the measures used are often subjective, so the validity of evaluation is limited. The focus of this research was to improve methods and measures that are used to evaluate the effectiveness of team training in healthcare. Two specific aims were defined: (1) to develop objective methods to score clinicians’ performance in simulated scenarios and in clinical practice and (2) to identify challenges and to highlight the importance of implementing Kirkpatrick’s training evaluation typology in the context of healthcare practice. Kirkpatrick’s typology is the most common typology for training evaluation and it consists of four stages: (1) immediate reaction of trainees to the training that they experienced, (2) learning and improvement in trainees’ skills and behaviors as measured in the training environment, (3) transfer of changed skills and behaviors to the practice environment, and (4) changes in organizational effectiveness. The area of practice chosen for this study is neonatal resuscitation. Neonatal resuscitation requires a team effort in which several clinicians provide intensive treatment to a neonate over a short time frame. At the time this research was carried out, there was no formal curriculum for team training in this area of practice. Therefore, it was necessary to develop training interventions likely to improve teamwork between clinicians whose scope of practice includes neonatal resuscitation. Yet, as indicated above, the focus was always to test the evaluation methods and the measures and not to test the effectiveness of the training interventions themselves. Brunswik’s probabilistic functionalism and the Accuracy Score, a measure emerging from judgment analysis studies, were used to develop an additional measure of teamwork effectiveness. Clinicians’ ability to monitor and assess the clinical state of a patient during stressful emergency situations is challenged, but it is argued that effective teamwork, where information is shared effectively, can compensate for such challenges. The Accuracy Score reflects how accurate clinicians’ assessments are compared with the actual state of a simulated patient. The accuracy of clinical assessments that clinicians made about a simulated patient is proposed as a measure to test teamwork effectiveness. The research consisted of two main parts: (1) a study in the training environment to evaluate Kirkpatrick’s Stages One and Two, and (2) a study in the practice environment to evaluate Kirkpatrick’s Stages Three and Four. For the first part, 17 clinicians from Mater Mothers’ Hospital (MMH), Brisbane, Australia, participated in a simulator-based experiment designed to test novel team performance measures. Participants were divided into three experimental groups that each experienced a different type of training. At Stage One, participants completed questionnaires probing their response to the training. Satisfaction was high across the three groups with no significant differences. At Stage Two, participants provided Apgar scores to describe the clinical state of the mannequin after viewing recorded scenarios and after hands-on scenarios. Hands-on scenarios took place before and after the training. The Accuracy Score was used to compute the accuracy of clinicians’ Apgar score assessments. The Accuracy Score was sensitive to variations in task demands (p<0.01) and to variation in the content and the delivery method of the training (p<0.05). For the second part, a study was conducted on the evaluation of training effectiveness with respect to debriefings that were mainly intended to improve teamwork and clinical practice. All clinicians performing neonatal resuscitations in the MMH were invited to attend training in the form of weekly debriefings that included presentation of actual resuscitations. Discussion that followed the presentation was focused on teamwork. It was expected that there would be changes in team performance due to the debriefings. At Stage Three, a panel of experts reviewed recordings of resuscitations that were performed during the debriefing period. The experts were blinded to the date that each recording was made. The scores for all the items relating to teamwork showed trends toward improvement, but the score for only one item improved significantly (p<0.05). There was no clear trend for changes in clinical performance, but the score for one item deteriorated (p<0.05). At Stage Four, four measures were tested, but given the relatively limited change in performance at Stage Three, changes at Stage Four were not expected. Indeed, changes at Stage Four were not identified. At this point it cannot be concluded which measures are most suitable for evaluation of team training for neonatal resuscitation at Stage Four. The main achievements of this research corresponded to its two aims. First, a method that may increase the objectivity of scoring clinicians’ performance was successfully pilot tested. Second, Kirkpatrick’s typology was implemented to test measures for evaluating team training in neonatal resuscitation. This research provides new directions for studying team training. The outcomes of the research may increase the validity of team training evaluation and consequently may enhance clinical practice and patient safety. Future studies are needed to test the applicability of this research findings to other types of training and in other areas of practice.
Keyword Neonatal resuscitation
Team training
Judgment analysis
Accuracy Score
Video recording
Video analysis
Apgar scores
Neonatal Resuscitation Program (NRP)
Additional Notes 32-33, 42-43, 69-72, 109-115

Citation counts: Google Scholar Search Google Scholar
Created: Mon, 14 Nov 2011, 16:30:09 EST by Mr Izhak Nadler on behalf of Library - Information Access Service