A comparative assessment and gap analysis of commonly used team rating scales

    Authors

    C. M. Pugh; E. R. Cohen; C. Kwan;J. A. Cannon-Bowers

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    J. Surg. Res.

    Keywords

    Rating scale; Teamwork; Taskwork; Patient safety; Medical education; Performance assessment; SURGEONS NONTECHNICAL SKILLS; RELIABILITY; SYSTEM; Surgery

    Abstract

    Background: The purpose of this article was to conduct a gap analysis of important team constructs that may be absent in widely used team assessments. Methods and materials: Two assessment tools with known validity evidence (1) Non-Technical Skills for Surgeons (NOTSS) and (2) the Cannon-Bowers Scale were used to evaluate 11 teams of surgical residents (n = 33) performing simulated laparoscopic hernia repairs. Faculty raters' scores were used to compare the surveys and assess validity and reliability. Raters' detailed observation notes were used to indicate important behavioral constructs that were missing from the team rating scales. Results: When assessing inter-item correlations (reliability) four of five NOTSS' scale items had significant correlations (r = 0.9-1.0, P < 0.05) with the Cannon-Bowers items. While the correlations were only noted for three of six Cannon-Bowers items, in each instance the same four of five NOTSS items correlated with the three Cannon-Bowers items, thus providing further validity evidence for both scales. When evaluating the gap, key emerging themes included the need to focus on critical team errors, individual team member contributions, task performance, and overall team performance. These gaps, plus items from the NOTSS and Cannon-Bowers scales, were incorporated into a new rating scale. Conclusions: Despite continued evidence of validity and reliability, there were several behavioral constructs that were not represented when using the NOTSS and Cannon-Bowers scales. Critical team errors, individual team member contributions, task performance, and overall team performance appear important in our ability to understand teams and teamwork. (C) 2014 Elsevier Inc. All rights reserved.

    Journal Title

    Journal of Surgical Research

    Volume

    190

    Issue/Number

    2

    Publication Date

    1-1-2014

    Document Type

    Article

    Language

    English

    First Page

    445

    Last Page

    450

    WOS Identifier

    WOS:000339801800010

    ISSN

    0022-4804

    Share

    COinS