Assessing Multidimensional Complex Decision Making With Situational Judgment Tests

Keywords

complex decision making; situational judgment test; validity

Abstract

The decisions that humans make are often multidimensional and complex, and system AIDS have been developed to support decision-making. However it is often difficult to evaluate the decision making of both the human or these system decision AIDS as decision making is a skill that is difficult to quantify. Traditionally, decision-making skill is assessed primarily through tasks or questionnaires. The challenge with those approaches is that measures of decision-making skill are often unidimensional. The Situation Judgment Test (SJT) is multidimensional and comprises scenarios obtained from accounts of real-world experiences, each with response options from which respondents select the most effective response. Although SJTs often show criterion-validity, this is typically obtained from post-hoc analysis. The present study seeks to evaluate the dimensionality of SJT scenarios, which are based on real-world decisions that are akin to the type of decisions that decision support systems seek to aid with. Fifteen SJT scenarios were administered to 94 participants, along with several measures of dimensions deemed relevant to real-world decision making. Most of the dimensions were able to predict performance on at least one scenario. Certain dimensions seemed to predict performance on more scenarios than others did. The results indicated that the SJT scenarios were able to incorporate some dimensions relevant to decision making. Future research should examine other measures for evaluating SJTs prior to using them to assess decision making.

Publication Date

6-22-2016

Publication Title

2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support, CogSIMA 2016

Number of Pages

49-55

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/COGSIMA.2016.7497785

Socpus ID

84981295156 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84981295156

This document is currently not available here.

Share

COinS