Unmasking Impression Management: Large Language Models for Detecting Socially Desirable Responding in Open-Ended Situational Judgment Tests

Author

• Barret Vermilion, University of Central FloridaFollow

Keywords

impression management, socially desirable responding, situational judgment tests, large language models, natural language processing, personnel selection

Subject Categories

Human Factors Psychology | Industrial and Organizational Psychology | Psychology

Abstract

Socially desirable responding (SDR) — the tendency to present oneself in an overly favorable light — remains a persistent validity concern in industrial-organizational psychology. Contemporary theory frames SDR as multidimensional, distinguishing agentic impression management (AM; inflating competence, dominance, and achievement) from communal impression management (CM; inflating warmth, cooperation, and prosocial qualities). Existing measures of AM and CM rely on self-report, which captures these constructs only at the level of generalized trait-like tendencies rather than as they naturally unfold in applicants' own language. This dissertation proposes a text-based alternative: detecting AM and CM in open-ended situational judgment tests (SJTs) of Big Five personality using natural language processing (NLP). Study 1 developed and validated psycholinguistic dictionaries capturing linguistic features of AM and CM. Trained raters generated seed word lists, which were empirically expanded using WordNet and evaluated for semantic reliability and convergent/discriminant validity using transformer-derived cosine similarity. Study 2 experimentally induced AM and CM by instructing participants to present themselves as ideal candidates for prototypically agentic or communal roles when completing open-ended SJTs (N ≈ 880 per condition), which were trained on a numeric measure capturing AM and CM. Three contemporary NLP modeling strategies were evaluated: zero-shot decoder-only large language models (GPT-4o, Llama, Mistral), a fine-tuned encoder-only transformer (DeBERTa-v3-small), and a hybrid model augmenting fine-tuned transformer embeddings with the psycholinguistic dictionaries. Both encoder and decoder models demonstrated meaningful convergent and discriminant validity, with predicted scores differentiating AM from CM as a function of job type and SJT content. The custom dictionaries provided no incremental validity beyond transformer embeddings alone, suggesting that fine-tuned transformers already encode theoretically relevant linguistic content. These findings establish NLP-based scoring of open-ended responses as a viable, nuanced complement to self-report, with implications for how impression management is measured and understood in personnel selection.

Completion Date

2026

Semester

Summer

Committee Chair

Shiyang Su

Degree

Doctor of Philosophy (Ph.D.)

College

College of Sciences

Department

Psychology

Format

PDF

Document Type

Dissertation

Language

English

STARS Citation

Vermilion, Barret, "Unmasking Impression Management: Large Language Models for Detecting Socially Desirable Responding in Open-Ended Situational Judgment Tests" (2026). Graduate Studies Theses and Dissertations 2026. 372.
https://stars.library.ucf.edu/gradstudies_etd_2026/372