Keywords

Risk Management, Risk Communication, Confidence Level, Uncertainty

Abstract

A review of the literature regarding risk management and effective risk communications identified that very few researchers have addressed risk assessment confidence levels when using risk scoring methods. The focus of this research is to develop a definition of risk assessment confidence levels for use in internal project management communications and to evaluate its usefulness. This research defines risk assessment confidence level as "The degree of certainty that the likelihood or consequence score (assigned by the risk assessor) reflects reality." A specific level of confidence is defined based on the types of analyses that were conducted to determine the risk score. A survey method was used to obtain data from a representative sample of risk assessment professionals from industry and academia to measure their opinion on the usefulness of the defined risk assessment confidence levels. The survey consisted of seven questions related to usefulness--four questions addressed the importance of stating confidence levels in risk assessments and three addressed the usableness of the proposed confidence level. Data were collected on the role and experience level of each of the respondents and the survey also included a comment section to obtain additional feedback. The survey generated 364 respondents representing a broad variety of roles associated with decision making and risk management with experience levels from fairly new to experienced risk assessors. The survey data were analyzed by calculating the proportion of respondents who gave negative, neutral and positive responses to the survey questions. An examination of the roles of the survey respondents indicated that no single group was dominant. A non-parametric Kruskal-Wallis test generally failed to reject the hypothesis that the means of the survey response distributions were identical. There was one exception which indicated that there are differences based on role and by inspection of the responses, it appears that decision makers, academics, and others more strongly support the need for confidence level information to reduce the difficulty in making risk based decisions in projects. The survey responses at a confidence level of 95% have a range of errors from 3.84 to 4.97%. Based on the results of the survey, 77 - 83% of those surveyed indicated agreement that knowing the confidence the assessors have in their assessment is important and would improve a management decision. The survey showed that 60 - 86% of the respondents agreed that the confidence levels and their definitions as presented in the survey were usable. The question with the lowest agreement (60%) was related to the way in which the individual levels were defined. The ad-hoc comments provided in the survey were divided into eleven groups based on similarity of the subject of the comment and then examined for common themes. These added additional insight into the results and useful information for future research efforts. This research validates that the use of risk assessment confidence levels is considered to be useful in project risk management. The research also identified several potential areas for future work, including determining the appropriate number of confidence levels that should be defined, refining the definition of the individual confidence level definitions, examining historical perspectives of whether the risk assessments were accurate, examining the concept of shiftability of risk assessments, further research on communication of variability of risk assessments, and research into the usefulness of risk matrices.

Notes

If this is your thesis or dissertation, and want to learn how to access it or for more information about readership statistics, contact us at STARS@ucf.edu

Graduation Date

2008

Advisor

Pet-Armacost, Julia

Degree

Doctor of Philosophy (Ph.D.)

College

College of Engineering and Computer Science

Department

Industrial Engineering and Management Systems

Degree Program

Industrial Engineering

Format

application/pdf

Identifier

CFE0002352

URL

http://purl.fcla.edu/fcla/etd/CFE0002352

Language

English

Release Date

December 2008

Length of Campus-only Access

None

Access Status

Doctoral Dissertation (Open Access)

Download

Share

COinS