Keywords

electronic performance support, usability, human computer interaction, interface design, instructional technology

Abstract

This study evaluated the usability of two types of performance-support interfaces that were designed using informational and experiential approaches. The experiment sought to determine whether there is a relationship between usability and the informational and experiential approaches. The general population under study was undergraduate education major students from the University of Central Florida. From the general population of three educational technology instructor-led classes, 83 students were solicited to participate in the study by completing a class activity. From the general population, a total of 63 students participated in the study. By participating in the study, the students completed a task and a questionnaire. Students were predominantly English-speaking Caucasian female education majors between the ages of 19 and 20; most of them were sophomores or juniors working part time. They possessed moderately low to high computer skills and most considered themselves to have intermediate or expert Internet skills. An experimental posttest-only comparison group research design was used to test the hypotheses posited for this study. The participants were randomly assigned to either the informational interface group (X1) or the experiential interface group (X2), and the experiment was conducted electronically via a Web-based Content Management System (CMS). The observed data consisted of five outcome measures: efficiency, errors, intuitiveness, satisfaction, and student performance. Two instruments--a checklist and an online usability questionnaire--were used to measure the five dependent variables: efficiency, intuitiveness, errors, satisfaction, and student performance. The CMS was used as the vehicle to distribute and randomize the two interfaces, obtain informed consent, distribute the instructions, distribute the online questionnaire, and collect data. First, a checklist was used to assess the students' performance completing their task, which was a copyright issue request letter. The checklist was designed as a performance criterion tool for the researcher, instructor, and participants to use. The researcher and instructor constructed the checklist to grade copyright request letters and determine students' performance. The participants had the opportunity to use the checklist as a performance criterion to create the task document (copyright request letter). The checklist consisted of ten basic yet critical sections of a successful copyright request letter. Second, an online usability questionnaire was constructed based on the Purdue Usability Testing Questionnaire (PUTQ) questions to measure interface efficiency, intuitiveness, errors, and satisfaction. While these test items have been deemed important for testing the usability of a particular system, for purposes of this study, test items were modified, deleted, and added to ensure content validity. The new survey, University of Central Florida Usability Questionnaire (UCFUQ), consisting of 20 items, was implemented in a pilot study to ensure reliability and content validity. Changes to the PUTQ were modified to fulfill a blueprint. A pilot study of the instrument yielded a reliability coefficient of .9450, and the final online usability instrument yielded a reliability coefficient of .9321. This study tested two approaches to user interface design for the Electronic Performance Support (EPS) using two HTML interface templates and the information from an existing training module. There were two interventions consisting of two interface types: informational and experiential. The SPSS Graduate Pack 10.0 for Windows was used for data analysis and statistical reporting in this study. A t test was conducted to determine if a difference existed between the two interface means. ANOVA was conducted to determine if there was an interaction between the interface group means and the demographic data factored among the five dependent variables. Results of this study indicated that students at the University of Central Florida reported no differences between the two interface types. It was postulated that the informational interface would yield a higher mean score because of its implementation of HCI guidelines, conventions, and standards. However, it was concluded that the informational interface may not be a more usable interface. Users may be as inclined to use the experiential interface as the informational interface.

Notes

If this is your thesis or dissertation, and want to learn how to access it or for more information about readership statistics, contact us at STARS@ucf.edu

Graduation Date

2005

Semester

Fall

Advisor

Hirumi, Atsusi

Degree

Doctor of Education (Ed.D.)

College

College of Education

Department

Educational Research, Technology, and Leadership

Degree Program

Curriculum and Instruction

Format

application/pdf

Identifier

CFE0000807

URL

http://purl.fcla.edu/fcla/etd/CFE0000807

Language

English

Release Date

January 2006

Length of Campus-only Access

None

Access Status

Doctoral Dissertation (Open Access)

Download

Share

COinS