Sangam: A Confluence of Knowledge Streams

An examination of factors affecting the accuracy, reliability and validity of job analysis task ratings

Show simple item record

dc.contributor Mark A. Wilson, Chair
dc.contributor J. William Cunningham, Member
dc.contributor Bert W. Westbrook, Member
dc.contributor Michael S. Wogalter, Member
dc.creator Murphy, Kristin Freund
dc.date 2010-04-02T19:12:08Z
dc.date 2010-04-02T19:12:08Z
dc.date 2000-11-02
dc.date.accessioned 2023-02-28T17:09:17Z
dc.date.available 2023-02-28T17:09:17Z
dc.identifier etd-20001031-205007
dc.identifier http://www.lib.ncsu.edu/resolver/1840.16/5348
dc.identifier.uri http://localhost:8080/xmlui/handle/CUHPOERS/265815
dc.description This study examined several issues related to evaluating job analysis information. The first issue concerned the relationship between job analysis reliability and accuracy and the utilization of these variables to gain an estimation of job analysis validity. In this study, the performance on reliability and accuracy indices was compared to determine the extent to which such indices identify a consistent set of reliable and accurate raters. Similar to Green and Stutzman (1986), this study also compared the rating profile of the selected accurate and reliable raters with that of the entire rater population. The second issue concerned the impact of other individual differences concerning job tenure and experience on the validity of job analysis ratings. In addition, the effect of individual rater's fatigue may have on job analysis reliability was explored. Finally, the study addressed whether the distinction between in-role and extra-role behavior may help explain some of the variance in job analysis ratings. Thus, the study examined whether the nature of the task has any impact on job analysis reliability or accuracy. A significant relationship was found between reliability and accuracy. Highest mean reliabilities were found for the reliable raters as compared to accurate raters and the incumbent population. Thus, the impact of the reliability of raters on the validity of the instrument was demonstrated. A correlation analysis among reliability and accuracy scores and individual difference variables revealed a significant negative correlation between reliability and organization tenure. No significant relationships were found between education and reliability scores or between education and accuracy scores. An analysis of reliability over the course of a job analysis inventory showed that mean reliabilities trend downwards initially and slope slightly back upwards. The downwards trend may suggest that fatigue impacts the reliability of the rater over time. The sudden change in trend could indicate that point at which incumbents changed survey format (from computer to paper-pencil or vice-versa). Lastly, nature of the task appears have an impact on reliability of job analysis ratings as in-role task mean reliabilities were higher than extra-mean reliabilities. In sum, this study sought to explain the potential impact of the factors of accuracy, reliability, individual differences and the nature of the task on the validity of job analysis ratings. This study expands the knowledge base concerning inter-relationship among these factors and the extent to which this knowledge could lead to a model of selecting accurate and reliable job analysis subject matter experts (SMEs).
dc.rights I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to NC State University or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report.
dc.title An examination of factors affecting the accuracy, reliability and validity of job analysis task ratings


Files in this item

Files Size Format View
etd.pdf 111.6Kb application/pdf View/Open

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse