Reweighting AT-SAT to Mitigate Group Score Differences

Reweighting AT-SAT to Mitigate Group Score Differences Andrew R. Dattel Raymond E. King Civil Aerospace Medical Institute Federal Aviation Administration Oklahoma City, OK 73125

July 2006

Final Report DOT/FAA/AM-06/16Office of Aerospace Medicine

Washington, DC 20591

NOTICE

This document is disseminated under the sponsorship

of the U.S. Department of Transportation in the interest

of information exchange. The United States Government

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine technical reports are available in full-text from the Civil Aerospace Medical Institute’s publications

Web site:

1347be14866fb84ae45c8d30/library/reports/medical/oamtechreports/index.cfm

i

Technical Report Documentation Page

1. Report No.

2. Government Accession No.

3. Recipient's Catalog No.

DOT/FAA/AM-06/16

4. Title and Subtitle

5. Report Date

July 2006

Reweighting AT-SAT to Mitigate Group Score Differences

6. Performing Organization Code

7. Author(s)

8. Performing Organization Report No.

Dattel AR, King RE

9. Performing Organization Name and Address

10. Work Unit No. (TRAIS)

FAA Civil Aerospace Medical Institute

P.O. Box 25082

11. Contract or Grant No.

Oklahoma City, OK 73125

12. Sponsoring Agency name and Address

13. Type of Report and Period Covered

Office of Aerospace Medicine Federal Aviation Administration 800 Independence Ave., S.W.

Washington, DC 20591

14. Sponsoring Agency Code

15. Supplemental Notes

Work was accomplished under approved subtask HRR-523.

16. Abstract

The Air Traffic Selection and Training (AT-SAT) test battery is the selection tool for applicants for Air Traffic Control Specialist (ATCS) positions within the Federal Aviation Administration (FAA) who have not previously been employed as an air traffic controller. AT-SAT is an aptitude test developed to predict the likelihood of successfully learning ATCS skills.Before operational use, however, concerns were raised about the low passing rate of incumbent (who are fully trained and certified) ATCS personnel (who participated in the initial research) and score differences between groups, which could result in adverse impact (possible unfair discrimination). To address these concerns, the subscores of AT-SAT were

reweighted, and the additive constant was changed to yield a new total score. The present study compares the original and new scoring methods using data from 724 developmental ATCSs who volunteered to take AT-SAT. An average increase of 4.86 points was found with the new scoring method; the notional passing rate (achieving a score 70) changed from 58.8% to 80%. American Indian/Alaskan Native, Hispanic, and black participants showed the greatest average increase in overall scores, 6.97, 6.98, and 7.02, respectively. The increase in scores of Hispanic and black participants was significantly higher than the increase in scores for white participants [F(4, 689) = 6.186, p < .001]. However, a chi square analysis showed no differences between groups for the number of participants whose failing score with the original scoring method changed to a passing score with the new scoring method. Additionally, a Spearman rank correlation coefficient of .85 was found between the two scoring methods, indicating that the ranking of inpidual participants did not change significantly. Moreover, no differences were found between groups in rank ordering of the two scoring methods. No significant gender differences were found between the scoring methods, with the scores for males increasing an average of 4.58 points and scores for females increasing an average of 5.67 points under the new weighting method. This study found that the new weighting formula has benefited all groups and is likely to reduce the potential of adverse impact.

17. Key Words

18. Distribution Statement

Aptitude Testing, Computerized Testing, Personnel Selection

Document is available to the public through the Defense Technical Information Center, Ft. Belvior, VA 22060; and the National Technical Information Service, Springfield, VA 22161

19. Security Classif. (of this report)

20. Security Classif. (of this page)

21. No. of Pages

22. Price

Unclassified

Unclassified 12

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

本文来源：https://www.bwwdw.com/article/16vl.html