Dr. Bolton conducts research for the US Air Force
Published June 19, 2018 This content is archived.
Dr. Matthew Bolton has been awarded a grant to do research for the Air Force Research Laboratory. His research is on psychometric rating scales.
Bolton says, "humans use introspection to convert some attribute of their psychological state or subjective experience into a number on a predetermined scale. These scales are used everywhere from review scores on Amazon, to disease diagnoses, to the engineering and design of safety-critical HAI. They are also widely used in scientific research in psychology, medicine, and engineering. In this work, we are introducing a new method for determining what the level of measurement is of psychological concepts captured by psychometric rating scales.
This work is significant for three reasons. First, trust in automation, mental workload, and situation awareness are critical measures in the engineering of nearly any system that relies on humans for its safe and efficient use. This will continue to be the case as autonomous systems are given increasing amounts of responsibility and expected to perform as integral members of human-machine teams. The work is extremely important, as it will give analysts the ability to handle psychometrics at the level of measurement that will produce meaningful insights with the maximum amount of analytic power. Second, the psychometric ratings of trust, workload, and situation awareness have been used in significant amounts of research, including predictive modeling efforts at varying levels of measure. As such, this work has the potential to either validate or invalidate findings derived from these varying treatments of the measures. This could have a profound, and possibly disruptive, impact on the engineering of HAI moving forward. Third, our method for determining the level of measurement of psychological phenomena is novel and itself constitutes a significant contribution. By using this new approach to evaluate and compare psychometrics, we will gain insights into the relationships between these metrics as well as individual differences in the way people produce ratings. Because psychometric ratings are used throughout the psychological, engineering, medical, and testing disciplines, numerous scales could be evaluated using this new approach in the future. Thus, this research will open up many possible avenues of research."
