Data collection for the Transportation Security Administration
Recently, two projects for the Transportation Security Administration (TSA) have succeeded in collecting data from Transportation Security Officers (TSOs) that are, in many ways, unparalleled in the research community. These projects, Performance Decrement and Passenger Threat Detection Resolution (PTDR) were aimed at answering different questions for different duties at the checkpoint. As such, the methods for data collection were quite different.
The primary question in the Performance Decrement project had to do with how long TSOs can interrogate X-Ray images of passenger carryon items without a break before performance degrades. To address this question, we attempted to replicate the environment in which TSOs make these decisions as closely as possible. We created custom software that emulated the look and feel of the Rapiscan AT-2 systems located at checkpoint around the country, received over 1000 raw x-ray image files from TSA, and then captured 31 or 32 image products for each of those raw x-ray files using a Rapiscan emulator. We also wanted to understand how TSOs interrogated these images, so we integrated a 3-camera infrared eye tracking system into the emulator and we located the entire system at 6 checkpoints around the country over the course of 6 months (2 weeks per airport), collecting ~3 hours of data from 187 TSOs. This data collection resulted in over 80 million eye tracking data points and terabytes of human-emulator interaction data (image manipulation tool use, threat/no threat decisions, decision times).
The PTDR project was focused on identifying the characteristics of TSOs that are predictive of performance on the job in all checkpoint duties except the X-Ray. Thus, we are investigating whether any of 33 cognitive and personality measures (most normed and validated, two Sandia-created measures (touch aversion and Sandia Progressive Matrices)); such as the Big 5 personality traits, analogical reasoning, and perspective-taking; would predict TSO performance at the Travel Document Checker, the Divest Officer, the Dynamic TSO (who performs bag checks, pat-downs), the Advanced Imaging Technology, or the Walk-through Metal Detector duty stations. We collected over 1,000 responses on these measures from 385 TSOs at 8 different airports around the country using cellular iPads and Survey Monkey. This process took between 3.5 and 7.5 hours for each TSO to complete, and wound up totaling ~2200 man hours of TSO time. Data collection took approximately 6 months of calendar time. We are building statistical models relating their responses on these measures to performance metrics TSA collects about them including scores on practical skills examinations, knowledge of standard operating procedures, and (for divergent validity) scores on Threat Image Projections, which are an operational measure of X-Ray effectiveness.
Contributors to the success of these two projects (alphabetical order):
PI: Ann Speed
Glory Emmanuel Avina , Kristin Divis, Michael Haass, Mike Heister, Lauren Hund, Robert Kittinger, Kiran Lakkaraju, Jina Lee, Shanna McCartney, JT McClain, Monique Melendez, Kate Rodriguez, Fred Rothganger, Austin Silva, David Stracuzzi, Derek Trumbo, Michael Trumbo, Ed Vieth, and Christy Warrender, Andy Wilson