Smartphones are now capable of telling users’ bodies about their physical activity, heart rate or sleep amount. New research from the People Aware Computing Lab may be adding user alertness to that list.
For the past two years, a group of four researchers in the Cornell lab have been working on AlertnessScanner, an Android application that utilizes a smartphone’s front-facing camera to assess a user’s alertness levels.
AlertnessScanner is one of many research projects being carried out in the People Aware Computing Lab, headed by Prof. Tanzeem Choudhury, information science, which focuses on developing mobile sensing systems for capturing, learning and interpreting human activities.
“Alertness has a great impact on our performance,” said Vincent Tseng, a Ph.D. student working on the project. “We are interested in seeing if we can measure and understand the changes in people’s alertness unobtrusively using smartphones, which people regularly interact with.”
The app does so by measuring the change in a user’s pupil size through analysis of a burst of pictures captured by the smartphone’s camera when the user unlocks their phone.
According to the researchers, measuring pupil size has been shown to be effective in determining a person’s fatigue and alertness since the size of a person’s pupil is controlled by two complementary nervous systems — the sympathetic nervous system and the parasympathetic nervous system — that cause the pupil to dilate when a person is alert and to constrict when they are drowsy.
To account for the varying distances that a user’s eye may be from the camera at the time of image capturing, AlertnessScanner utilizes PIR, pupil-to-iris ratio, as a measure for pupillary response. Such method uses the user’s iris size, which remains constant in diameter, as a reference to determine pupil size.
As part of this task, the researchers developed a computer-vision-based algorithm that extracts the PIR measure of a user’s pupils using eye detection, iris segmentation and pupil segmentation.
The research team hopes that by making alertness information available, people can meaningfully influence and interact with the world around them.
“The goal is to know the peak and low point of our alertness,” Tseng said. “Then, we can also learn other things about ourselves. For example, students can determine their prime time for accomplishing tasks, and surgical clinicians or employees with demanding jobs can see when they experience highest level of alertness.”
So far, the team has conducted two in-the-wild studies involving Cornell students as research subjects to evaluate the effectiveness of the application.
Participants in the study were given Google Nexus 5 phones with AlertnessScanner downloaded to use. Over the course of three weeks, the app collected sleep data and captured images from the users. Additionally, users were asked to complete a Psychomotor Vigilance Task, an objective method of determining alertness. At the end of the study, the data collected by the app is compared with the data from collected from PVT.
The results of their studies suggest that AlertnessScanner is accurate and reliable in evaluating alertness and that it is possible for the application to replace other less convenient methods of alertness assessment.
In going forward, the team wants to utilize its findings from this project to gain better understanding about people’s productivity and performance.
“We’re currently in the process of developing a study such that we can get signals from phone data and behavior data to know when people perform better and see how the curve changes over time,” Tseng said.