Pancreatic cancer has one of the worst prognoses — with a five-year survival rate of — in part because there are no telltale symptoms or non-invasive screening tools to catch a tumor before it spreads.

Now, 91̽�� researchers are developing an app that could allow people to easily screen for pancreatic cancer and other diseases — by snapping a smartphone selfie.

uses a smartphone camera, computer vision algorithms and machine learning tools to detect increased bilirubin levels in a person’s sclera, or the white part of the eye. The app is described in a to be presented Sept. 13 at , the Association for Computing Machinery’s International Joint Conference on Pervasive and Ubiquitous Computing.

One of the earliest symptoms of pancreatic cancer, as well as other diseases, is jaundice, a yellow discoloration of the skin and eyes caused by a buildup of in the blood. The ability to detect signs of jaundice when bilirubin levels are minimally elevated — but before they’re visible to the naked eye — could enable an entirely new screening program for at-risk individuals.

In an initial clinical study of 70 people, the BiliScreen app — used in conjunction with a 3-D printed box that controls the eye’s exposure to light — correctly identified cases of concern 89.7 percent of the time, compared to the blood test currently used.

“The problem with pancreatic cancer is that by the time you’re symptomatic, it’s frequently too late,” said lead author , a doctoral student at the Paul G. Allen School of Computer Science & Engineering. “The hope is that if people can do this simple test once a month — in the privacy of their own homes — some might catch the disease early enough to undergo treatment that could save their lives.”

BiliScreen builds on earlier work from the UW’s , which previously developed , a smartphone app that screens for newborn jaundice by taking a picture of a baby’s skin. A  in the journal Pediatrics showed BiliCam provided accurate estimates of bilirubin levels in 530 infants.

In collaboration with 91̽��Medicine doctors, the UbiComp lab specializes in using cameras, microphones and other components of common consumer devices — such as smartphones and tablets — to screen for disease.

The blood test that doctors currently use to measure bilirubin levels — which is typically not administered to adults unless there is reason for concern — requires access to a health care professional and is inconvenient for frequent screening. BiliScreen is designed to be an easy-to-use, non-invasive tool that could help determine whether someone ought to consult a doctor for further testing. Beyond diagnosis, BiliScreen could also potentially ease the burden on patients with pancreatic cancer who require frequent bilirubin monitoring.

In adults, the whites of the eyes are more sensitive than skin to changes in bilirubin levels, which can be an early warning sign for pancreatic cancer, hepatitis or the generally harmless . Unlike skin color, changes in the sclera are more consistent across all races and ethnicities.

Yet by the time people notice the yellowish discoloration in the sclera, bilirubin levels are already well past cause for concern. The 91̽��team wondered if computer vision and machine learning tools could detect those color changes in the eye before humans can see them.

“The eyes are a really interesting gateway into the body — tears can tell you how much glucose you have, sclera can tell you how much bilirubin is in your blood,” said senior author , the Washington Research Foundation Entrepreneurship Endowed Professor in Computer Science & Engineering and Electrical Engineering.  “Our question was: Could we capture some of these changes that might lead to earlier detection with a selfie?”

BiliScreen uses a smartphone’s built-in camera and flash to collect pictures of a person’s eye as they snap a selfie. The team developed a computer vision system to automatically and effectively isolate the white parts of the eye, which is a valuable tool for medical diagnostics. The app then calculates the color information from the sclera — based on the wavelengths of light that are being reflected and absorbed — and correlates it with bilirubin levels using machine learning algorithms.

To account for different lighting conditions, the team tested BiliScreen with two different accessories: paper glasses printed with colored squares to help calibrate color and a 3-D printed box that blocks out ambient lighting. Using the app with the box accessory — reminiscent of a headset — led to slightly better results.

Next steps for the research team include testing the app on a wider range of people at risk for jaundice and underlying conditions, as well as continuing to make usability improvements — including removing the need for accessories like the box and glasses.

“This relatively small initial study shows the technology has promise,” said co-author Dr. , a professor in the 91̽��Medicine Department of Pediatrics whose father died of pancreatic cancer at age 70.

“Pancreatic cancer is a terrible disease with no effective screening right now,” Taylor said. “Our goal is to have more people who are unfortunate enough to get pancreatic cancer to be fortunate enough to catch it in time to have surgery that gives them a better chance of survival.”

Co-authors include Allen School undergraduate student Megan A. Banks, research study coordinator Lauren Phillipi and assistant professor of medicine .

The research was funded by the National Science Foundation, the Coulter Foundation and endowment funds from the Washington Research Foundation.

For more information, contact the research team at uwbiliscreen@gmail.com or at atm15@cs.washington.edu.

Newborn jaundice: It’s one of the last things a parent wants to deal with, but it’s unfortunately a common condition in babies less than a week old.

Skin that turns yellow can be a sure sign that a newborn is jaundiced and isn’t adequately eliminating the chemical . But that discoloration is sometimes hard to see, and severe jaundice left untreated can harm a baby.

91̽�� engineers and physicians have developed a smartphone application that checks for jaundice in newborns and can deliver results to parents and pediatricians within minutes. It could serve as a screening tool to determine whether a baby needs a blood test – the gold standard for detecting high levels of bilirubin.

“Virtually every baby gets jaundiced, and we’re sending them home from the hospital even before bilirubin levels reach their peak,” said , a 91̽��professor of pediatrics and medical director of the newborn nursery at 91̽��Medical Center. “This smartphone test is really for babies in the first few days after they go home. A parent or health care provider can get an accurate picture of bilirubin to bridge the gap after leaving the hospital.”

The research team will present its results at the Association for Computing Machinery’s in September in Seattle.

“This is a way to provide peace of mind for the parents of newborns,” said , a 91̽��associate professor of computer science and engineering and of electrical engineering. “The advantage of doing the analysis in the cloud is that our algorithms can be improved over time.”The app, called , uses a smartphone’s camera and flash and a color calibration card the size of a business card. A parent or health care professional would download the app, place the card on her baby’s belly, then take a picture with the card in view. The card calibrates and accounts for different lighting conditions and skin tones. Data from the photo are sent to the cloud and are analyzed by machine-learning algorithms, and a report on the newborn’s bilirubin levels is sent almost instantly to the parent’s phone.

A noninvasive jaundice screening tool is available in some hospitals and clinics, but the instrument costs several thousand dollars and isn’t feasible for home use. Currently, both doctors and parents assess jaundice by looking for the yellow color in a newborn’s skin, but this visual assessment is only moderately accurate. The 91̽��team developed BiliCam to be easy to use and affordable for both clinicians and parents, especially during the first several days after birth when it’s crucial to check for jaundice.

Jaundice, or the yellowing of the skin, can happen when an excess amount of bilirubin collects in the blood. Bilirubin is a natural byproduct of the breakdown of red blood cells, which the liver usually metabolizes. But newborns often metabolize bilirubin slower because their livers aren’t yet fully functioning. If left untreated, severe jaundice can cause brain damage and a potentially fatal condition called .

The 91̽��team ran a clinical study with 100 newborns and their families at 91̽��Medical Center. They used a blood test, the current screening tool used in hospitals, and BiliCam to test the babies when they were between two and five days old. They found that BiliCam performed as well as or better than the current screening tool. Though it wouldn’t replace a blood test, BiliCam could let parents know if they should take that next step.

“BiliCam would be a significantly cheaper and more accessible option than the existing reliable screening methods,” said , lead author and a 91̽��doctoral student in computer science and engineering. “Lowering the access barrier to medical applications can have profound effects on patients, their caregivers and their doctors, especially for something as prevalent as newborn jaundice.”

The researchers plan to test BiliCam on up to 1,000 additional newborns, especially those with darker skin pigments. The algorithms will then be robust enough to account for all ethnicities and skin colors. This could make BiliCam a useful tool for parents and health care workers in developing countries where jaundice accounts for many newborn deaths.

“We’re really excited about the potential of this in resource-poor areas, something that can make a difference in places where there aren’t tools to measure bilirubin but there’s good infrastructure for mobile phones,” Taylor said.

Within a year, the researchers say BiliCam could be used by doctors as an alternative to the current screening procedures for bilirubin. They have filed patents on the technology, and within a couple of years hope to have Federal Drug Administration approval for the BiliCam app that parents can use at home on their smartphones.

Other members of the research team are and , 91̽��doctoral students in computer science and engineering; of Southern Methodist University; and of the 91̽��pediatrics department. Southern Methodist University is a collaborator on the machine learning design and app development.

This research is funded by the Coulter Foundation and a National Science Foundation Graduate Research Fellowship.

###

For more information, contact the research team at bilicam@cs.washington.edu.