If a sound is made and there is no smartphone present to record it, did it really occur? A growing body of researchers seem intent on that being an impossible riddle to unravel as they increasingly move already-ubiquitous smartphone monitoring into places that might cause one to scratch their head, including the bathroom and bedroom.
The smartphone’s technology dominance has, at times, created some concern. Some bemoan a loss of personal privacy and security, and a perceived detrimental impact on direct interpersonal communications. A growing body of researchers, however, believe the smartphone provides an "everyman platform" to record the sounds and movements of our mundane daily experiences and assess our lifestyle choices.
"This is similar to the business model of Facebook, which on the surface is offering a useful service, and in the background is analyzing user data for various purposes," says Takuya Maekawa, an associate professor of Osaka University’s Graduate School of Information Science and Technology, and one of the authors of "Evaluating Tooth Brushing Performance with Smartphone Sound Data," presented at the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. "Users can be asked to confirm their consent prior to transmitting data off the device," he adds.
By moving life’s ongoing evaluation process into the bathroom, the researchers believe an individual’s tooth-brushing technique can be improved with a smartphone app that monitors brushing sounds to provide individualized brushing efficiency assessments.
"We think that healthcare applications are a promising area for future research in the use of audio data collected by smartphones," Maekawa says. "There are several studies that try to detect breathing and/or estimate breathing performance to provide feedback to users when exercising and participating in sports. We think this is the first [study] to evaluate brushing performance by solely using sound data."
Maekawa and his colleagues are not alone on the smartphone audio monitoring bandwagon. Another group of researchers in Japan recently used smartphones to monitor and quantify snoring and sleep apnea, demonstrating how the devices can be a "useful tool" allowing users to self-monitor modifications such as changes in sleeping posture and fluctuations in body weight.
Also, researchers at the Georgia Institute of Technology have studied the use of smartphones to monitor the sounds of a person eating, in order to aid dietary self-monitoring and encourage healthier eating habits. "It might be possible to determine whether individuals are eating alone or with friends, and whether they are eating while working or watching television," they wrote, also suggesting participants should have the right to review the recordings and remove content before it is reviewed by professionals.
Maekawa says he began considering tooth brushing as an area for study when he noticed the sounds made when brushing various parts of the teeth varied depending on the individual and the area being brushed. He spoke to others in the Ubicomp (ubiquitous computing) field, including co-author Kazunori Nozaki, an assistant professor at Osaka University Dental Hospital. They discovered a growing body of research into the qualitative assessment of daily activities using a specialist’s knowledge to interpret sensor data.
Maekawa’s colleagues in the study, who also include Hiroo Tamagawa, an associate professor at Osaka University Dental Hospital, and Joseph Korpela, a second-year Ph.D. student at Osaka University Graduate School of Science and Technology, are currently applying for research funding to create a marketable smartphone app.
The researchers want to apply the process of gamification (the use of game design elements in non-game contexts) to the toothbrush. "We plan to create a game that gives users feedback in a more enjoyable way than simply displaying a numerical analysis of their performance," Korpela says. "Gamification encourages better habits in users."
The sound detection method outlined in the paper is intended to point out the weak points in the user’s tooth brushing. Specifically, a score is created for each region of the user’s teeth to enable an evaluation of overall stroke coverage.
The method uses hidden Markov models (HMMs) for the recognition of tooth brushing actions and to generate estimated scores of effectiveness. The next step is to apply a deep learning approach to improve accuracy. Recent studies have shown high sound recognition performance when using deep neural network-based HMMs.
In the research evaluation, 94 sessions of tooth brushing audio data were taken from 14 research participants. The HMMs were tailored to recognize seven tooth brushing activities, such as "outer surface of front teeth, rough stroke (FO-Rough)," or "inner surface of back teeth, fine stroke (BI-Fine)," or worse, "no tooth brushing activity (None)."
A dentist watched videos of the tooth brushing participants and assigned rankings based on coverage, stroke, and duration to the recorded user efforts. Up to two points were awarded for each criteria in each region, for a maximum of six points per region.
Korpela believes when combined with a dentist’s guidance and a meaningful motivation (likely gamification), the app he and his colleagues are developing will "encourage better performance in health-related behaviors [that] can affect long-term improvements."
Just don’t forget to floss, and wash your hands.
Darrell Dunn is a freelance writer and editor based in the Dallas-Fort Worth area.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment