We read and hear much about the promise of mobile health. Crucial in the acceptance of mobile health by the clinical community is clinical credibility. And now, clinical credibility is synonymous with evidence, and just “evidence” but reliable, solid evidence. I’ve blogged before about studies of the quality of mental health smartphone apps. I missed this piece from Nature which, slightly predictably, is titled “Mental Health: There’s an app for that.” (isn’t “there’s an App for that a little 2011-ish though?) It begins by surveying the immense range of mental health-focused apps out there:
Type ‘depression’ into the Apple App Store and a list of at least a hundred programs will pop up on the screen. There are apps that diagnose depression (Depression Test), track moods (Optimism) and help people to “think more positive” (Affirmations!). There’s Depression Cure Hypnosis (“The #1 Depression Cure Hypnosis App in the App Store”), Gratitude Journal (“the easiest and most effective way to rewire your brain in just five minutes a day”), and dozens more. And that’s just for depression. There are apps pitched at people struggling with anxiety, schizophrenia, post-traumatic stress disorder (PTSD), eating disorders and addiction.
The article also has a snazzy infographic illustrating both the lack of mental health services and the size of the market:
The meat of the article, however, focuses on the lack of evidence and evaluation of these apps. There is a cultural narrative which states that Technology = Good and Efficient, Healthcare = Bad and Broken and which can give the invocation of Tech the status of a godterm, pre-empting critical thought. The Nature piece, however, starkly illustrates the evidence gap:
But the technology is moving a lot faster than the science. Although there is some evidence that empirically based, well-designed mental-health apps can improve outcomes for patients, the vast majority remain unstudied. They may or may not be effective, and some may even be harmful. Scientists and health officials are now beginning to investigate their potential benefits and pitfalls more thoroughly, but there is still a lot left to learn and little guidance for consumers.
“If you type in ‘depression’, its hard to know if the apps that you get back are high quality, if they work, if they’re even safe to use,” says John Torous, a psychiatrist at Harvard Medical School in Boston, Massachusetts, who chairs the American Psychiatric Association’s Smartphone App Evaluation Task Force. “Right now it almost feels like the Wild West of health care.”
There isn’t an absolute lack of evidence, but there are issues with much of the evidence that is out there:
Much of the research has been limited to pilot studies, and randomized trials tend to be small and unreplicated. Many studies have been conducted by the apps’ own developers, rather than by independent researchers. Placebo-controlled trials are rare, raising the possibility that a ‘digital placebo effect’ may explain some of the positive outcomes that researchers have documented, says Torous. “We know that people have very strong relationships with their smartphones,” and receiving messages and advice through a familiar, personal device may be enough to make some people feel better, he explains.
And even saying that (and, in passing, I would note that in branch of medical practice, a placebo effect is something to be harnessed, not denigrated – but in evaluation and study, rigorously minimising it is crucial) there is a considerable lack of evidence:
But the bare fact is that most apps haven’t been tested at all. A 2013 review8 identified more than 1,500 depression-related apps in commercial app stores but just 32 published research papers on the subject. In another study published that year9, Australian researchers applied even more stringent criteria, searching the scientific literature for papers that assessed how commercially available apps affected mental-health symptoms or disorders. They found eight papers on five different apps.
The same year, the NHS launched a library of “safe and trusted” health apps that included 14 devoted to treating depression or anxiety. But when two researchers took a close look at these apps last year, they found that only 4 of the 14 provided any evidence to support their claims10. Simon Leigh, a health economist at Lifecode Solutions in Liverpool, UK, who conducted the analysis, says he wasn’t shocked by the finding because efficacy research is costly and may mean that app developers have less to spend on marketing their products.
Like any healthcare intervention, an App can have adverse effects:
When a team of Australian researchers reviewed 82 commercially available smartphone apps for people with bipolar disorder12, they found that some presented information that was “critically wrong”. One, called iBipolar, advised people in the middle of a manic episode to drink hard liquor to help them to sleep, and another, called What is Biopolar Disorder, suggested that bipolar disorder could be contagious. Neither app seems to be available any more.
And even more fundamentally, in some situations the App concept itself and the close relationship with gamification can backfire:
Even well-intentioned apps can produce unpredictable outcomes. Take Promillekoll, a smartphone app created by Sweden’s government-owned liquor retailer, designed to help curb risky drinking. While out at a pub or a party, users enter each drink they consume and the app spits out an approximate blood-alcohol concentration.
When Swedish researchers tested the app on college students, they found that men who were randomly assigned to use the app ended up drinking more frequently than before, although their total alcohol consumption did not increase. “We can only speculate that app users may have felt more confident that they could rely on the app to reduce negative effects of drinking and therefore felt able to drink more often,” the researchers wrote in their 2014 paper13.
It’s also possible, the scientists say, that the app spurred male students to turn drinking into a game. “I think that these apps are kind of playthings,” says Anne Berman, a clinical psychologist at the Karolinska Institute in Stockholm and one of the study’s authors. There are other risks too. In early trials of ClinTouch, researchers found that the symptom-monitoring app actually exacerbated symptoms for a small number of patients with psychotic disorders, says John Ainsworth at the University of Manchester, who helped to develop the app. “We need to very carefully manage the initial phases of somebody using this kind of technology and make sure they’re well monitored,” he says.
I am very glad to read that one of the mHealth apps which is a model of evidence based practice is one that I have both used and recommended myself – Sleepio:
One digital health company that has earned praise from experts is Big Health, co-founded by Colin Espie, a sleep scientist at the University of Oxford, UK, and entrepreneur Peter Hames. The London-based company’s first product is Sleepio, a digital treatment for insomnia that can be accessed online or as a smartphone app. The app teaches users a variety of evidence-based strategies for tackling insomnia, including techniques for managing anxious and intrusive thoughts, boosting relaxation, and establishing a sleep-friendly environment and routine.
Before putting Sleepio to the test, Espie insisted on creating a placebo version of the app, which had the same look and feel as the real app, but led users through a set of sham visualization exercises with no known clinical benefits. In a randomized trial, published in 2012, Espie and his colleagues found that insomniacs using Sleepio reported greater gains in sleep efficiency — the percentage of time someone is asleep, out of the total time he or she spends in bed — and slightly larger improvements in daytime functioning than those using the placebo app15. In a follow-up 2014 paper16, they reported that Sleepio also reduced the racing, intrusive thoughts that can often interfere with sleep.
The Sleepio team is currently recruiting participants for a large, international trial and has provided vouchers for the app to several groups of independent researchers so that patients who enrol in their studies can access Sleepio for free.
This is extremely heartening – and as stated above, clinical credibility is key in the success of any eHealth / mHealth approach. And what does clinical credibility really mean? That something works, and works well.
A Medical Education 