A New Approach to Evaluating Mental Health Apps

Based on software engineering, informatics, and clinical knowledge and experiences, a group of researchers has proposed an evaluation framework to assist clinicians in evaluating the utility, safety, and efficacy of smartphone apps for psychiatry.

More than 165,000 healthcare-related mobile apps are currently available, among which the largest category of apps focuses on mental health disorders, covering everything from addiction to depression and schizophrenia. “Evaluating an app requires new considerations that are beyond those employed in evaluating a medication or typical clinical intervention,” write the researchers in a commentary published in The Journal of Clinical Psychiatry.

However, the efficacy of most smartphone apps have not undergone rigorous scientific review, according to commentary co-author Peter Yellowlees, Professor of Psychiatry at the University of California at Davis and an expert in technology use in the clinical setting. “While patients have access to an exponentially increasing number of apps, the research literature has not kept pace,” He said. “But this lack of data has not held back the high level of industry and consumer interest.”

Among this ever-growing collection of mental health-focused apps, only 14 that cover major depressive disorder or bipolar disorder were examined in a recent literature review. Of them, only half had been reviewed for people with psychosis. And for those seven, little efficacy, safety, or clinical outcome data existed in the published literature.

According to Yellowlees and colleagues’ commentary, psychiatrists are left with two options for considering apps and other consumer devices for clinical care:

1. Avoid using these apps and suggest to patients that they do the same because of the lack of evidence supporting their utility or efficacy.
2. Accept that many patients already use these apps and will continue to do so.

For those who choose the second option, the team recommends a framework to consider when evaluating all “ASPECTS” of an app: whether the app is Actionable, Secure, Professional, Evidence-Based, Customizable and TranSparent:

• Actionable -- To be actionable, an app should collect data, but it must be data that can be valuable and clinically useful. A psychiatrist should consider how app data will be incorporated into clinical decision-making and how the data will inform care. He noted that in the future, it will be increasingly valuable for some categories of apps to seamlessly integrate with electronic health records and complement clinical practices.
• Secure -- Laws mandate that health information be secure, among them the Health Insurance Portability and Accountability Act (HIPAA). Psychiatrists should examine whether apps are password protected or biometrically authenticated. Patient data should be encrypted in case the mobile device is stolen or hacked.
• Professional -- Apps should be in line with professional standards for clinical use, including legal and ethical standards. HIPAA is a federal law and in part requires strict protection and confidential handling of protected health information, and severe penalties for violation. Other laws protecting privacy may vary state to state.
• Evidence-Based -- Apps with little or limited data may be risky to use. There are already documented cases in which apps designed for reduction in alcohol intake led to increased alcohol use. Caveat emptor -- let the buyer beware.
• Customizable -- One size does not fit all where apps are concerned. When considering an app for clinical use, psychiatrists should look for those that offer more customizable and flexible features. Patients and clinicians are more likely to be invested in and adhere to something they created together.
• TranSparent -- Apps should openly report how data is collected, stored, analyzed, used and shared. This is critical in selecting an app for clinical care. If there is uncertainty about how an app is using a patient's health care data, then there is uncertainty in any conclusions or recommendation that app may offer.

“The framework presented here is important, as it offers a flexible tool that clinicians and patients can use together to make more informed decisions about whether to use or not use a smartphone app or other mobile health technology,” said lead author John Torous, a clinical fellow in psychiatry at Beth Israel Deaconess Medical Center and the Harvard Medical School and chair of the American Psychiatry Association Workgroup on Smartphone App Evaluation.. “While both patients and clinicians know the right questions to ask about a new medication or pill, sometimes they may not be aware of all the best questions to ask about an app. With this framework we hope to guide them towards a more informed discussion.”