Suicide Prediction Models in the Military Health System

Photo by Cpl. Tyler Viglione
By Brad Belsher, Ph.D.
September 30, 2019

There is growing interest for the Department of Defense (DoD) to implement computer-generated algorithms that use health care data to identify service members at heightened risk for suicide. The Psychological Health Center of Excellence conducted a systematic review of suicide prediction models (SPMs), and the findings were recently published in the journal “JAMA Psychiatry.” The review synthesized all of the existing research to provide decision makers and the public with an overview of the current evidence on SPMs.

To put the results of this review in context, it’s important to note the accuracy of any prediction, be it based on a clinical screening tool or complex computer model, depends on the frequency of the condition being predicted. In general, frequent events are easier to predict, relatively rare events are harder to predict, and extremely rare events are essentially unpredictable. Suicide mortality among active-duty service members falls in the relatively rare category at 21 deaths per 100,000 service members (2017 Department of Defense Suicide Event Report) and, therefore, is very difficult to predict. This means that any randomly identified service member will have 0.02 percent chance of dying from suicide. Suicide screeners attempt to increase this accuracy but do not perform very well.

SPMs attempt to increase prediction accuracy by using large-scale data sets – with hundreds to thousands of variables – and complex statistical procedures to identify a cohort of high-risk patients that have a greater likelihood of dying by suicide. For example, DoD investigators developed a prediction model that improved prediction accuracy to 1 percent. This means if this algorithm were implemented in standard practice, one out of every 100 soldiers flagged as high-risk would likely die by suicide, and conversely, 99 out of every 100 soldiers flagged would not die by suicide. Notably, more than half of service members who die by suicide would not be flagged by this model.

Our review found SPM prediction accuracy was similar across military, veteran, and civilian health systems. We also found no research that had rigorously evaluated the benefits, harms, feasibility, or costs of actually applying SPMs to patients (see Table 1 for examples of these and other considerations). Based on these findings and the lack of research beyond model development, we concluded that it would be premature to implement SPMs across large health systems.

Table 1. Considerations that Should be Addressed Prior to SPM Implementation

 

Domains

Examples

Acceptability

Will clinicians, clinic leaders, and other staff support SPMs and the associated interventions given the increased burden this may place on resources? What are the attitudes toward SPMs among service members and family members?

Feasibility

What is the feasibility of implementing this intervention based on current resources and workflows (e.g., SPM management, information sharing, care management procedures for positive flags and negative flags)?

Effectiveness

Does this intervention reduce suicide mortality?

Harms

Are there harms to using SPMs given the high rates of misclassifications and the familial/medical/occupation ramification of being flagged positive? Are there potential harms to patients who are flagged as lower risk?

Cost-effectiveness

Are the benefits and resource requirements of this approach a good value compared to current practices or alternative strategies?

Ethical considerations

What are the rights of the individuals whose data is being used to communicate their risk status? 

Since publication of our systematic review, some who commented on the article deemed our conclusions to be pessimistic. We, however, consider them to be responsibly cautious. Some have argued treatment recommendations exist for other conditions with lower prediction rates. For example, cancer-reducing medications, such as tamoxifen, are recommended for women with at least a 3 percent risk for breast cancer. Research has also demonstrated these medications are relatively safe and can reduce cancer risk. Unfortunately, we do not have something comparable to tamoxifen for suicide prevention. The recently updated VA/DoD clinical practice guideline on suicide makes only one strong recommendation for treatment of suicide risk: cognitive behavioral therapy (CBT), which may be effective in reducing suicide attempts. The cost-benefit differences between dispensing medication and offering up to 12 sessions of CBT across large patient cohorts is considerable.

Another major concern is the potential for adverse consequences of falsely flagging service members as at high risk for suicide. Service members’ employment is interlinked with their health care system, and there are different rules about health information than in the Department of Veterans Affairs or civilian health care systems. This enables health information to be shared between the medical side and line side to ensure service members are fit for duty. In the DoD environment, flagging large cohorts of service members as high risk for suicide based on a 1 percent predictive accuracy rate could have harmful effects on their careers (and by extension, their families) and their general well-being. Before any SPM goes live in the military health system, more research is needed to better evaluate these potential harms and benefits. 

The history of health care is full of examples of initiatives rolled out prior to adequate assessments of benefits, harms, and costs. Decision makers must make use of the best evidence available at the time, along with other factors, to determine whether or not to fast-track an intervention into routine care. Based on the results of our review, we’re cautious about advocating for the full-scale implementation of SPMs at this time. We’re also cautiously optimistic science will continue to advance our ability to more effectively prevent suicide, especially in light of the unprecedented amount of ongoing suicide research.

Dr. Belsher is the chief of research translation and integration at the Psychological Health Center of Excellence. His primary areas of focus include deployment-related mental health, systematic review methodologies, health services research, dissemination of evidence-based mental health practice, and collaborative care.


The views expressed in Clinician's Corner blogs are solely those of the author and do not necessarily reflect the opinion of the Psychological Health Center of Excellence or Department of Defense.


Comments

  • I am very interested in this method of predicting suicidal behavior

  • It’s about time

Add new comment

PHCoE welcomes your comments.

Please do not include personally identifiable information, such as Social Security numbers, phone numbers, addresses, or e-mail addresses in the body of your comment. Comments that include profanity, personal attacks, or any other material deemed inappropriate by site administrators will be removed. Your comments should be in accordance with our full comment policy regulations. Your participation indicates acceptance of these terms.

All ideas will be considered, but may not be accepted.