Carol Espy-Wilson, , Philip Resnik (UMIACS/Linguistics), John Dickinson (UMIACS/CS), Deanna Kelly (UMSoM), Shuo Chen (UMSoM)

Funding Agency

National Science Foundation




Carol Espy-Wilson is the principal investigator on a four-year, $842,431 NSF “Smart and Connected Health” grant, Using Multi-Stage Learning to Prioritize Mental Health. Her co-PIs are Philip Resnik and John Dickerson.

Deanna Kelly is funded on a separate, related $307,565 NSF grant, Using Multi-Stage Learning to Prioritize Mental Health Risk Using Evidence from Speech and Text. Associate Professor Shuo Chen of UMSoM is Kelly’s co-PI.

The team is developing technology that can monitor mental health conditions in the time between clinical appointments. The app one day could be useful to stay on top of conditions such as depression, anxiety and bipolarism.

From her years of research in signal processing and her expertise in the mechanics of speech production, Espy-Wilson has learned that the complex and neurologically based act of speaking can be a good way to detect and assess mental health issues. She and her colleagues have been working for several years to bring to market a smart phone app that can monitor speech for signs of mental illness and alert a care provider to deteriorating conditions between visits.

The new NSF funding, spread across two grants, will allow the researchers to conduct simulations, the next phase of the research.

This new phase proposes a fundamental shift in how machine learning is used to approach the problem of mental health detection and monitoring, with a technological investigation that brings together speech analysis, language analysis, and machine learning research, informed by deep clinical experience and expertise and fueled by ethically collected data.

A tiered multiarmed bandit framework will be used to provide a highly flexible way to evaluate multiple kinds of evidence in settings where there can be diverse methods for assessment that vary in cost and the value of the information they provide. As such, it is an excellent fit for the real-world problem of mental health assessment in resource-limited settings.

Investigations will include simulations of patient monitoring between clinical visits that will be informed by realistic, real-world assumptions and team members' clinical experience treating patients with schizophrenia, depression, and risk of suicide.

At the core of this project's technical approach is the recognition that the “multi-armed bandit” problem in machine learning is a good fit for the real-world scenario that mental health providers face when monitoring a population of patients in treatment: what is the best way to allocate limited resources among competing choices, given only limited information? This project develops a tiered multi-armed bandit formulation, where a succession of stages is applied to a population of patients in order to best allocate different types of resources, each with different per-patient impact but also cost.

Conceptually, tiered approaches are familiar in current medical practice. For example, patient contact typically progresses from a receptionist, to a nurse or intake coordinator, perhaps to a certified nurse practitioner, to a primary care doctor, ultimately to a specialist—each step involving corresponding increases in both the cost of the professional involved and their degree of expertise.

The tiered multi-armed bandit model developed by this award includes concerns of stochastic and adverse selection, where patients at one tier do not proceed deterministically to the next, even when explicitly selected. It also incorporates complex (e.g., non-linear such as monotone submodular) objective functions that better capture within-cohort interactions. One core strength of the tiered model is that it provides a flexible way to incorporate multiple kinds of evaluative evidence in settings where there can be diverse methods for assessment that vary in cost and the value of the information they provide. Toward that end, this project also includes both text analysis and speech analysis components that make use of ethically collected language and speech data and clinically validated assessments of mental condition.

Techniques developed under this award, while directly motivated by and tested in the mental health setting, will be useful in other settings in both healthcare as well as other settings where a "prioritization funnel" is in play, including talent sourcing and customer acquisition.