Health Care Reform, Research, and Child Mental Health

Advancing Children’s Mental Health: Structures, Incentives and Paradigms to Close the Research to Practice Gap

* 41 million uninsured Americans exhibit consistently worse clinical outcomes than the insured, and are at increased risk for dying prematurely (Institute of Medicine, 2002; Institute of Medicine, 2003a)

* The lag between the discovery of more effective forms of treatment and their incorporation into routine patient care averages 17 years (Balas, 2001; Institute of Medicine, 2003b)

* 18,000 Americans die each year from heart attacks because they did not receive preventive medications, although they were eligible for them (Chassin, 1997; Institute of Medicine, 2003a)

* Medical errors kill more people per year than breast cancer, AIDS, or motor vehicle crashes (Institute of Medicine, 2000; Centers for Disease Control and Prevention; National Center for Health Statistics:  Preliminary Data for 1998, 1999)

* More than 50% of patients with diabetes, hypertension, tobacco addiction, hyperlipidemia, congestive heart failure, asthma, depression and chronic atrial fibrillation are currently managed inadequately (Institute of Medicine, 2003c; Clark et al., 2000; Joint National Committee on Prevention, 1997; Legorreta et al., 2000; McBride et al., 1998; Ni et al., 1998; Perez-Stable and Fuentes-Afflick, 1998; Samsa et al., 2000; Young et al., 2001)  log

Problem Statement, Background, & Our Assumptions:

Failure to apply recent research findings about effective treatments is a critical problem that spans all areas of medicine, including child and adolescent mental health services.  According to the Institute of Medicine, the lag between discovery of a new or more effective form of treatment and its widespread application averages 17 years across all areas of medicine. In the mental health arena specifically, experts estimate that more than 50% of adults with depression do not receive appropriate care.  In the child mental health area, McLennan and colleagues noted that research-practice gaps can be considered to fall into 4 different categories – in essence one sin of omission and 3 sins of commission: 1) failing to implement procedures that have been shown to be effective; 2) implementing procedures that have been shown to be harmful; 3) implementing procedures that have no effect; and 4) implementing approaches that not been studied (McLennan et al., 2006).

In this blog we take a “big picture view” of what we believe are some of the key policy, structural, and economic problems in closing the research practice gap, and offer several potential pathways for future progress, in terms of necessary new policies, structures, incentives, and conceptual paradigms that may enable the child mental health field to shorten the research-practice gap.

Background:

Despite the general awareness of the field of the quality chasm in translating research findings into practice, in the child mental health services area there has been little explicit study of the extent and nature of this research-practice gap, in terms of formally documented reasons for the gap, and the extent to which it applies across specific childhood disorders, treatment types, provider disciplines, clinical populations, or environmental contexts.  Even within the best-studied disorder area of ADHD, evidence indicates that clinicians do not apply recent research findings developed by multiple professional and advocacy organizations.  While accreditation measures have been developed for ADHD, the National Committee on Quality Assurance studies indicate widespread failure to implement basic standards.  No measures have been developed or applied for any other area of child mental health diagnosis and treatment services, not even common problems such as adolescent depression across most settings or plans.

Why is Research Often Not Applied to Practice? In other areas of medicine, the research to practice translation gap has been more thoroughly examined (e.g., cancer, diabetes).  Generally speaking, four major reasons for translation difficulties have been identified: 1) Intervention characteristics (e.g., cost, time demands, level of staff expertise required, difficulties learning the intervention, failure to package “manualize” the approach, failure to develop the intervention considering users’ needs, failure to consider how to make the intervention self-sustaining, failing to make the intervention modular or customizable, intervention specificity to a particular setting, etc.); 2) problems with studies’ research designs (e.g., study populations not relevant or representative, failure to identify critical outcome variables for the intervention; failure to study key variables needed by policy makers and communities prior to adoption, such as cost, reach, implementation, maintenance, and/or sustainability factors); 3) problems with intervention adoption settings (e.g., competing demands, program imposed from outside, financial/organizational instability, specific needs of clients and setting, limited resources, time, or organizational support; misaligned incentives or regulations; competing prevailing practices; and challenges to intervention implementation quality), and 4) Interactions among the three above barrier types (participation barriers reduce program reach or participation, inflexible interventions, interventions not appropriate for the target population; organization and intervention philosophies not aligned, etc.).

While we agree with the general outline of these “reasons” for translation difficulties, these reasons tend to be rather molecular in focus, and in fact might mislead healthcare system organizers, policy-makers, and planners to simply continue to tweak existing interventions in order to try to close the research-practice gaps, piece by piece, bit by bit.  While we believe that such an approach has some merit, we also believe that such an approach will yield modest (if any!) results in closing the gap.  The current list of reasons does not fully consider the backdrop of larger clinical and community factors and forces, against which new evidence-based interventions must compete, if they are to be successfully implemented.  Taking these factors into account must be considered as paramount for a successful strategy in closing the research practice gaps.

We also note that the research-practice gap also partly overlaps with other known gaps in access and effectiveness, such as health care disparities as a function of ethnicity, region, and income.  These gaps must also be considered as a part of the overall difficulties in improving care and closing the research-practice gaps.

Four Key Assumptions:

The strategies we outline below are based on 4 key assumptions, or beliefs.  While we believe our assumptions are reasonable and likely correct, each assumption itself might be the subject of study and debate;  this is a task beyond our immediate scope of responsibility.

1. “Teaching Old Dogs New Tricks”: New Research Often Not Implemented By Clinicians, Even in the Absence of Barriers. Many current “evidence-based practices” in mental health assessment, prevention, and psychosocial and pharmacologic treatment are available, but are not well-implemented.  Even in instances where health care barriers are minimal (e.g., with 100% insurance and health care access), clinicians do not deliver (and families fail to receive) treatments consistent with the most recent evidence. To illustrate,

Within the well-studied disorder area of ADHD, decades of research findings are available, and treatment standards based on recent research findings have been developed by multiple professional and advocacy organizations and the NCQA.  Yet in a recent NCQA validation study across 6 health plans for the “ADHD diagnosis initiation quality indicator,” applying the simple standard of requiring an ADHD diagnosis to be captured in the follow-up claim, with >2 additional follow-up visits in the next 11 months, plans’ average visit compliance rates were 19 percent and 23 percent for commercial and Medicaid plans, respectively. Across all commercial plans providing information 2005-2007, 90% of insurers averaged less than 45% on the quality measure of achieving minimal follow-up visits after ADHD diagnosis, with the mean percentile across all health plans of less than 33%.  No evidence of change was noted across the years 2005-2007.

This example of ADHD reflects only a limited view of what is widely believed by most scientists as the much larger research-practice translation problems across all of medicine.  We note, however, that the IOM’s assumption of a 17-year research-practice gap across all of medicine may or may not be correct, and this concept of a 17-year research-practice gap has never been operationalized and rigorously applied to child mental health.  We believe that we lack any specific details of the research-practice gap, in terms of which areas are most affected, i.e., populations, disciplines, disorders, settings, etc.  For child mental health researchers, policy makers, and the general public to fully tackle this problem, the gap needs to be defined and operationalized with publicly vetted and accepted scientific measurement approaches, so that the gap indicator itself can be used as a feedback tool for the field to monitor its difficulties and measure its progress (or lack thereof).

2.  Developing New Types & Models of Health Care Practice Organizations. We believe that the current alignment of federal, state, and local economic incentives that a) pay for health and social problems but not their prevention; b) pay for clinical and educational training models that consider children’s mental health and educational outcomes as discrete rather than inter-twined, c) stove-pipe funds and responsibilities for care and outcomes within but not across systems; d) focus solely on treatment of individuals vs. the management of populations; and e) assume patients to be passive recipients of an expert-delivered, short-term, finite/discrete product or service –  all 5 of these factors distort/prevent the application of more efficient, evidence-based procedures.  Health care organizations as bureaucracies are usually organized around these distorted economic incentives, so rather than taking advantage of new information systems and findings from social neuroscience about how to best design optimal human service organizations, health care systems tend to perpetuate the older, familiar models. We believe that without fundamental changes that tie incentives to outcomes-based feedback and performance for individuals, populations, and health care organizations, change and application of EBPs will spotty at best.

To test the assumption that economic incentives will facilitate the adoption of EBPs seems reasonable, but it is important to note that previous studies indicate an uncertain relationship between economic incentives and subsequent provider/clinician behavior.  Perhaps even more importantly for child EBPs, the questions of whether EBPs make economic sense might be evaluated using a range of tools, including benefit-cost analyses (BCA) and cost-effectiveness analyses (CER). BCA provides a full accounting of the resource implications of an intervention, policy, or program. One measures both the costs and benefits of the intervention and then calculates net benefits—that is, the benefits of the intervention less its costs. If the net benefits are positive, then the intervention or treatment is desirable. Unlike BCA, cost-effectiveness analysis (CEA) does not require one to measure outcomes in dollar terms. Rather, the outcome measures remain in their natural metric (e.g., a 1-point difference on a symptom checklist or a percentage point reduction in the number of teenagers giving birth). The analyst then compares interventions or programs in terms of their added (or incremental) costs per added unit of the outcome measure (Zerbe & Dively, 1994). One could calculate such ratios for a variety of outcome measures, comparing a standard treatment vs. an EBP.

Other areas might also be examined vis-a-via EBPs, such as mining various state, insurance, or federal data sets to examine the vast differences in children’s mental health services from county to county and state-to-state, or within and across service systems, to determine factors such as cost-shifting when funds are constrained in one setting, yet result in additional costs in other settings.  Optimal vs. non-optimal treatments for disorders like ADHD might be analyzed in various data sets to show how costs accrue over time to various service settings among families with unaddressed risk factors.  Likewise, studies need to be mounted that examine how changes in the organization of how care is paid for (e.g., payment to a provider for treatment of discrete illness episodes vs. payment to a health care organization in terms of preventing illness in a population of patients).

The current lack of coupling of children’s mental health services, outcomes, financial incentives, and economic understanding of this decoupling may be most evident in current situations where only persons with substantial economic resources may receive mental health care from a child psychiatric specialist, yet may or may not have any better outcomes!

3. Moving Research from Ivory Towers to Earthen Trenches: Translating EBPs to “the Real World.” Researchers who have developed EBPs often have little know-how how to take well – established programs out into “the real world”.  In fact, follow-up interviews of well-established investigators who have developed EBPs suggest that they often have disincentives for doing so, ranging from 1) lack of understanding how to adapt interventions to make them feasible and sustainable in the real world, 2) how to develop fiscal and business models to sustain EBPs deployment, to 3) ethical restrictions imposed by IRBs and universities if the possibility exists that researchers might profit from continuing to study and deploy the intervention.   In several instances, researchers have developed a business model (sometimes in partnerships with the university), have drawn upon SBIR funding, or have set up independent businesses.  These various strategies each have a number of advantages and disadvantages, and each entails a whole new set of activities which must be learned and mastered, for which many investigators appear unprepared or unwilling to pursue.

In the area of evidence-based school mental health interventions, for example, a review by Forman, Olin, Hoagwood et al. (in press) identified 25 EBPs for which 24 intervention developers were interviewed, fiscal stability was consistently cited as the most critical factors. For example, one intervention developer stated in describing a self-perceived lack of knowledge and skill to deal with implementation and sustainability issues, “I’m just a college professor who created a program.”  The unexpected and complex processes that are set in motion when a new program is introduced require preparation, a long-term commitment to overcoming numerous challenges, and many partners, much more like starting a successful business than conducting a narrowly defined research study. Additional efforts are needed to ensure that future EBPs can be implemented and sustained in practice settings..   We believe that stable business-savvy dissemination organizations and mechanisms that can assist researchers to “take research to market” are very much needed or at the least, integrated delivery systems with motivation to provide the best possible care must be linked with scientists who create such programs.

4. Rethinking  children’s mental health: a child psychotherapist for every family, or designing healthier communities? The child and adolescent mental health system with its emphasis almost exclusively on the individual clinician patient-encounter is seriously flawed.  Attempts to close the research-practice gap by focusing solely on this encounter will not succeed, due to range of factors that will prevent full implementation of EBPs, such as stigma about identification and treatment of mental health problems, problems with access to clinical services separated from community locations such as schools, etc.  We believe that effective strategies to close the research-practice gap (as well as other treatment disparities) will ultimately require a public health approach (e.g., see IOM Prevention Report, 2009), and other research-to-practice translational efforts will be incremental at best.

Studies that could be mined to help establish this point might include several of the recent child mental health clinical trials (MTA Study, TAD Study, and others), where data show that extra-treatment factors such as parent depression, family stress, social and community factors usually explain the largest bulk of variance in outcomes, particularly over longer-term periods. RCT data from studies such as the MTA also show that access, stigma, family concerns with specific treatments, etc., often undermine optimal deployment in clinical settings.

The Big Question

In this blog, we hope to capture your ideas about 1) the problems as we see them, and 2) what you see as possible solutions!

Peter S. Jensen, MD

President and CEO

The REACH Institute

co-authored with Kelly Kelleher, Marc Atkins, and Mike Foster

It’s time for a change…

I recently had the opportunity to attend the Twenty-fourth Annual Rosalynn Carter Symposium on Mental Health Policy in Atlanta, Georgia.  At this remarkable meeting 200+ persons from across the US convened to revisit the progress that we have made as a country since Jane Knitzer’s seminal Unclaimed Children report over a quarter-century ago. This meeting highlighted presentations and discussions based on recently completed research conducted by Janice Cooper, Jane Knitzer, and their colleagues at Columbia University, Unclaimed Children Revisited: The Status of Children’s Mental Health Policy in the US.

According to Cooper and colleagues’ findings, we have witnessed meaningful shifts in the culture of care in the last 25 years, often in specific regions and as a function of strong state leadership.  However, more in-depth analysis revealed that “changes, while promising, are often severely limited in scope and shallow in depth due to lack of concerted strategic plans.”

I agree.  It is ironic, even tragic, that NIH has spent over 2 billion dollars on child mental health-related research since 1990,  but getting this information and new methods out into our communities and to our families has hardly progressed whatsoever.  We have made remarkable strides in terms of developing better diagnostic tools and more effective treatments, but as a country we have not put into place the necessary mechanisms that take correct principles and proven programs to their ultimate destination — families.  And it truly is not “rocket science!”   Successful examples exist that indicate that the necessary steps are fairly straightforward.  When a discovery is made or a new finding is developed, four steps are needed to take it to market, at least in the child health and human services arena:

• Identifying and validating successful programs with the key stakeholder groups; a discovery won’t last long if it is repudiated by other scientists, or is felt to be unacceptable, unaffordable, infeasible, or unsustainable within larger communities.
• Adapting them for wider dissemination so that they fit the cultural needs of new communities, with those communities’ buy-in.  Rarely is an innovation fully ready for prime time right out of the university investigators’ laboratory.  Lots of input, adaptation, and modifications are often needed, where the researcher learns what the community really needs and wants…which may not be exactly what the researcher did.
• Disseminating the programs with the leverage of those communities, and assisting those communities to bring additional funding sources to the table as needed; and
• Empowering those communities to continue delivering the latest, scientifically updated versions of the programs through some kind of central, non-profit, publicly oriented support function.

These 4 steps often take years, and without careful assistance, they result in what the Institute of Medicine concludes is an 18-year science-to-service translational gap in putting science to work for families.  So the likelihood of any child or youth getting state-of-art, up-to-date care for mental health problems is small, unless the family travels long distance to one of the 4-5 universities doing the cutting edge research in that area.

However, in our experience at the REACH Institute, these four steps, done strategically and systematically, can be completed in as little as 3 years, when all the key stakeholders are quickly brought together, essentially forming a science to service dissemination pipeline.

Why the delays?  When I was at NIMH from 1990-2000, where I served as the Associate Director of Child & Adolescent Research, we often saw that the first two steps (identification and adaptation) would take years, often because most of our scientists naturally do what they were hired to do, and in fact do best: test and compete various programs in the market place of ideas, and then publish these findings in scientific journals.  As scientists we tend to debate for years in the pages of our journals about best methods, while families and community health providers remain confused about best practices.  In fact, while I was at NIMH, we only hosted one NIH- sponsored Consensus Conference during the entire decade while I was there…for ADHD in 1998.  So how are competing professional scientific discplines going to reach consensus on thorny issues like teen depression, or use of medications vs. psychotherapies?   And even if consensus can be reached with scientists and consumer stakeholders, who is there for the second step?  — to adapt the beta-versions (university lab prototypes) so that they are readied for distribution across the  science-to-service pipeline (if it existed).  Adapting the interventions to “the real world” and then helping others apply them, was not then, and is not now, a common function of universities.

While still at NIMH, Lisbet Shore’s book “Common Purpose” had a great impact on me.  She described the case stories of successful national dissemination programs, and detailed the factors associated with successful disseminaton.  One common factor was that successful programs usually had a hands-on, national training and technical assistance center that systematically helped new sites learn the program, adapt it to their communities, and then provided ongoing support and evaluation.

This is why the REACH Institute was formed in 2006.  Because the ultimate success of any program depends upon whether it actually “reaches” and helps children and families, key leaders from NAMI, CHADD, MHA, and FFCMH, serve on REACH’s Board to guide this process. In addition, because of the somewhat fractious nature of our field historically, as well as concerns about commercial interests, we established REACH with a great deal of interdisciplinary input, and obtained IRS approval as a 501c3 non-profit.  Its 501c3 status allows generous individuals invested in child mental health system reforms to make non-profit donations to bring the latest methods to their local communities’ health/mental health providers.  In addition, my own time on behalf of REACH is donated pro-bono, in order to avoid any misinterpretations about personal financial gain.

Now that REACH is up and going, I believe it has something to offer all of us interested in transforming children’ mental health and reaching all children across the country.  Speaking on behalf of REACH, we are are eager to partner with individuals and organizations at local, state and national levels, and across the range of provider disciplines and consumer voices that need to be involved to solve our problems. We are eager to make all of our tools and programs available to anyone committed to transforming children’s mental health across the country.

A brief one-pager describing some of REACH’s current programs is available at the link www.TheReachInstitute.org/documents/REACHintro.pdf.  Also, please drop me an email note if you have ideas on programs and partnerships that we might follow up on together.  Or to get a more extended view, please go the homepage and browse to your heart’s content, www.TheReachInstitute.org.

Happy Holidays!    peter jensen

If you are interested other comments on these issues, go to blog catchservices.wordpress.com and add your comments about how change might be accomplished.