Submitted by Adrian Sandler, MD
The mismeasure of medicine: the promise and tyranny of measurement
Every three months, I receive patient satisfaction data based on post-visit telephone surveys. Once again I am trying to make sense of a graph entitled "Key Drivers of Excellence" displaying the quality of my care. I think it says that 71% of the sampled patients rated me "excellent." This seems pretty good, but puts me only at the 36th percentile for percent excellent - compared with whom? Are most other doctors much more excellent than I am? Many of my patients thought I was only very good. I am conflicted. On the one hand, I am genuinely motivated to sharpen my empathic listening skills. On the other hand, I suspect that one or two of those who considered me to be less than excellent were doctor-shopping and drug-seeking and didn’t get exactly what they wanted from me. On balance, I have to say that my colleagues and I do not give these data much credence. We all want to take excellent care of our patients, and it is hard to see that these patient satisfaction measures add to that intrinsic motivation.
Our hospital has moved from straight salary to production based compensation (measuring RVU widgets of work) to a complex formula based on RVUs and quality metrics. The quality metrics – accounting for 10-15% of our compensation – include some system-wide priorities (improving access to care, performing med reconciliation) and some metrics that each group develops each year. In the past 3 years, our DBP practice has chosen metrics such as increasing the use of Vanderbilt scales at follow-up ADHD visits, increasing the percentage of children <6 years of age with ADHD referred to behavior therapy first, and improving our motivational interviewing skills. This commitment to ongoing QI is a very positive development in healthcare. The opportunity each year to set a worthwhile and meaningful goal, commit ourselves to doing things differently, measure our progress, and receive an incentive payment for doing so is undoubtedly a win-win proposition. The problem is that it takes an army of managers to implement projects, and to monitor and report data. The data are usually weak and unreliable, but the managers and providers seem happy with the results and the exercise is repeated the next year. "Performance-based reimbursement" seems a grand name for all this.
Clearly, there is a critical need in our field for discussion and consensus about meaningful process and outcome measures. Although a broad range of outcome measures may be needed to monitor and report outcomes of care, a generic approach that emphasizes functional impairments and participation may be applicable to the vast majority of children with different diagnoses. The World Health Organization’s International Classification of Functioning, Disability and Health (ICF) provides a useful framework for measurement of outcomes (http://www.who.int/classifications/icf/en/). Our 2017 guest lecturer Dr Peter Rosenbaum will address this in depth in Cleveland Oct 13-16. Save the dates!
There are some promising developments on the measurement front in SDBP. The DBP Research Network is conducting research on patient-reported outcome measures (PROMIS) http://www.healthmeasures.net/explore-measurement-systems/promis. Stay tuned for more information. My group is developing a Clinical Functional Improvement Scale (CFIS), and we hope to disseminate this freely for others to use in their clinics. SDBP is engaged in the development of a clinical guideline on the care of children with complex ADHD, under the leadership of Bill Barbaresi. This work is critically important, but progress is slow and incremental.
Some pediatric professional societies, including American Academy of Neurology/Child Neurology Society (AAN/CNS) and American Academy of Cerebral Palsy and Developmental Medicine (AACPDM) have moved to develop measurement sets and common data elements to capture process and outcomes. https://www.childneurologysociety.org/resources/resources-detail-view/child-neurology-quality-measurement-set-public-comment-(respond-by-sept-28)
How do you measure if a health care provider is really going that extra mile for their patients, providing the exemplary individualized care that truly makes a difference? Will these measurement sets give rise to robust measures that can be reliably reported? Will these drive us to excellence in our clinical care? Or will clinicians tweak their EHRs to appear “compliant” and to create the illusion of progress? I am reminded of Stephen Jay Gould’s seminal book The Mismeasure of Man https://en.wikipedia.org/wiki/The_Mismeasure_of_Man. Gould cautioned us about reducing complex concepts into measurable small entities and then placing too much stock in those flawed entities. Will the tyranny of measurement ultimately undo its promise in health care? Time will tell...but in the meantime I would be interested in your responses on the Discussion Board!
Volume 20, Issue 3, 2016
|Past President's Column: Esther H. Wender, MD|
|Past President's Column: Karen Olness, MD|
|SDBP/JDBP Media Highlights Blog|
|IDPA Congress Mumbai 2017|
|Childhood Disintegrative Disorder (CDD)|
|From the Trenches|