Who Knows best

An interesting article in a recent New York Review of Books. The article, “Health Care: Who Knows Best”, by Jerome Groopman, a professor of medicine at Harvard. The article focuses on an internal Executive Office dispute on the central issue of how best to secure what is known in medicine as “Best Practices”, or “Evidence-based Medicine”. The basic concept is not much in dispute. Evidence-based medicine is the seemingly unarguable idea that medical practice ought to be based on the best clinical evidence available, evidence most usefully obtained through rigorous clinical trials. What is the alternative, one might ask? Well, much of medical practice is based on what might be called anecdotal evidence, i.e., the physician’s professional experiences over time shape that physician’s approach to treatment. Physicians acquire a substantial amount of both theoretical and clinical knowledge during their extensive education and training. On entering their own practices, they begin applying that knowledge base. As they gain clinical experience, their clinical knowledge base expands, as does their theoretic knowledge from their continuing medical education.

This growing knowledge base sometimes departs from the best evidence-based practice guidelines for many reasons. One of the major reasons for the departure is what has come to be known as “defensive medicine”, i.e., the practice of medicine in which physicians practice so as to minimize the risks of malpractice lawsuits. Another increasingly important distorting effect is health insurance—specifically what will be covered and what will not be covered by health insurance. Other factors include the physician’s personal experiences with certain treatment modalities, or therapeutic interventions. Over time, then, some practices begin to diverge from what the best evidence-based medical approaches would suggest.

So, then what is the dispute on this issue? Cass Sunstein, a professor of law and head of Obama’s Office of Regulatory Affairs, and his colleague Richard Thaler, a professor of behavioral science and economics at the University of Chicago, argue that what they call a “nudge” approach should be employed in any health care overhaul measure that attempts to infuse evidence-based medical approaches into the health care delivery system. By “nudge”, they argue that a strictly “libertarian” approach should emphasize the advantages of evidence-based practice, but that the final choices ought be left to the people themselves. They suggest that nudges are not mandates, and that behavior should not be forcefully directed by changing economic incentives.

On the opposite side of this argument is Peter Orszag, currently head of the Office of Management and Budget. Orzsag argues precisely the opposite. He would implement a system in which comparative effectiveness research (evidence-based medicine) would be combined with aggressive promulgation of standards and changes in financial and other incentives. Doctors and hospitals that follow the federal standards of effective medicine would receive more money and favorable public assessments. Those who deviate would receive less money and be rated as providers of “poor medicine.”

When I read this article, my thought was, "Oh no . . . OMB strikes again.” Mr. Orszag in his preferred approach, is following a line of thought long practiced within his federal agency, and one that, in my view, eventually destroys the effectiveness of whatever approach is being pursued.

OMB is not only the President’s budget manager (they gather up all the budgets proposed by the federal agencies and then impose their own will to construct a single federal budget). In addition, the “M” part of OMB is charged with introducing and overseeing “evidence-based” management approaches throughout government. During the eight years I spent running an evaluation office in the Department of Health and Human Services, I watched OMB implement and then eventually kill off a number of at least theoretically interesting management concepts, Two come to mind immediately. Peter Drucker, the management guru, had advocated an approach he called “Management by Objectives”. In that approach, managers would direct resources in such a way as to maximize the probability of achieving stated, measurable management objectives. The idea was that they would negotiate with their superiors on the management objectives considered useful and necessary to secure higher level goals. Arriving at an agreed-upon objective plan, the managers would then set to work devising plausible plans to achieve their objectives. Upper level managers would oversee by measuring achievement of objectives. This approach was a potentially useful approach to managing almost any venture, so OMB at some point decided to mandate the approach throughout government. But OMB’s approach to mandating was to promulgate a directive telling the agencies they were henceforth to implement, in this case, the Management by Objectives” approach. Since OMB never had the staffing, the energy, or the wits to effectively oversee their implementation, the approach eventually fell into disrepair. Agencies proposed objectives that were often as not, not real objectives but rather means to some objectives, and the entire approach fell into the category of a bad joke. It turned out to be easier to game the system than it was to implement it faithfully, and game it they all did.

When Jimmy Carter became president in 1976, he ordered federal agencies to implement a system called “Zero-Based Budgeting.” Again, the idea was not especially arguable. His approach suggested that one would obtain more effective federal agency budget if each year, the agency would construct their budgets with a zero-base in mind. That is, consider your budget anew each year. Are the issues/problems still real, is your approach/program(s) still plausible? Are the problems being resolved/managed? So, following the president’s lead, OMB mandated the Zero-Based Budgeting approach amidst much fanfare about how this would transform the federal budgets. But like most OMB management directives, this one followed the same path. Agencies over time figured out how to construct their budgets with OMB formulaic procedures, and nothing much changed.

And it is not the case that federal managers are opposed to good management practices. Like most managers, they would prefer doing a good job and being rewarded for doing so. But they also are realistic about their jobs, and they will always opt for courses of action that minimize risk to themselves and their agencies over riskier courses. And, as pointed out by Dr. Groopman, in his article, such approaches tend to work best in limited circumstances, and on issues favoring fairly mechanical approaches. He argues that, for example, the procedure whereby a catheter is inserted into a blood vessel can be reduced to a standard approach that does in fact minimize the risks of infection. But he argues, “once we depart from such mechanical procedures and impose a “best practice” on a complex malady, our treatment is too often inadequate. He cites a number of problems that beset a system when standardized procedures are imposed on complex situations:

• Overconfidence bias – the tendency to overestimate our ability to analyze information, make accurate estimates and project outcomes (see almost anything deduced and acted upon by the CIA)

• Confirmation bias – the tendency to discount contradictory data, staying wed to assumptions, despite conflicting evidence.

• Focusing illusion – the tendency to base our predictions on a single change in the status quo.

Having worked in the field of program evaluation over a long period, I observed all of these biases both within government and externally as a consultant to government. Our own observation of the failures of George W. Bush’s “No Child Left behind Act”, a dismal failure by almost any measure, is a classic case in point. The unwillingness to fully fund the program and the striking use of negative incentives brought on gaming approaches up to and including simply faking the numbers.

Many of these sweeping mandates, intended to reduce complex social systems to simple, unitary ideas fail partly because one cannot deal effectively with such complex organisms as our public education system, or our health care system with such incentives-based systems. But they also fail through inattention. I call it the “Autopilot Bias”. Under the autopilot bias, we devise some system, implement it through a directive, and then walk away from it, as though the problem is solved. We need to understand that the real world does not operate according to autopilot rules. We need to manage our affairs every hour of every day. We need to continuously bring in objective data, employ that data to challenge our current approaches and then modify them so as to achieve a new balanced approach—the way a real pilot flies an airplane in heavy weather.

A final thought. Mr. Orszag’s approach would also introduce the very thing the detractors of health care reform are most fearful of—that their health care would be removed from the mind and hands of their doctor, and placed in the hands of some bureaucrat in Washington. That we can live without.