Checklists (The Sequel), 5 Million Lives, and the Magic of Measurable Results

Last month’s New England Journal included another astounding checklist study, an international extravaganza that found nearly 50% reductions in mortality and complications after implementation of pre- and post-op surgical safety checklists.


Wow.


Coincidentally, I read the study, conducted by a research team led by surgeon/author extraordinaire Atul Gawande, on my way home from a meeting at the headquarters of the Agency for Healthcare Research and Quality (AHRQ). The AHRQ gathering brought together the advisors to a new rollout of the Hopkins/Michigan checklist program to prevent central line-associated bloodstream infections (CABSI) to 10 additional states. You remember that study, published in the NEJM in 2007: implementation of a simple 5-item checklist in more than 100 Michigan ICUs led to over 1000 lives saved.


(Parenthetically, this was the study, led by Genius-Award Winner Peter Pronovost, that the Feds tried to shut down over the failure to obtain informed consent, an action that caused me to blow my gasket last year. The subsequent broo-ha-ha, fueled in large part by readers of this blog, led to a change in the federal policy regarding informed consent for quality improvement programs. Accordingly, the new 10-state roll-out of the CABSI program is considered “quality improvement” and thus exempted from the need for individual consent from providers and patients, Heaven be Praised.)


Last month’s surgical checklist study, one of the first initiatives of the WHO’s World Alliance for Patient Safety (spearheaded by the UK’s indefatigable and charismatic Chief Medical Officer, Sir Liam Donaldson) is, if anything, even more amazing than the Hopkins/Michigan effort. Why? Instead of changing the behavior of intensivists and ICU nurses, this one involves surgeons. And instead of changing practices in ICUs in a single U.S. state, this one did it in hospitals in 8 different cities around the world, ranging from Seattle to Manila, and Auckland to Amman. It was an audacious effort, which makes its results all-the-more-remarkable.


(In fact, it is hard to fully explain the tremendous decrease in surgical deaths and complications based on the nature of the intervention, leading some to question whether the results owe to a Hawthorne effect and are replicable. We’ll see, but for now I think the study is impressive and the intervention is likely to work elsewhere, albeit perhaps not quite as well as reported in the NEJM. OK, that’s enough parenthetical paragraphs for one posting.)


Gawande articulated (a particularly apt word in the case of this gifted writer) his zeal for checklists in his New Yorker article, “The Checklist.” And Pronovost is passionate about everything – he could make you joyful about doing your laundry. But what is so impressive about both these superb leaders and the projects they spawned was something more wonky than stirring: their insistence on a rigorous measurement strategy.


Which brings me to the recent wrap-up of the 5 Million Lives Campaign, the Institute for Healthcare Improvement’s (IHI) much-touted sequel to its 100,000 Lives Campaign. As you may recall, I critiqued the earlier campaign for using fuzzy math in estimating its “lives saved,” among other things. I worried aloud that the short-term high that came from celebrating the 100,000 Lives Campaign’s achievements might prove to be as ephemeral as that achieved by crack cocaine: without measurable results, it is awfully hard to generate sustainability, and the day-after hangover can be nasty.


And so it was with the 5 Million Lives Campaign, which ended with a relative whimper, sans the press hoopla that accompanied the 100,000 Lives effort. To IHI’s credit, there was no preening this time about lives saved, or defending of largely indefensible statistics. In fact, as the Boston Globe reported in December after the campaign’s end,


[the IHI] does not have numbers to measure the effect of its efforts over the last two years, IHI vice president Joseph McCannon said… But stories of reduced infection rates, improved medication management, and better cardiac care have been flowing in from the more than 4,000 hospitals participating in the campaign, he said…

The next stage, added McCannon, was the addition of three new “planks” to the 5 Million Lives practices: linking hospitals’ patient safety efforts to cost savings, preventing nosocomial urinary tract infections, and promoting the WHO surgical checklist program described above. In addition, IHI launched a tool, known as the “Improvement Map,” to help hospitals keep track of their various quality and safety initiatives.


This is all fine, but I get the sense that the absence of results took some wind out of the IHI’s sails. (Of course, that’ll be nothing compared to the wind removed if – as has been reported in the blogosphere – IHI founder and CEO Don Berwick becomes head of the Centers for Medicare & Medicaid Services).


What is the lesson in all of this? While baking rigorous measurement into large-scale tests of quality improvement or patient safety practices costs some time and money (doubtless the reason why IHI chose a Nike-like, “Just Do It” implementation strategy), the choice to eschew such measurement comes at a high cost. When the Michigan ICU study ended in a seminal New England Journal paper, there was tremendous momentum to do more, based largely on the unassailable evidence of lives saved. Not only did AHRQ jump at the opportunity to fund a rollout to 10 more states, but two private philanthropists gave Pronovost another $2.5 million to disseminate it even more widely! We haven’t heard the upshot of the WHO surgical checklist study yet, but you can bet that it will spark more efforts to promote this and similar interventions. Not just because the premise (marrying the use of checklists with culture change) made sense or because Gawande is a wonderful writer and a great guy, but because they demonstrably worked.


At the end of the AHRQ meeting last month, all the advisory board members were asked to identify the highlight of the day. Mine was this: I was floored by the blend of passion and scientific rigor shown by Pronovost and the other members of the project team (he’s being aided by the American Hospital Association’s Educational/Research Trust, along with several leaders from the Michigan effort).


Why did I focus on this combination of passion and scientific rigor? Because too little of the former and you can’t get projects this complex and messy off the ground. And too little of the latter, you risk finishing the project and not knowing whether you’ve accomplished anything of importance.


In other words, this is a Goldilocks problem and both Gawande and Pronovost, and their superb teams and funders, have gotten it just right.

4 Responses to “Checklists (The Sequel), 5 Million Lives, and the Magic of Measurable Results”

  1. menoalittle February 13, 2009 at 1:39 pm #

    Bob,

    I share your enthusiasm for the work of these ingenious and robust researchers. When a process as simple as a checklist results in an attention to detail, lives are saved and patient safety is optimized.

    When one observes the reports of complications that occurred or may have occurred from processes of computerized medical care that have not been methodologically studied for overall outcomes (http://www.pittsburghlive.com/x/pittsburghtrib/s_611120.html), and may in fact interfere with the requisite attention to clinical detail, one can not help but wonder if financial resources are being misapplied by our government leaders at the behest of lobbyists of an industry that stands to profit from widespread use of unproven technology.

    Best regards,

    Menoalittle

  2. DrJaymez February 14, 2009 at 1:27 pm #

    I really enjoyed this post. Isn’t Atul Gawande an amazing figure? He has studied and written extensively on QI, including his book, “Better.” Now he is transforming his own field from within. What is our hospitalist equivalent of this checklist? Is it the daily internal monologue: “Does this patient still need their foley?” “Can that line come out?”?

  3. danwalter February 14, 2009 at 3:38 pm #

    Here is my Johns Hopkins patient safety experience: http://adventuresincardiology.com/

  4. Maxine May 22, 2009 at 6:18 pm #

    I’m new to the SHM and these blogs so playing catch up…

    I agree that Dr. Gawande is an amazing individual, exceptional in what he does, a clear, thoughtful and forward thinker. I have read all that he has written and implemented some of his ideas in my own hospital sphere of influence

    President Obama would do well to consider him for the Surgeon General post.

Leave a Reply