Bureaucracy Run Amok: Can Checklists Kill?

As you may know, I’ve argued that that the quality and safety of healthcare have traditionally been underregulated. But regulators are like patients with Parkinson’s: it’s hard to get them unglued, but once they’re moving, it’s hard to stop them.

Welcome to Exhibit A.

Last month, I described Atul Gawande’s thrilling New Yorker article recounting the seminal work by Peter Pronovost and his Johns Hopkins colleagues in Michigan. By implementing checklists to prompt people to follow safety procedures (like using barrier precautions during catheter insertions), they were able to slash the infection rate in over 100 Michigan ICUs. The results were tremendous savings – of over 1500 lives and $200 million. The original Pronovost study was published a year ago in the New England Journal of Medicine – AHRQ Patient Safety Network, the federal government’s patient safety portal, deemed it an “instant classic,” a status granted only 1-2 articles each year.

So you’d think we’d be talking today about how to migrate this work to other states, or to operating, delivery, or emergency rooms. Instead, the news is far more depressing. The study has been shut down by a handful of bureaucrats, acting in the name of patient protection. Huh?

The shocking saga was recounted by Gawande in a Sunday NY Times op-ed piece, and analyzed nicely in a handful of blog postings by Maggie Mahar (here and here) and others (here). In a nutshell, a whistleblower – apparently concerned that patients and providers were not consented and IRB approval was not obtained at each of the 70 hospitals – alerted the federal Office of Human Research Protections, the office whose job it is to protect patients from the next Tuskegee, Nuremberg, or Vioxx. But from checklists?

As Maggie (who has done some behind-the-scenes digging) describes, OHRP began its crusade by sending threatening letters to Michigan and Hopkins. Hopkins investigators and campus leaders protested, arguing that they had received approval for the study from their own Institutional Review Board (IRB), which waived patient and provider consent requirements or the need for individual hospital IRB approvals. But OHRP strongly disagreed, insisting that the study needed to be approved by every participating hospital’s IRB, and that every participating patient and provider needed to provide informed consent.

So instead of saving more lives, Pronovost and his colleagues are presently jumping through bureaucratic hoops, getting approval from 70 Michigan IRBs. Some hospitals doubtless lack IRBs (since they have no research infrastructure) and will choose not to participate. Meanwhile, the study’s data collection is more frozen than Lake Huron.

Peter Pronovost is a can-do, eternally optimistic guy (you can’t be as effective as he is if you’re not), and he and Hopkins have the wherewithal to plow ahead. I’m more worried about the chilling effect of this ruling on quality improvement work elsewhere. Gawande shares this concern.  In a series of email exchanges with me, he made the following points (all quotes are Atul’s, except for my insertions in italics):

“(1) The central difficulty for those of us trying to improve care is to change how people practice medicine day to day. We’re trying to ingrain in medical and surgical culture that we will more consistently adhere to at least the minimum standards of care, and to do this we will likely need to use checklists and other strategies, track our performance, learn from the results, and share the lessons as a regular part of everyday practice.
[Parenthetically, I’ll add that Atul made this point brilliantly in another New Yorker article.]

(2) This should not require IRB approval. It’s hard enough to get clinicians to do this as it is. For example, following Peter’s lead, I am working on introducing a surgical safety checklist and a key part is helping everyone with data collection to ensure success. My local Harvard School of Public Health IRB considered this non-research. The federal ruling disagrees with this logic. If the IRB had insisted on review, we’d have to get approval in every hospital signing up, which would be impossible. Similarly, Peter’s efforts to extend his program to the whole of the US, Spain, Netherlands will be simply impossible – just from requiring local IRB review. (Keep in mind, Peter did submit to his own IRB – it judged his work non-research and that judgment was the target of the federal ruling). That’s not what we want for patient care.
[A clarification: according to Pronovost, the Hopkins IRB actually did deem the Michigan work to be “research” – it’s just that they did not feel that it was feasible or necessary to get informed consent from the involved participants or individual hospital IRB approval.]

(3) The agency’s language in finding against Johns Hopkins is forceful in saying all involved institutions IRBs must review and that the agency thinks the failure to obtain consent from patients and providers is itself a violation. The question of patient consent is NOT the central issue. I’m sure they’d back down on that if local IRBs came down differently. The fundamental issue, though, is whether being systematic about clinical care requires doctors and hospitals to get special permissions from all the local IRBs – and it shouldn’t. This would be crazy, a hindrance to good care, and costly. 

(4) This does not mean that quality improvement is ethics free. The responsibility for clinical care and quality have always been with clinicians – including responsibility to protect privacy. Just because we make a shift from doing things by the seat of our pants to following clinical standards more scientifically does not absolve us of those same ethical clinical responsibilities. But it should not require people to have to seek additional permissions. This is the way care should be expected to be done – and that should not be made more difficult to do.

(5) HOW then can we act? Part of the question is whether there is room under current legislation for the agency to issue a different clarification of the rules than the one it is currently preparing. The Hastings Center thought there is room. So I actually do think pressure from Congress and higher ups at HHS would have real impact. Specifically, we’d want to ask congressional representatives to request that the Office for Human Research Protections let quality improvement research proceed unencumbered and reverse the ruling on Peter’s program [I have some addresses for you at the end of this posting]. Work to monitor, study and improve performance should become the norm in medicine, and doctors and hospitals should be expected to pursue this work – without federal research regulations blocking the way.”

I’ve already heard from several colleagues doing QI projects who now wonder whether they need to stop collecting data – or even discontinue their project altogether – if they haven’t consented the patients or providers or obtained IRB approval at every practice site. Atul describes a Nigerian physician learning statistics in a Harvard Public Health School class. “As a class project, he came to me for mentorship in looking at the patterns of his hospital’s surgical death and complication rates… Given the ruling against Johns Hopkins, however, we were advised (when I stupidly inquired) that he could NOT do this research work without submission to an IRB – likely at both Harvard and at each institution. It’s hard to see how this ruling could mean otherwise. But this is a 4-week class. The result: the project was dropped. That can’t be good for patients.”

Just think about it. We used to believe that hospital-acquired infections were an inevitable consequence of hospital care – bad things happen to good people. Then research, mostly the collection of day-to-day clinical data by committed people looking for improvement opportunities, demonstrated that certain commonsensical interventions, like more religious hand washing, could largely prevent these infections. Furthermore, we learned that using antibacterial alcohol gel rubs works well and is easier to perform than traditional hand washing.

Let’s say I now want to put an alcohol hand gel dispenser in every patient’s room, and encourage people to use it – with checklists, teamwork training… hell, I’ll drink the stuff if that’s what it takes. Do I need to obtain IRB approval before I install the dispensers? Or before I measure my infection rates after installation? Do I need the consent of the docs and nurses to “participate” in this “intervention”? How about the patients?

I think you can see where this takes us.

Having written about the unanticipated consequences of quality and safety practices and critiqued (along with Pronovost) some aspects of the IHI’s 100,000 Lives Campaign for vigorously promoting practices with weak evidence, I recognize that these are not easy questions. Even commonsensical practices can backfire, and Pronovost believes, as do I, that serious quality improvement interventions – when accompanied by a hypothesis and data collection – are research. The question here is one of balance and feasibility. If the OHRP ruling stands, then what we’ve ostensibly done is place a huge bureaucratic tax on quality improvement work – a tax extracted in money, time, hassles, and frustration. The results will be predictable: less good QI work, fewer participating providers and institutions, fewer smart young people drawn to careers in quality and safety, and ultimately more harm and deaths.

In the end, the question is this: will more patients benefit from the protections afforded by extra bureaucratic obstacles to QI work, or by wider dissemination of rigorous quality improvement projects and studies? If you want to see a close race, look to Huckabee vs. McCain, or Obama vs. Clinton. But don’t look here: this contest is a landslide.

***
If you agree, there’s something you can do. Please write to one or all of the following:

  • Your US representative (click here)
  • Your US senator (click here)
  • And the Office for Human Research Protections:

Kristina Borrer, PhD
Director, Division of Compliance Oversight, OHRP
kristina.borror@hhs.gov    AND

Ivor Pritchard, PhD
Acting Director, OHRP
ivov.pritchard@hhs.gov

10 Responses to “Bureaucracy Run Amok: Can Checklists Kill?”

  1. Brad F January 11, 2008 at 10:07 am #

    Bob,
    You write:

    The agency’s language in finding against Johns Hopkins is forceful in saying all involved institutions IRBs must review and that the agency thinks the failure to obtain consent from patients and providers is itself a violation. The question of patient consent is NOT the central issue. I’m sure they’d back down on that if local IRBs came down differently.

    Can you further clarify further. For the little guy doing QI work at local hospitals, the central question IS whether we need to obtain consent from each participant. As things stand now, if IRB’s are keeping up with this story, they will turn down virtually every request for an intervention, as I think you allude to, if we dont get the individual consent. For the end user, that is the rub. Why do you think local IRBs would “come down differently?” They are apprehensive like everyone else.

    Thanks
    Brad

  2. Brad F January 11, 2008 at 11:58 am #

    Forgot additional comment: does the OHRP’s pronouncement only apply to CHECKLIST based QI protocols?

    Thanks again,
    Brad

  3. andrewmc January 11, 2008 at 12:20 pm #

    Hi,

    Great summation of the situation.

    How do you suggest that the issues raised are resolved? How do you ensure that the QI work that is taking place is of sufficient quality that its results are meaningful without having some form of oversight.

    I completely agree that having oversight adds a bureaucratic burden but the options appear to be to either have some form of oversight which adds the tax that you talk about or to allow people to continue to work as they have done and accept that some of the work that will be done will be of a poorer quality.

    My preference is the latter in that change will occur faster but a cost of this will be that somethings are simply not as well done as others. The former guarantees that what is done is of a certain quality (not necessarily good though) but that most people will simply not participate due to the bureaucratic burden.

    Are you not better off having a program like the Saving 100k Lives and accepting that it is not ideal, than to put in place regulatory restrictions and not having the program at all?

    I don’t have the answers and I’m not sure that what I have said above is clear but it does seem to me that we want to “have our cake and eat it”, we want to be able to continue to improve quality with little bureaucratic input whilst at the same time make sure that those are doing it are ensuring it is of sufficient quality with little bureaucratic input and I am not sure that you can do both.

    Regards

    Andrew

  4. Bob Wachter January 11, 2008 at 4:22 pm #

    All great questions — thanks!

    Re: Brad’s first point, let me clarify that I’ve quoted extensively from my friend Atul Gawande (with Atul’s permission; they come from his emails to me, not from previously published work). So, “The question of patient consent is NOT the central issue. I’m sure they’d back down on that if local IRBs came down differently.” is actually Atul’s writing, not mine. (I’m flattered to have them confused, since Atul writes so elegantly).

    I’ll pass along your question to him — and hope that he’ll consider engaging in the discussion. Personally, I worry about BOTH burdens: the macro-burden of getting IRB approval (x 1 or 100) when you’re doing this type of work, and the micro-burden of potentially requiring the informed consent of EVERYBODY (can you imagine — every patient, doc, nurse, respiratory therapist) when you’re implementing something as benign as a checklist.

    And Andrew’s point is really important. QI projects CAN have negative consequences, and those engaged in them need to be thoughtful in their planning, execution, and measurement. But at some point, individuals and institutions have to have the freedom to go through PDSA cycles — knowing that some things won’t work — in order to create true improvement. If the OHRP ruling stands, I can’t imagine why it would be just about checklists. It seems to me that it implies that you need patient and provider consent any time you’re changing a care process or structure and measuring its impact. That’s why I used the example of implementing a new process for hand hygiene.

    One might argue that Peter’s project was “research” and that imposes a special burden, over and above daily QI and management work. Yes, I believe that is true, which is why Peter did the right thing taking the protocol to his IRB. And the Hopkins IRB did the right thing by blessing it and waiving the individual hospital IRB and individual patient/provider consent requirements. And then OHRP did the wrong thing.

    Like most ethical questions, it isn’t that there is an absolute “right answer” in these situations. Rather, one needs to weigh benefits and burdens in formulating a preferred approach. In my judgment, the burdens of what the OHRP is asking for in this case — particularly when one thinks about its massive multiplier effect as it pings its way through the QI universe — far exceed any possible benefit. It is really bad policy and precedent.

  5. bev M.D. January 13, 2008 at 4:53 am #

    Actually I have already written the OHRP at the above email address (Dr. Borror), with copy to Dr. Francis Chesley at AHRQ, who funded the study. (Francis.Chesley@ahrq.hhs.gov) I then discovered that Dr. Borror’s boss is the acting chief of OHRP,; his email address is ivov.pritchard@hhs.gov.

    I received a thoughtful response from Dr. Chesley, but nothing from the OHRP. I am waiting for their response (if any) and will post all of it when received. I don’t want to quote anything out of context now. I do urge readers to write to these people also so they know it’s not just one nut who wrote them (me). (:

  6. Bob Wachter January 13, 2008 at 4:50 pm #

    Thanks, Bev, for pointing out Dr. Pritchard, the acting director of the OHRP, who should receive correspondence (ivor.pritchard@hhs.gov). I found an interesting article about him (including an interview with him) here. Although he does not appear to have any significant background in healthcare, he does seems like a thoughtful person of integrity. It gives me hope. A few relevant quotes from the piece:

    Dr. Pritchard is known not only for his keen intelligence, bone-deep principles and integrity, and prodigious work ethic, but also for his wry and warming sense of humor and his amazing talent as a folk guitarist and harmonica virtuoso. Dr. Pritchard is the closest thing the research ethics field has to a rock star, and he has written and performed songs at several conferences over the past 10+ years. (by the author of the article)

    And in Dr. Pritchard’s own words:  

    …we hope to encourage institutions to pay attention to the most important elements of the subject protection process, rather than be distracted by bureaucratic requirements that don’t seem to make sense or add value.

    The first point I’d like to make is that the system for the regulation of human subjects’ research is fundamentally a decentralized one.

    There are some theorists of American democracy who say that our system has lasted as long as it has because the governmental apparatus is so slow-moving that it only adopts change when the value of that proposed change has been proven on a wide scale. The same thing could be said about the way in which research is developed, implemented, and improved.  Most of the best ideas and real changes usually begin at the grass roots level, and then, ten years later, the federal government sees how good those changes are only after they have been adopted by those working in the field.

  7. bev M.D. January 13, 2008 at 9:01 pm #

    Oops, Bob; thanks for correcting my typo on Dr. Pritchard’s email address. (couldn’t read my own writing – dr’s handwriting, you know!) Your quotes from him do indeed give me hope and I will send along whatever I hear from him or Dr. Borror.

  8. bev M.D. January 16, 2008 at 6:07 am #

    Well, it doesn’t appear that I will receive a response to my letter from the OHRP, so here is the text of Dr. Chesley’s letter. At least he had the courtesy to respond. Also, thanks to Bob for sending Paul Levy a copy of the letter the AHA sent to Secretary Leavitt. (At least, I think it was Bob that sent it, I read so many blogs….)

    Oddly enough, I was unable to copy and paste the text of the letter, so I have retyped it verbatim below. Good thing I took typing in high school!

    “Good afternoon Dr. (bev, my last name redacted):

    Thank you for your letter of January 6, 2008 regarding the recent opinion piece in the New York Times by Atul Gawande, M.D. As you are aware, the Agency for Healthcare Research and Quality (AHRQ) provides funding for health services research to improve the quality, safety, efficiency, and effectiveness of health care. Research funded by AHRQ is subject to the regulations at 45 CFR part 46, for which the Office for Human Research Protections (OHRP) has regulatory oversight.

    AHRQ concurs with the decisions by the OHRP relating to the determination that the activities described in the New England Journal of Medicine by Pronovost, et al represented research involving human subjects. The funds provided by AHRQ to conduct these activities were intended to be utilized for research purposes to address the questions described in the grant application.

    The work described by Pronovost, et al supports the hypothesis that an evidence-based intervention could provide a sustained reduction in the rate of catheter-related bloodstream infections in intensive care patients. The results of the study provide substantial evidence for the effectiveness of the intervention and the potential for its widespread use. AHRQ encourages hospitals to incorporate these practices in their intensive care units, as needed, to provide better care for their patients.

    AHRQ notes that your letter suggests that the determinations made by OHRP forbid the use of the intervention described by Pronovost, et al in hospitals to prevent catheter-related bloodstream infections. Please bear in mind that the determinations of OHRP involve only the research study conducted by John (sic) Hopkins and carried out in hospitals in Michigan. Should hospitals wish to use this intervention to provide better care for their patients, there is no prohibition on doing so. The use of this intervention in this manner would be outside the scope of 45 CFR part 46.

    AHRQ recognizes the need for greater guidance on when quality improvement activities fall into the category of research involving human subjects. AHRQ also believes that there is need for better communications between those conducting quality improvement activities (including research), regulators, and funding agencies to understand the factors that influence when the regulations at 45 CFR part 46 apply to particular studies.

    AHRQ appreciates your concern in this area and looks forward to continued dialog on these issues.

    Francis D. Chesley, Jr. M.D.
    Director, Office of Extramural Research, Education, and Priority Populations
    AHRQ”

  9. cmhughes January 16, 2008 at 8:13 pm #

    Hi, Bob,

    Any chance you’d share your letter to legislators with us that we might be able to use as a template for our own?

    Cheers,

  10. Bob Wachter January 16, 2008 at 9:33 pm #

    My letters referenced and linked to my blog, so they are a bit too personal to use as a template. I know that the Society of Hospital Medicine is working on such a letter even as we speak. In the meantime, here is the letter that the President of the American Hospital Association sent the HHS Secretary Mike Leavitt last week — it would be a good starting point:

    January 4, 2008

    The Honorable Michael Leavitt
    Secretary, The U.S. Department of Health and Human Services
    200 Independence Avenue, S.W., Room 615F
    Washington, D.C. 20201

    Dear Mr. Secretary:

    In a December 30 New York Times op-ed piece, Dr. Atul Gawande raises important questions about a misguided and potentially dangerous policy of the Office for Human Research Protections that would impose an unprecedented deterrent to quality improvement efforts across the country. I am writing to ask you to immediately retract any statements from the Office of Human Research Protections that imply that quality improvement efforts should undergo review by Institutional Review Boards (IRBs), and that consent should be obtained from all patients before changes could be incorporated.

    As you know, hospitals across the nation are engaged in a variety of activities aimed at redesigning health care delivery systems to ensure that our patients get the best possible care we can deliver. Some of these activities are organized by hospitals, such as the Michigan Health and Hospital Association’s Keystone project that Dr. Gawande cites. Others include projects initiated by the Institute for Healthcare Improvement, the Quality Improvement Organizations funded by the Centers for Medicare & Medicaid Services, and the work of several professional societies and organizations such as the American College of Surgeons and the American College of Cardiology.

    As Dr. Gawande points out, research to determine which drugs or procedures will benefit patients requires appropriate oversight by an IRB and informed consent by the patients. However, those efforts are far different from the quality improvement efforts exploring the use of checklists, computerized reminders, teamwork training, and other steps to ensure that the care we intend to deliver is actually delivered.

    It is worth noting that hospitals and health care professionals are not the only ones engaged in such projects. The quality transparency efforts in which the AHA, the Department of Health and Human Services (HHS), and several other organizations have spearheaded, the local value exchanges your department has fostered, and the value-based purchasing initiatives you have championed are other examples. Yet, HHS has, quite reasonably, sought no IRB review or informed consent for these changes, because they, too, are intended simply to improve the delivery of care.

    As quality improvement efforts become more standardized and rigorous, and as the data collection efforts that support this work become more extensive, it would be right and appropriate to contemplate how we can collaborate to ensure that the welfare of patients remains the central concern and that patient privacy is protected. It also would be appropriate to consider effective ways for hospitals and other providers to communicate with the public about their quality improvement efforts. However, it would be wholly inappropriate and detrimental to the patients and communities we serve if the measures apparently championed by the Office for Human Research Protections were to force hospitals and others to discontinue their quality improvement efforts.

    On behalf of America’s hospitals and the patients who depend on us, I urge you to ensure that the essential quality improvement efforts underway across the nation continue unabated.

    Sincerely,

    Rich Umbdenstock
    President and CEO
    American Hospital Association

Leave a Reply