Checklists don’t work* (*sometimes, particularly if you get implementation wrong)

 B17 Flying Fortress Checklist, 1944
B17 Flying Fortress Checklist, 1944

This isn’t an anti-checklists post.

It’s an illustration of why picking up an example of good practice in one location, and dropping down in another doesn’t always work. No matter how brilliantly conceived, beautifully constructed or obviously ‘good’ the original good practice might be.

So, next time a politician or clever speaker at a conference tells you “it’s just a simple matter of transferring good practice”, please ask them; “what about Atul Gawande and the Hospital Checklists?” (and then get sacked or thrown out of the conference).

Checklists are a good thing. The thinking behind checklists is difficult to argue against. Basically a checklist breaks down a complicated procedure into a series of logical, easily understood steps. This helps a person/operator successfully complete the procedure.

The origins of the modern checklist (which is a fascinating story) track back to 1935 and the adoption of the B17 Flying Fortress Bomber by the U.S. Airforce. The B17 checklist helped to reduce accidents caused by human error, and versions of it have been ubiquitous in the aircraft industry for the last 80 years.

Checklists are so effective that they aren’t just restricted to aviation. They have a role in everything from; ‘packing for your round the world trip’ through to auditing and ‘shutting down your nuclear reactor in an emergency’. Useful in any complicated situation where the human brain could become confused or muddled, particularly in a stressful situation. So, it’s no surprise that the medical world (eventually) picked them up.

IMG_4093Atul Gawande helped us love checklists. You’ve probably heard of Atul Gawande, author of the 2009 bestseller, The Checklist Manifesto? (Yes, I do own a copy, a birthday present from my Father in Law).

The book talks about how eight hospitals took part in a pilot of the  World Health Organisation (WHO), 19 Point, Surgical Safety Checklist during 2007/8. The results were impressive:

  • Post operative complications reduced by almost a third.
  • Death rates reduced by almost a half.

The WHO went on to recommend that the Surgical Safety Checklist (or something similar) was adopted by all hospitals. The UK National Health Service (NHS) required that all its facilities use the checklist, and by 2012 over 2000 facilities worldwide had tried checklists. An impressive example of ‘evidence based good practice’ being rolled out across organisations.

If you want to get a feel for just how enthusiastically this example of good practice has been ‘rolled out’, have a look at You Tube and search for ‘WHO Surgical Safety Checklist’. You might be surprised by just how many diverse organisations and groups have uploaded videos, I was. You can lose a few hours browsing the 3000+ search results. ‘How NOT to do the WHO Surgical Safety Checklist’ is a particularly uncomfortable yet compelling video to watch.

Checklists haven’t worked like they should. This feels like a bit of a spoiler, but do have a look at this article from Nature; Hospital checklists are meant to save lives – so why do they often fail?

The Nature article uses an example of 200,000 procedures carried out at 101 hospitals in Canada, where there was no evidence of any reduction in complications and deaths following use of checklists. It also details the failure to match results of the Michigan Checklist, which was used to prevent problems associated with the introduction of catheters into veins (eurrgh!).

Source: http://www.nature.com/news/hospital-checklists-are-meant-to-save-lives-so-why-do-they-often-fail-1.18057
Source: Nature

For me, the article is making a very clear point that; checklist aren’t the problem, it’s how the people who receive the ‘new’ checklist choose to use it. There are a number of ‘issues’ used to illustrate this: (have a look at the helpful graphic)

  • Staff resisted, or failed to complete the checklist. Sounds like a bit of the ‘not invented here syndrome’ to me.
  • The checklist was illogical or inappropriate. Fair point, one size rarely fits all.
  • The checklist was seen as a waste of time. A bit like the first point, people don’t see it as useful.
  • ‘Parachuted in’ solutions. It was just another ‘initiative’ dropped on front line staff by Managers and Administrators.
  • It felt ‘Imposed’. People do have a tendency to resist things that are imposed upon them, regardless of how much it is the ‘right thing to do’ (I know I do).
  • It didn’t fit the local context. The requirements of a checklist developed in a well-resourced American hospital might not apply in an under-resourced hospital in a conflict zone. You may not have the same standard of equipment or even the same number of people available to perform the task.

It’s all about the implementation. The Nature article goes to talk about the need for careful thinking about how you transfer good practice and the growth of Implementation Science.

As an example of proven good practice (which saves lives) the WHO Surgical Safety Checklist stands out. It’s been enthusiastically adopted in many places, just have a look at You Tube. However there are places where it has failed to have the desired impact. Based on this experience I think there is a case for thinking more widely about the process of implementation in the transfer of good practice and the role of Implementation Science. Good practice transfer is not always a simple case of ‘just do it’.

So, what’s the PONT?

  1. Checklists are a very effective way of reducing human error in complicated (and straightforward) procedures. An example of ‘transferable good practice’.
  2. Issues around ‘implementation’ the WHO Surgical Safety Checklist have prevented them from being fully effective in some settings.
  3. Universal acceptance and use of the Surgical Checklists might take time (Aviation checklists are after all 80 years old).

Related Post: Why is Good Practice such a bad traveller? https://whatsthepont.com/2012/08/19/why-is-good-practice-such-a-bad-traveller/

About whatsthepont

The things I’m currently interested in are: 1.How people learn and share knowledge; 2.Social Media, Web2.0 whatever you want to call the world of the internet; 3.Better public services.

11 Responses

  1. Had no idea the humble checklist originated in a B17! Live and learn. 🙂

    You could replace the word ‘checklists’ with ‘computer systems’ and this would all be equally valid! There’s often a strong urge to implement a system because it’s working well elsewhere.

    And just in the same way that people HATE having checklists thrust upon them, they really despise having to bend their working day to new software they don’t understand or think they need.

    Last week I was pondering how context is EVERYTHING – particularly when trying to implement change. It’s one of the reasons that I think Working Out Loud is so vital – it’s an opportunity to narrate the journey of problem solving and co-create the solution together (thus defusing some resistance along the way).

  2. The big mistake people make is to use checklists to enforce compliance and attempt to control the actual work. Crap! Checklists are best used before a long boring sequence of tasks to mentally prepare, or just after a short boring task to make sure you haven’t forgotten anything.

    The idiots try to apply them by saying they make up for weaknesses or deficiencies in perfect teleologicall processes, so called human factors. That’s wholely inaccurate and only said by people who don’t really understand the cognitive science. People reject being patronised by an idiot, they rarely reject a handy list of boring stuff!

    There’s also growing evidence that what it says on the list is mostly irrelevant, it’s the reframing experience that creates safety, by ritualising the transition from one social context to another. It’s more important that the theatre team perform the ‘checklist’ together to heighten collective situational awareness as they move from social norms, to formal roles around the operating table, than what it actually says on the checklist. #mindbombed

      1. You do not need to read some arsehole professor’s view on this malarkey to understand the difference between a two dimensional abstraction (the list) and a three dimensional methode plonge (doing it yourself). Tomorrow morning, brush your teeth using your other hand! The toothbrush pattern is embedded in your usual hand. The unusual hand requires a checklist!

  3. Healthcare and aviation are two very different industries with two very different cultures. Checklists (and Crew Resource Management generally) were transplanted from one to the other despite this poor tissue match and, not surprisingly, were rejected!
    The underlying genetics, however, are sound but require a Genetic Engineering approach to transfer them. Small changes at the coalface with the support of the staff will enable any benefits to spread through a unit or hospital by natural selection (staff generally want to do their best and will support initiatives which help them to do so) whilst failures will simply be bred out of the system.
    Dr Lucian Leape told us way back in 1999 that “To Err is Human” but Error Management Training failed to be implemented with the result that our results now are little better than back 16 years ago.
    Aviation needed the loss of almost 600 lives in the Tenerife disaster in 1977 to make it focus. What will healthcare need to force it to do likewise…??

  4. Clover

    I guess that why human beings are fall safe for broken checklists – maybe invest more in them and use the checklists as aide memoirs for effective good practice transfer.See David Denyer’s presentation at @Academi’s summer school 2014. http://www.daviddenyer.com/wp-content/uploads/2014/07/map.jpg. Black Hawk (1994) https://en.wikipedia.org/wiki/1994_Black_Hawk_shootdown_incident. The investigating officer couldn’t believe how this hadn’t happened sooner.

  5. Hi, there’s a book called Blackbox Thinking by Michael Syed that I’ve downloaded a sample chapter of, and in the very first chapter he goes in depth into the difference between the aviation culture and their thinking about learning and failure, and the very different medical culture of cover up and deference to hierarchy. It’s very interesting but the thing I’ve realised is that they’ve got skin in the game, literally, compared with doctors and nurses. And its impossible to hide an aviation disaster compared with a sick patient dying. That’s what sick people do, right?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s