It's all about me, me, me: the problem with advocacy-inquiry in debriefing

The importance of the debrief

It is safe to say that those of us involved in simulation believe that the debrief is a very important part of the learning experience. Many of us (1) would say it was the most important part. A slide taken from the SCSC's faculty development (train the trainers) course helps us see why (Figure 1).

Fig. 1: Scenario and Debrief mapped onto Kolb's learning cycle

When we map the simulation activity onto Kolb's learning cycle, we can see that three-quarters of the process is supposed to occur during the debrief. The debrief is important therefore because much of the learning is supposed to take place during this period.

In 2007, Fanning and Gaba wrote an article entitled "The Role of Debriefing in Simulation-Based Learning". They explain how adult learners do not find "linear teaching models" (i.e. didactic teaching) very effective and instead benefit from active participation (i.e. experiential learning as shown in Kolb's learning cycle above). The role of the teacher is to facilitate the learners' progress around the cycle (hence "facilitator") through a debrief of the events which occurred in the simulated scenario. This role is an important one, as Fanning and Gaba state:

"Data from surveys of participants indicates that the perceived skills of the debriefer have the highest independent correlation to the perceived overall quality of the simulation experience." (2, p.118)

Dismukes and Smith identify three levels of facilitation (3):

1. High level facilitation: participants more or less debrief themselves
2. Intermediate level facilitation: somewhere between high and low
3. Low level facilitation: facilitator directs the entire debrief

Fanning and Gaba state that we should use the highest level possible and, in their paper, go on to list some of the debriefing styles (e.g. funnelling, framing) and techniques (e.g. plus-delta, target-focused) which are used.

Debriefing with good judgment (Advocacy-inquiry)

In conversations with some facilitators at simulation-focused conferences I have been struck by their belief that "debriefing with good judgment" is the "one, true" debriefing style. After all, who does not want to debrief with good judgment?
In their two (similar) papers Rudolph et al discuss the theory and practice of debriefing with good judgment (4,5). Drawing on "a 35-year research program in the behavioral sciences on how to improve professional effectiveness through 'reflective practice'" they cite three elements in their approach:

1. Uncovering the participant's knowledge, assumptions and feelings (internal frames) allows the facilitator to "reframe" and improve future actions and behaviour
2. The facilitator has a stance of "genuine curiosity" about the participant's internal frames
3. The facilitator uses the "advocacy-inquiry" conversational technique to bring his/her judgment and the participant's frames to light.

The first 2 elements are not controversial, but the third gives me some concern. Rudolph et al define advocacy as:

"A type of speech that includes an objective observation about and subjective judgment of the trainees’ actions" (4, p.49)

To illustrate what they mean, they provide the following worked example:

I have highlighted the problem (as I see it) with the worked example below:

It's "me, me, me" and then "I, I, I". For a facilitated debrief there seems far too much focus on the facilitator. Here's my alternative to the above debrief:

Debriefer: So, how was that?
A group member: I was a bit confused by what was happening.
Debriefer: Really? In what way?
A group member: I wasn't sure who was in charge or what I was supposed to do.
Debriefer: Okay, did anybody else feel that way?
Group: Several members agree.
Debriefer: Did the confusion have any effect on how you dealt with the patient?
etc. etc.

The above is still too facilitator-driven but at least removes the facilitator as the pivot around which the conversation flows. The advocacy-inquiry technique seems to be at best an intermediate-level facilitation and at worst a low-level facilitation where the participants rely on the facilitator to discuss what he/she thought were the important points.

Should we get rid of "debriefing with good judgment"? No. Much of what Rudolph et al discuss is valid. They are correct to say that there is no "non-judgmental" debriefing. Being "genuinely curious"is also extremely important. However, I would argue that one can be genuinely curious without focusing the conversation on the facilitator.

Is there a place for "advocacy-inquiry"? Yes. Fanning and Gaba state "the debriefing techniques employed need to take individual learning styles into consideration" (2, p.117) High-level facilitation should be used whenever possible, then perhaps stepping down to advocacy-inquiry if the participants need more direction.

In their paper, Rudolph et al follow the debriefer/trainees conversation above with the debriefer saying:

I think that question needs to come much earlier.



References:
1) Issenberg SB, McGaghie WC, Petrusa ER, et al: Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher 2005;27:10–28.
2) Fanning RM and Gaba DM: The role of debriefing in simulation‐based learning. Simulation in Healthcare 2007;2(2):115-125. (Article available for free here.)
3) Dismukes R, Smith G: Facilitation and debriefing in aviation training and operations. Aldershot; UK: Ashgate, 2000
4) Rudolph JW, Simon R, Dufresne R, et al: There’s no such thing as “Nonjudgmental” debriefing: A theory and method for debriefing with good judgment. Simul Healthcare 2006;1:49–55.
5) Rudolph JW, Simon R, Rivard P, et al: Debriefing with Good Judgment: Combining Rigorous Feedback with Genuine Inquiry. Anesthiol Clin 2007;25:361-376. (Article available for free here.)

A train in Spain

On the 25th July 2013 a train travelling from Madrid to Ferrol derailed near Santiago de Compostela. Out of 218 passengers, 79 died. A BBC report quotes the president of the railway firm:

The train had passed an inspection that same morning. Those trains are inspected every 7500km... Its maintenance record was perfect.

One of the train drivers, according to the same BBC report, told the control room that he took the bend at 190km/h; the bend's speed limit is 80km/h. He is also reported to have kept repeating "We're human, we're human." The train was running five minutes late.

A later BBC report explores the safety systems in place on trains; the European Train Control System (ETCS) which can prevent speeding and the more basic ASFA (Anuncio de Senales y Frenado Automatico) which warns the drivers when the speed limit is exceeded but cannot stop speeding. The track where the train derailed only had the ASFA system in use. The train driver received 3 audible warnings to reduce speed, the last warning was 250m (or 4.6 seconds) before derailment. The BBC report also quotes a Spanish journalist who says that there had been concerns about this section of track since it opened, as it required the driver to reduce the train's speed from 200km/h to 80km/h "in just a matter of seconds". According to unnamed officials, the bend does not need additional safety measures "because of the proximity of a major urban center, which requires that drivers slow down trains regardless." Writing in the Guardian, Miguel-Anxo Murado tells us that:

There were arguments for having that section of the route remade completely, but Galicia's particular land tenure regime makes expropriations an administrative nightmare. So the bend was left as it was, and speed was limited there to 80km/h

On the 27th July the train driver is discharged from hospital and taken to a police station for questioning. The Interior Minister accuses him of reckless manslaughter. The driver has 30 years experience, became a fully qualified driver in 2003 and had been driving on that route for over a year.

On the 31st July we learn that the driver was on the phone to train company staff and/or the train's ticket inspector at the time of the crash. He would normally start braking 4km before the bend (at 200km/hr he has approximately 72 seconds to decrease his speed to 80km/hr).

The Guardian provides us with additional information:

Renfe is among the firms bidding for a €13bn contract to build a high-speed rail link in Brazil. The terms of the tender reportedly exclude firms involved in the running of high-speed train systems where an accident has taken place in the preceding five years.

In the latest BBC report dealing with the crash, the Public Works Minister is quoted as saying: "Everything is under review, everything is subject to proposals for improvements". There is discussion of beacons on sections of track which require rapid braking, the use of satellite technology and a review of the physical and psychological requirements of train drivers.

The train driver has been charged with involuntary homicide due to "professional recklessness".

The aftermath of this train disaster followed the same course as that of the sinking of the Costa Concordia: initial focus on the “sharp end” of the captain/driver, with immediate denials from the corporate offices of any wrong-doing or failure on their part. Then, once more information comes to light, there is an appreciation that perhaps there were other problems contributing to the sinking or crash.

Instead of an immediate denial of culpability, it would be refreshing if the president or CEO of a company would instead express sorrow at the loss of life and regret at the injuries, coupled with a promise to explore all factors leading up to the event. It is almost inevitable that any such large-scale disaster will have a number of causes and missed opportunities for prevention. However, in a world where such sentiments would affect stockmarkets and bids for contracts, this may remain wishful thinking.

Book of the month: Why we make mistakes by Joseph Hallinan

"Why we make mistakes" is another "light" read for this month following on from last month's "Set phasers on stun". Joseph T. Hallinan is a former writer for the Wall Street Journal and winner of the Pulitzer Prize. The Pulitzer Prize was for investigative reporting on medical malpractice in Indiana, USA, and so it seems apt that he has now written a book on mistakes.

Who's it for?

This book is written for a general audience and, as summer holidays draw to a close, could be squeezed into the last few days off, now that the kids have gone back to school. Hallinan covers a number of human factors terms and concepts such as hindsight bias (p. 5, p.65), different types of mistakes (p. 8), framing (p.92), anchoring (p. 103) and illusion of control (p.162). He also mentions some of the big names in human factors research such as Simons and Levin (p. 14), Kahnemann (p.93, p.206), Ericsson (p. 172) and Gaba (p. 192).

I haven't got time to read a whole book...

Read the introduction, chapter 1, chapter 5 and the conclusion. (For a description of these chapters see below)

What's good about this book?

Hallinan starts off well, stating:

"When something goes wrong the cause is overwhelmingly attributed to human error.... And once a human is blamed, the inquiry usually stops there. But it shouldn't - at least not if we want to eliminate the error."(p.2)

In the subsequent 13 chapters, he goes on to look at different causes of mistakes and offers advice on avoiding them.

The tabletop on the left is obviously narrower and longer...

In the first chapter, "We Look but Don't Always See," Hallinan refers to vision, perception and change blindness. The "door" study by Daniel Simons and Daniel Levin may be a useful example to use when discussing this aspect of human factors in a lecture or workshop. This chapter also uses the two tabletops image to show that even when we know that something is true we still cannot force ourselves to see the truth.

In this first chapter, Hallinan then goes on to provide us for another reason that radiologists miss dancing gorillas (and tumours). The less frequently something occurs, the more likely we are to miss it. Because tumours are infrequent radiologists tend to miss them (this is another argument for not requesting tests such as chest x-rays on the off chance that something may be picked up.) Baggage screeners fare little better for the same reason, as Hallinan states, in 2006 at "Los Angeles International Airport screeners missed 75 percent of bomb materials." A reassuring thought for the next time you're standing in line at the airport with your belt and boots in one hand and the other hand holding up your trousers.

The next chapter which has a decent shot at dealing with a human factors problem is chapter 5: "We Can Walk and Chew Gum - but Not Much Else". Hallinan explores the myth of multi-tasking and looks at task saturation, trying to do too many things at the same time. An interesting concept here is that we can walk and chew gum at the same time because neither of these tasks requires conscious thought. We can drive a car and have a conversation because (once we are proficient) driving the car no longer requires our undivided attention. This idea may be useful to present in a human factors workshop or talk; i.e. repeated practice of managing crises in a simulator will allow one to use less mental workload on the basics during a real crisis.

Would you work for this man?

A final concept which may prove of interest is that constant change at the top of an organisation may not be helpful to the organisation itself. Hallinan discusses Warren Buffett's company, Berkshire Hathaway, where none of the CEOs have voluntarily left for other jobs in its 42 year history (p. 160). Hallinan proposes that the CEOs stay in post long enough to receive feedback and learn from their mistakes. Might this have lessons for the NHS?

Hallinan's advice for making fewer mistakes is given in the conclusion and that is to "Think small". Unfortunately he doesn't then quite manage to explain what this means, but instead goes on to provide other, more useful, advice:

1. Calibrate yourself. Be honest when you consider decisions that you have made and what their effect has been. (We have a tendency to think we performed better than we actually did.)
2. Think negatively. By considering what can go wrong you can prepare for that eventuality.
3. Ask others for advice. A spouse, friend or colleague may provide insight into a problem.
4. Sleep. Fatigued people perform less well.
5. Be happy. Happy people make decisions more quickly and are more creative with their problem-solving.

What's not so good about this book?

Areas which could be improved include Hallinan's introduction to System 1 and System 2 thinking which is somewhat clumsy, quoting Paul Slovic: "Our perception of risk lives largely in our feelings, so most of the time we're operating on system No. 1" (p. 95) He also digresses somewhat to spend a significant number of pages discussing the direction sense of men and women (p. 143-148).

Additionally Hallinan shows a slight lack of experience when he discusses how time affects decisions, stating:

"Many factors can affect the way we frame our decisions. One of the least obvious is time."

Anybody who has worked in a simulator or in an environment where time is critical, will be aware of how great an effect time has on on decision-making. Hallinan also mentions looking for "the" root cause which Sidney Dekker would have something to say about, namely:

"Asking what is the cause (of an accident), is just as bizarre as asking what is the cause of not having an accident. Accidents have their basis in the real complexity of the system, not their apparent simplicity. (p. xiii)" (The Field Guide to Understanding Human Error"

A final criticism of Hallinan's book (and perhaps much of human factors research) is that so many of the conclusions are based on research carried out on American university (Princeton, Ohio Wesleyan, Duke, Carnegie Mellon, Yale...) students; are these truly representative of the rest of the population?

To conclude

This is another book which should be borrowed, not bought. It has some material that may be useful for those who prepare human factors workshops. For the human factors novice it provides an easy-to-read introduction to some of the concepts and big thinkers in the field.

Putting patient safety first

Is your hospital adopting a safety culture? For both patients and staff? Below is a message sent to all hospital staff by the Interim Medical Director at Forth Valley Royal Hospital, Larbert. This is the sort of message we need to receive and spread in order to promote safety.

Dear Colleague

I am grateful to all of you who contribute to teaching and supervision of junior doctors and I am sure I can count on your wholehearted welcome and support to new colleagues starting this week.

Following recent guidance from the GMC and NES all new doctors at induction are being reminded of their duty to raise any concerns regarding patient safety as soon as possible.   This should be done as soon as possible after the perceived safety risk  -  with their supervisor or with another consultant in the department; with the Director of Medical Education (Dr David McQueen);  an AMD or;  with the Medical Director.   Concerns should be raised timeously in the knowledge that they will be considered seriously, without prejudice and will be acted upon appropriately.

The learning environment we provide should ensure that their training should take place in a supportive environment where undermining or bullying behaviour is not tolerated.   There is a ‘zero tolerance’ of undermining and we hope this will not be experienced at any time.   If the trainee feels that this is an issue for them, they are asked to urgently report it to a member of the consultant staff (if possible, to their clinical or educational supervisor or to a College Tutor, or equivalent).

Thank you again for your invaluable support.

Sent on behalf of

Dr Peter Murdoch

Interim Medical Director  

Forth Valley Royal Hospital