Checking firefighters for carbon monoxide - recent studies, persistent concerns.

Carbon monoxide (CO), we can all agree, is bad. The symptoms are vague, the patients sometimes can't give a good history, and definitive diagnosis requires a needle stick. 

"Do I have to? I can't stand having my blood drawn!" (source)

Of particular concern, firefighters can be exposed to high levels of CO in the course of their duties. For example, one small study suggested that FFs can double their CO levels during overhaul, a period when many personnel are not using SCBA (see abstract #4).

In response, many fire departments have started to screen for elevated carbon monoxide during the rehab phase of a fire. For the most part, EMS agencies and FDs use devices made by Masimo Corporation that monitor CO levels noninvasively, and are quite portable, whether as a stand-alone device (in the picture below), or available as an option in some monitor-defibrillators. 

Name 4 things done incorrectly in this ad.
Answers below!

Sounds great! What's the problem?
They may not be accurate enough to screen FFs.

A large study done by Touger et al. in 2010, conducted in an ED in the Bronx, looked at the ability of these devices to diagnose high CO levels. They compared the CO values obtained by the traditional blood draw method with the painless finger probe on the RAD-57.

They checked out a bunch of patients who came to the ED with suspected CO poisoning, and found that the RAD-57 was usually correct if it predicted a high level, but ...

... it missed a lot of cases too.

In fact, it missed about half of the cases where the CO level was ≥ 15%. In 3 cases where the patients had CO levels over 15%, the RAD-57 gave a value of "zero." The authors concluded that 

"Our results do not support use of this device to replace standard laboratory measurement or as an out-of-hospital triage tool."

Why did they choose 15% CO as the cutoff?
Most toxicologists will tell you that they don't care as much about the specific CO level, as they do about the symptoms. A patient with a level of 17%, but only feels some mild dizziness, is less concerning than a patient who passed out for 10 minutes, but has a "lower" level of 11%.

Instead, the 15% threshold comes from the National Fire Protection Association (NFPA) guidelines. Here's an example of what their training curriculum says about using CO levels.

Rehabilitation and Medical Monitoring: A Guide for Best Practices : an Introduction to NFPA 1584

The implication of the Touger study is that half of CO-poisoned firefighters might not be identified during screening in the rehab sector. Not good.

Can we use a different cut-off?
Sure! A few studies have looked at this, but the results show that there is a trade-off.

How about a CO level of 1o%?
A 2012 Candian study (Zaouter) used human volunteers, instead of ED patients. This study was conducted in a lab setting, where the researchers were able to precisely deliver CO to the volunteers, and then test them using both the RAD-57 and the standard blood test.

• The Good Part: They were able to look at the performance of the RAD-57 over a range of well-controlled CO levels.  
• The Bad Part: You can't really give human volunteers more than 15% CO blood levels - too dangerous.

The subjects breathed in CO until they reached levels of 10 - 15%, and then the researchers measured the CO level, simultaneously using the finger probe and a blood draw. How did the RAD match up?

Not so hot. Using a cut-off of ≥ 10%, they found that (like Touger), that it missed about half of the cases. Of the 24 patients who had CO levels of 10 - 14%, only 13 were identified by the RAD-57. The authors note that:

"In light of this low sensitivity, it has been advocated that the RAD-57 cannot be used to exclude CO poisoning in any patient with an appreciable risk of being intoxicated."

How about a CO level of 6.6%?
However, an Austrian study from 2011, (Roth) seemed to show that the RAD-57 could pick up almost all the cases of CO poisoning. Like the studies done by Touger et al. and by Sabbane et al., it was done in an ED, and involved simultaneous blood draws and use of the RAD-57.  So why the difference?

Because they ended up using a lower CO level of 6.6%, not 15%. They calculated this level after running the tests, and then calculating receiver operating curves, statistics, etc. What this table also says, is that, if you ran 100 FFs with CO levels above 6.6% through rehab, and checked them with the RAD, you would catch 94 of them.

So, is the answer to use a cut-off of 7% instead of 15%? We don't really know. Problem was, almost no one in this study had CO poisoning - only 1.1% of the patients. With so few "positives," the researchers couldn't say much about the rate of "false-positives." As the authors noted:

"The opportunity for false-negative results was limited. Because a false-negative reading could have serious medical consequences, this device should be tested in a much larger number of poisoned patients to confirm the generalizability of our stated cut-off values."

How about a CO level of 9%?
A French team of researchers (Sebbane) was also conducted in the ED, examining suspected CO-poisoned patients. The decided to try using different cut-offs for smokers and non-smokers, and defined CO toxicity as a blood level of ≥ 5% in non-smokers, and ≥ 10% in smokers. 

After studying a bunch of ED patients, and doing the same sort of statistical stuff as the Austrian team above, they found that using RAD-57 levels of ≥ 6% and ≥ 9%, respectively, they still missed a number of folks, all of them non-smokers. It was worse if they use a cut-off of ≥ 9% for everyone.

The researchers concluded that:

"Subjects with suspected CO poisoning and first-line, positive RAD-57 testing (SpCO 9% in smokers, or 6% in non-smokers) could benefit from immediate care. However, a negative RAD-57 test will not exclude standard blood COHb measurement to confirm CO poisoning."

Large study using the RAD-57, with firefighters, in real-life!
Lastly I have to talk about the big study, the one that enrolled a pretty large number of active firefighters.

In this "real-world" test of the RAD-57, the researchers enrolled all firefighters coming through the rehab station at major structure fires. They were able to simultaneously measure the blood levels of CO at the same time as using the RAD-57. Instead of being done in a lab setting, or even in an emergency department, they tested it in the challenging environment of a true fire scene. The firefighters were dirty, their fingers were cold, and the real firefighters (not research staff) checking the RAD-57 measurements didn't have time to obtain multiple measurements - they had to get it right the first time.

So how did the RAD-57 measure up? How many CO-poisoned FFs did the device pick up? How many did it miss, and, as a result, let go back to their duties?

Well, we don't know. Because no such study exists. Despite the heavy marketing to fire departments around the country, telling them this device saves lives, there are no studies showing that the RAD-57 can function accurately in this environment. 

The Bottom Line 

Looking over all of these studies (plus a few more), a few results are found reasonably consistently:

• On average, the RAD-57 is fairly accurate. 
• However, the "average" can hide a lot of variation.
• You can usually trust a "high" reading. 
• A "negative" test, however, can very often be false. 
• Most importantly, you cannot use the RAD-57 to "rule-out" CO toxicity.

Most toxicologists are very skeptical about using RAD-57-type devices to make clinical decisions. A very recent article, written by 3 critical-care physicians, noted that 

"Until well-performed trials demonstrate that these devices provide consistently accurate measurements, we cannot recommend their routine clinical use."

With these results in mind, I'm not sure that fire departments can use this to safely return firefighters to duty.

Answer to picture question

1. The rehab/medical monitoring station is set up right next to the fire and a diesel engine? This seems like a bad place to evaluate the asymptomatic firefighter.
2. Perhaps the FF is suspected to be symptomatic for suspected CO toxicity. In that case, where is the high-flow oxygen?
3. Since the FF is still fully dressed, and his face is covered in soot, we can assume that he hasn't had time to wash up. This could be a problem, since some researchers have found that dirty fingers can lead to false CO measurements (Piatkowski 2009)
4. Lastly, the CO level is being checked in direct sunlight. The operators manual states that "direct sunlight, directed on the sensor, may not allow the Rad-57 to obtain readings."