Can we believe every H7N9 seroprevalence study we see?

Special Guest writer: @influenza_bio

A little over a year ago, the first known human patient got sick with avian influenza virus(H7N9). The number of H7N9 cases rose and fell in the spring of 2013, and a total of 134 people were known to have contracted H7N9 before June, 2013. Since then, sporadic cases appeared in the summer and fall, and by the end of December, 2013, new cases started to pick up again. We have now seen a second wave rise and fall, although several new cases still being reported each week. As of the time of this writing, just over 400 people are known to have been infected with H7N9. The case fatality rate (CFR) � roughly speaking, the percentage of people infected with H7N9 who die from it � for these known cases is almost 40%.

One question that is on a lot of people's minds is, how many other H7N9 cases are out there that we don't know about? How many mild cases are there that never get tested? How many asymptomatic cases are there that are missed? If there were a lot of undetected cases out there, that would mean that H7N9 is a lot less fatal than the known cases would make us think. On the other hand, if we were somehow miraculously seeing every single actual case, then the CFR would be as bad as all of these cases make it out to be. (And imagine what the CFR would be like without hospitals, ventilators and oseltamivir!)

How do we find out if there are cases that we're missing? One way is to do what is called a seroprevalence study. This means collecting blood samples from as wide a swath of a population as possible and testing to see how many of these samples have antibodies to H7N9. Antibodies are molecules that are made by cells of the immune system and that stick to specific pathogens to help our bodies to rid themselves of these pathogens. If someone gets sick with H7N9 influenza, his or her body would most likely continue to produce a significant amount of antibodies specifically against that strain for at least a good number of months after infection and possibly much longer. In general, people who are infected with influenza but who do not develop symptoms will also produce such antibodies, but their bodies will make fewer of them, and, on average, they won't make as many of them for as long. We don't know exactly what the pattern of antibody production is for people who are infected with H7N9 but don't develop symptoms, though, because researchers haven't identified enough of these individuals to study.

It is very important that we get these seroprevalence studies right. If they're done wrong and we miss a lot of cases, then we will simultaneously underestimate how common H7N9 cases are and overestimate how deadly the strain is. On the other hand, if seroprevalence studies are done wrong and we think a lot of people were infected with H7N9 when they weren't, then we will overestimate how common H7N9 cases are but underestimate how deadly the strain is. Facts can help us to respond to H7N9, and if we get the facts wrong, then we can't respond properly. For example, if we come to think mild H7N9 cases are far more numerous than the severe ones that actually get diagnosed, then we might not worry as much about H7N9 as we should.

What I'd like to talk about here are some of the important ways that seroprevalence studies can go wrong. To answer my title question, no, we cannot always believe the conclusions of every seroprevalence study we see. Scientists make mistakes, just like everyone else, and sometimes things just go wrong, too. I'd like for you to understand just how some of these mistakes can arise, so that you can better judge for yourself whether a study is likely to be reliable or not, or so that you can at least know that there are things out there that can go wrong.

How are seroprevalence studies done?

There are 2 types of laboratory assays (tests) that are usually used in seroprevalence studies (although there are others): hemagglutination inhibition (HI) assays¹and microneutralization (MN) assays.² (For more information about the HI assay in general, see a nice description by Dr. Racaniello.³) MN assays are considered better (more sensitive and specific) than HI assays, but they are harder to do. MN assays require a significant amount of extra work at the end that HI assays don't. But, more importantly for H7N9 studies, HI assays can be done with either "killed," modified or "live" virus, whereas MN assays require "live" H7N9 virus. In other words, HI assays can be done in almost any lab, but MN assays require a BSL-3 lab. A neutralization assay⁴ has been developed that uses a "pseudovirus" instead of live H7N9 and is therefore far less hazardous to work with, but formal WHO diagnostic criteria still require standard HI and/or MN assays.

First, blood samples are collected. Each blood sample is drawn into a tube, and after 15-30 minutes, the tube is centrifuged to separate clotted red blood cells from the rest of the blood. The red blood cells are discarded; what's left is called serum, and that's what's studied. The serum samples should then be put in a refrigerator if they'll be studied within a few days; if they'll be studied later, they should be frozen. Once a researcher is ready to study the serum samples, the serum samples are thawed. Virus is also used for the assay, so one or more tubes of virus are thawed, too. Different types of mammalian or bird cells are prepared: typically horse, turkey or chicken red blood cells for HI assays, or a special type of dog kidney cells ("MDCK" cells) for MN assays. Various solutions are prepared. Serum samples, virus preparations and cells are diluted as needed, and everything is transferred into little wells in a plastic "plate" in just the right way. In the HI assay, the plate then sits at room temperature for 1 hour, after which it is "read" by eye. In the MN assay, the plate then sits at 37�C (body temperature) for 19-21 hours, after which it is read by a machine (an "ELISA reader"). The assay is done. The results of the assay are then written down and analyzed, and voil�, a paper appears in the scientific literature.

What could possibly go wrong with these blood tests?

Let's start with some things that can go wrong with the lab work:

1. If blood samples are left sitting around for a long time without being centrifuged, the red blood cells will start to break apart, and enzymes released from the red blood cells will start to destroy antibodies (and everything else) in the blood samples. This happens even faster if blood samples are not refrigerated.
2. If serum samples are left in the fridge too long, things can start to deteriorate, just like food in your fridge would. The antibodies that you would like to measure start to be broken down. (Sometimes, for many different kinds of studies, people study serum samples left over after patients' blood tests at hospitals. Those samples sometimes sit around in a fridge for quite a while. Some of them can even be green from stuff growing in them while they're sitting around. Yuck.)
3. If plasma (what's left in blood after unclotted red blood cells are removed) is used instead of serum (what's left in blood after clotted red blood cells are removed), then the assay can read artificially high. Serum should always be used, not plasma.
4. Every time serum is frozen and thawed, some of the antibodies are effectively destroyed. This should not be done over and over. Serum samples should be put into the right size tubes that the researcher will want to use, so that the samples are put through only 1-2 "freeze-thaw cycles" before they are tested. And all serum samples should go through the same number of freeze-thaw cycles.
5. The same thing is true for virus samples used in MN assays. A single freeze-thaw cycle can reduce virus infectivity by a factor of 10. Virus samples also need to be kept on ice when they're being worked with.
6. The plate can be read wrong. It's hard to imagine reading an HI assay plate wrong, but a special procedure (ELISA) and special equipment (ELISA plate reader) are used in the MN assay, and ELISA assays can go wrong.

But, hopefully all of that was done right. Not all researchers, students and technicians are created equal, but hopefully the lab "PI" (Principal Investigator; the person running the lab) is competent and ensures that everyone is doing things correctly.

What could go wrong with the data analysis?

What else could go wrong? The data analysis might not be done correctly. And it's here where perfectly good data can be ruined and where you have to look at seroprevalence studies most closely.

Suppose you've measured your antibody amounts ("titers") in your serum samples. How do you decide which titers mean the sample came from someone who was infected with H7N9, and which titers mean they didn't? Do you just pick a number out of thin air? If you don't have data to tell you which titers mean what, then all you are doing is measuring antibody levels in a population, and you can make no interpretation about what those levels mean. You can't say that they mean any people have or have not been infected with H7N9 at all.

Instead, you need actual measurements using serum samples from people who are known to have been infected with H7N9 to tell you what your titers mean. Someone has to study a number of patients to see what their actual H7N9 antibody titers are, and then a mathematical analysis of that data is done to come up with a threshold titer value, above which serum samples can be said to have come from people infected with H7N9 with some large degree of certainty, and below which they are thought to have come from people who were not infected. We've seen almost no asymptomatic cases (cases with no symptoms), so we really can't say much about them. So we have to go with data from H7N9 patients who have had symptoms. Here's a great graph showing antibody titers, as measured using the HI assay, in serum samples from H7N9 patients:⁵

Figure 1. H7N9 HI
Euro Surveill. 2013 Dec 12;18(50):20657

As you can see in the graph above (Figure 1), by around 3 weeks after infection onset, all samples from patients whose HI titer was measured had titers =40.

The graph below (Figure 2), from a different study,⁴ shows that the HI titer for all H7N9 samples studied by this set of authors was also =40. In addition, this graph shows titers from "control" samples (i.e., samples from people who did not have H7N9 infections); all control samples had titers that were <40.

Figure 2: H7N9 IC50 HI4
Emerg Infect Dis. 2013 Oct;19(10):1685-7

Finally, below (Figure 3) is another nice graph, from a third study,⁶ showing anti-H7N9 antibody levels ("IgG"), "HI" assay results and MN assay ("NAb") results for several H7N9 patients, again showing that all samples from the H7N9 patients studied had HI titers =40. This graph also shows that all H7N9 patient serum samples had an MN titer of =20, if samples were taken after enough time had elapsed since their infections had started.

Figure 3. H7N9 IgG HI NAb.
Emerg Infect Dis. 2014 Feb;20(2):192-200

In other words, if an individual's anti-H7N9 antibody titer is =40 by the HI assay or =20 by the MN assay, these data suggest that we could pretty safely say that he or she has had a symptomatic H7N9 infection within the past few months, and if the HI or MN titers are below those cutoffs, then the individual probably hasn't had a symptomatic H7N9 infection. We don't know to what extent asymptomatic H7N9 infections will be captured by these cutoffs, but it is likely that some asymptomatic cases would be missed using these cutoffs. It is also possible that some mild infections could be missed using these cutoffs. However, it would be a great step forward just to get estimates of what percentages of any regional population or occupational group of people have had any kind of H7N9 infection. A comparison of antibody titers for asymptomatically infected and symptomatically infected H5N1 cases may be instructive when thinking about H7N9.⁷

WHO guidelines are even stricter than the cutoffs discussed in the paragraph above. WHO guidelines say that, using the HI assay, only single samples with titers of =160 can be considered "seropositive": "Paired sera (acute and convalescent sera) with a 4-fold rise in HI titer or single sera collected in convalescent phase with HI titer of =160 could be considered as H7N9 HI antibody positive. Sera with HI titer of 20-80 should be confirmed by MN or WB assay."¹ For the MN assay, however, the WHO does not give specific cutoffs: "With single-serum samples, care must be taken in interpreting low titers such as 20 and 40. Generally, knowledge of the antibody titers in an age-matched control population is needed to determine the minimum titer that is indicative of a specific antibody response to the virus used in the assay."²

Now, it should be noted that WHO assay instructions recommend the use of horse red blood cells for the HI assay, and not everyone uses horse red blood cells. Some people use chicken, turkey, guinea pig or other kinds of red blood cells. That starts making comparisons between different groups' assays difficult. Horse red blood cells are better to use than turkey red blood cells for H7N9 because they have more a2,3-linked ("bird") sialic acids (influenza receptors); HI results are more sensitive with horse red blood cells. In other words, it may take less antibody in the assays to get the same result using horse red blood cells than it would using turkey red blood cells. This would translate into a higher number, when discussing H7N9 patient titers, for HI assays using horse red blood cells, compared to assays using turkey red blood cells. I have not seen direct comparisons of titers obtained using different types of red blood cells in HI assays specifically for H7N9, but the situation is probably similar to that for H5N1.⁸

Figures 1 and 3 above were made with HI data obtained using horse red blood cells. Figure 2 used guinea pig red blood cells. Are they completely comparable? No. Are they pretty comparable? Yes.

Are you getting a feeling for how complicated it is to interpret a seroprevalence paper? And for how difficult it is to compare results across studies?

Why does all of this matter?

It matters because some seroprevalence studies don't use appropriate cutoffs. And because it can be hard to determine even what an appropriate cutoff is when red blood cells from different species are used in an HI assay. This is where the reader has to be really careful. Cutoffs for seropositivity have been a big issue⁹ with H5N1 seroprevalence studies; some researchers have used cutoffs that were too low, and hence they have almost certainly overestimated how common H5N1-specific antibodies were in the populations studied.

So far, only one H7N9 serology paper published to date has reported probable seropositive samples, and this paper simply reported HI titers without using any specific threshold for seropositivity. Only one used study horse red blood cells in HI assays. The one paper that used an MN assay did use appropriate cutoffs. It should be noted that the new WHO HI guidelines were only published in December, 2013, after a couple of these papers were already published.

Here are the studies that have been published so far (I hope I haven't left any out):

1. Bai et al.¹⁰ looked at serum samples collected before November, 2012 from poultry workers in eastern China and found no H7N9-positive samples. The study used HI and MN assays. Turkey red blood cells were used in the HI assay. Appropriate cutoffs were used for the MN assay.
2. Hsieh et al.¹¹ studied 14 close contacts of the first H7N9 case in Taiwan. The authors took blood samples within 18-28 days after the contacts' earliest exposures. The authors used an HI assay but not an MN assay. They used turkey red blood cells for the HI assay. They found all contacts to have an HI titer �10, and declared all to be seronegative. The HI titer for the H7N9 patient in their study was 1:80. These conclusions seem very sound.
3. Yang et al.¹² looked at serum samples from 1129 people from regions of China in which H7N9 cases had been seen, and from 396 poultry workers from 10 districts in which H7N9 cases had been seen. None of the samples from the general population was found to be seropositive, whereas >6% of the poultry workers were found to be seropositive. The authors also examined serum samples from several H7N9 patients. The study used an HI assay but not an MN assay. The authors used a cutoff of =80, along with turkey red blood cells, for the HI assay. Because the authors examined serum samples from H7N9 patients using their methods and got results that are reasonably similar to other results, their cutoffs are most likely reasonable, and their conclusions are probably quite sound. The authors report:
• "Of the 1129 serum samples collected from individuals (age range, 1�88 years) in the general population, 9 (0.8%) had an HI titer of=40 to in?uenza A(H7N9), but no serum samples with an HI titer of=80 were found (Table 1). In contrast, among poultry workers, 13.9% (55/396) and 6.3% (25/396) had in?uenza A(H7N9) antibody titers of =40 and =80 (20 had an HI titer of 80, and 5 had an HI titer of 160), respectively."
  
  It is hard to imagine that an HI titer of 160 can be a spurious finding ("non-specific," to the initiated). Thus, these data strongly suggest that at least some H7N9 cases have been going undetected among poultry workers. Suppose we consider only the poultry workers with HI titer =80, or 6.3% of the poultry workers. If we then consider how many poultry workers there are, total, in districts from which H7N9 cases have emerged, then this study suggests that it's possible that quite a large number of poultry workers have been exposed to H7N9. Still, this study examined only a very small number of people, and we should be cautious about reading too much into these results.
4. Qiu et al.¹³ looked at 3 H7N9 patients and 3 close household contacts of the patients who were exposed before infection control practices were put in place. The authors looked for viral RNA using a sensitive test (PCR) and examined serum samples drawn 15-26 days post-exposure using both an HI assay and a pseudovirus-based neutralization assay. They found no contacts to be seropositive. The H7N9 patients had HI titers that reached 160-640 during this time, and the patient contacts all had titers <10. The authors used horse red blood cells for the HI assay. These findings also seem sound.

To summarize, the conclusions from all of these papers do seem sound. But, it would be wise to keep all of these issues in mind as subsequent studies appear over time.

An additional study¹⁴looked at antibody titers in 1723 serum samples collected in Vietnam using a very different kind of assay (a protein microarray). Because seropositivity cutoff levels had not been determined with authors' assay methods using actual H7N9 patient samples, these authors were appropriately very careful not to attempt to draw any conclusions about H7N9 seroprevalence from their data:

"Because titers calculated from our assay are not directly comparable to HI or microneutralization tests, no cutoff is chosen to represent positivity or clinical protection. It is not possible to associate these titers with past exposure or past infection, as serological assays have not yet been validated for H7N9."

For the future

So, as new H7N9 serology studies gradually come out, you be the judge. Figure out whether they're believable or not. Ask yourself the following:

1. What assay(s) were used? Did the authors use an MN assay? They get bonus points if they did. 
• If only an HI assay was used, then the conclusions are slightly less certain than if an MN assay was used.
2. If the authors used an HI assay, what species were the red blood cells from?
• If horse red blood cells weren't used, then HI titer cutoffs lower than 160 are probably appropriate, but there is also more uncertainty about what an appropriate cutoff would be.
3. What cutoff(s) did they use for seropositivity in their assay(s)? Do these cutoffs mesh with WHO guidelines? Do they mesh with what we know about H7N9 patient HI and MN antibody titers?

References

1. http://www.who.int/influenza/gisrs_laboratory/cnic_serological_diagnosis_hai_a_h7n9_20131220.pdf
2. http://www.who.int/influenza/gisrs_laboratory/cnic_serological_diagnosis_microneutralization_a_h7n9.pdf
3. http://www.virology.ws/2009/05/27/influenza-hemagglutination-inhibition-assay/
4. Qiu C, Huang Y, Zhang A, Tian D, Wan Y, Zhang X, Zhang W, Zhang Z, Yuan Z, Hu Y, Zhang X, Xu J. Safe pseudovirus-based assay for neutralization antibodies against influenza A(H7N9) virus. Emerg Infect Dis. 2013 Oct;19(10):1685-7
5. Zhang A, Huang Y, Tian D, Lau EH, Wan Y, Liu X, Dong Y, Song Z, Zhang X, Zhang J, Bao M, Zhou M, Yuan S, Sun J, Zhu Z, Hu Y, Chen L, Leung CY, Wu JT, Zhang Z, Zhang X, Peiris JS, Xu J. Kinetics of serological responses in influenza A(H7N9)-infected patients correlate with clinical outcome in China, 2013. Euro Surveill. 2013 Dec 12;18(50):20657 
6. Guo L, Zhang X, Ren L, Yu X, Chen L, Zhou H, Gao X, Teng Z, Li J, Hu J, Wu C, Xiao X, Zhu Y, Wang Q, Pang X, Jin Q, Wu F, Wang J. Human antibody responses to avian influenza A(H7N9) virus, 2013. Emerg Infect Dis. 2014 Feb;20(2):192-200
7. Buchy P et al., PLoS One. 2010 May 27;5(5):e10864
8. See, e.g., Table 4 in Pawar SD et al., Virol J. 2012 Oct 30;9:251
9. Osterholm MT and Kelley NS, MBio. 2012 Feb 24;3(2):e00045-12
10. Bai T et al., N Engl J Med. 2013 Jun 13;368(24):2339-40
11. Hsieh SM et al., J Infect. 2013 Nov;67(5):494-5
12. Yang S et al., J Infect Dis. 2014 Jan 15;209(2):265-9
13. Qiu C et al., J Clin Virol. 2014 Feb;59(2):129-31
14. Boni MF et al., J Infect Dis. 2013 Aug 15;208(4):554-8

NOTE: I did not have a hand in writing this post and thus take no credit for it. This was entirely the work of the Guest Writer.