Health Care Anecdotes — And Data

One of the things I’m genuinely trying the hardest to work my way through with regard to the health care debate is the delicate balancing act that goes into how much credence to give first-hand accounts (i.e. – anecdotes) regarding various health care experiences. Here’s what’s been bugging me: here in the American south, I hear virtually nothing but negative stories about the system of VA hospitals, and also about any other country’s health care systems. Keep in mind, this is all anecdotal. Similarly, people who’ve lived in other countries for a few years will tell you horror stories about those countries’ health care systems.

However, I know too many people, both here and elsewhere, who also have anecdotes about how terrible this or that insurance company here in the states treated them, as well – or this or that hospital, etc. One thing that’s obviously true is that, with something as large as health care, no system will be perfect. But it’s nearly impossible on a personal level to tally up the stories, one against another, and come up with a clear picture of which system is worst (or best).

For example, despite the almost-universal anecdotal panning of the VA system, a 2005 study by the conservative-leaning RAND Corporation (pdf) discovered that the VA actually scored better on many metrics of care – and patient satisfaction – than private insurance companies. So, who to believe: the RAND Corporation, or the mountain of anecdotal evidence? In this case, I’m going to have to go with the RAND Corporation. Why? Well, it’s been said that the plural of anecdote is not data. Of course, it’s also been said that the plural of anecdote is data. I actually think both are true, depending upon what you’re considering. The original quote was that the plural of anecdote is data. What was meant by this is simply that the process of collecting numerous anecdotes eventually amounts to a data set. This seems to be quite an elementary formulation; it’s the way polls are conducted, at the most basic level: ask enough people what they think about ____ and you can get a pretty good idea of what the general population thinks.

Unfortunately, while it’s undoubtedly true that you can get a decent idea of what the general population thinks by asking a subset, it’s not a certainty that you will get a reliable idea of what the general population thinks just by asking some people a question. Polling is a very detailed science/art, about which a great deal has been written from many perspectives, but in essence: if it were easy to get a good idea about what the general population thinks on a subject just by asking a few people a question, then there wouldn’t be any variance between the results obtained by two different polling firms who ask the same question during the same time-frame. But there always is, right? Some polls show one thing, some another. What it boils down to is that there are a virtual galaxy of variables which have to be taken into account in order for a poll’s data to be considered reliable. Even then, humans being what they are, the results are never 100% certain. It matters how many people you ask, how you frame the question (i.e. – specific wording), whether the people you ask are representative of the population in terms of gender balance, age, ethnic background, etc, etc, etc…way too many factors to list.

Anyway, all of those hard-to-pin-down factors are what led to the inverse of the original quote: that the plural of anecdote is not, in fact, data. Or at least not reliably. The meaning of that rather pithy inverse-quote is that, when trying to extrapolate over a large enough data set (say, the people in the United States who consume at least some health care annually – which is virtually all 316,000,000 of us), second-hand anecdotes from one person’s experience aren’t really a worthwhile gauge of the perceptions and mood of the general population. Even two or three similar anecdotes aren’t. Yes, if you meticulously adjust for that galaxy of variables I mentioned, and you ask a large enough group of people, then you’ve probably got a set of data which is pretty reliable. But I think that the tongue-in-cheek inverse quote is meant to signify that just because someone tells you that they heard it from a guy that people hate (or love) ______, isn’t representative of anything but that individual’s opinion.

The one good thing about data is that, provided you have the time and energy to dig into the details to discover its validity, you can always assess whether a given data set – or poll – is worth listening to. There are plenty of unscrupulous political operatives of all stripes who engage in what’s called “push-polling”; in other words, asking loaded questions to a slanted subset of the population, in order to get the “answer” that the poll’s data provides to be the answer they wanted to hear. It’s not always easy to figure out which polls are bogus or slanted, and which are likely to be pretty accurate, but spend a little time doing it, and you usually can tell.

Not so with anecdotes. And it seems to me lately, perhaps especially not so with anecdotes regarding health care, health insurance, and potential reforms. Everybody’s got a story. Maybe you have one yourself. If not, chances are good that you know someone who does – often a close family member or friend. Perhaps a co-worker. Although the age-old cliché tends to be true: that people remember – and re-tell – negative or bad stories more often and more vividly than good or positive ones, if you pay attention, you’ll hear all sorts of anecdotes, both positive and negative, about this or that health insurance company, the VA, what health care was like when so-and-so lived in England or France or Canada, etc. How relevant are they? Again, it all depends (and no, I’m not going to give you a way to tell which ones are good and which aren’t, because there isn’t any reliable way). Most people tend to trust anecdotes from people they know well – provided that the person in question is someone they consider wise and/or trustworthy. People put a lot more stock in a vividly-described anecdote from a friend or relative than they do in “abstract” polling data by some big firm. But should they?

I’ll reiterate my own bias here: I believe that soundly-done, large-scale data are always better than even the most moving anecdote from a trusted friend, assuming the goal is to figure out attitudes regarding something that large numbers of people have experienced. That’s not limited to health care reform options or Presidential candidates, either: if a good friend tells me that hang gliding is fun and not scary or frightening, but a comparison of injury rates among sports tells me that hang gliding is as likely to cause injury as tackle football, I tend to believe the data, and take the friend’s opinion with more of a grain of salt.

Obviously, as this August has shown everyone, when it comes to health care reform options, the stakes — and the passions — are high. There are huge corporate (and other) interests which all have a stake in one or another outcome of any eventual reform (or in opposing all reform). There are huge incentives for the sort of biased push-polling I mentioned earlier. And, among individuals, there is plenty of bias and preference based upon personal experience, political indoctrination and other factors, as well. How to sort it out? Well, as I said with regard to data: the steps don’t change. Just look at who’s saying what, and see if their methodology makes sense and conforms with reality. That’s why I tend to go with the RAND Corporation’s meticulous study, rather than the avalanche of negative anecdotes regarding the VA I hear. Am I discounting those anecdotes out-of-hand? Am I being a data-nerd who looks at only numbers and not people? I don’t think so. Though polls can be made to show things which don’t reflect reality, when done by people with an agenda and few scruples, a good representative sampling is hard to argue with.

But there’s even another set of anecdotal conclusions I wish to disagree with: the near-universal awfulness of other countries’ health care systems. I’ve already linked to a set of horror stories about our own, private health care system’s insurance companies. It’s far from the only one: such stories abound on the Internet. In fact, before the politics truly became engaged with regard to people’s opinions about the private health insurance companies in America – say prior to the release of Michael Moore’s “Sicko” – it was difficult to find people who were thrilled with our system of health insurance. In fact, health insurance companies ranked right down there with used-car salespeople and genital warts on the anecdotal popularity scale. To be fair, much of that may be due to exactly what I described above: people remembering the bad things more than the good things. I’m not going to try to discount negative anecdotes about the VA by saying they’re biased, and then turn around and use the same bias to support the conclusion that people think US health insurance companies stink. However, it is worth noting that the anecdotal evidence – until the last two or three years when the politics of health care reform set in – was disproportionately against the private health insurance companies.

When it comes to other countries’ health care delivery and national health insurance/health care, though, the anecdotal evidence has, if anything, actually gotten worse since the politics of national health care kicked in. If anything, people (especially people here in the south) who have had experience with other countries’ health insurance systems view those countries’ systems even more negatively. But again, the question to ask is: how much credence should we give the anecdotal evidence? And in this case, I’m tempted to wonder if the correct answer is: even less.

Why? Because in addition to the notion that people tend to remember horror stories more than great ones (when it comes to dealing with large organizations), there’s something else at play. I’ve observed for years that people who have an opinion that a new proposal or idea or trend is a bad thing tend to seize upon instances of that idea/trend having failed as at least evidence, if not conclusive proof, that the idea/trend is a failure. Example: it’s not at all uncommon to hear someone opine, upon reading or hearing about a gay or polyamorous relationship that ended badly, that this “goes to show” that homosexuality or polyamory “doesn’t work,” despite the fact that although such an occurrence could be an indicator of such a thing, it’s nowhere near “proven.” Contrariwise, you almost never hear someone, upon learning of a friend or acquaintance’s divorce, that this is “proof” that marriage or monogamy itself – and certainly not heterosexuality – “has failed.”

Yet people rarely notice this: very few people would even think to wonder whether marriage, monogamy or even heterosexuality were “failures” just because someone got divorced, because all three of those things (marriage, heterosexuality and monogamy) have been the cultural norms for as long as anyone alive can remember. It just doesn’t occur to people to wonder such a thing. But open and unapologetic homosexuality, polyamory or partnership without marriage is FAR from the norm today, and even less so as we go back in American history. Thus, depending upon the upbringing, experience and biases of specific individuals, it’s much more likely to hear people opine – and others to agree – that the failure of a specific gay relationship, or polyamorous relationship, is “proof” that either of those concepts “can’t work” or “fails,” when, in fact, nothing of the kind is necessarily true.

When it comes to health care and health insurance delivery mechanisms, I’ve been wondering whether a similar effect might be in play regarding the anecdotal evidence. In addition to all the factual data which indicates that the United States pays far, FAR more than any other country per person for our health care and gets lower-quality care in many if not most respects, I wonder if another additional reason to view the torrent of negative anecdotes about other countries’ health care is due to the fact that in America, our private health insurance system is the norm; it’s what we know. Whether it’s good or bad (relatively), it benefits from the “devil we know” effect: people in general have to be VERY dissatisfied with the status quo before they’re willing to risk the unknown. Even if it’s not untested (as universal, single-payer style health care is not untested), it’s still unknown personally to most Americans. So, when someone who’s been born and raised in America winds up living in France or Canada or England for a few years, in their 30s, they approach that country’s health care system with a built-in, invisible bias against it, and in favor of the system in which they grew up.

That’s not to suggest that people are slaves to experience or can’t think for themselves. But the weight of inertia and familiarity – as I demonstrated with the marriage example – definitely do play some part in people’s perceptions. So, if they are misdiagnosed in a foreign hospital, or they have some sort of problem with one of their doctors, nurses or other aspect of the health care system, many people are unconsciously far more likely to extend that bad experience into an indictment of the entire health care system. While, back here in America, even the pages upon pages of make-you-shudder anecdotes regarding domestic insurance companies denying payment or coverage, bad mistakes in care, etc, don’t result in an overall, systemic indictment of the health care system even to the people who experienced the horror story. You are much more likely to hear someone who’s had a bad experience here say “I will never trust or use _______ insurance company (or hospital, or doctor) again, than you are to hear them say “this just goes to show that the for-profit system of private health insurance we have in the United States doesn’t work.” But when Americans live/travel abroad, it seems sometimes (anecdotally, LOL) that almost the reverse is true when they have bad experiences with other countries’ health care.

Of course – to be fair – I have no actual data which proves any of this…I just wonder to what degree we are conditioned to…if not prefer, then at least be comfortable with, whatever we’re familiar with, so that when it fails to perform well, we don’t even think to indict the entire system.