HowTo: Reporting bias. Or, why do little kids prefer apples from McDonald’s?

The story:

“McFood” Better Than Food, Kids Say

Robinson and colleagues studied 63 low-income children enrolled in Head Start centers in California. The kids ranged in age from 3 years to 5 years.

Told they were playing a food-tasting game, the kids sat at a table with a screen across the middle. A researcher reached around either side of the screen to put out two identical food samples: slices of a hamburger, french fries, chicken nuggets, milk, or baby carrots.

The only difference between the pairs of food samples was that one came in a plain wrapper, cup, or bag, and the other came in a clean, unused McDonald’s wrapper, cup, or bag. The kids were asked whether they liked one of the foods best, or whether they tasted the same.

I love the idea of some brainless reporter going ‘newspaper’ with the heading, yet still using the word “kids”, rather than, say, “children”.

The results of the study!

• 77 percent of the kids said the same french fries, from McDonald’s, were better in a McDonald’s bag than in a plain bag (13 percent liked the ones in the plain bag; 10 percent could tell they were the same).
• 61 percent of the kids said milk tasted better in a McDonald’s cup (21 percent liked milk in a plain cup; 18 percent could tell it was the same).
• 59 percent of the kids said chicken nuggets tasted better in a McDonald’s bag (18 percent liked them in a plain bag; 23 percent could tell they were the same).
• 54 percent of the kids said carrots tasted better in a McDonald’s bag (23 percent liked them in a plain bag; another 23 percent could tell they were the same).
• 48 percent of the kids liked hamburgers better in a McDonald’s wrapper (37 percent liked them in a plain wrapper; 15 percent could tell they were the same).

Now, I’m vegan, so my familiarity with old whatshisarches is a little rusty – but aren’t hamburgers their business? To add context:

Kids who preferred “McFood” tended to live in homes with a greater number of television sets and tended to eat at McDonald’s more often than kids not influenced by the McDonald’s brand name.

I’m not surprised the study got that the right way around, but I am pleasantly surprised that the reporter kept it so. The rest of the article goes on about marketing. Apparently McDonald’s provides parents with the “safest food” – whatever that is supposed to mean.

Now, here’s the problem. The results of this study are potentially two things, and probably a mixture of both. The first thing, the phenomen being measured and presumed to have been found, is revealed preference. For all that economists draw demand curves and insist our neo-classical models explain the world, one cannot actually measure demand, only consumption. Thus we need to measure revealed preferences, rather than preferences directly. Hence, little children are revealing a preference for foods in McDonald’s wrappers.

Here’s the thing, though. The children are tasting the food, then saying which they thought tasted nicer, as opposed to selecting one (in which case the preference for McDonald’s brands would be revealed). If the food is the same, this is irrational, but there’s a slight difference in what’s being measured and what we are concluding:

Children do not necessarily like milk better given that it is in a McDonald’s container, they just say they like it better given that they have seen it in a McDonald’s container.

That is, we aren’t measuring what the children like, but what they say they like. This exposes their responses to reporting bias. I’m a health economist, so I find this in that context. I give you reporting bias in Self-Assessed Health.

Self-Assessed Health (or SAH) is how health is often measured. It’s too damned expensive to have every respondent in the Medical Expenditure Panel Survey see a doctor. So our surveys ask something like, “how would you rate your own health?”, and give you, as options, “Very Good”, “Good”, “Fair”, “Poor” and “Very Poor”.

Now. Suppose the question did not specify that you consider only people your own age, income bracket, choice of sport (NFL players?), etc.”

• My grandparents are in their mid-80s. My grandmother has had, I think, everything replaced one can have, without having had plastic surgery. Compared to their friends, however, my grandparents both rate their health as very good. If you or I (assuming you aren’t 80, while reading this) had their health, we’d think otherwise.
• Poor people will rate the same level of health as better for them than a rich person – because their expectations are different.
• A diabetic might rate their health relative to their disease state. So they will say their health is Good, while we think, “Dude – you’ve lost a leg!” Again, different health-expectations.

This is reporting bias. Two excellent papers on this (full disclosure: both include as co-authors attractive former colleagues, and one my PhD supervisor): Reporting Bias and Heterogeneity in Self-Assessed Health. Evidence from the British Household Panel Survey; and Does Reporting Heterogeneity bias The Measurement of Health Disparities?. I remember the latter very well because using “bias” as an active verb always struck me as a little wobbly. But like I said, the colleague is very attractive. There are different standards for pretty girls (I’m incredibly superficial).

When is reporting bias a problem? If one is modelling health care demand, health insurance purchasing, etc., it is not that much of a problem. People consume according to their perception of their needs. If their perception is wrong, that’s their misfortune; only that the link between what they think they need and what they purchase is properly measured. If, on the other hand, one is modelling inequalities in health, then it is that much of a problem. As above, if a poor person rates a given level of (relatively low?) health higher than a rich person, measuring income-related inequalities in health becomes very tricky indeed. Same for international comparisons.

The latter of the two papers to which I linked is a good starting point, if you’re interested. It explores the use of anchoring vignettes to get some standard comparators in terms of health, and see how different ages, genders, incomes, nationalities affect the rating of that given health state (identified descriptively in terms of mobility, pain, etc.), and then use those as ‘handicaps’, to correct for bias in individual responses. It works fairly well.

Back to the story of the children and the clown! How sophisticated are little children, anyway? How much of a game might they be playing? Are they honestly saying, “I like this better because of the brand, and because I’m wilfully irrational”? I say this because, absent the surrounds of a McDonald’s restaurant, I don’t see how a wrapper in isolation can increase utility (in which case it might not be irrational, although it still does not affect the taste).

Might they also be saying, though, that they like the food wrapped in McDonald’s paper because they remember McDonald’s food as tasting better than other meals, including home-cooked? Following that, might they also be saying they liked it better in the hopes that their rating will get them more McDonald’s food in the future?

I submit that both are very possible. Children are certainly sophisticated enough to play their own games in a study like this one. Remember also the socio-economic gradients tentatively identified amongst the children. Split them up poor/wealthy, and imagine the food that their households consume from a supermarket, the other places they might eat, etc.:

• Odds are, the wealthier families are having better ingredients (even down to, say, frozen food) end up on their plate, so that the disparity between home-cooked and McDonald’s cooked food is greater.
• There is also potentially an exposure effect – wealthier children being exposed to a greater range of non-home eating, including proper restaurants. This would devalue the association with McDonald’s.
• In terms of the memory of a preference for McDonald’s, even the environment in which they eat is a potential factor. For a relatively poorer child, McDonald’s is nicer than his home, perhaps. Not so for a wealthier child, who may even have McDonald’s brought home, more than in a restaurant.

See? Reporting bias. Ultimately I’m saying this study is information that should be considered in a wider context. It doesn’t simply mean that McDonald’s is winning some war against parents, or that McDonald’s will be worse than ever, now that it thinks it can market milk and carrots successfully. Alternatively it is, and we should be thinking about socio-economic gradients in consumer behaviour of this type (it’s not really my area of health economics).

For me, McDonald’s will always be the back-up public bathroom if I can’t find a Starbucks. I heart New York.

« Extreme weather the norm across globe

Maybe a Chinese taxi driver can save Australia »