2-4-6-8 Don’t be so quick to correlate

EATING OR DRINKING    IS LINKED TO           P-VALUE
Raw tomatoes                       Judaism                        <0.0001
Egg rolls                                  Dog ownership           <0.0001
Energy drinks                       Smoking                        <0.0001
Potato chips                          Higher score
                                                   SAT math v verbal      <0.0001
Soda                                         Weird rash                    <0.0002
Shellfish                                 Right-handedness      <0.0002
Fried/breaded fish             Democrat                       <0.0007
Beer                                         Frequent smoking      <0.0013
Coffee                                      Cat ownership              <0.0016
Salt                                           Likes ISP                        <0.0014
Steak w fat trimmed          Atheism                          <0.0030
Bananas                                 Higher score
                                                   SAT verbal v math      <0.0073
Cabbage                                  Innie bellybutton         <0.0097

This spurious [and pretty damn funny] correlations table is from a smart, thoughtful article recently posted on FiveThirtyEightScience. It’s about why it is so very difficult to get trustworthy, consistent information about diet and nutrition.

Here’s the problem. Or rather, the problems:

There’s lack of consensus about what makes for a healthy diet. Yep, it’s that basic. There are raw foodists and calorie-restrictors, vegans and paleos, gluten-frees and dairy-frees, Mediterranean fans and Asian followers. There’s scientific evidence to support the health benefits of all these regimens. No one is making a case for a high-sugar, low-fiber, processed food diet, but the inclusion (or exclusion) of meat (yes beef, no beef? grass fed?), dairy (milk no, but yogurt yes?), various grains (demon wheat, angelic quinoa?), fruit (blueberries as panacea?), vegetables (kale kale kale…really?), coffee (no! yes!) all get big media attention, millions of adherents – and (confusingly) credible research backing.

That’s because scientific research on diet and nutrition is flawed. I’m not talking about the Beef people tweaking numbers or the Dairy lobby funding its own studies. I am talking about the underlying method used by top-notch researchers. Studies on the possible connection between certain foods or dietary regimens and health are retrospective. That is, researchers ask eaters to keep food diaries or fill out questionnaires about what and how much they eat after the fact. As anyone who has ever tried to keep a food diary can attest, this is not easy — unless you always eat at home and prepare your own food. Also, the farther back you have to remember, the less accurate you are (but a number of studies ask you to respond to “in the last month, how often have you eaten…”) It is also a well known phenomenon in the research that people like to report they eat healthier than they do.

The potential for flawed data continues when researchers ask a zillion other questions about the eater’s habits and lifestyle. The more data collected, the more possible it is to find correlations between responses that may in fact have no connection at all outside the realm of statistics. In fact, it’s ridiculously easy to link individual foods with reported behaviors or conditions. As  as a computational physiologist says in the article I urge you to read: These connections are nothing more than circular reasoning.  “You’re taking one type of subjective report and validating it with another form of subjective report.” And so we get corrections between use of table salt and one’s level of satisfaction with an internet provide. Or consumption of potato chills and SAT math scores. (So that’s why my math score was so low!)

Read the article. It’s funny and fascinating. And sobering.