clock menu more-arrow no yes

Filed under:

Medical journal’s bogus investigation could derail better dietary guidelines

The BMJ published an article that makes erroneous statements

National Cancer Institute / wikimedia commons

Every five years, the US government issues new dietary guidelines for Americans. These guidelines have a big impact on food labeling and scientific research, as well as on the way Americans eat. The formulation of guidelines starts with a report written by a group of experts — this year’s committee consisted of 14 scientists convened by agricultural and health regulators — who review the latest evidence and come up with suggestions regarding food that consists "a healthy diet" and reduce the risk of chronic disease.

The 2015 guidelines will be released later this year; the report that informs the guidelines was published on February 19th. Now, as an October Congressional hearing approaches, the British Medical Journal has published an "investigation" that attacks the US dietary advisory committee for what is calls an "overall lack of sound science and proper methods." Unfortunately, the denunciation suffers from many of the same flaws that the journalist who wrote the piece, Nina Teicholz, accuses the US report of having. A copy of the article sent to reporters prior to its publication also did not disclose Teicholz’s own conflict of interest. (She is the author of a book called The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet.)

The BMJ alleges that the report didn't look at all the "relevant scientific evidence."

The article in BMJ alleges that the US dietary committee’s report "does not take account of all the relevant scientific evidence." The article goes on to make erroneous statements, like that the US report has "deleted meat" from its recommended food list, and "abandoned established methods for most of its analyses." Reading the US committee’s report, though, shows that not only is meat not "deleted" but that some of the studies the BMJ article faults the report for ignoring are, in fact, irrelevant.

But because the BMJ is a well-respected medical journal, the investigation might still get some traction — especially with Republican senators and representatives who signed letters criticizing the committee’s stance on red meat. Efforts to disparage the committee’s report could have an effect on the guidelines that the US government eventually puts out. (Lobbying around the dietary guidelines is perennially intense.)

"These guidelines are hugely influential, affecting diets and health around the world," the BMJ's editor-in-chief, Fiona Godlee, said in the journal's press release for the investigation. "The least we would expect is that they be based on the best available science." And one would expect the same of the BMJ.

The Verge contacted the BMJ by email and by phone multiple times to inquire about how these mistakes made it into the journal, but no editor responded to us prior to publication — despite promises from press officers that someone would. Teicholz, the author, did reply.

Deleting meat

The BMJ’s investigation condemns the US dietary advisory committee for "deleting meat from the list of foods recommended as part of its healthy diets." This would be outrageous — if it were true. It’s not. The list to which the BMJ’s investigation refers actually states the following:

The overall body of evidence examined by the 2015 DGAC identifies that a healthy dietary pattern is higher in vegetables, fruits, whole grains, low- or non-fat dairy, seafood, legumes, and nuts; moderate in alcohol (among adults); lower in red and processed meats; and low in sugar-sweetened foods and drinks and refined grains.

Additional strong evidence shows that it is not necessary to eliminate food groups or conform to a single dietary pattern to achieve healthy dietary patterns. [Emphasis ours]

And in a footnote, the advisory committee report states quite clearly that "lean meats can be a part of a healthy dietary pattern." So yes, the committee does think that reducing the intake of two kinds of meat — red meat and processed meat— and eating more grains, dairy, fruits, and vegetables is a good idea. But in no way are they suggesting that people should "delete meat" from their diet. In fact, the amount of meat, poultry, and eggs that the committee recommends — 26 ounces per week for a 2,000-calorie diet — remains unchanged from 2010.

"In no way are they suggesting that people should 'delete meat' from their diet."

This actually isn’t the first time that Teicholz has misrepresented the report. In February, she wrote in a New York Times op-ed that "the committee’s new report also advised eliminating ‘lean meat’ from the list of recommended healthy foods, as well as cutting back on red and processed meats." This is demonstrably untrue. As of publication, no correction has yet appeared in the Times.

"The report always says to lower the intake of red and processed meat," says Anna Maria Siega-Riz, a nutritional epidemiologist at the University of North Carolina at Chapel Hill and a co-author of the advisory committee report. "It never says eliminate it."

When asked about the wording in the BMJ article, Teicholz said that she made a fair assessment of the report’s content. "There’s something called a ‘list of healthy foods,’ and they delete it from that list," she says. "There are areas where they suggest reducing meat and they model it differently; they have a footnote contradicting themselves that overall shows to me that the report does not have a clear sense of what it wants to say."

"The report does not have a clear sense of what it wants to say."

Teicholz also claims the committee ignored "relevant scientific evidence" in making dietary recommendations. According to her, studies that should have been included in the dietary committee’s report were wrongly omitted. These are serious allegations, if true, and worth examining in detail.

Two studies she cites as an omission are a "meta-analysis and a critical review" that "concluded that low carbohydrate diets are better than other nutritional approaches for controlling type 2 diabetes." In fact, the studies are not included. There is a reason for that: if these studies had been included in the committee’s analysis, that would have been overstepping the bounds of the report. That’s because the guidelines aren’t supposed to be used to manage or treat an illness; they’re supposed to provide Americans with advice on how to remain healthy. Once someone develops an illness like diabetes, the guidelines aren’t applicable anymore.

"The report was having to do with prevention of health outcomes, so once you have the health outcomes, that's a totally different issue," says Siega-Riz. "Her mention of that [study] has no relevance whatsoever."

"Her mention of that [study] has no relevance whatsoever."

Teicholz disagrees. "I guess that your point is [the committee] has a mandate to prevent diabetes and not cure diabetes," Teicholz told The Verge in a telephone interview. "Well I think if you talk to any scientist they will [say] you don’t have one approach for preventing a disease, another for fighting a disease, and another for curing a disease — it’s all part of the same approach of what is a healthy diet." She is essentially faulting the committee for doing what they were asked to do.

In another instance, Teicholz condemns the advisory committee for "omitting" a "large controlled clinical trial, the Women’s Health Initiative, which included nearly 49,000 people" from their review on saturated fats. The study was omitted because of its design; it was not designed to look at saturated fat, according to USDA and the US Department of Health and Human Services spokespeople. "Since it examined all types of fat and did not look at saturated fat in isolation, this study was not used in 2010 to answer questions relating to saturated fat," they told us. From the study:

The primary aim of the Women's Health Initiative (WHI) Dietary Modification Trial was to test whether behavioral intervention intended to produce a dietary pattern low in total fat, along with increased intakes of vegetables, fruits, and grains, would decrease the incidence of breast and colorectal cancer in postmenopausal women. A secondary aim was to test whether such a dietary intervention, which did not focus on the intake of specific fats, would also reduce the risk of CVD. [Emphasis ours]

The study is not designed to test hypotheses about saturated fats. Which, of course, means that it can’t use it to draw conclusions about saturated fats; omitting it was correct. Amusingly, the Women’s Health Initiative was included in a different review used by the committee one for overall diets.

Faulting the committee for doing what they were asked to do

Teicholz doesn’t agree that the study wasn’t designed to look at individual fats, like saturated fat. The Women’s Health Initiative "intended to reduce saturated fats to below 7 percent, which is in the original protocol of the paper, and a reduction was successfully achieved, as noted in the reference in my BMJ piece," Teicholz says. But that isn’t right.

The study wasn’t set up to evaluate saturated fats, but the authors were aware that reducing the total fat that people consume would likely also affect the amount of saturated fat they ingested. (The 7 percent figure Teicholz cites comes from the study itself, and the somewhat wonky language is this: "It was presumed that by reducing total fat intake to 20 percent kcal, intake of saturated fat would also be reduced (7 percent energy intake).") Saturated fat was never the focus of the research.

Methodological flaws

Teicholz also suggests that the committee failed to use "Nutrition Evidence Library" reviews to answer "over 70 percent" of the questions in the report. The NEL, which is run by the USDA, is charged with performing "systematic reviews, or pulling together the best available research to answer important food- and nutrition-related questions." These reviews are considered very rigorous. But the NEL methods aren’t relevant for close to half of the questions asked. Essentially Teicholz is scolding the dietary committee for ignoring irrelevant studies.

Of the 83 questions asked by the committee, 20 related to trends in food consumption. To answer these questions, the committee needed original analysis of the data in the most recent national health and nutrition surveys, as well as data relating to the most common foods consumed in the US. NEL reviews involve already-published studies, but no other studies conducted using this survey met the needs of the committee — so a brand new analysis had to be performed.

The committee ignored irrelevant studies

"There is no NEL review or alternative approach that could possibly supersede the approach that the 2015 advisory committee used to answer these questions," says Rafael Perez-Escamilla, an epidemiologist at Yale University and a co-author of the advisory committee report.

The committee relied on a 2008 government report issued by the physical activity guidelines committee to answer an additional 18 other questions because physical activity is beyond the scope of the USDA’s Nutrition Evidence Library. Because the physical activity guidelines committee is only supposed to issue new recommendations about once every 10 years, the dietary committee had to use the first and most recent report. "We used the physical activity report done by the government — there was no [other] data that superseded it," Siega-Riz says.

Once the questions that the NEL can’t answer are eliminated, it becomes clear that the committee actually cited NEL methods as the "source of evidence" for 22 of the 45 questions, or 48 percent of the time.

"I was just reporting on what they said in the report."

Teicholz says that this distinction — that some questions couldn't be answered using NEL methods — isn’t specified in the report. This is an oddly incurious response for someone purporting to be an investigative journalist. "I was just reporting on what they said in the report," Teicholz says. "I think that what I stated was accurate."

Conflicts of interest

The BMJ’s misleading attacks weaken the article’s legitimate criticism. For instance, the advisory committee members did not publicly disclose their conflicts of interests.

"Much has been written about how industries try to influence nutrition policy, so it is surprising that unlike authors in most major medical journals, guideline committee members are not required to list their potential conflicts of interest," Teicholz writes in the BMJ. When Teicholz asked committee chair Barbara Millen about this, Millen’s reply was simple: members were vetted by the federal government — a fact that was confirmed to The Verge by three committee members as well as by spokespersons from the HHS and USDA.

"All special government employees are required to file confidential financial disclosure reports," two HHS and USDA spokespeople told us (they declined to be identified). These financial disclosure reports are used to assess whether committee members "are able to perform their expected job duties as [advisory committee] members" while complying with the government's ethics laws. But these disclosures aren’t subject to the Freedom of Information Act, and they are not released to members of the public.

As Teicholz points out, committee chair Barbara Millen is "president of Millennium Prevention, a company based in Westwood, Massachusetts, that sells web-based platforms and mobile applications for self health monitoring." The advisory report mentions "mobile health" as an "emerging area." Given that Millen works in a field that’s mentioned in the report, the disclosure seems relevant.

"Conflict of interest is always a problem with food industry funding of academics."

"Conflict of interest is always a problem with food industry funding of academics, mainly because the effects of industry funding are, according to research, unconscious, unintentional, and unrecognized," says Marion Nestle, New York University food scientist and author of Food Politics: How the Food Industry Influences Nutrition and Health. (Nestle was not involved in the current report but was senior nutrition policy advisor in the Department of Health and Human Services between 1986 and 1988.) "Such conflicts need to be fully disclosed, and I’m surprised that they weren’t."

Teicholz also has conflicts of interest to disclose. The "competing interests" section of the article that was sent to reporters prior to publication did not mention that she is the author of a book entitled The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet. The book clearly shows that she has some expertise in the area of food science, but it also shows that she has a certain point of view — one that seems relevant, given the misleading way she characterizes the committee’s response to meat. Teicholz has her own agenda to push and book to sell. That’s relevant to a reader who might rely on the BMJ for information. (In response to The Verge’s questions about Teicholz’s conflict of interest, her authorship is disclosed on the version of the article appearing online.)

"This journalist has a big conflict of interest."

"This journalist has a big conflict of interest," says Perez-Escamilla. "She has written a book where she's trying to push the idea that it’s okay to have as much butter as you want, as it’s part of a healthy diet. Where did she come from with that evidence? How does she feel that this is stronger evidence than 19 months of work from 14 scientific experts in the field?" Charles Mueller, an NYU clinical nutritionist who didn’t work on the committee, characterized Teicholz as having "an axe to grind herself."

Teicholz says that disclosing a book like hers in an article like this is "not standard." (Nor, it should be noted, is it standard for the dietary guidelines committee to disclose theirs. That doesn’t make it right.) "All it shows is that I have a stated professional investment in this point of view," she says. And "including a mention of my book would ultimately help sales."

It seems a shame that no editor at the British Medical Journal chose to respond to our questions. Obviously, greater oversight was needed for this article, or else such egregious mistakes would not appear in an "investigative" report in the journal’s pages. Now that the investigation has been published, it may end up being used as ammunition by people — most obviously, the meat lobby — who would alter the eventual guidelines.