A week ago, the British Medical Journal published an "investigation" of US Dietary Guidelines Advisory Committee's February report that The Verge found was full of errors. The BMJ's investigation went after a number of aspects of the diet committee's report, which will be used to issue new federal dietary guidelines this year. As a result, a number of news outlets — including Time, Newsweek, and Mother Jones — wrote stories that reported the investigation's errors as fact.
Now, the BMJ has published "clarifications" rather than corrections or retractions (in medical journals, a clarification doesn’t change anything in the body of an article or admit error). These clarifications address two of the issues that The Verge revealed last week in our initial coverage of the Meat Delete affair — but they miss the mark entirely. In fact, they make additional errors.
"It’s not much of a clarification."
"The clarifications are totally inadequate; it doesn't improve upon the original," says epidemiologist Barbara Millen, chair of the committee and president of Millennium Prevention, a mobile health startup. "It’s not much of a clarification," says Marion Nestle, New York University food scientist who wasn't involved in this year's committee, but was on the committee in 1995. The committee has raised "substantive objections" to the BMJ article in its response — and "the BMJ is not addressing these," Nestle says.
Every five years, the US government issues new dietary guidelines for Americans. Given how influential the guidelines are — they affect school lunches, food labelling, and even scientific research — they often face push-back from various lobbying groups. But this year, a congressional hearing has been called to address issues that some representatives say exist in the committee's report.
"The 'witch hunt'... is totally without precedent."
This is very unusual, according to Nestle and Rafael Perez-Escamilla, an epidemiologist at Yale University and a co-author of this year's advisory committee report. "Although there is always lobbying around the [committee's] report, the 'witch hunt' level that has been reached this time around is totally without precedent," Perez-Escamilla says. The hearing will take place on October 7th, and the BMJ's faulty investigation, which reads like pro-fat propaganda, could end up being used by the meat lobby. That’s because the committee wants people to eat less red and processed meats. So by issuing clarifications, rather than a correction or a retraction, the BMJ isn’t alerting unwary readers to its bad reporting — making it easier to use the BMJ's "investigation" for political propaganda.
For members of the committee, that means that the BMJ has decided to forgo truth, in favor of a political agenda. The "BMJ made the choice to publish this quite convoluted and confusing report without having given the DGAC an equal opportunity to rebut it at the same time the article was posted online," Perez-Escamilla says. "I don't think that the release of this BMJ 'report' so close to the upcoming congressional hearing on the [committee] is a coincidence."
Meat Delete 2015
The first item on the docket is the "deleted meat" clarification. The author of the BMJ investigation, Nina Teicholz, goes after the Dietary Guidelines Advisory Committee for "not only deleting meat from the list of foods recommended as part of its healthy diets, but also actively counseling reductions in 'red and processed meats.'" (Teicholz is the author of a book entitled The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet.) But as The Verge reported last week, this is incorrect. BMJ editor Rebecca Coombes admits this by "clarifying" that what Teicholz meant to write was "deleting lean meats." In her own words: "We accept that the article would have been clearer if it had used the phrase 'deleting lean meat' rather than 'deleting meat.'" 1
The full clarification reads: The article sought to report how the DGAC has dropped lean meat from the list of foods recommended for a healthy diet. Although lean meats are recommended in the 2010 guidelines, they no longer appear in the committee’s proposals for the updated 2015 guidelines.The article says: "New proposals by the 2015 report include not only deleting meat from the list of foods recommended as part of its healthy diets, but also actively counselling reductions in ‘red and processed meats.’" We accept that the article would have been clearer if it had used the phrase "deleting lean mean" rather than "deleting meat."
It's already odd that a piece labeled as an "investigation" would fail to distinguish between types of meat. After all, that there are good fats and bad fats is part of the crux of the case for fats in the first place; nuance would seem important here. But that’s not all that’s strange here. What’s stranger is that this sentence is demonstrably wrong. Compare, below, the list of recommended foods from 2010 (left) and 2015 (right):
These images show that the 2010 report actually recommends that people should eat "only moderate amounts of lean meats." In contrast, the 2015 report simply suggests reducing the quantity of red and processed meat — and says nothing of lean meat. So it's hard to see how the 2015 report, which states quite plainly that "lean meats can be a part of a healthy dietary pattern," could be deleting meat — lean or otherwise.
In addition, the 2015 report contains a table that shows that in two of the three of dietary patterns that the committee considers "healthy," meat is included. In fact, the suggested intake for meat alone is 12.5 ounces per week.
2015 Dietary Advisory Guidelines Committee report
This so-called clarification is nothing of the sort
This so-called clarification is nothing of the sort. It is, instead, an extension and amplification of the original error. It is factually incorrect that the committee thinks Americans should not eat meat.
Honesty here demands a correction.
The BMJ also "clarified" that when Teicholz wrote that the committee "did not use NEL reviews" — a very rigorous kind of systematic review carried out by the USDA's Nutrition Evidence Library — "for more than 70 percent of the topics," what she really meant to say was that "the portion of questions requiring a systematic review that did not receive one is 63 percent."
Originally, Teicholz claimed the rigorous NEL reviews weren’t used for 70 percent of the topics. This is a red herring — in some cases those reviews couldn’t answer the questions being posed. In the cases where the reviews did apply, they were used about half the time. In "clarifying" the original reporting, Coombes changes the language from NEL review to "systematic review," and claims it wasn’t used 63 percent of the time. Using this criteria and counting the questions where this larger category of reviews applied, we found that they weren’t directly used as a source of evidence 29 percent of the time.2
Teicholz’s original accusation — that NEL reviews weren’t used in 70 percent of cases — seems troubling on the face of it. So we checked. In fact, that specific type of report couldn’t answer all the questions the committee was posing; only 45 of the 83 questions the committee was considering could be answered using NEL methods. (The rest were answered using an original analysis and information from a 2008 report from the government’s physical activities guidelines committee.) Of the 45 that could be answered using that specific type of report, 22 questions, or 48 percent, were answered using NEL methods. The clarification doesn’t address this. Instead, Coombes is slippery with language, writing that 63 percent of questions that could be answered using a "systematic review" were not. But in fact, of those 45 questions we counted from the report, another 10 were answered using existing systematic reviews, which are distinct from NEL reviews in that they are peer-reviewed, high-quality reviews that were not conducted by the NEL. Simple math shows that an accurate representation of the committee’s work using "systematic reviews" specifically is 32 of the 45 questions that could be answered that way. In other words, only 29 percent of the questions that could be answered using a systemic review weren’t answered that way. Obviously, 63 percent and 29 percent are significantly different numbers.
In short, the BMJ appears to want its readers to know that its editors have either not read the 2015 report, or that they do not understand its contents. The Verge emailed Coombes and Fiona Godlee, the journal's editor-in-chief, to ask where the "63 percent" came from. We have yet to hear back.
What still needs to be 'clarified'
This inadequate clarification wholly ignores several other significant problems with the previous report. For instance, Teicholz takes the committee to task for excluding three studies. The studies were excluded rightly; they are irrelevant to the committee’s task. Here they are:
- A "meta-analysis and a critical review" that, in Teicholz's own words, "concluded that low carbohydrate diets are better than other nutritional approaches for controlling type 2 diabetes." As The Verge has previously reported, studies done on people who already have an illness — such as type 2 diabetes, for instance — aren’t included in the reviews used by the committee. That’s because the food recommendations are meant to lower the risk of chronic disease, not help patients who are already ill manage an illness.3
- A "large controlled clinical trial, the Women’s Health Initiative, which included nearly 49,000 people." Teicholz thinks this study should have been included in the committee's review of saturated fats. But there's a very good reason for why this study was omitted: it was never supposed to look at the effect of specific fats — only total fat. This means that omitting the study from that review was correct, and as a medical journal, the BMJ should know that. (For what it's worth, the WHI study was included in another review used by the committee.)4
The committee's report states that the goals of the dietary guidelines for Americans are to "provide science-based advice on how nutrition and physical activity can help promote health across the lifespan and reduce the risk for major chronic diseases in the US population ages 2 years and older."
The authors of the WHI study state in the protocol that "the primary aim of the Women's Health Initiative (WHI) Dietary Modification Trial was to test whether behavioral intervention intended to produce a dietary pattern low in total fat, along with increased intakes of vegetables, fruits, and grains, would decrease the incidence of breast and colorectal cancer in postmenopausal women. A secondary aim was to test whether such a dietary intervention, which did not focus on the intake of specific fats, would also reduce the risk of CVD." [Emphasis ours]
It took the BMJ almost a week to publish an error-laden "clarification" to an error-laden article, and the editors still didn’t bother to fix all the things the original report got wrong. (One wonders what, precisely, they were doing with all that time.)
The health of all Americans is "at stake."
Given the upcoming congressional hearing, the BMJ's failure to correct or retract the piece is blatantly irresponsible. "It is the public health of all Americans that is at stake when journals do not carefully fact check what they publish," Perez-Escamilla says. "I sincerely hope that our legislators in DC have the wisdom to understand the enormously dangerous precedent that it will represent trying to 'legislate' scientific guidelines based on inflammatory misleading journalistic articles that — for reasons that I just can't understand sometimes — end up being published in reputable scientific journals."