Illegal and unconstitutional: that’s what a sudden, blistering report from an obscure government committee has called the NSA’s phone-record database. The Privacy and Civil Liberties Oversight Board was conceived in the wake of 9/11, the event we’ve used to justify almost any level of surveillance. For five of its ten years, it hasn’t even been operational. But on Thursday, it came out with perhaps the harshest take yet we’ve seen on the NSA’s phone-record database, drawing a combination of sharp disagreement and immediate disregard from the president. In contrast to the vague promises of reform in Obama’s speech last week, though, the oversight panel’s 238-page report takes direct aim at the problems posed by the bulk collection of phone metadata — even if all it gets is a rhetorical victory.

Like Obama’s other NSA review panel, the oversight board made its decision based partly on classified documents, but the report follows a long trickle of disclosures about what the agency really does. Over the past six months, for example, the NSA’s claim that metadata collection foiled 54 terrorist plots has been largely debunked, leaving only a few cases where phone-record searches provided hints. It’s believed to have unmasked precisely one potential terrorist: Basaaly Moalin, who was convicted in 2013 not of planning an attack but of sending money to militant group al-Shabaab. The report says that while there was "critical value" in cutting off these funds, it’s telling that over seven years of data collection, Moalin’s case remains the only example of the program getting a lead on a suspect. It even suggests that the FBI may have found the link without the NSA's tip-off, and that the NSA could probably have gotten the same information without the database, even if it took more time.

Over those same years, phone companies inadvertently sent the NSA credit card numbers, the NSA accidentally kept thousands of call-record files for over the five-year limit, and technicians created automated alert lists that sucked in records without authorization, then tried and failed to fix the problem multiple times before shutting down the alerts altogether. Declassified documents so far have only revealed a dozen cases where analysts intentionally misused the database, but accidental violations keep piling up.

The report doesn’t say that the program was useless, but it effectively damns it with faint praise. The board agrees that having so much information could let agencies "triage" terrorist threats by using the database to quickly hunt down and prioritize leads. It also allows "negative reporting" — being able to see every phone communication can let agents know if a terrorist has not been in contact with anyone in the US, whether that’s for peace of mind or to focus resources somewhere else. Has having these options open helped close cases or prevent attacks, though? The worth of peace of mind is difficult to measure.

What is easy to point to is the list of plots the program hasn’t foiled. The report compares the database to a security system or insurance policy that could be valuable even if it never goes off. "Yet, it is worth noting that the program supplied no advance notice of attempted attacks on the New York City subway, the failed Christmas Day airliner bombing, or the failed Times Square car bombing." Given these failures, it concludes, it’s unlikely to provide "significant value, much less essential value, in safeguarding the nation in the future."

Obama’s ad-hoc review panel of "outside experts," which released its report back in December, said something similar: the database had never actually disrupted a plot, but it could help confirm where attacks weren’t taking place. Its take, though, proved gentler than that of a committee whose members Obama didn’t even bother to nominate during most of his first term. After the press quoted the December panel report’s conclusion that the program was "not essential to preventing attacks," chair (and former CIA deputy director) Michael Morell followed up with a scathing editorial, saying that the records database could "prevent the next 9/11" and "needs to be successful only once to be invaluable." The 9/11 claim has been disputed by other members, and the panel explicitly recommended moving information out of a government-owned database, but Morell’s argument stayed at the forefront: the program hasn’t helped before now, but it might help someday, and that’s enough.

Morell’s panel punted anything close to a legal question into the realm of the courts. But the oversight board’s biggest finding is that not only is the database almost certainly ineffective, it’s also probably illegal. The program is justified by the Patriot Act’s Section 215, which lets agencies request evidence that’s relevant to a national security investigation. But the panel says the current program "bears almost no resemblance" to this description. If a tool like the database is seen as relevant to investigations as a whole, this definition opens the door to any new, more invasive technology. "The word ‘relevant,’" the report says, "becomes limited only by the government’s technological capacity to ingest information and sift through it efficiently."

Defenders have pointed out that Congress knew about the program and voted twice to continue it, but the panel says legislators "merely postponed the sunset dates" and often had virtually no information about how the law was actually being used. On a broader level, if Congress signed off on the law, the report says it would still violate the First and Fourth Amendments: collecting the information counts as an unreasonable search, and freedom of speech suffers when the government can theoretically analyze the patterns of almost anyone’s life. Once again, metadata matters.

For anyone who’s been frustrated by the slow progress of NSA reform, this is all gratifying to hear. But does it actually mean anything? The White House has already said it believes the board’s opinion is wrong — Obama even got to propose his reforms without a public report castigating their legal basis, insisting that the program was both effective and legal. The intelligence community and attorney general will be coming back in March to present alternatives to the current government database, but they’ll likely make only the most modest changes possible.

Bills like the USA Freedom Act are meant to rein in metadata collection, and Section 215, which authorizes the phone-record collection, will come up for debate next year — it’s set to expire on June 1st, 2015. Any significant changes, though, will meet with heavy opposition. Senate Intelligence Committee Chair Dianne Feinstein (D-CA) quickly shot down the idea that Congress would go much further than Obama on reforms. "The president has very clearly said that he wants to keep the capability," she said on an NBC Meet the Press appearance. "So I think we would agree with him. I know a dominant majority of the — everybody, virtually, except two or three, on the Senate Intelligence Committee would agree with that."

What the oversight board recommendations can do is help set the tone for these debates and, because the board was given access to more information than the public will ever get, provide meaningful clues about what the NSA can do. But if we’re interested in arguments about how the program is illegal, we should look to Judge Richard Leon, who recently called bulk phone record collection an "Orwellian" overreach that was likely unconstitutional. Leon’s opinion is balanced by that of another judge, who ruled in favor of the government in a different trial. Both cases, though, will go to appeals, potentially setting the stage for a Supreme Court showdown. The chances that the judicial branch will crack down on metadata collection are far from certain, but it’s got a better shot than a report that the president decided to ignore before we even got to see it.