Theranos’ proprietary tech wasn’t vetted by federal inspectors for two years

The more we learn about Theranos, the more questions we have

Theranos, the Silicon Valley blood testing company, launched its first blood-sampling center at the end of 2013. But for most of the two years that followed, federal officials didn’t inspect Theranos’ inventions at all, The Verge has learned.

This suggests one of two possibilities: either Theranos’ customers received test results generated by equipment that no federal official checked for accuracy, or Theranos wasn’t using the proprietary technology behind its $9 billion valuation to return results to patients. A recent report from The Wall Street Journal pointed toward an extreme version of option number one: that Theranos actively hid its proprietary technology from lab inspectors, which would explain why the inventions were never checked. And Theranos has publicly proclaimed it used its technology on patients, which means either that the company’s lying or patients received tests that had not been validated by any federal authorities.

Theranos first came under scrutiny after a report in the Journal in October, which said that Theranos’ inventions were only used on a small number of tests sold to patients. On Sunday, the paper reported that during an inspection by Centers for Medicare and Medicaid Services auditors, employees who worked in the "Normandy" section of the Theranos lab were told not to enter or exit the lab; the inspectors toured the section of the lab that housed traditional lab instruments Theranos bought from other companies but never saw the Edison machines — Theranos’ highly publicized blood analyzers. This account fits with lab inspection documents that The Verge obtained from CMS, which do not mention Theranos’ proprietary technology at all. There’s a reason for that: "CMS did not inspect the Theranos proprietary technology," a spokesperson for CMS told The Verge in an email. Consequently, at no point between Theranos opening its first wellness center in 2013 and a surprise inspection by the FDA in August did any federal official examine the company’s proprietary technology — either its Edison machine, which reads samples, or its "Nanotainer" device, which stores blood from pinprick testing.

The lack of inspection was possible in part because Theranos registered its technology in categories that have minimal oversight. Theranos sold its tests under a designation that even the US Food and Drug Administration calls a regulatory loophole: the "lab-developed test." Under this designation, no pre-market FDA approval is required for the company’s blood tests — as it typically is for most drugs and devices. The category was meant for research hospitals, which sometimes adapt commercial tests to suit patients’ needs, and Theranos is one of several companies using the loophole to sell its tests to the public. Instead of the FDA, regulation of Theranos’ labs fell to the Centers for Medicare and Medicaid Services.

Theranos also registered its blood containers as FDA Class I medical devices in 2013, according to the FDA. That category of medical device doesn’t require an FDA inspection. No FDA inspectors entered a Theranos facility until August of this year, during a round of surprise inspections. In the FDA’s inspection report, it referred to Theranos’ Nanotainer as an "uncleared medical device" and altered its registration to a Class II medical device, which requires greater oversight.

CMS declined to provide us with additional information about Theranos. The FDA provided some information about Theranos’ regulatory status — including the fact that Theranos registered its containers as Class I in 2013 — but declined to reveal anything further about the company.

Theranos: CMS Auditors were "proactively briefed."

Theranos says CMS inspectors have looked at documents relating to the proprietary machines and tests. "[CMS] auditors and personnel have been proactively briefed on Theranos' laboratory-developed tests and Theranos' proprietary platform on multiple occasions, including reviewing validation reports and other documents about our LDTs," Theranos spokesperson Brooke Buchanan told The Verge in an email. She also points to a section of a December 2013 lab inspection report which mentions the lab’s "alternative assessment procedures," an in-house verification procedure that the company says it used to check the accuracy of its proprietary tests.1 "There were no observations related to [alternative assessment procedures] or otherwise related to our LDTs" in that report, Buchanan says, meaning the inspectors didn’t note any problems with the company’s methods. But the CMS report that mentions an alternative form of verification in the lab inspection doesn’t reference the company’s inventions. In addition, this type of verification process isn’t conducted by an external agency so it’s up to Theranos to make sure it’s doing it correctly.

Alternative assessment procedures (or AAP) are used by companies because their machines are too different to be used with standard proficiency testing samples. Under AAP, Theranos gathers its own blood sample and then runs it on two different machines — a commercial machine and one of its own analyzers — inside its own lab. Then it compares the two results and documents the process. No outside agencies are involved, so Theranos has a lot more control than it would during proficiency testing. And as The Verge previously reported, the decision to perform AAP came from Theranos — not CMS. In addition, using AAP does not let companies skip out on portions of a lab inspection.

It’s unclear why CMS didn’t verify whether Theranos’ proprietary tests were accurate. Inspections don’t just involve "briefing" assessors on machines and reviewing documents. Theranos' labs fall under the Clinical Laboratory Improvement Amendments, or CLIA. Under CLIA, CMS is supposed to make sure labs are checking the accuracy and precision of each test performed in their lab. Also among CMS inspectors’ tasks: ensuring that correct results are reported to patients and that a blood sample tested on one day will return the same results on the test at a later date. To do that, inspectors check a company’s maintenance procedures, patient test reports, and quality control data. Inspectors are supposed to walk the lab floor, says James Nichols, a pathologist at Vanderbilt University and the government relations committee chair for the American Association for Clinical Chemistry. "Inspectors typically will read the procedure, review the data supporting method validation, and then tour the lab particularly to see the instrument workstation and speak with the technologists performing the testing to ensure that what they are doing matches what is written in the test policy." And while this type of verification can’t determine if a test really does detect a health condition, it does help establish that the test is functioning properly.

(Michael Kovac / Getty Images)

There are several possible explanations for the fact that CMS says it never inspected Theranos’ inventions. If a machine or a test isn’t being used to return results to patients, CMS doesn’t have to check it. It’s possible that despite claims made by Theranos in the media, these machines were never used to return results to patients (a scenario that resembles conclusions in The Wall Street Journal’s initial report). Federal regulators may also have failed to do their jobs correctly. Or, as The Wall Street Journal has most recently suggested, the devices could have been hidden during inspection.

If "they have a shelf in their freezer that says ‘research,’ I won't check anything."

It’s not unusual for CMS to ignore experimental technology if a company is using conventional technology to deliver patient results, says Michael Murphy, president of Conatus Consulting, a medical device regulatory consulting firm. These companies rely on conventional machines to generate revenue, while their scientists work out the kinks in their new technologies; a company might use an invention on patient samples, but only in parallel with other methods. When these companies hand out test results, they only use those generated by FDA-cleared machines. In those instances, technologies that are in development aren’t included in CMS inspections.

"If I'm inspecting a lab, and they have a shelf in their freezer that says ‘research,’ I won't check anything on that shelf because those materials and reagents are not being used for patient testing," says Ted Schutzbank, technical director of the specialized testing and microbiology laboratories for the St. John Providence Health System in Michigan. (He also occasionally conducts lab inspections for the CMS-accredited College of American Pathologists.) "If there's an instrument that says ‘research, not for patient testing,’ I won't check that instrument either." This practice is quite common — but companies that operate this way don’t tout their revolutionary tests to the public and press, Murphy says.

"A lot of companies would probably be more conservative and not advertise that their new collection device is ready for prime time if, in fact, they're just using it for comparison to a standard method — which is what it sounds like that they’re doing," Murphy says. "I wouldn't be making such bold claims, especially given the notoriety of their founder and especially their board members. They don't need to brag about something that they aren’t doing."

Theranos insists that it used its proprietary technology to conduct patient tests. In its 6,000-word response to the initial Wall Street Journal report, Theranos said that by the end of 2014, Theranos offered "more than 80 of the tests on our online test menu via finger-stick." Of those, "all ran using proprietary technologies." In addition, the company stated that "by the fourth quarter of 2014, 57 percent of guests got lab tests run on finger-stick samples."

If Theranos has been using its own inventions, then uninspected results — of totally unknown accuracy — may have been returned to patients. By some estimates, test results influence 60 to 70 percent of medical decisions (although that number has been disputed). "Lab tests are essential to the everyday practice of medicine," says Richard Novick, the Recanati Family Professor of Science at New York University’s school of medicine. "It is critical that they be accurate." Inaccuracies can put patients’ health at risk.

It’s possible that CMS was just out of its depth. Most tests that fall under the category of "lab-developed tests" consist of slight alterations to existing commercial tests. But Theranos took LDTs to an extreme when it decided to build a brand-new system with no peer-reviewed studies to back up its methods. However, if CMS failed to inspect novel technologies that were returning results to patients, that’s bizarre, says Stephen Master, chief of clinical chemistry at Weill Cornell Medical College. "If there was an inspection of a lab that's focused on novel technology and it didn’t explicitly examine the validation data for those assays, it would raise some significant concerns."

In August, Theranos’ inventions were finally looked over by federal officials from the FDA. The circumstances that surround those inspections are murky. Theranos’ FDA-cleared herpes device had not yet come up for routine inspection when the agency conducted surprise inspections at Theranos' facilities in August, a spokesperson for the FDA told us. That inspection may therefore have been a simple, random inspection, or the result of someone tipping the FDA off about the company. "Something made them want to inspect — but I don’t know what it is," says Mary Pendergast, a lawyer and regulatory expert who is also a former deputy commissioner at the FDA. "Your guess is as good as mine." Theranos’ Buchanan declined to comment on the reason for the inspection.

Theranos says it halted tests on its proprietary technology following the August FDA inspection — except for the one test that has FDA approval. Still, that doesn’t address what happened at the company during the two previous years. Either Theranos never used its proprietary technology to return patient results — presenting itself as a revolutionary startup when the company was actually a conventional lab — or the company gave out results with unverified accuracy to patients. Neither option looks good for the 3.5 million tests that belonged to patients who trusted Theranos with their health.