When Pierre D. Martin was arrested on robbery charges, it didn’t seem like an unusual case. There were nearly 12,000 robberies reported to the Chicago Police Department last year, and Martin was charged in two of them after allegedly carrying out daytime stickups on commuter trains. But a July 2013 Chicago Sun-Times article pointed out why Martin’s case stood out: he had been identified using facial recognition technology.

Martin was convicted in June this year. At that time, the state’s attorney, Anita Alvarez, told the Sun-Times: "This case is a great example that these high-tech tools are helping to enhance identification and lead us to defendants that might otherwise evade capture." But those tools — purchased by CPD through a $5.4 million federal grant — may not be as necessary as Alvarez would have you believe.

Martin's attorney told The Verge that facial recognition tech wasn’t even mentioned at trial. And a Freedom of Information Act request with CPD revealed "no responsive records" related to the department’s use of facial recognition technology in any arrests, including Martin’s. That could mean CPD is concealing arrests tied to facial recognition. But Hanni Fakhoury, a staff attorney at the Electronic Frontier Foundation, offers another possibility: "Maybe ultimately facial recognition isn’t really all that effective," he wrote in an email to The Verge. Maybe "all this money being put into it is a waste."

A perfect match

About a year before Martin’s arrest, CPD installed a facial recognition system. The system, called NeoFace, was built by the NEC Corporation, to which CPD had previously awarded at least $13 million in automated fingerprint-identification system contracts going back to 1986. The company claims NeoFace will clearly and precisely match CCTV screen captures with police photos, and CPD, a longtime testing ground for cutting-edge law-enforcement technology, was happy to give it a try.

On Jan. 28, 2013, that partnership supposedly captured its first suspect.

According to the Sun-Times, a Chicago Transit Authority surveillance camera picked up an image of a man who had just taken another man’s cellphone. CPD detectives ran the photo through NEC’s NeoFace computer program, and Pierre D. Martin’s name came up as the most likely suspect. The rest was history. Martin was arrested and charged. "This was our first success," said Jonathan Lewin, a CPD commander in charge of information technology, to the Sun-Times in reference to Martin’s case being CPD’s first facial recognition arrest. A year later, in June this year, Martin was convicted of robbery and sentenced to 22 years in prison.

Martin’s conviction isn’t raising any questions. (He admitted to the crime.) But the surrounding story certainly does.

First of all, a cursory look at Martin’s rap sheet shows that he’s 25 years old — born in September 1988 — not 35, as the Sun-Times claimed. And, at least according to his public defender, John J. Kwon, Martin’s conviction had nothing to do with facial recognition. "I don’t know where that information came from," he told The Verge. Kwon stressed that Martin was identified in a lineup — and that the young man admitted guilt to investigators. An arrest report confirms both these things.

Curious whether there were other cases like Martin’s where NeoFace tools were used with surveillance footage to identify perpetrators, The Verge filed a Freedom of Information Act request with CPD for the number of arrests, number of closed cases, and the names of defendants in closed cases in which facial recognition technology "has been credited with apprehending a suspect and / or assisting with the apprehension of a suspect."

CPD’s response:

Your request was reviewed by … the Public Safety IT section, the Bureau of Detectives, the Records Services Division, and the Crime Prevention Information Center. A search, based on the information provided … has yielded no responsive records.

Separate from the FOIA request, CPD’s public affairs office did not respond to repeated requests for comment about the department’s use of facial recognition technology. Emails to NEC Corporation’s marketing department went unreturned. And while Commander Jonathan Lewin assured The Verge that facial recognition was being used by the department "as a tool to help solve crimes" — both in Martin’s case and generally, in accordance with the department’s directives for facial recognition technology — he declined to say how many cases have been helped by facial recognition technology or the kinds of cases in which it’s being used.

Hanging low

In addition to confirming that Martin had been identified in a lineup and later admitted guilt, his arrest report includes one additional telling line: Martin was "identified through CTA surveillance footage…." Though it says nothing about facial recognition, one can suppose that’s how CPD officers have decided to label investigations helped by NeoFace software.

Let’s assume that’s the case — that CPD officers simply do not acknowledge that facial recognition software was used when they write arrest reports. Even so, Martin’s criminal record shows that he had been arrested at least nine times — starting in June 2000, when he was picked up as an 11-year-old for retail theft — prior to his most recent arrest.

Furthermore, this was a man who, according to police records, was identified in CPD’s gang-monitoring system as a possible member of the Latin Kings street gang. He also had two prior convictions. This was a man very much on CPD’s radar.

As such, Martin’s case doesn’t really tell us anything about facial recognition’s effectiveness. What it shows is that NeoFace can do investigative work that pretty much any competent police officer could easily do — that is, if it was used at all.

"The camera was able to identify Martin because he had a long rap sheet with plenty of mug shots to link his picture to," Fakhoury said. "So the cameras allowed Martin (not someone else) to be arrested because of facial recognition. But if you’ve got a long history of arrests," he continued, "it’s going to be easy to circumstantially connect you to and prosecute you for the crime."

Drawing a blank

Facial recognition is being rolled out in stores, in airports, at border crossings, at Olympic games, and now in an increasing number of cities such as Seattle, Daytona Beach, and San Diego. Cities in the UK are now experimenting with NeoFace tech specifically.

It's important to know how well these systems work, because cities stand to spend a lot of money on futuristic-seeming technology that might not work at all. A 2002 ACLU investigation found that a facial recognition program in Tampa "not only has not produced a single arrest, but it also has not resulted in the correct identification of a single person from the department’s photo database…." Things have changed significantly since then, but how much have they really changed?

"Facial recognition has always been very bad at using pictures taken in the wild," said Jay Stanley, an ACLU senior policy analyst who co-wrote the 2002 study. "For facial recognition to work well, it needs to take place in environments that are in very controlled conditions, in uniform lighting, with a full frontal picture, and preferably with a neutral expression. Even a smile can mess up the algorithms."

Even if Martin’s case was a success, more information is needed, Stanley said. "For every success, how many failures are there," he said. "How many wild goose chases are there? How many people come under the microscope of the police for no reason?"

Travis Hall, a research fellow at NYU’s Information Law Institute who contributed to a 2009 report on facial recognition technology for NYU’s Center for Catastrophe Preparedness and Response, had similar concerns. Facial recognition technology has improved "exponentially over the last few years," Hall wrote in an email to The Verge. But "even with those gains in accuracy it will be a long time before [facial recognition] acquires the status of fingerprinting in evidentiary standards."

Cases like Martin’s "are often trotted out as major successes, without going into detail as to whether or not traditional police tools helped to capture the suspect or could have drawn the same conclusions on their own," Hall continued. "It is also important to look at the amount of data collected and money spent in proportion to the number of crimes solved by the particular technology. If Chicago has been using this tech for a year, and this is its first and only success story, that’s not a great record."