Skip to main content

NYC's 'geek squad' combines public agency info with big data techniques

NYC's 'geek squad' combines public agency info with big data techniques

Share this story

New York City Manhattan (STOCK)
New York City Manhattan (STOCK)

In an article this weekend, The New York Times takes a look at New York City's Office of Policy and Strategic Planning, a small team of data-science specialists that combine public data with brute server power to help city agencies get a little smarter. In the last three years, they've guided city workers to trees felled by Hurricane Sandy, tracked bootleg cigarettes, and directed understaffed housing inspectors towards the most urgent fire code violations.

They're working with the same Bayesian data-mashing techniques as companies like Microsoft and Foursquare, but as a city agency, they've got a lot of data to work with: over a terabyte of public archive information each day, ranging from tax returns to building code reports. From there, they pull even more info from city-affiliated businesses like ConEd, which provides data on power usage and heating units. Put it together, and you've got a comprehensive view of the whole city, from commuting patterns to noise complaints.

As a result, when the city notices a buildup of illegally dumped grease in the city's drains, they can work backward from a map of restaurants that pay for grease-carting services, and come away with a list of likely suspects. It's not exactly Minority Report, but as other cities deal with shrinking municipal budgets, it's an increasingly attractive proposition.