SXSW Talk on Sensors & Journalism

At SXSW 2013 in Austin, I'm moderating and speaking on a panel looking at how sensors can play a role in journalism called "Sensoring the News: Detector-Driven Journalsm."

Joining me on the panel are:

Sarah Williams, Assistant Professor and Director the Civic Data Design Project at MIT. Sarah's slides are here.

Nadav Aharony, co-founder and CEO at Behavio.

Matt Waite, Professor and head of the Drone Journalism Lab at the University of Nebraska-Lincoln. Matt's video presentation is here.

And my slides are here.

Talk: Data News on the Fly

Today I'm talking about how the WNYC Data News team has done some quick-turnaround projects, especially around Sandy, at the IRE Computer Assisted Reporting conference

Here are the slides for my presentation, including links I reference. 

I also wrote a quick outline about our thinking on how we could best serve our audience during the storm, called "Predicting Questions, Building Answers."

Counting the Jay-Z subway crowd

Saturday morning we did something fun: We counted the number of people who took the subway to the opening-night Jay-Z concert at Brooklyns new Barclays Center the night before.

Or at least got pretty close.

Traffic and transit were closely watched for the new arena, as it the 19,000 or so concertgoers would have just 541 parking spaces. So we decided to grab data from subway turnstiles to measure the crowds leaving the Atlantic Ave-Barclays Center station for the show.

How we did it

Turning around the data overnight took a little planning. Here's how we pulled it off:

Every Saturday morning, the MTA posts turnstile data for the previous week. Fortunately for us, the last reading is 8 p.m. Friday, the scheduled start time for the concert.

The data files contain the entry and exit counter readings for each turnstile in the system as a sort of "odometer" reading. The data is a little tricky to use, though it does have a regular structure.

So Steve Melendez, our Data News Team programmer, wrote some Python code that grabs the data files and puts the individual readings into a SQLite database. He then sorted the readings by station (using this chart), and calculated how many exit clicks were logged for the Atlantic Avenue station from 4 p.m. to 8 p.m.

We suspected there would be a jump in the time period before the concert. So earlier in the week, we ran the numbers for each Friday for much of the year and calculated those averages (I ended up using just September, because they were higher, post-summer vacation readings). Then, Saturday morning, Steven got up really early Saturday and ran the program again, including the newly posted numbers.

He sent me the latest values, and I added them to the chart in a taxi on the way to the station. At 8:35 a.m., I was on the air talking about how it appears about a third of the concert-goers took the subway.

It could be more: Some people could have left the system at another station. And if anyone left through an emergency exit, or if they showed up after 8 p.m., they wouldn't be in our turnstile data.

But it's a place to start, and we'll be watching how these numbers change for future concerts and for Brooklyn Nets games.