The Secret Decoder Ring

Today our education reporter had a bunch of data from New York State she was trying to match to schools in New York City. But the school codes used by the two governments look radically different.

For example, PS 15 on the Lower East Side is known to the state as 310100010015; the city calls it 01M015.

I once made a nifty formula to make the conversion(!), but a more straightforward and official approach involves the Excel spreadsheet found here. It lists all of the city schools, along with their addresses, various codes, and more. For a data cruncher, that's a secret decoder ring.

What made me smile was that the only reason I knew this document even existed was because of a little prototype I tried during the first swine flu outbreak. That experiment wasn't robust enough to make it beyond this blog, but it taught me a lot ... including where to find this ring!

Movies & Demographics

What a great visualization of Netflix movie-rental data from the New York Times! Love how you can see how different movies play across the city.

It's even more interesting when you know something about the demographic makeup of the zip codes. Look how the Harlem River between upper Manhattan and the South Bronx is a bright dividing line for almost every movie.

How about a mashup that would reflect this info and demographic data simultaneously?


(tip via Nate Westheimer @innonate)

Connecting Journalists and Technologists

Tuesday night I'll have the fantastic privilege of making a quick presentation at the NY Tech Meetup about our news-technology efforts at WNYC. I'll also invite folks to connect with us in the interest of, well, the public good.

Huge thanks to Nate Westheimer for his interest in our work and the opportunity to say a few words.

Acknowledgment where due: WNYC's public radio news-tech projects, including the Super Simple Mapping tool, are supported by the John S. and James L. Knight Foundation and the Corporation for Public Broadcasting.

Super Simple Mapping Tool

Making collaborative maps is easier than ever. But maybe not quite easy enough.

One of the projects I'm working on at the moment is a super-duper-simple tool to help public radio and television stations (and pretty much anyone else) collect and map local information from their audience.

We're in the design phase right now, and we've mocked it up for feedback. The video is below.

Whaddya think? Let us know!

Presenter's Notes

The person crafting the overall sound and content of your local public radio station usually is the program director, and I have the honor of speaking at an annual gathering of PD's from across the nation this week.

Uncommon Indicators

My first talk is about a WNYC community crowdsourcing project called Your Uncommon Economic Indicators, which began just about a year ago when the economy collapsed. It focuses on getting people to contribute insights about the economy from their neighborhoods.

It has grown to include some special side projects. One is Halted Development, a look at unfinished or vacant housing in New York City (link to big map is here). Another is a video contest, in which this video took first place.

The slides I used in my presentation are available as zipped PowerPoint and Keynote files (both about 30MB) and as a pdf (3.5MB).

Collaboration as Dating

The second presentation is with Tom Detzel of ProPublica about the great partnership WNYC has had with ProPublica, including how it came to be and how we've make some great journalism together.

Part of my discussion is stolen from my earlier blog post using dating as a guide to successful collaborations. The handout I'm giving to people in the room is here as a pdf.

I'll update this post if/when audio or video of the talks are made available.

Paint Sticky Data (Please)

I'm into info. I want it accurate, relevant and clear.

On the radio, we try to paint clear, understandable, and journalistically-sound images of the mind -- the vivid mental pictures you see while listening to good storytelling.

Actual images can tell rich stories, too. The best photojournalism certainly does. Some pictures hit you in the chest.

But images drawn from data -- infographics, or visualizations -- rarely tell a story so well.

And they almost never hit me in the chest.

Why not? With all of the technology available, why can't we create really good visualizations that project understanding, timeliness, utility and ... dare I say ... stories?

I'm on the lookout. And I'm defining what I want to see.

For that definition, I've made a checklist based on one of my all-time favorite books, Made to Stick, by Chip Heath & Dan Heath (Random House, 2007). The initial words come from their Six Principles of Sticky Ideas; the rest is my application of their concepts.

For me, the best information images are ...
Simple: Non-geeks can absorb it within a few seconds
Unexpected: It fills a gap in our knowledge
Concrete: It takes advantage of our senses and understandings
Credible: It is journalistically sound, from a trusted source, without bias
Emotional: It hits you in the chest, you feel the data
Story: It tells one
And I'll add one more:
Relevant: It is timely, current and useful
Got examples that ring all seven bells? Maybe even four? Share them in the comments here or email me: john (at) designAgitator.com.

-----

The map detail above is from my favorite example at the moment, which is on Slate. Jump over there, take a look, and then run it through the checklist:

Simple? Once you know that blue is jobs gained and red is jobs lost, you're set. Just press play.
Unexpected? Seriously so. The speed of change is amazing.
Concrete? The familiar map orients me at a glance; I respond quickly to the circle sizes, colors and densities.
Credible? Bureau of Labor Statistics, Slate.
Emotional? Oh yeah. I saw someone actually shiver while watching it.
Story? Definitely.
Relevant? Yup.
Rings my bell.

The Man Behind The Zipper

I've joined an exclusive club of New York City microbloggers.

Twitter? Ha. Facebook? Kidstuff. We're talking bricks and mortar, baby.

Quite literally.

My missives scroll across the facade of WNYC's building in west SoHo, zipping into your field of view as a parade of little red lights. It's the WNYC News Zipper.

As you walk to work or sit in traffic on Varick Street, I've got your eyeballs.
NY terror-plot suspects indicted
None of this 140-character stuff. Better to use just five words; seven max. (I used a nonessential adjective clause once. Lost everyone by the second comma.)
Media banned from covering Iran protests
And I know where you are, no fancy GPS required.
Building collapse on Reade Street, up ahead
Even if it's partly cloudy in the Bronx, I am absolutely certain you're in a downpour.
This rain ends by evening
User customization? Easy. I can sense you're in line for the Holland Tunnel on your evening commute home. So how about a little news about your governor and his chief rival?
Corzine, Christie speak to biz group tonite
It's tempting to simply repurpose our tweets or web headlines, feeding them automatically to the sign. But it's also clear that wouldn't be as special. Or impactful. Or memorable. So I've been recrafting our material specifically for my particular version of a hyperlocal, mobile user.

I've been doing this for a few weeks as a prototype, and soon WNYC's editors, producers and hosts will feed lines to the sign. What I've learned by writing -- and watching -- those little red words will help our staff craft the phrases that catch your eye as you zip by.

Rapid News Visualizations: Prototype 1

In my quest for timely, interesting, understandable info-graphics, I've set up a prototyping challenge for myself: Upon finding news data, turn it into something visual, compelling and useful ... as fast as possible.

I'm prepared to fail quickly and often.

In this case, "as fast as possible" was three days to make, another two to find time to post it. The result is not wholly useful. And you can't absorb it quickly. And it's a little misleading.

But it's a start.

Here is what I made. It's a visual representation of the attendance rates for every public school in New York City on Thursday, May 21, 2009. The New York City Department of Education started posting this data the previous day as the Swine Flu/H1N1 outbreak was causing kids to stay home.

It tells me two things off the bat:
1) Queens and Brooklyn schools had much lower attendance rates than Manhattan and Staten Island schools.

2) Teens skip school on nice May days.
No. 2 is apparent because almost every red square is a high school, which have notoriously low rates this time of year. For a better indication of potentially flu-related absences, I'd chart the difference between these absentee rates and a typical May day at each school ... which is info I don't have. Yet.

Initially I published this in Google Maps, which was interactive and allowed you to click on schools for specific info. But Google Maps only plotted about 100 or so schools, and there are more than 1,000 here. Instead, I did it in Google Earth on my own computer and took a snapshot. Here's another.

Kinda cool. Was fun to do.

Next!

----------

Anatomy of the process:

Daily absentee data from the school system is here.
An Excel spreadsheet with general data on each school is here.
I crossed these two data sets in Access to match school numbers with addresses.
I got the latitude and longitude for each address, in 500-line batches, here.
I spent a lot of time learning about KML files, writing them, failing, trying again.
I made colored icons in Photoshop, and used Excel to assign each school the correct icon.
I put all of the relevant data into one spreadsheet and fed it to this little helper ...
Which gave me this KML file ...
Which I fed to Google Earth, running on my Mac.