Weaving a Patchwork Map ... in Real Time

We did something a little creative and unique at WNYC this past election night: We mapped the vote by "community type."

This revealed the diversity of the vote across New York State -- from the cities to the suburbs, boom towns and "service worker" centers -- in real time, on the air and on the WNYC home page.

And the diversity is striking. Despite Democratic wins in every statewide race, the Republicans running for state attorney general and comptroller "won" every community type outside "Industrial Metropolis" and "Campus & Careers" counties.

Patchwork Nation's Dante Chinni talked about this on air during WNYC's coverage election night, and has written more about it since.

The live map was a mashup of Patchwork Nation's unique take on the nation and the Associated Press's live vote totals. At the request of WNYC, Patchwork Nation programmers dove into the AP test results and quickly wove them into a new map based on PN's existing county maps -- customizing them for the event and adding real-time percentages by community type.

Bringing the Threads Together

In the months before the election, I had wondered how we might better understand the early returns -- those that come in typically between 9 p.m. and 10:30 p.m. -- which often don't match the final results. I wanted more clarity.

At a Hacks/Hackers Open-Source-a-Thon, I started playing with the election data with help from Al Shaw (then at TalkingPointsMemo, now at ProPublica) and Chrys Wu (of Hacks/Hackers and ONA fame).

That evolved into a little program I wrote in Sinatra that generated vote-total map at the left, shading counties darker as more of their precincts reported. It also helped me better understand how the data were structured, how to retrieve the numbers and what it might take to make a live map.

So when Chinni asked if WNYC had any county-level data sets we'd like to put through the Patchwork Nation treatment, I had the perfect candidate.

Open All Night: The Great Urban Hack NYC

For 26 hours this weekend, a bunch of journalists and coders got together to make lots of great things designed to help the citizens of New York City.

My blog post summarizing the event and all of the resulting projects is upon Hacks/Hackers.

I helped Jenny 8. Lee, Chrys Wu and Stephanie Pereira organize the event. Then I joined a team working with digital heaps of NYC taxi trip data to make data visualizations and start some other projects. My favorite one is here (with a detail below), which is a representation of taxi usage for 24 hours, set around a clock. Beautiful. It was built by Zoe Fraade-Blanar using Processing and data crunched by the other teammates.

Click image for full view

I came away from the event with many new connections, excitement about learning Processing, some more skills in Sinatra and a note to check out Bees with Machine Guns(!)

Creatively Covering NY's New Ballot

New York switches to a new paper tomorrow -- Primary Day -- and ballot designers say voters likely will be confused.

At WNYC, we're covering this story in several ways that go beyond audio and written text.

Ballot Markup

For one, we had ballot designers help us annotate a ballot using DocumentCloud.

Sourcing Through Texting

For two, we're welcoming voters to share their experiences with the ballot via text. They (or you!) sign up by texting BALLOT to 30644.

Video Fun

The whole thing started when our host Brian Lehrer and reporter Azi Paybarah actually tried the sample ballot and made several mistakes. Which leads me to the third component: A video about using the new ballot:

Movies & Demographics

What a great visualization of Netflix movie-rental data from the New York Times! Love how you can see how different movies play across the city.

It's even more interesting when you know something about the demographic makeup of the zip codes. Look how the Harlem River between upper Manhattan and the South Bronx is a bright dividing line for almost every movie.

How about a mashup that would reflect this info and demographic data simultaneously?


(tip via Nate Westheimer @innonate)

Paint Sticky Data (Please)

I'm into info. I want it accurate, relevant and clear.

On the radio, we try to paint clear, understandable, and journalistically-sound images of the mind -- the vivid mental pictures you see while listening to good storytelling.

Actual images can tell rich stories, too. The best photojournalism certainly does. Some pictures hit you in the chest.

But images drawn from data -- infographics, or visualizations -- rarely tell a story so well.

And they almost never hit me in the chest.

Why not? With all of the technology available, why can't we create really good visualizations that project understanding, timeliness, utility and ... dare I say ... stories?

I'm on the lookout. And I'm defining what I want to see.

For that definition, I've made a checklist based on one of my all-time favorite books, Made to Stick, by Chip Heath & Dan Heath (Random House, 2007). The initial words come from their Six Principles of Sticky Ideas; the rest is my application of their concepts.

For me, the best information images are ...
Simple: Non-geeks can absorb it within a few seconds
Unexpected: It fills a gap in our knowledge
Concrete: It takes advantage of our senses and understandings
Credible: It is journalistically sound, from a trusted source, without bias
Emotional: It hits you in the chest, you feel the data
Story: It tells one
And I'll add one more:
Relevant: It is timely, current and useful
Got examples that ring all seven bells? Maybe even four? Share them in the comments here or email me: john (at) designAgitator.com.

-----

The map detail above is from my favorite example at the moment, which is on Slate. Jump over there, take a look, and then run it through the checklist:

Simple? Once you know that blue is jobs gained and red is jobs lost, you're set. Just press play.
Unexpected? Seriously so. The speed of change is amazing.
Concrete? The familiar map orients me at a glance; I respond quickly to the circle sizes, colors and densities.
Credible? Bureau of Labor Statistics, Slate.
Emotional? Oh yeah. I saw someone actually shiver while watching it.
Story? Definitely.
Relevant? Yup.
Rings my bell.

The Man Behind The Zipper

I've joined an exclusive club of New York City microbloggers.

Twitter? Ha. Facebook? Kidstuff. We're talking bricks and mortar, baby.

Quite literally.

My missives scroll across the facade of WNYC's building in west SoHo, zipping into your field of view as a parade of little red lights. It's the WNYC News Zipper.

As you walk to work or sit in traffic on Varick Street, I've got your eyeballs.
NY terror-plot suspects indicted
None of this 140-character stuff. Better to use just five words; seven max. (I used a nonessential adjective clause once. Lost everyone by the second comma.)
Media banned from covering Iran protests
And I know where you are, no fancy GPS required.
Building collapse on Reade Street, up ahead
Even if it's partly cloudy in the Bronx, I am absolutely certain you're in a downpour.
This rain ends by evening
User customization? Easy. I can sense you're in line for the Holland Tunnel on your evening commute home. So how about a little news about your governor and his chief rival?
Corzine, Christie speak to biz group tonite
It's tempting to simply repurpose our tweets or web headlines, feeding them automatically to the sign. But it's also clear that wouldn't be as special. Or impactful. Or memorable. So I've been recrafting our material specifically for my particular version of a hyperlocal, mobile user.

I've been doing this for a few weeks as a prototype, and soon WNYC's editors, producers and hosts will feed lines to the sign. What I've learned by writing -- and watching -- those little red words will help our staff craft the phrases that catch your eye as you zip by.

Rapid News Visualizations: Prototype 1

In my quest for timely, interesting, understandable info-graphics, I've set up a prototyping challenge for myself: Upon finding news data, turn it into something visual, compelling and useful ... as fast as possible.

I'm prepared to fail quickly and often.

In this case, "as fast as possible" was three days to make, another two to find time to post it. The result is not wholly useful. And you can't absorb it quickly. And it's a little misleading.

But it's a start.

Here is what I made. It's a visual representation of the attendance rates for every public school in New York City on Thursday, May 21, 2009. The New York City Department of Education started posting this data the previous day as the Swine Flu/H1N1 outbreak was causing kids to stay home.

It tells me two things off the bat:
1) Queens and Brooklyn schools had much lower attendance rates than Manhattan and Staten Island schools.

2) Teens skip school on nice May days.
No. 2 is apparent because almost every red square is a high school, which have notoriously low rates this time of year. For a better indication of potentially flu-related absences, I'd chart the difference between these absentee rates and a typical May day at each school ... which is info I don't have. Yet.

Initially I published this in Google Maps, which was interactive and allowed you to click on schools for specific info. But Google Maps only plotted about 100 or so schools, and there are more than 1,000 here. Instead, I did it in Google Earth on my own computer and took a snapshot. Here's another.

Kinda cool. Was fun to do.

Next!

----------

Anatomy of the process:

Daily absentee data from the school system is here.
An Excel spreadsheet with general data on each school is here.
I crossed these two data sets in Access to match school numbers with addresses.
I got the latitude and longitude for each address, in 500-line batches, here.
I spent a lot of time learning about KML files, writing them, failing, trying again.
I made colored icons in Photoshop, and used Excel to assign each school the correct icon.
I put all of the relevant data into one spreadsheet and fed it to this little helper ...
Which gave me this KML file ...
Which I fed to Google Earth, running on my Mac.

Feeling Information

Information and raw data are piling up faster than our ability to absorb it. And the tools available to access, understand, visualize and feel that information are woefully inadequate.

I believe journalism, design thinking and information technology can be wielded to express these stories in ways never before considered. And I'm part of a small posse poised to do just that.

If this interests you, or if you'd like to join our rag-tag group, write me: john (at) designAgitator.com.

In the meantime, assume some of the gaps in designAgitator postings mean we're hard at work helping to explain the world(!)

Election Night Design: Gaming and Planning

ITEM 1: There's a solid writeup of electoral vote scenarios at FiveThirtyEight.com expressing much of what we've been gaming out in our newsroom.

ITEM 2: Several weeks ago we worked to select a few Counties That Count, half-remembering* a prescient article that suggested a few Florida counties could decide the 2000 election.

Andrea Bernstein has counted our counties among her stops for The Takeaway, and points out that in the last few days Barack Obama has been doing the same -- visiting Clark County, Nevada; Pueblo County, Colorado; Prince William County, Virginia; and Palm Beach County, Florida.

This weekend, the journo-programmers at The Takeaway are working to make sure you can track the real-time election results from our Counties That Count on the show's website.

ITEM 3: To get more info in our face, I set an overhead monitor to flip through several websites automatically, for free. Here's how:

-- I put the Firefox 3 browser on a computer attached to the monitor
-- Next I installed the ReloadEvery Firefox add-on, which auto-reloads websites
-- Installed the Tab Slideshow add-on, which cycles through tabs at a set interval
-- Installed the Full Fullscreen add-on, which hides the Firefox toolbars
-- Pulled up several sites in separate tabs
-- Set ReloadEvery to 1 minute, and Tab Slideshow to 5 seconds
-- Turned on the Full Fullscreen feature to hide the toolbars
-- ... and voila!

* Anyone who remembers the specific article, please let me know!