johnkeefe.net

Google Fusion Tables can handle huge amounts of data -- and seem designed for that. But a great little secret is that they're fantastic for making fast maps. Even little ones.

And it's surprisingly easy.

At WNYC, we used fusion tables for this quickie map of 63 taxi relief stands. My colleague Jim Colgan whipped together these plowed-streets maps (including the one below) from listeners' texted-in reports -- while he was sick in bed!

Some reasons we've been drawn to mapping with Google Fusion Tables:

Simple uploads. All you need is a comma-separated table (csv) or a spreadsheet made in Excel or Google Docs. Each "point" goes on a row. If you have even basic Excel skills, you're more than ready to go.

Embedded geocoding. Put addresses in one of your columns, and Google will geocode them for you -- doing the work of finding the latitude and longitude for your pin. If you already have the coordinates, that's fine, too. Here's the help page on this for more.

Customizable icons. You can designate one of your columns as the icon column, and use this map of available icons to pick names to put in that column for each point There are some really clear instructions for this. 

Custom popups. You can define what appears in a pin's pop-up bubble. Doing this is a little tricky, but just. In the "map" visualization, click on "Configure Info Window." I find the default templates confusing, so I choose "Custom" from the drop-down menu. You can then use text, html and the table info {in_curly_brackets} to craft a custom bubble.

Easy embeds. Zoom and position the map as you like it and then click the "Get Link" button for a link to what you see. Or click the blue "Get embeddable link" link to get the embed code. (Design note to Google folks: It's confusing that one of these is a button and one is an html link!)

Easy updates. You can add more data points easily, either with additional uploads or just typing your additions or fixes in your browser. 

Privacy controls. As with other Google products, you can click the "Share" button to control who can view and/or edit each table and map -- which is really nice for working in teams.

News maps on news time.  That's been working for us.

Update: Jim Colgan, who put together the snowplow map, talks about how he did it with the folks at Mobile Commons, who run the platform we use for texting projects.

We did something a little creative and unique at WNYC this past election night: We mapped the vote by "community type."

This revealed the diversity of the vote across New York State -- from the cities to the suburbs, boom towns and "service worker" centers -- in real time, on the air and on the WNYC home page.

And the diversity is striking. Despite Democratic wins in every statewide race, the Republicans running for state attorney general and comptroller "won" every community type outside "Industrial Metropolis" and "Campus & Careers" counties.

Patchwork Nation's Dante Chinni talked about this on air during WNYC's coverage election night, and has written more about it since.

The live map was a mashup of Patchwork Nation's unique take on the nation and the Associated Press's live vote totals. At the request of WNYC, Patchwork Nation programmers dove into the AP test results and quickly wove them into a new map based on PN's existing county maps -- customizing them for the event and adding real-time percentages by community type.

Bringing the Threads Together

In the months before the election, I had wondered how we might better understand the early returns -- those that come in typically between 9 p.m. and 10:30 p.m. -- which often don't match the final results. I wanted more clarity.

At a Hacks/Hackers Open-Source-a-Thon, I started playing with the election data with help from Al Shaw (then at TalkingPointsMemo, now at ProPublica) and Chrys Wu (of Hacks/Hackers and ONA fame).

That evolved into a little program I wrote in Sinatra that generated vote-total map at the left, shading counties darker as more of their precincts reported. It also helped me better understand how the data were structured, how to retrieve the numbers and what it might take to make a live map.

So when Chinni asked if WNYC had any county-level data sets we'd like to put through the Patchwork Nation treatment, I had the perfect candidate.

For 26 hours this weekend, a bunch of journalists and coders got together to make lots of great things designed to help the citizens of New York City.

My blog post summarizing the event and all of the resulting projects is upon Hacks/Hackers.

I helped Jenny 8. Lee, Chrys Wu and Stephanie Pereira organize the event. Then I joined a team working with digital heaps of NYC taxi trip data to make data visualizations and start some other projects. My favorite one is here (with a detail below), which is a representation of taxi usage for 24 hours, set around a clock. Beautiful. It was built by Zoe Fraade-Blanar using Processing and data crunched by the other teammates.

Click image for full view

I came away from the event with many new connections, excitement about learning Processing, some more skills in Sinatra and a note to check out Bees with Machine Guns(!)

New York switches to a new paper tomorrow -- Primary Day -- and ballot designers say voters likely will be confused.

At WNYC, we're covering this story in several ways that go beyond audio and written text.

Ballot Markup

For one, we had ballot designers help us annotate a ballot using DocumentCloud.

Sourcing Through Texting

For two, we're welcoming voters to share their experiences with the ballot via text. They (or you!) sign up by texting BALLOT to 30644.

Video Fun

The whole thing started when our host Brian Lehrer and reporter Azi Paybarah actually tried the sample ballot and made several mistakes. Which leads me to the third component: A video about using the new ballot:

Fantastic presentation of GPS data from New York City Taxis in the New York Times today. So many stories embedded here, giving a wonderful sense of the city.

The ability to "play" the data is especially key. And the treatment itself hits all of my buttons. You just get it.

Beautiful.

What a great visualization of Netflix movie-rental data from the New York Times! Love how you can see how different movies play across the city.

It's even more interesting when you know something about the demographic makeup of the zip codes. Look how the Harlem River between upper Manhattan and the South Bronx is a bright dividing line for almost every movie.

How about a mashup that would reflect this info and demographic data simultaneously?

(tip via Nate Westheimer @innonate)

I'm into info. I want it accurate, relevant and clear.

On the radio, we try to paint clear, understandable, and journalistically-sound images of the mind -- the vivid mental pictures you see while listening to good storytelling.

Actual images can tell rich stories, too. The best photojournalism certainly does. Some pictures hit you in the chest.

But images drawn from data -- infographics, or visualizations -- rarely tell a story so well.

And they almost never hit me in the chest.

Why not? With all of the technology available, why can't we create really good visualizations that project understanding, timeliness, utility and ... dare I say ... stories?

I'm on the lookout. And I'm defining what I want to see.

For that definition, I've made a checklist based on one of my all-time favorite books, Made to Stick, by Chip Heath & Dan Heath (Random House, 2007). The initial words come from their Six Principles of Sticky Ideas; the rest is my application of their concepts.

For me, the best information images are ...

Simple: Non-geeks can absorb it within a few seconds
Unexpected: It fills a gap in our knowledge
Concrete: It takes advantage of our senses and understandings
Credible: It is journalistically sound, from a trusted source, without bias
Emotional: It hits you in the chest, you feel the data
Story: It tells one

And I'll add one more:

Relevant: It is timely, current and useful

Got examples that ring all seven bells? Maybe even four? Share them in the comments here or email me: john (at) designAgitator.com.

-----

The map detail above is from my favorite example at the moment, which is on Slate. Jump over there, take a look, and then run it through the checklist:

Simple? Once you know that blue is jobs gained and red is jobs lost, you're set. Just press play.
Unexpected? Seriously so. The speed of change is amazing.
Concrete? The familiar map orients me at a glance; I respond quickly to the circle sizes, colors and densities.
Credible? Bureau of Labor Statistics, Slate.
Emotional? Oh yeah. I saw someone actually shiver while watching it.
Story? Definitely.
Relevant? Yup.

Rings my bell.

I've joined an exclusive club of New York City microbloggers.

Twitter? Ha. Facebook? Kidstuff. We're talking bricks and mortar, baby.

Quite literally.

My missives scroll across the facade of WNYC's building in west SoHo, zipping into your field of view as a parade of little red lights. It's the WNYC News Zipper.

As you walk to work or sit in traffic on Varick Street, I've got your eyeballs.

NY terror-plot suspects indicted

None of this 140-character stuff. Better to use just five words; seven max. (I used a nonessential adjective clause once. Lost everyone by the second comma.)

Media banned from covering Iran protests

And I know where you are, no fancy GPS required.

Building collapse on Reade Street, up ahead

Even if it's partly cloudy in the Bronx, I am absolutely certain you're in a downpour.

This rain ends by evening

User customization? Easy. I can sense you're in line for the Holland Tunnel on your evening commute home. So how about a little news about your governor and his chief rival?

Corzine, Christie speak to biz group tonite

It's tempting to simply repurpose our tweets or web headlines, feeding them automatically to the sign. But it's also clear that wouldn't be as special. Or impactful. Or memorable. So I've been recrafting our material specifically for my particular version of a hyperlocal, mobile user.

I've been doing this for a few weeks as a prototype, and soon WNYC's editors, producers and hosts will feed lines to the sign. What I've learned by writing -- and watching -- those little red words will help our staff craft the phrases that catch your eye as you zip by.

In my quest for timely, interesting, understandable info-graphics, I've set up a prototyping challenge for myself: Upon finding news data, turn it into something visual, compelling and useful ... as fast as possible.

I'm prepared to fail quickly and often.

In this case, "as fast as possible" was three days to make, another two to find time to post it. The result is not wholly useful. And you can't absorb it quickly. And it's a little misleading.

But it's a start.

Here is what I made. It's a visual representation of the attendance rates for every public school in New York City on Thursday, May 21, 2009. The New York City Department of Education started posting this data the previous day as the Swine Flu/H1N1 outbreak was causing kids to stay home.

It tells me two things off the bat:

1) Queens and Brooklyn schools had much lower attendance rates than Manhattan and Staten Island schools.

2) Teens skip school on nice May days.

No. 2 is apparent because almost every red square is a high school, which have notoriously low rates this time of year. For a better indication of potentially flu-related absences, I'd chart the difference between these absentee rates and a typical May day at each school ... which is info I don't have. Yet.

Initially I published this in Google Maps, which was interactive and allowed you to click on schools for specific info. But Google Maps only plotted about 100 or so schools, and there are more than 1,000 here. Instead, I did it in Google Earth on my own computer and took a snapshot. Here's another.

Kinda cool. Was fun to do.

Next!

----------

Anatomy of the process:

Daily absentee data from the school system is here.
An Excel spreadsheet with general data on each school is here.
I crossed these two data sets in Access to match school numbers with addresses.
I got the latitude and longitude for each address, in 500-line batches, here.
I spent a lot of time learning about KML files, writing them, failing, trying again.
I made colored icons in Photoshop, and used Excel to assign each school the correct icon.
I put all of the relevant data into one spreadsheet and fed it to this little helper ...
Which gave me this KML file ...
Which I fed to Google Earth, running on my Mac.

Information and raw data are piling up faster than our ability to absorb it. And the tools available to access, understand, visualize and feel that information are woefully inadequate.

I believe journalism, design thinking and information technology can be wielded to express these stories in ways never before considered. And I'm part of a small posse poised to do just that.

If this interests you, or if you'd like to join our rag-tag group, write me: john (at) designAgitator.com.

In the meantime, assume some of the gaps in designAgitator postings mean we're hard at work helping to explain the world(!)