tag:johnkeefe.net,2013:/posts johnkeefe.net 2019-06-15T19:36:24Z John Keefe tag:johnkeefe.net,2013:Post/1420535 2019-06-15T19:34:53Z 2019-06-15T19:36:24Z Converting videos to images for machine learning

This week I kept to my summer of training plan, however the model-building I did was for a Quartz project we're not ready to share. But! I learned something super useful in the process: how to quickly turn videos into many still images.

For our latest project, I'm training a model to identify specific objects available to me – much like how I trained a model to identify items in the office.

The fastest way to get lots of images of an object is to take a video of it. And a quick way to turn that video into images – called an "image sequence" – is ffmpeg. It seems to convert from many formats like .mp4, .mov, .avi to lots different image formats such as .jpg and .png.

There's plenty more detail in the ffmpeg docs, but here's what I did that worked so quickly on my Mac:

brew install ffmpeg

I use Homebrew to put things on my Mac, so this went pretty quickly. I had to update my Xcode command line tools, but Homebrew is super helpful and told me exactly what I needed to do.

Next, I did this from the Terminal:

ffmpeg -i IMG_1019.MOV -r 15 coolname%04d.jpg

Here's what's going on:

  • -i means the next thing is the input file
  • IMG_1019.MOV is the movie I Airdropped from my phone to my laptop
  • -r is the flag for the sample rate.
  • 15 is the rate. I wanted every other image, so 15 frames every second. 1 would be every second; 0.25 every 4th second.
  • coolname is just a prefix I picked for each image
  • %04d means each frame gets a zero-padded sequence number, starting with 0001 and going to 9999– so my image files are named coolname0001.jpg, coolname0002.jpg, coolname0003.jpg, etc.
  • .jpg is the image format I want. If I put .png I got PNGs instead.

In mere moments I had a dozens of JPG files I could use for training. And that's pretty great.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1418405 2019-06-09T22:19:57Z 2019-06-15T19:35:08Z Artisanal AI: Detecting objects in our office

Off-the-shelf services like the Google Vision are trained to identify objects in general, like car, vehicle, and road in the image below.

But many of the journalism projects we're encountering in the Quartz AI Studio benefit from custom-built models that identify very specific items. I recently heard Meredith Broussard call this kind of work "artisanal AI," which cracked me up and also fits nicely.

So as an experiment, and as part of my summer training program, I trained an artisanal model to identify between the three objects at the top of this page from the Quartz offices: A Bevi water dispenser, a coffee urn, and a Quartz Creative arcade game (don't you wish you had one of those?!)

I also made a little website where my colleagues and I can test the model. You can, too — though you'll have to come visit to get the best experience!

The results

The model is 100% accurate at identifying the images I fed it — which probably is not all that surprising. It's based on an existing model called resnet34, which was trained on the ImageNet data set to distinguish between thousands of things. Using a technique called transfer learning, I taught that base model to use all of its existing power to distinguish between just three objects.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1417028 2019-06-06T04:49:41Z 2019-06-06T17:11:26Z Making music with my arms

The brilliant Imogen Heap performed in New York a few weeks ago, and I got to experience live how she crafts sounds with her arms and hands

It was a great night of beautiful music and technology, both.

One mystery I couldn't solve from the audience was how her computer detected the position of her arms. Unlike in her early videos, I didn't see something akin to a Kinect on stage.

Now I think maybe I know.

That's because this week I took a workshop from Hannah Davis on using the ml5.js coding library, which touts itself as "friendly machine learning for the web," letting me use machine learning models in a browser. The class was part of the art+tech Eyeo Festival in Minneapolis.

One of the models Davis demonstrated was PoseNet (also here), which estimates the position of various body parts — elbows, wrists, knees, etc — in an image or video. I'd never seen PoseNet work before, let alone in JavaScript and in a browser.


Inspired by Heap, I set out to quickly code a music controller based on my arm movements, as seen by PoseNet through my laptop camera.

Try it yourself

It's pretty rough, but you can try it here. Just let the site use your camera, toggle the sound on, and try controlling the pitch by moving your right hand up and down in the camera frame!

I put it on Glitch, which means you can remix it. Or take a peek at the code on Github.

There are lots more ml5.js examples you can try. Just put the index.html, script.js, and models (if there's such a folder) someplace on the web where the files can be hosted. Or put them on your local machine and run a simple "localhost" server.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1417003 2019-06-06T02:12:07Z 2019-06-06T18:03:35Z My summer training program

This summer is all about training. Yes, I'm trying to run regularly, but I'm actually talking about training machine-learning algorithms.

I've been trying to learn machine learning for about three years — only to feel hopelessly overwhelmed. It was as though someone said, "With a chicken, a cow, and a field of wheat, you can make a lovely soufflé!"  

I took online classes, read books, and tried to modify sample code. But unless I devoted myself to the computer version of animal husbandry, it seemed, I was stuck.

Then someone at work mentioned fast.ai. It's a machine-learning library for Python that got me to the eggs-milk-flour stage, and provided some great starter recipes. Thanks to free guides and videos, I was soon baking algorithms that actually worked.

Now I want to get good, and experiment with different flavors and styles.

So this summer, I'm setting out to train and use new machine learning models, at least one each week. I'll try several techniques, use different kinds of data, and solve a variety of problems. It's a little like my Make Every Week project, providing constraints to inspire and motivate me.

I'll share what I learn, both here and at qz.ai where the Quartz AI Studio is helping journalists use machine learning, and I get to practice machine learning at work. 

In the fall I'll be teaching a few workshops and classes that will incorporate, I hope, some of the things I've learned this summer. If you'd like to hear about those once they're announced, drop your email address into the signup box on this page and I'll keep you posted.

Time to train!

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1388405 2019-03-21T18:03:08Z 2019-03-21T18:16:54Z You can't make your NY Times subscription online-only online

Our family believes in paying for good journalism, so we have a few subscriptions – including the New York Times.

When we signed up, we got online access along with physical papers delivered on the weekend. But we almost never read the paper version anymore, and thought it a waste. So today I went online to change my subscription to all-digital.

But you can't.

You must actually call the New York Times and speak to someone. I had to call two phone numbers, speak to two robots, and two people. All together, it took me 15 minutes. Not forever, but the user experience was a C-minus at best.

Here's what I did:

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1356763 2018-12-24T22:27:09Z 2018-12-24T22:28:59Z Beginning as a practice: The movie

Well, "the 5 minute video" at least.

Here is an Ignite talk at Newsgeist 2018 this past autumn in which I make the case for beginning repeatedly and intentionally, and even making it a practice.

By the way, the AI Studio, which I ask the audience to keep secret, has since been announced

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1342595 2018-11-10T20:37:33Z 2018-11-10T20:38:21Z A bot now updates my Slack status

One of my closest collaborators is a teammate far away — I'm in New York and Emily Withrow is in Chicago.

We stay connected chatting on Slack. But recently Emily asked if I could regularly update my Slack status to indicate what I was doing at the moment, like coding, meeting, eating. It's the kind of thing colleagues in New York know just by glancing toward my desk.

Changing my Slack status isn't hard; remembering do it is. So I built a bot to change it for me.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1323095 2018-09-19T02:20:06Z 2018-09-19T02:41:03Z I Bought Civil Tokens Today. I think.

I'm pretty sure I purchased Civil tokens today — literally buying into an experiment to put journalism on the blockchain.

After the sale, There were no tokens in my wallet and no indication my purchase was "on its way." Just a blank screen.

Unsettling, but I'm not actually worried.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1321673 2018-09-14T16:38:10Z 2019-03-06T14:45:29Z New Kid on the Blockchain

UPDATED at 7:45 pm ET on 9/17/2018 with new information. See the end of the post for details.

It's my time to go crypto.

I've followed blockchain technology, principles and trends for years without getting involved, but now have couple of reasons to get real: A new blockchain-based journalism project is about to launch, and my employer, Quartz, just launched a new cryptocurrency newsletter.

It also seemed perfect for my practice of beginning new things repeatedly.

The inspiration

Earlier this year, friends Manoush Zomorodi and Jen Poyant left their public radio jobs to join a new journalism … thing … called Civil. I had heard snippets about Civil, and started listening to Manoush's and Jen's podcast, ZigZag, part of which attempts to explain it.

After weeks of being pretty confused, I think I get it. Here's my attempt: Civil is a system designed to foster and reward quality journalism in a decentralized way, in contrast to platforms like Facebook and Google upon which so much journalism rests today.

The system’s backbone is the blockchain-based Civil token, abbreviated CVL. Holders of tokens can start news organizations in the system, challenge the membership of other news organizations in the system and/or cast votes when such challenges arise.

I have no idea if it will work. But I’m interested, and I’d rather participate than watch from the sidelines. So I’m willing to give it a whirl and okay with losing a little money in the process.

To participate, I just needed to buy some CVL ... though it turns out there's no just about it. But that's okay, too.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1320562 2018-09-12T22:39:11Z 2019-01-04T00:20:18Z Beginning as a Practice

[I recently presented this post as a 5-minute Ignite talk.]

On a morning flight some years back, the pilot's cheerful voice came over the speakers.

"I'm glad you're flying with us. This is the first time I've flown a Boeing 747,” the captain said with a pause. “Today."

We all laughed, of course. Who’d want to be on a pilot’s maiden flight?!

Not us. We want experts. Society counts on them. Companies pay them better. Spectators watch them play. Vacationers rely on their forecasts. We attend educational institutions and work long hours to become them — the qualified, the trusted, the best.

Nobody likes being a beginner.

Except that I do.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1320798 2018-09-11T21:05:06Z 2018-09-11T21:42:36Z Eyeo Festival Videos: Check them out

The Eyeo Festival just posted all of the videos from the 2018 festival, which is such a great service. Above is my talk on how we in the Quartz Bot Studio tell stories with conversational interfaces.

The festival had so many great speakers, and it's literally impossible to see them all live.

Here are some of my favorites I did see at the time, and highly recommend:

Check them out! 


]]>
John Keefe
tag:johnkeefe.net,2013:Post/1267726 2018-04-01T20:41:33Z 2018-04-01T20:51:10Z Giving Better Weather to Alexa

Nearly every day, someone in our family asks Alexa for the day's weather. The default response is fine -- high temp, low temp, sun or rain.

But given our three nor'easters, intense wind chills, and high-wind days, that wasn't enough. How much rain? When will it start? How much snow? How cold will it feel?

We needed something better.

Fortunately the US National Weather Service does a fantastic job writing up little descriptions of what's in store for every spot in the country. It's been my go-to source for years. Could I get Alexa to say that?

Short answer: Yes, I could. And now you can add "Better Weather" to your Alexa, too. For free. (In the US only, for now.)

For a longer description of how I made it, read on. 

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1177135 2017-07-26T02:52:41Z 2017-07-26T19:28:43Z Chibimojis: A dad and his daughter walk into the App Store

We made adorable manga faces you can add to iPhone messages!

They're iMessage "stickers," a fairly obscure feature of Apple's texting system that, it turns out, are pretty easy to make – and make public.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1171792 2017-07-09T04:01:00Z 2017-07-09T04:38:27Z Heel, Rotson! My list of computer-generated dog names

Shadoopy. Dango. Ray-Bella. Figgie.

If I told you those were names of actual dogs in New York City, would you believe me?

They're not. They were generated by a machine-learning algorithm mimicking dog names after it "studied" a list of 81,542 dogs registered in NYC.

The experiment, which took just a few hours Saturday, was something I've wanted to try since I saw the playful, awesome work of Janelle Shane and her experiments using neural networks to generate paint colors, guinea pig names and Harry Potter fan fiction.

I happened to have some free time, and decided to give it a shot. Along the way I:

  • built, in mere minutes, a computer in the cloud powerful enough for machine learning
  • made and played with a recurrent neural network
  • learned a little more about machine learning
  • had a lot of fun

The program generated lots of names, including many that existed in the original data. Once I filtered those out, I had almost 400 computer-created, mostly plausible dog names. Here are some of my favorites:

Rotson
Dudly
Lenzy
Murta
Cookees
Geortie
Dewi
Chocobe
Sckrig
Booncy
Cramp
Dango
Ray-Bella
Santha
Coocoda
Satty
Bronz
Shadoopy
Mishtak
Figgie
Grimby
Phince
Bum-Charmo
Soma
Blant
Snowflatey

If you'd like to geek out about how I did this, read on. You can do it, too.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1139036 2017-03-15T21:38:04Z 2017-03-15T22:06:43Z Qs and As About Bots for News

In my new job as a bot-maker and product manager at Quartz, I've been asked lots to share my thots about bots.

For a deep dive about conversational interfaces and what they mean to journalism (according to me), you can check out this Nieman Lab interview.

If you'd like a quicker scan, here are some questions and answers I prepared ahead of a panel about bots organized by the New York City chapter of the Online News Association. Here are those notes, and some links, too:


What should you consider before you start working on a bot?

  • You're entering uncharted territory! Have fun, explore, try new things.
  • There are no obvious places to find your bot. Bot makers talk about "discoverability" of bots, which is pretty problematic everywhere at this point.
  • So consider building where people already are interacting with you.
  • Make a "worker bee" bot that does a particular task well -- not a "know it all" bot like Siri or Alexa.
  • Is the information being exchanged sensitive? If so, think carefully. Making bots often means sending conversations through one or more 3rd-party services.
  • Play!

What are the specific design questions you need to keep in mind?

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1125283 2017-01-23T16:53:05Z 2017-02-10T22:31:24Z A New Role: Bots and Apps at Quartz

It's been an amazing run.

For nearly 16 years I've been at radio station WNYC, working with dedicated, talented people to inform New Yorkers every day and to help them navigate elections, blackouts, hurricanes and terror attacks.

Most recently I've helped mix code, design and reporting into new forms of journalism with brilliant colleagues on the WNYC Data News Team.

Along the way I've been tinkering with bots, chat systems and artificial intelligence. These explorations, together with my lifelong interest in journalism technology, have led me to a new role at Quartz.

I'll be building bots in the new Quartz Bot Studio and managing future iterations of Quartz's breakthrough iPhone and Android apps.

It's such an honor. I've been a fan of Quartz's executive editor and VP of product Zach Seward for many years, and I'm always impressed by how well Quartz crafts its site, newsletters, tools and apps to be super useful and exceptionally user-friendly. I feel so fortunate to be joining that team.

This all begins two weeks from today, which won't leave nearly enough time to get through my goodbyes and recount all of my memories at WNYC. But I'm excited about what's ahead, and I'll always be a listener and a member.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1121337 2017-01-08T01:40:00Z 2018-03-06T00:06:32Z Building a "Build-A-Bot" Workshop

I've been playing a lot with bots lately, and recently had a great opportunity to help others play, too.

It was part of the Future.Today conference in New York City last month. Futurist and organizer Amy Webb planned deep discussions about artificial intelligence and human-machine interactions on the main stage. In a side room, she wanted to give the audience tactile bot experiences — and asked me to help. Could I create a "Build-A-Bot" workshop?

The idea was to get conference-goers building chatbots over lunch -- making them easily, without code, and in a way people could "take" their bots home to work on further.

We ended up making nearly 100.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1109067 2016-11-18T14:29:38Z 2016-11-18T14:29:59Z Fast Company on WNYC's Storytelling Experiments

Fast Company writer John Paul Titlow did a great job capturing the spirit of experimentation at WNYC -- and me doing an ill-advised live demo on stage:

"Anyone who thinks old-school media can't be stealthy and innovative has never seen John Keefe text a room full of people from a command line on his laptop. But tonight, the senior editor for data news at WNYC—a public radio station founded in 1924—is showing off some things he built to help his colleagues tell stories."

Read the whole story here.
]]>
John Keefe
tag:johnkeefe.net,2013:Post/1101119 2016-10-22T20:47:03Z 2016-10-22T21:15:08Z Alexa Baked in a Pi

You can put Alexa in a Raspberry Pi, and that is pretty cool.

Alexa is Amazon's intelligent agent, like Siri for your living room. Standing nearby, you speak to it with a question or a command, and it responds verbally.

Normally Alexa lives inside a $180 device called an Amazon Echo, or the new $50 Echo Dot. But Emily Withrow at Northwestern University's KnightLab told me it was possible to put the Alexa code inside a cheap Raspberry Pi hobby computer. And I happened to have an old Pi lying around.

So I gave it a whirl!

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1093837 2016-09-27T04:39:16Z 2016-09-27T12:59:14Z Making Liza's Fireflies

Several friends recently planned a party for maker, e-textiler and all-around awesome person Liza Stark — and I wanted to celebrate her with blinkies appropriate for the occasion.

The event was to take place in a rented house with a porch overlooking a slice of woods.

I decided to fill the trees with digital fireflies.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1085860 2016-09-13T01:33:00Z 2016-09-14T00:31:16Z Now Available: Family Projects for Smart Objects!

It exists! I can't believe it!

My first book, Family Projects for Smart Objects: Tabletop Projects That Respond to Your World is in my hands. The Kindle edition is available now, and Amazon is taking preorders for the paperback edition that comes out September 24, 2016. 

It's a collection of 11 projects designed to introduce beginners to Arduinos, sensors and "internet of things" things. I tried to make it as accessible as possible, with clear instructions intended for girls, boys, women and men who have never done anything like this before.

The book grew out of my attempt to make something every week for a year (and blog about it).

Other fun facts:

  • I wrote much of it on my phone riding the NYC subway to work.
  • I promised myself I'd never write a book (thanks to Quinn Heraty for talking me into it).
  • It's published by the folks who make Make Magazine.

If you're into making and live in the NYC area, come out to the World Maker Faire October 1-2. I'll be there both days, demonstrating some projects and talking about the book!

"Family Projects for Smart Objects" at the World Maker Faire, NY Hall of Science:

Saturday, October 1, 2016 at 2:45 p.m. — Zone 3 Make: Show & Tell Stage

Sunday, October 2, 2016 at 11 a.m. — Zone 3 Make: Show & Tell Stage

C'mon out!



]]>
John Keefe
tag:johnkeefe.net,2013:Post/1077047 2016-07-29T01:43:27Z 2017-09-15T17:53:07Z Tracking Harlem's Heat with Sensor Journalism

harlem_heat_ba_Credit-JohnKeefe-WNYC

How hot is a Harlem apartment?

We're trying to find out.

There are now DIY sensors in about 20 apartments, measuring the indoor heat and humidity -- in the middle of a heat wave.

It's the latest sensor journalism project from WNYC's Data News Team, in a collaboration with blog AdaptNY, community group WEACT and observation platform ISeeChange

And this week we worked with maker space HackManhattan, which hosted a soldering party to build more sensors.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/1076090 2016-07-26T04:19:53Z 2016-09-14T00:31:42Z Book Making: Done

Between the last post and this one, I made a book!

Family Projects for Smart Objects is a collection of 11 projects based on Arduino hobby computers. They’re DIY activities designed for beginners who want to learn about sensors and make “Internet of Things” things.

Or just make stuff with people you love.

The book is currently with a team of designers who are working to make it beautiful, useful and fun.

Game plan is to publish in time for the World Maker Faire in New York this fall. If you’d like a note when it’s published, just jump onto my mailing list.

Ridiculously, that’s not all I’ve been doing. More posts ahead about cool (and hot) things I’m exploring at work and on the side.



]]>
John Keefe
tag:johnkeefe.net,2013:Post/973499 2016-01-18T23:39:42Z 2016-01-18T23:40:30Z Make Every Week Begets a Book


About this time last year I set out to make something every week in 2015.

In the end, it was actually “Make Every 1.7 Weeks.” But two exciting things happened along the way:

  • I made many, many more things than I would have otherwise, learning a ton.
  • I was invited to write a book.
]]>
John Keefe
tag:johnkeefe.net,2013:Post/950422 2015-12-15T14:35:56Z 2015-12-15T14:35:56Z Adventures in Minecraft & Parenting

When it comes to screen time, there are two activities where we give our daughters a lot of latitude: coding and Minecraft.

This morning I published a post on Medium called "Gardening at Night: One Dad's Guide to Minecraft." It's something I've been noodling on for a while, inspired by my daughters and by a few other parents who wanted to know how our family got started with the game.

Please let me know if it's useful to you or anyone else you know!

Separately, I've been collaborating with Jodi Jefferson to create a meetup geared for girls who play Minecraft called Girls Who Mine. We've met a couple of times, and are making it public with an event we're crafting for January. If you're interested, and live in the New York City area, jump over and add your name to the mailing list. We'll keep you posted.


]]>
John Keefe
tag:johnkeefe.net,2013:Post/940522 2015-11-29T04:00:53Z 2015-11-29T04:03:17Z DIY River Sensors: The 5 Minute Summary

Here's the high-wire act in which I describe the West Virginia University sensor-journalism project with 20 slides that advance automatically every 15 seconds. This took place in a room of brilliant thinkers at Newsgeist 2015 earlier this month in Phoenix.

More details about the sensor project are available on the StreamLab site and in an earlier blog post.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/933600 2015-11-14T16:47:21Z 2015-11-14T16:57:21Z Make Every Week: Programs in Python

“Daddy, I want to learn Python,” announced my 12-year-old daughter a couple of weeks ago. Boys in her youth group know it, she said. She wanted to, too.

Say no more.

I’ve introduced my daughters to a variety of friendly programming platforms, including Kids Ruby, Hopscotch, Codea and Lua in Minecraft. They’ve sweetly tolerated my programatic prodding. This was the first direct request.

I quickly ordered two paper copies of “Learn Python the Hard Way,” by Zed A Shaw, and we’ve been walking through each lesson together — one every week.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/916076 2015-10-13T03:52:30Z 2015-11-14T19:37:23Z Make Every Week: Distance Sensor Demo

I stumbled on a fun, visceral way to show how Arduinos can sense and respond.

In preparation for a presentation at the Online News Association Conference in Los Angeles, I grabbed a Ping distance sensor I had in a bin. The Ping works like a bat — it emits an inaudible, high-frequency sound, and listens for the sound to bounce off an object. The round-trip time between ping and reflection reveals the distance.

]]>
John Keefe
tag:johnkeefe.net,2013:Post/913886 2015-10-07T03:55:51Z 2016-07-06T14:56:14Z Monitoring the Monongahela

Yesterday the Streamlab class put do-it-yourself water monitors into Gatorade bottles and anchored them in the Monongahela River near Morgantown, West Virginia. They’re now texting their data readings live.

The link to the live chart is here, and the raw data is here.

We’re sensing conductivity, which is a good indicator of dissolved solids in the water, and temperature. The locations are: upstream of an industrial site, downstream of the same site and further downstream below the Morgantown lock and dam. 

]]>
John Keefe
tag:johnkeefe.net,2013:Post/908924 2015-09-24T03:44:55Z 2015-11-11T20:31:48Z Make Every Week: Message From a Bottle

A summer of tinkering has culminated with a conductivity and temperature sensor that texts its data from inside a Gatorade bottle.

The contraption consists of a Riffle, which is an Arduino-like board designed to fit through the mouth of a water bottle and a Fona cell-phone board. And a bottle.

The plan is to submerge several of these along a stretch of the Monongahela River as part of a sensor-journalism class at West Virginia University. It’s a work in progress, but you can [see how things are going]. My job was to build a working conductivity sensor that would report its findings live. Here are the components and how I made it go.

Update: We actually deployed some of these sensors in a river!

]]>
John Keefe