Category Archives: Crafting with Data

Revisiting book data

For Crafting with Data today, our assignment was to gather preliminary data for our Discovery Seeker project:

Use what you’ve learned to gather sufficient samples for your purpose. Explore again, check for significances, patterns, correlations or trends to share with the class.

Since my project is to try to approach the abandoned Bookalator from a new angle, I didn’t actually set out to collect new data right away. Continue reading Revisiting book data

The Project That Would Not Die

Vesalius spread

Hallowe’en is coming, which must be why the Bookalator has been stirring again, gathering its strength to once more stalk the earth.

Back in May, I made a summer “to do” list which was long enough to keep me busy for the next ten years. Among the proposed tasks was “finishing” the perpetually stalled Bookalator project. I then, of course, proceeded to spend the next three months mostly sitting around feeling bad about not doing stuff. But the constant demand for Ideas, Ideas, Ideas, and my constant lack of same, has made spring’s doleful mess look like fall’s Golden Ticket.
Continue reading The Project That Would Not Die

Self-Portrait in Data

Our assignment last week for Crafting with Data was as follows:

“Self-Portrait: Data is factual information; science finds its story. But data isn’t only about science so we can turn that on its head. Find data for your own story. Create a self-portrait, using data.”

I thought this would be a cinch, as I keep somewhat obsessive records about a lot of stupid stuff. However, I underestimated the inconsistency of my recordkeeping, and the difficulty of presenting awkward data in some coherent fashion, so when the fab Nicholas Felton came to talk to our class and see what we’d done, I had nothing yet to show. Bummer. But here’s what I finally came up with.


Self-Portrait in Data: Foods

To give you some perspective on how poorly I eat. These are servings, over the course of September 2009. Data came only from my diary, in which some days are blank and others obviously incomplete. But I think the overall gist is accurate. To wit: I had cheese about every three days, less than one serving of fruit or vegetables per day, and more than one dessert per day, much of it cake.

Mmmm, cake.


Self-Portrait in Data: Beverages

I do much better with drinking than with eating, especially if you consider that I rarely remember to keep track of glasses of water—it’s my favorite beverage, and I definitely drink more of it than is shown here. Data came only from my diary, in which some days are blank and others obviously incomplete. So, according to my crappy records, I had only thirty-nine eight-ounce servings of water over the course of September 2009; I’d guess the real figure is at least three times that. The tea counts—38 black, 11 green, 4 herbal—are probably also way too low. Everything else is probably pretty accurate.

I know that I had exactly three beers, for instance (two bottles of Brooklyn Brown and a can of Murphy’s Stout), and I can’t have had more than three glasses of wine, because it was on the same evening as two of the beers—at a wedding—and I would have fallen down if I’d had more. Cheap date.

Update, 9/10

Rob wanted to know why I made this a pie chart. I guess the reason was that I felt that the relationship of each part to the whole was the most important; I was thinking in percentages, and a pie chart is, for me, the most intuitive way to visualize percentages. Which probably just means my vocabulary of types of charts is too limited.


Self-Portrait in Data: Social
(Click for a bigger image.)

Here are my interactions with friends and family during the month of September 2009. I was surprised that face-to-face interactions outweighed all others, since I feel like I hardly ever see anybody, but I think the e-mail numbers would have kicked face-to-face’s ass had I not gone to a wedding where I knew a lot of people.

The height of each block represents days of interaction (DoI). That is, if I exchanged a dozen e-mail messages with Elisabeth on Monday and three on Tuesday, and also on Tuesday I got two from Jack, it counts as three DoI: Elisabeth x 2 + Jack x 1.

The width of each block represents the number of people I interacted with using the method in question. There were seventy-six people altogether; I met thirty-one face to face, and only one sent a note through LinkedIn. Four people called, texted, or left Flickr comments.

This data came from my diary, from copies of DMs in my e-mail account, and from Twistory, which shows all the tweets I sent to people on my Google Calendar. I did not have the patience to page back through all the actual Twitter messages, so I may have failed to count some tweets that were sent to me. I delete most text messages from my phone upon reading them, so those are noted only when I knew I exchanged some—e.g., when I was coordinating with friends about wedding transportation, and when Elisabeth was visiting New York (yay!). I think I counted Flickr comments on only one of my two accounts; oops.

Update, 9/10

So, in class on Thursday, people seemed to feel that the way I’d presented this data was . . . misleading? Not sure that’s the right word. In particular, Rob had trouble with the notion of Days of Interaction, I think. People would rather have seen real quantities of e-mail and so forth, separated into in/out? Yes, I’d like to see that, too, but I didn’t want to spend a million hours gathering the data; as it was, this took me more time than I’ve spent on any other homework so far this semester. And probably it should be a double-bar chart instead of using the number of people in each category to influence the area of the bar.

I felt when I made this that it wasn’t the right sort of chart to represent this information, but to me it’s meaningful enough. It tells me that I see a lot more people than I think I do, and that I really ought to get rid of my land line, now that I’ve managed to get Mom to call my Google Voice number 99 percent of the time. I expect the SMS numbers will start going up, now that I’m using a phone with a keyboard; it was just too slow before, and I’d sometimes get halfway through a message and then delete it and just call, though I hate talking on the phone.


So, I’m a near-teetotaler on the verge of getting scurvy who mostly stays in touch with people electronically.

Living in uninteresting times

don’t press the red button

This week’s assignment for Crafting with Data was

Use an Arduino to gather 500 samples of interesting data in two different conditions (1000 samples total). Post your data online so it can be downloaded.

Now, see, I did this assignment uncharacteristically promptly—days ago!—without having noticed the adjective interesting, so originally what I had was some profoundly dull numbers from a thermistor, measured (a) under my fluorescent proofreading lamp and (b) while I held the sensor between my fingers. You will thank me, therefore, for rereading the assignment, noting my error, and, at the last minute, using the Pong controller I made for Understanding Networks to gather some slightly less dull information from the x axis of an accelerometer: (a) walking on level floor, and (b) climbing up and down stairs.

Now, with charts!

accelerometer readings while walking on level floor

accelerometer readings while climbing stairs

The guts of my beautiful data collection device look like this:

Pong controller guts
(Click the photo for more details at Flickr.)

I used two Processing sketches—for the thermistor data, Rob’s code; for the accelerometer data, a quick-and-dirty mod of the almost wholly unoriginal code I used for the Understanding Networks assignment.

[sourcecode language=”java”]/*
Serial String Reader + Simple net Client
Language: Processing

Reads in a string of characters from a serial port until
it gets a linefeed (ASCII 10). Then splits the string into
sections separated by commas. Then converts the sections to ints,
and prints them out.

* Starts a network client that connects to a server on port 8080,
* sends any keystrokes pressed.

created 2 Jun 2005
modified 6 Aug 2008
by Tom Igoe
modified 24 September 2009 by India Amos

import processing.serial.*; // import the Processing serial library
import*; // import the Processing networking library

Serial myPort; // The serial port

Client myClient; // instance of the net Client
String data; // string to hold incoming data
String ipAddress = “”; // address of the server
String myKey1; // the virtual keypress generated by the x-pin of the accelerometer
String myKey2; // the virtual keypress generated by the y-pin of the accelerometer
String myKey3; // the virtual keypress generated by the switch
String myKeyBuffer; // all the virtual keys pressed by a single bleat from the Arduino

float bgcolor; // Background color

void setup() {
// establish the background and foreground:
size(200, 200);

// Connect to server on port 8080
myClient = new Client(this, ipAddress, 8080);

// List all the available serial ports

// I know that the first port in the serial list on my mac
// is always my Arduino module, so I open Serial.list()[0].
// Change the 0 to the appropriate number of the serial port
// that your microcontroller is attached to.
myPort = new Serial(this, Serial.list()[0], 9600);

// read bytes into a buffer until you get a linefeed (ASCII 10):

void draw() {

// serialEvent method is run automatically by the Processing applet
// whenever the buffer reaches the byte value set in the bufferUntil()
// method in the setup():

void serialEvent(Serial myPort) {
// read the serial buffer:
String myString = myPort.readStringUntil(‘\n’);
// if you got any bytes other than the linefeed:
if (myString != null) {

myString = trim(myString);

// split the string at the commas
// and convert the sections into integers:
int sensors[] = int(split(myString, ‘\t’));

// print out the values you got:
for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) { print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t"); } // add a linefeed after all the sensor values are printed: println(); // Convert integers into behavior. // A change of +/-50 on sensor 1 means a roll forward (down) or backward (up), respectively. Flat is 520. // A change of +/-50 on sensor 2 means a tilt coounter-clockwise or clockwise, respectively. Flat is 505. myKey1=""; myKey2=""; myKey3=""; if (sensors[0] > 570) // rolling forward, so paddle down
myKey1 = “d”;

if (sensors[0] < 470) // rolling backward, so paddle up { myKey1 = "u"; } if (sensors[1] > 555) // rolling counter-clockwise, so paddle left
myKey2 = “l”;

if (sensors[1] < 455) // rolling clockwise, so paddle right { myKey2 = "r"; } if (sensors[3] > 0 ) // button pressed
myKey3 = “x”;

myKeyBuffer = myKey1+myKey2+myKey3;

// If there’s incoming data from the client:

if (myClient.available() > 0) {
// get the data:
data = myClient.readString();

if (myKeyBuffer != null) {
// send out anything that’s typed:
if (keyPressed) {
// send out anything that’s typed: