Revisiting book data

For Crafting with Data today, our assignment was to gather preliminary data for our Discovery Seeker project:

Use what you’ve learned to gather sufficient samples for your purpose. Explore again, check for significances, patterns, correlations or trends to share with the class.

Since my project is to try to approach the abandoned Bookalator from a new angle, I didn’t actually set out to collect new data right away. Continue reading Revisiting book data

The Project That Would Not Die

Vesalius spread

Hallowe’en is coming, which must be why the Bookalator has been stirring again, gathering its strength to once more stalk the earth.

Back in May, I made a summer “to do” list which was long enough to keep me busy for the next ten years. Among the proposed tasks was “finishing” the perpetually stalled Bookalator project. I then, of course, proceeded to spend the next three months mostly sitting around feeling bad about not doing stuff. But the constant demand for Ideas, Ideas, Ideas, and my constant lack of same, has made spring’s doleful mess look like fall’s Golden Ticket.
Continue reading The Project That Would Not Die

Pongbox

For week 2 of Understanding Networks, a newly created class that keeps shifting between abstraction and application, leaving me sadly uncertain as to whether I understand what Tom’s talking about at any given moment, our terrifying assignment was as follows:

Socket exercise: communicating in realtime. I will give you a game platform and the protocols to log in and communicate with it. Make a client to log in and play. Work in Processing, Flash, Arduino or whatever environment is comfortable to you. You will not play with your own client, but you’ll show someone else how to use it.

Continue reading Pongbox

Drawing Machines, week 1

Doing some catch-up documentation, here. I was finally allowed to replace this class with Mashups on Monday, but I thought I might as well post my work from the first month, since, you know, I did it and all.

The assignments for week 1 were,

  1. Make 3 drawings that explore line & contour according the instructions given in the Nicolaides reading.
  2. Photograph an image that explores line & contour.
  3. Describe an abstract process that explores line & contour.

Continue reading Drawing Machines, week 1

Self-Portrait in Data

Our assignment last week for Crafting with Data was as follows:

“Self-Portrait: Data is factual information; science finds its story. But data isn’t only about science so we can turn that on its head. Find data for your own story. Create a self-portrait, using data.”

I thought this would be a cinch, as I keep somewhat obsessive records about a lot of stupid stuff. However, I underestimated the inconsistency of my recordkeeping, and the difficulty of presenting awkward data in some coherent fashion, so when the fab Nicholas Felton came to talk to our class and see what we’d done, I had nothing yet to show. Bummer. But here’s what I finally came up with.

Foods

Self-Portrait in Data: Foods

To give you some perspective on how poorly I eat. These are servings, over the course of September 2009. Data came only from my diary, in which some days are blank and others obviously incomplete. But I think the overall gist is accurate. To wit: I had cheese about every three days, less than one serving of fruit or vegetables per day, and more than one dessert per day, much of it cake.

Mmmm, cake.

Beverages

Self-Portrait in Data: Beverages

I do much better with drinking than with eating, especially if you consider that I rarely remember to keep track of glasses of water—it’s my favorite beverage, and I definitely drink more of it than is shown here. Data came only from my diary, in which some days are blank and others obviously incomplete. So, according to my crappy records, I had only thirty-nine eight-ounce servings of water over the course of September 2009; I’d guess the real figure is at least three times that. The tea counts—38 black, 11 green, 4 herbal—are probably also way too low. Everything else is probably pretty accurate.

I know that I had exactly three beers, for instance (two bottles of Brooklyn Brown and a can of Murphy’s Stout), and I can’t have had more than three glasses of wine, because it was on the same evening as two of the beers—at a wedding—and I would have fallen down if I’d had more. Cheap date.

Update, 9/10

Rob wanted to know why I made this a pie chart. I guess the reason was that I felt that the relationship of each part to the whole was the most important; I was thinking in percentages, and a pie chart is, for me, the most intuitive way to visualize percentages. Which probably just means my vocabulary of types of charts is too limited.

Social

Self-Portrait in Data: Social
(Click for a bigger image.)

Here are my interactions with friends and family during the month of September 2009. I was surprised that face-to-face interactions outweighed all others, since I feel like I hardly ever see anybody, but I think the e-mail numbers would have kicked face-to-face’s ass had I not gone to a wedding where I knew a lot of people.

The height of each block represents days of interaction (DoI). That is, if I exchanged a dozen e-mail messages with Elisabeth on Monday and three on Tuesday, and also on Tuesday I got two from Jack, it counts as three DoI: Elisabeth x 2 + Jack x 1.

The width of each block represents the number of people I interacted with using the method in question. There were seventy-six people altogether; I met thirty-one face to face, and only one sent a note through LinkedIn. Four people called, texted, or left Flickr comments.

This data came from my diary, from copies of DMs in my e-mail account, and from Twistory, which shows all the tweets I sent to people on my Google Calendar. I did not have the patience to page back through all the actual Twitter messages, so I may have failed to count some tweets that were sent to me. I delete most text messages from my phone upon reading them, so those are noted only when I knew I exchanged some—e.g., when I was coordinating with friends about wedding transportation, and when Elisabeth was visiting New York (yay!). I think I counted Flickr comments on only one of my two accounts; oops.

Update, 9/10

So, in class on Thursday, people seemed to feel that the way I’d presented this data was . . . misleading? Not sure that’s the right word. In particular, Rob had trouble with the notion of Days of Interaction, I think. People would rather have seen real quantities of e-mail and so forth, separated into in/out? Yes, I’d like to see that, too, but I didn’t want to spend a million hours gathering the data; as it was, this took me more time than I’ve spent on any other homework so far this semester. And probably it should be a double-bar chart instead of using the number of people in each category to influence the area of the bar.

I felt when I made this that it wasn’t the right sort of chart to represent this information, but to me it’s meaningful enough. It tells me that I see a lot more people than I think I do, and that I really ought to get rid of my land line, now that I’ve managed to get Mom to call my Google Voice number 99 percent of the time. I expect the SMS numbers will start going up, now that I’m using a phone with a keyboard; it was just too slow before, and I’d sometimes get halfway through a message and then delete it and just call, though I hate talking on the phone.

Conclusion

So, I’m a near-teetotaler on the verge of getting scurvy who mostly stays in touch with people electronically.

Living in uninteresting times

don’t press the red button

This week’s assignment for Crafting with Data was

Use an Arduino to gather 500 samples of interesting data in two different conditions (1000 samples total). Post your data online so it can be downloaded.

Now, see, I did this assignment uncharacteristically promptly—days ago!—without having noticed the adjective interesting, so originally what I had was some profoundly dull numbers from a thermistor, measured (a) under my fluorescent proofreading lamp and (b) while I held the sensor between my fingers. You will thank me, therefore, for rereading the assignment, noting my error, and, at the last minute, using the Pong controller I made for Understanding Networks to gather some slightly less dull information from the x axis of an accelerometer: (a) walking on level floor, and (b) climbing up and down stairs.

Now, with charts!

accelerometer readings while walking on level floor

accelerometer readings while climbing stairs

The guts of my beautiful data collection device look like this:

Pong controller guts
(Click the photo for more details at Flickr.)

I used two Processing sketches—for the thermistor data, Rob’s code; for the accelerometer data, a quick-and-dirty mod of the almost wholly unoriginal code I used for the Understanding Networks assignment.

[java]
/*
Serial String Reader + Simple net Client
Language: Processing

Reads in a string of characters from a serial port until
it gets a linefeed (ASCII 10). Then splits the string into
sections separated by commas. Then converts the sections to ints,
and prints them out.

* Starts a network client that connects to a server on port 8080,
* sends any keystrokes pressed.

created 2 Jun 2005
modified 6 Aug 2008
by Tom Igoe
modified 24 September 2009 by India Amos
*/

import processing.serial.*; // import the Processing serial library
import processing.net.*; // import the Processing networking library

Serial myPort; // The serial port

Client myClient; // instance of the net Client
String data; // string to hold incoming data
String ipAddress = “128.122.151.178”; // address of the server
String myKey1; // the virtual keypress generated by the x-pin of the accelerometer
String myKey2; // the virtual keypress generated by the y-pin of the accelerometer
String myKey3; // the virtual keypress generated by the switch
String myKeyBuffer; // all the virtual keys pressed by a single bleat from the Arduino

float bgcolor; // Background color

void setup() {
// establish the background and foreground:
size(200, 200);
background(50);
fill(200);

// Connect to server on port 8080
myClient = new Client(this, ipAddress, 8080);

// List all the available serial ports
println(Serial.list());

// I know that the first port in the serial list on my mac
// is always my Arduino module, so I open Serial.list()[0].
// Change the 0 to the appropriate number of the serial port
// that your microcontroller is attached to.
myPort = new Serial(this, Serial.list()[0], 9600);

// read bytes into a buffer until you get a linefeed (ASCII 10):
myPort.bufferUntil(‘\n’);
}

void draw() {
}

// serialEvent method is run automatically by the Processing applet
// whenever the buffer reaches the byte value set in the bufferUntil()
// method in the setup():

void serialEvent(Serial myPort) {
// read the serial buffer:
String myString = myPort.readStringUntil(‘\n’);
// if you got any bytes other than the linefeed:
if (myString != null) {

myString = trim(myString);

// split the string at the commas
// and convert the sections into integers:
int sensors[] = int(split(myString, ‘\t’));

// print out the values you got:
for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) { print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t"); } // add a linefeed after all the sensor values are printed: println(); // Convert integers into behavior. // A change of +/-50 on sensor 1 means a roll forward (down) or backward (up), respectively. Flat is 520. // A change of +/-50 on sensor 2 means a tilt coounter-clockwise or clockwise, respectively. Flat is 505. myKey1=""; myKey2=""; myKey3=""; if (sensors[0] > 570) // rolling forward, so paddle down
{
myKey1 = “d”;
}

if (sensors[0] < 470) // rolling backward, so paddle up { myKey1 = "u"; } if (sensors[1] > 555) // rolling counter-clockwise, so paddle left
{
myKey2 = “l”;
}

if (sensors[1] < 455) // rolling clockwise, so paddle right { myKey2 = "r"; } if (sensors[3] > 0 ) // button pressed
{
myKey3 = “x”;
}

myKeyBuffer = myKey1+myKey2+myKey3;
println(myKeyBuffer);

// If there’s incoming data from the client:

if (myClient.available() > 0) {
// get the data:
data = myClient.readString();
println(data);
}

if (myKeyBuffer != null) {
// send out anything that’s typed:
myClient.write(myKeyBuffer);
}
if (keyPressed) {
// send out anything that’s typed:
myClient.write(key);
}

}

}
[/java]

OMFG! I can’t believe this actually works!

Book grid sorted by word count

I made a Flex application!!!1111!!!!

Okay, so what is this thing?

Well, each green block represents the total word count of a book; that count is also given by the big number beneath each book. The orange stripe represents the number of unique words in the book, case insensitive (so The and the count as one). It’s all a continuation of this data visualization project I’ve been working on all semester.

And what’s it made of? Aside from snips, snails, and puppydog tails, it comprises some really stupid code, four static XML docs, some CSS, and one . . . class? I think it’s what passes for a class in Flex. God only knows. Here’s the filthy, embarrassing source code.

Yes, I am well aware that this is some of the most fucked-up, redundant, unnecessarily hard-coded shit you’ve ever seen, and that the radio buttons don’t work right on the first click, but considering that I only started learning Flex on, like, Thursday, and that I didn’t start trying to code this thing in earnest until the wee hours of Monday morning, I think it’s Oh. Kay.

And, yeah, no, I couldn’t figure out how to get rid of the gap between the orange and green blocks. CSS in Flex is really weird and undernourished.

Next steps:

  • Make this code not suck.
  • Add more data (I’ve got about 20 more books on hand to process, and then it’s time to hit Bittorrent).
  • Make the code for pulling out the word counts not suck. Right now it’s case-sensitive, which I don’t want it to be (I’ve been batch-converting the text to lowercase before processing it), and I’ve been manually deleting all words that start with numbers or that look likely to be roman numerals. I’ve also been doing this from Terminal, one file at a time, when it really ought to be able to process in batches. This proves that I am not nearly lazy enough, otherwise I would have dealt with this weeks ago, in order to spare myself a lot of tedious busywork.
  • Make the design not suck (e.g., get rid of those gaps, and replace the radio buttons with something less nasty).
  • Add more views—for example, something should actually happen when you mouse over or click on a book thumbnail, besides it lighting up in hideous powder blue. There are a lot more ways I want to slice up this data, and I still want to be able to compare books or sets of books. That will require building the word-counting code into the Flex app somehow.

But in the meantime, w000t! It works!

India’s ITP blog