Color Survey Results

Who in the rainbow can draw the line where the violet tint ends and the orange tint begins? Distinctly we see the difference of the colors, but where exactly does the one first blendingly enter into the other? So with sanity and insanity.
—Herman Melville, Billy Budd

Orange, red? I don’t know what to believe anymore!
—Anonymous, Color Survey

I WILL EAT YOUR HEART WITH A FUCKING SPOON IF YOU AKS ANY MORE QUESTIONS ABOUT COLORS
—Anonymous, Color Survey

Thank you so much for all the help on the color survey.  Over five million colors were named across 222,500 user sessions.  If you never got around to taking it, it’s too late to contribute any data, but if you want you can see how it worked and take it for fun here.

First, a few basic discoveries:

  • If you ask people to name colors long enough, they go totally crazy.
  • “Puke” and “vomit” are totally real colors.
  • Colorblind people are more likely than non-colorblind people to type “fuck this” (or some variant) and quit in frustration.
  • Indigo was totally just added to the rainbow so it would have 7 colors and make that “ROY G. BIV” acronym work, just like you always suspected. It should really be ROY GBP, with maybe a C or T thrown in there between G and B depending on how the spectrum was converted to RGB.
  • A couple dozen people embedded SQL ‘drop table’ statements in the color names. Nice try, kids.
  • Nobody can spell “fuchsia”.

Overall, the results were really cool and a lot of fun to analyze.  There are some basic limitations of this survey, which are discussed toward the bottom of this post.  But the sheer amount of data here is cool.

Sex

By a strange coincidence, the same night I first made the color survey public, the webcomic Doghouse Diaries put up this comic (which I altered slightly to fit in this blog, click for original):

It was funny, but I realized I could test whether it was accurate (as far as chromosomal sex goes, anyway, which we asked about because it’s tied to colorblindness) [Note: For more on this distinction, see my follow-up post]. After the survey closed, I generated a version of the Doghouse Diaries comic with actual data, using the most frequent color name for the handful of colors in the survey closest to the ones in the comic:

Basically, women were slightly more liberal with the modifiers, but otherwise they generally agreed (and some of the differences may be sampling noise).  The results were similar across the survey—men and women tended on average to call colors the same names.

So I was feeling pretty good about equality.  Then I decided to calculate the ‘most masculine’ and ‘most feminine’ colors.  I was looking for the color names most disproportionately popular among each group; that is, the names that the most women came up with compared to the fewest men (or vice versa).

Here are the color names most disproportionately popular among women:

  1. Dusty Teal
  2. Blush Pink
  3. Dusty Lavender
  4. Butter Yellow
  5. Dusky Rose

Okay, pretty flowery, certainly.  Kind of an incense-bomb-set-off-in-a-Bed-Bath-&-Beyond vibe.  Well, let’s take a look at the other list.

Here are the color names most disproportionately popular among men:

  1. Penis
  2. Gay
  3. WTF
  4. Dunno
  5. Baige

I … that’s not my typo in #5—the only actual color in the list really is a misspelling of “beige”.  And keep in mind, this is based on the number of unique people who answered the color, not the number of times they typed it.  This isn’t just the effect of a couple spammers. In fact, this is after the spamfilter.

I weep for my gender.  But, on to:

RGB Values

Here are RGB values for the first 48 out of about a thousand colors whose RGB values (across the average monitor, shown on a white background) I was able to pin down with a fairly high degree of precision:

The full table of 954 colors is here, also available as a text file here (I have no opinion about whether it should be used to build a new X11 rgb.txt except that seems like the transition would be a huge headache.)

The RGB value for a name is based on the location in the RGB color space where there was the highest frequency of responses choosing that name.  This was tricky to calculate.  I tried simple geometric means (conceptually flawed), a brute force survey of all potential center points (too slow), and fitting kernel density functions (math is hard). In the end, I used the average of a bunch of runs of a stochastic hillclimbing algorithm.  For mostly boring notes on my data handling for this list, see the comments at the bottom of the xkcd.com/color/rgb/ page.

Spelling and Spam

Spelling was an issue for a lot of users:

Now, you may notice that the correct spelling is missing.  This is because I can’t spell it either, and when running the analysis, used Google’s suggestion feature as a spellchecker:

A friend pointed out that to spell it right, you can think of it as “fuck-sia” (“fuch-sia”).

Misspellings aside, a lot of people spammed the database, but there were some decent filters in place.  I dropped out people who gave too many answers which weren’t colors used by many other people.  I also looked at the variation in hue; if people gave the same answer repeatedly for colors of wildly varying hue, I threw out all their results.  This mainly caught people who typed the same thing over and over.  Some were obviously using scripts; based on the filter’s certainty, the #1 spammer in the database was someone who named 2,400 colors—all with the same racial slur.

Map

Here’s a map of color boundaries for a particular part of the RGB cube.  The data here comes from a portion of the survey (1.5 million results) which sampled only this region and showed the colors against both black and white backgrounds.

The data for this chart is here (3.6 MB text file with each RGB triplet named).  Despite some requests, I’m not planning to make a poster of any of this, since it seems wrong to take advantage of all this volunteer effort for a profit; I just wanted to see what the results looked like.  You’re welcome to print one up yourself (huge copy here), but keep in mind that print color spaces are different from monitor ones.

Basic Issues

Of course, there are basic issues with this color survey.  People are primed by the colors they saw previously, which adds overall noise and some biases to the data (although it all seemed to even out in the end).  Moreover, monitors vary; RGB is not an absolute color space.  Fortunately, what I’m really interested in is what colors will look like on a typical monitors, so most of this data is across the sample of all non-colorblind users on all types of monitors (>90% LCD, roughly 6% CRT).

Color is a really fascinating topic, especially since we’re taught so many different and often contradictory ideas about rainbows, different primary colors, and frequencies of light. If you want to understand it better, you might try the neat introduction in Chapter 35 ofThe Feynman Lectures on Physics (Vol. 1), read Charles Poynton’s Color FAQ, or just peruse links from the Wikipedia article on color.  For the purposes of this survey, we’re working inside the RGB space of the average monitor, so this data is useful for picking and naming screen colors. And really, if you’re reading this blog, odds are you probably—like me—spend more time looking at a monitor than at the outdoors anyway.

Miscellaneous

Lastly, here are some assorted things people came up with while labeling colors:

Thank you so much to relsqui for writing the survey frontend, and to everyone else who sacrificed their eyeballs for this project.  If you have ideas and want to analyze these results further, I’ve posted the raw data as an SQLite dump here (84 MB .tar.gz file). It’s been anonymized, with IPs, URLs, and emails removed.  I also have GeoIP information; if you’d like to do geocorrelation of some kind, I’ll be providing a version of the data with basic region-level lat/long information (limited to protect privacy) sometime in the next few days. Note: The ColorDB data is the main survey.  The SatOnly data is the supplementary survey covering only the RGB faces in the map, and was presented on a half-black half-white background.)

And, of course, if you do anything fun with this data, I’d love to see the results—let me know at xkcd@xkcd.com.

Color Name Survey

I’d like your help for a color name survey!  The survey shows you colors, and you type a name (word or phrase) you might use for that color.  The names can be as broad or specific as you want.  The survey is here:

http://aram.xkcd.com/color/

My friend Elizabeth (relsqui on #xkcd) wrote the frontend. I’m doing the analysis, though I won’t go too deeply into the details or purpose (to the extent that it has one) for now so as not to bias peoples’ answers.  Of course, RGB is a small and relative color space which varies depending on the device displaying it, so this survey has its limits, but it’s produced some cool data so far.

If any of  you want to help, you can fill out a few quick questions (don’t worry if you don’t know the answers—they’re highly optional and just help with calibration) and then come up with color names.  There’s no end to the survey; the more you answer, the better the data.  Thanks!

P.S. I’ve added a few much-requested prints and two new posters (Movie Narrative Charts and Gravity Wells) in the xkcd store.  The prints of Comic #150 (Grownups) are marked backordered right now while I replace a batch that was printed wrong, but they should be shipping again within a week.

Math Puzzle

This cool puzzle (and solution) comes from my friend Mike.

Alice secretly picks two different real numbers by an unknown process and puts them in two (abstract) envelopes.  Bob chooses one of the two envelopes randomly (with a fair coin toss), and shows you the number in that envelope.  You must now guess whether the number in the other, closed envelope is larger or smaller than the one you’ve seen.

Is there a strategy which gives you a better than 50% chance of guessing correctly, no matter what procedure Alice used to pick her numbers?

I initially thought there wasn’t, or that the problem was paradoxically defined, but it turns out that it’s perfectly valid and the answer is “Yes.”  See my first comment for an example of one such winning strategy.

This puzzle is similar to, but not the same as, the two envelopes paradox.  For more time-devouring reading, see Wikipedia’s List of Paradoxes.

Android Bug Reports, Songs, Rovers

The following is a partial list of bugs in Android (or associated software) which have impacted my actual life in some way.  Some may have been fixed since I last encountered them.

  • When navigation is muted, but you hit volume down just to be safe, it unmutes and starts blaring, waking other passengers on the bus.
  • Sometimes the GPS stops getting locks on satellites until the phone is rebooted.  (This may be related to the GPSStatus app, installed to avoid this kind of thing.)  To be fair, satellites are very small and far away, so you can hardly blame it for having trouble.
  • You can say “Call <contact name>”, and contact names+addresses show up in the navigation search with “navigate to” options, but you can’t say “Navgiate to <contact name>” like you can with all other major actions.  This means you have to type their full address to find out which exit to take from the traffic circle, and it’s really hard to type while holding the wheel steadily to the left like that.
  • Sometimes, when arranging home screen icons, you feel sad and you’re not sure why.
  • If you follow Google’s guidelines and have your own Jabber DNS records set up in a particular way on your Google Apps server, it screws up Google Talk’s ability to authenticate, which in turn (for some bizarre reason) causes all app downloads to hang midway through.  This is rare enough that the error messages are not Googleable.  But don’t worry, it’s easy to figure out as long as you’ve read all of the source code to Android and to Google’s servers, which is good practice when you get any product.
  • Sometimes the home screen icons all disappear.  The only thing which fixes this is a restart, going through five menus to kill the “Home screen” process, or crying quietly for hours until the icons start to feel sorry for you and come back.
  • Occasionally, when swiping the lock sideways to unlock the phone, the lock button images are rotated by 90 degrees.  This is probably connected to your Jabber server somehow.
  • Sometimes an Android user will think they hear someone say their name, but they’re not sure, so they say ‘Yes?’, but then it turns out it was something else.
  • Maps Navigation doesn’t cache your route, so if you drive through an area with bad cell coverage, it may silently stop notifying you of upcoming turns, and you won’t realize anything’s wrong until you discover you’ve taken NH Route 2 all the way to the White Mountains, which are very cold.
  • Latitude doesn’t update well on either end.  Often times, it will tell you someone was in a particular place “5 minutes ago”, and then 10 minutes later — after multiple restarts — it will still say it got that last update from them “5 minutes ago” even if they’re actively using Maps, so you have no idea whether they’ve left.  This can continue for hours as you slowly run out of air in the closet.
  • The phone occasionally locks on full brightness and turns off the keyboard backlight, which is fixed by un-checking and then re-checking the “automatic brightness” checkbox, also about five menus deep, which is hard to see when you’re squinting because the screen is so bright.  Fortunately, it never gets very dark where I live in Boston.  Unfortunately, it’s extremely dark at night in the White Mountains.
  • Navigation instructions silence things like podcasts players, but don’t pause them.  Thus, to hear that missing sentence or two, you have to switch over to the podcast app and hit ‘back’ several times, then swerve your car to avoid the stupid stop sign that shouldn’t be in the middle of the sidewalk to begin with.
  • Google Voice doesn’t do push notifications, so you often get voicemails quite some time after the caller leaves them, sometimes after you’ve already called them back.  This can make you call your doctor back again and have a really confusing conversation where you accidentally get a second prescription. Which you can then get filled and sell on the street. Come to think of it, this bug might be a feature.
  • If you stop for gas, sometimes navigation suspends, but doesn’t resume when you start driving again (or just disappears without notifying you), so you miss the upcoming turn and think you’re already on I-95, and by the time you discover your mistake and turn around you’ve lost enough time that you totally get to the conference too late to catch Richard Stallman doing his acapella Bad Romance cover which is the whole reason you paid the entry fee in the first place.
  • If you have several Google accounts, there’s no way to select which one to use for Google Checkout purchases.  It just picks whichever one it notices first, even if it’s one from an old Google account tied to your mom’s credit card, so now she might know that you bought the app to turn the phone into a vibrator.  (The app doesn’t really work since the vibrate motor is too weak, but the reviews by people who don’t understand its purpose are hilarious.)
  • If you have a secondary account that it’s decided to use for Checkout, and you want to delete it so the phone will use your primary one, you can’t.  Why?  Because that account is used for an essential service and so can’t be deleted.  As far as I can guess, that service is Google Checkout.  This bug report is dedicated to Joseph Heller.
  • Google Chrome in OS X follows the Apple guidelines concerning the green “+” button, and has it make the window no bigger than it deems necessary to fit the current page’s contents.  This annoys a lot of people. Since there are no OS X Chrome extensions, the best workaround seems to be registering maximizechrome.com and putting a giant <div> there.  Whenever you need to maximize, you load that page before hitting the green button.  (This bug is not, technically, related to Android in any way.)
  • There’s a Fantastic Contraption app on iPhone, but not Android. This is probably a feature rather than a bug, since it means Android users can spend time on things other than playing Fantastic Contraption.

These issues aside, I’m really happy with my Droid.  The screen is incredible, it’s much faster and easier to use than the G1, and I wouldn’t trade away the physical keyboard and persistent SSH for anything.


I just about fell out of my chair when I saw this.  Thank you to everyone who appeared in it, and to Olga and Elaine for doing whatever black magic they did to get them all together. <3. (Side note: I met Neil Gaiman once, back when I was in college, when his book tour came near my town. At the signing afterward, I talked awkwardly to him for what seemed like several minutes while he signed stuff for our group. Later, my friends pointed out that I was speaking too quietly for him to hear—or, in fact, to notice that I was standing there at all. I’ve been quietly embarrassed about that ever since and this video makes me feel better.)


If anyone is still sad about my Spirit comic, maybe this person’s rewrite – author unknown, found on a random image page – might provide some comfort.  Also, I got a couple of nice letters from members of the Spirit/Opportunity teams, and it’s very clear how proud they are of these little rovers. Next, Europa!

Books and Laptops and Bugs

Hey!  To anyone in the Boston area who wants to get an xkcd book signed, I’ll be doing a signing event at MIT this Thursday (the 17th).  It’s in Room 26-100 from 6:00 to 8:00 PM.  Details are here.  The event is free, and there will be books for sale there.  I’ve been pretty busy signing prints for the store to keep up with the massive flood of Christmas orders, but I wanted to get in a free non-charity event for people who wanted to get books signed but couldn’t afford tickets.


Speaking of which, here are pictures of the xkcd school nearing completion in Laos:

Thanks for all suggestions for the dedication plaque thing! Here’s what we ended up going with:

“Do not train children to learning by force and harshness, but direct them to it by what amuses their minds, so that you may be better able to discover with accuracy the peculiar bent of the genius of each.” – Plato.

This school is a gift from the readers of XKCD, an internet comic strip. The world is full of exciting things to discover. We hope you find some of them.

Thanks, everyone.  <3


I posted a while back about getting an x200s laptop.  A big part of my decision was based on its durability — reviews touted the roll cage and carbon-fiber composite casing.  Sadly, it doesn’t seem to hold up very well in my daily use.  The case has cracked in five or six places, and some of these breaks have warped the frame.  It also just feels flimsy.  The related computers in the line (the x301, the x200, etc) are reportedly a lot sturdier.

Long and boring story short, I decided to repair and resell it, and use the money to get a 13-inch Macbook Pro.  It’s significantly heavier than the x200s, but I have a Droid now (which I quite like) and it’s helped remove a lot of my need for an ultraportable laptop.  I’ve been using an OS X desktop (a Mac Mini) as my main desktop machine for a couple years now, and I’m tired of rebooting my laptop to Windows when traveling to get to Photoshop (Wine et. al. don’t quite cut it).  The things I use Linux for (programming and playing with random toys/projects) are, on the other hand, easily done in VMs.  So for me, an OS X laptop seems like a logical choice, and so far I’ve been largely happy with it.

I love the multitouch trackpad (I’ve installed jitouch to expand the set of available gestures), and VMs seem to run nicely.  Supposedly the battery’s lifespan is much improved from the old 250 cycles.  If not, expect a post in about a year and a half about me blowing up my apartment while attempting to replace the battery with an off-brand one.  I may die, but at least I’ll prove the TSA wrong. This laptop has the bonus of being more powerful than my old desktop, which I can now resell to another friend (making the laptop my primary computer), and actually come out ahead on this whole affair.


So long as I’m blabbing about recent gadget experience: Amazon’s support/replacement for broken Kindles is amazing (“We’ll rush-ship you a new one immediately! Just send the old one back to us whenever. So sorry!”). The Droid is proving excellent, though it seems like each day brings a hilarious new Android bugs. For example, it turns out having certain kinds of DNS records for the Jabber server for your Google Apps account for your domain can, starting last week, through a complicated series of events, cause Market downloads to freeze and keep you from updating any apps on the phone.  That was a fun bug to try to track down, but it’s trumped in sheer weirdness by the Droid’s 24.5-day Autofocus bug, which is itself the weirdest bug story since The Case of the 500-Mile Email.

Signed Book Timeline

When I put my book up for sale, I thought it would be neat to offer the option of a signed copy to people who ordered in the first 24 hours. It was a popular option (more popular than we expected), and some people are wondering why they took so long to get there (most should have arrived last week, the last few American copies should have arrived by now, and overseas orders will be shipping outas soon as I finish signing them).  I had put a note on the order page warning that the signed copies would take longer to ship; if anyone’s interested in why they took as long as they did, here’s the story:

The regular copies were shipped immediately, and continue to ship on time.  The people who ship the books get them out fast — orders are generally shipped within 36 hours of when we get them.  The signed books started shipping out within a few days of the initial sale, and generally arrived promptly.  The books were being printed right up to the publication date, so I didn’t actually have a large stock of them beforehand.  Since I had no idea how popular the signed copies would be, I put up a note saying that the signed books might take longer to ship, since I’d be signing them after they were ordered.  This meant they had to be shipped to me first, before going to the shipping people.  It turned out (to my delight and horror) that they were extremely popular, and we had to raise the price just to make sure there wouldn’t be more orders than I had time to sign.

Since I was going on the book tour soon after the book went up for sale, we had the bulk of the books sent out to the reddit offices in San Francisco.  For the week after my book tour, I sat up late at one of their desks signing books (and working my way through the entire run of The West Wing).  I got all those signed by the end of the week (leaving an impressive callous on my right middle finger) and left the reddit office full of boxes for FedEx.  They picked them up Monday the 28th and took them to the shipping place in Virginia.  Because of the weight, they had to go ground, so they took all of that week to go there.  Those books shipped out within 36 hours of arriving, so they should have arrived late last week.

I got back to Boston, and later that week the next set of books arrived there.  I signed a large set of those, which finished off the US orders, and they went to the distribution office for shipping out sometime last week.  For the last week, I’ve been signing boxes of mainly European orders (followed by Australia and other overseas, which take longer to ship), and I should be finishing those up in the next handful of days.  Then there’s a four-or-five day delay as they’re sent to the shipping people (too heavy to send by anything but FedEx Ground), and they’ll go out then.  I apologize to anyone whose signed books took longer than they expected to arrive.  If you have any questions or problems regarding your order, email orders@xkcd.com.

Thank you so much to everyone who ordered a book.  If you’re interested, there are some pictures of the xkcd school site in Laos over on the BreadPig blog.

School

Just a note: thank you to everyone who made it to the events. We raised enough money to build a school!  It’ll be in the Salavan Province, on Road No. 13 South.  We’re not going to torment the kids’ ability to learn phonetics by calling it The XKCD School, but we will be writing a dedication which will go on a plaque.  You guys made this happen — does anyone have any suggestions for what it should say?

Also, I’m currently happily hiding out for a while working on drawing and other projects, but I’ll hopefully be doing some less fundraise-y free signing events around Boston in the next month or two.  I’ll let you know once we have any places or dates!

Physics for Entertainment

Update: I’m finally home after a month or so of nonstop events, including several xkcd book fund-raisers/signings.  I met tons of cool people, we raised a lot of money for the EFF and Room to Read, and at one point I signed a book for a robot.  Thank you to everyone who ordered a copy, by the way!  I hope you like it.  They’re shipping out nicely, and we’re about ready to order a second printing.

The events and travel were a huge amount of fun, and I loved getting to talk to (or at) so many of you cool people.  But I’m an introvert at heart, and after doing that much socializing I feel a powerful urge to hide in my room for about a month.  At some point in my travels I seem to have picked up a cold that’s been keeping me down for a couple days, so it’s just as well that I don’t have any more events on the immediate calendar.  There’s no fever, so it’s not swine flu, but it’s keeping me awake at night and I’m going through a lot of tissues and cough medicine.  But it should blow over in a couple days, and then I’ll get to spend a while quietly working on new projects!

While I’m doing that, here’s a bit about a neat book I found recently:

Physics for Entertainment:

Physics for Entertainment was written by Yakov Perelman in the 1920′s (in Russian) and updated periodically through the 1930′s.  There are actually two parts to it, but Volume 1 is long out-of-print (though findable online — more on that later).  The book I have is a 1975 translation of Volume 2. The book is a series of a few hundred examples, no more than one or two pages each, asking a question that illustrates some idea in basic physics.

It’s neat to see what has and hasn’t changed in the last century or so.  Many of the examples he uses seem to be straight out of a modern high school physics textbook, while others were totally new to me.  And some of the answers to the questions he poses seem obvious, but others made me stop and think.  The diagram to the right shows a design for a fountain with no pump — it took me a while to get why it works.  (For an easier-to-build variant, click here.)  Later in the book, he explains the physics of that drinking bird toy.

It’s written in a fun, engaging, conversational style, as if he’s in the room chatting with you about these neat ideas.

There are a lot of diagrams:

And it’s hard not to like the guy:

“If you’ll bear with me for a moment, let’s analyze this fairy tale from a physics standpoint …”  That’s a man after my own heart.

He also spends a lot of time discussing why various perpetual-motion machines won’t work.  it’s interesting to see that there was as thriving a community of free energy people a century ago as there is now, many of their designs based on the same misapplications of physics.

Lastly, when he talks about space travel — from a pre-space-age perspective — he turns starry-eyed and poetic:

I alternate wildly between thinking that it’s totally crazy that we clawed our way up out of the atmosphere and walked on the moon, and thinking that it’s a shame that it turned out to be so boring.  But I really desperately want to see more missions to places like the Jovian moons. If it turns out one of them is teeming with life, we’re gonna feel awfully silly about how long we spent shuffling around in the Martian dust.  Also, Kepler is really exciting, putting us in a much better place to speculate about life in the galaxy.

You can get the printed Volume 2 on Amazon, while Volume 1 was supposedly unavailable for translation or reprint.  However, I mentioned this book at one of the events recently, and reader Matthias Kübel emailed me to let me know Volume 1 is available free online!  I’m looking forward to reading through it.

xkcd: volume 0

The xkcd book is now officially available in the store! (There are also a handful of new shirts available for preorder, and we’ve got the signed prints back in stock).

It’s been fun putting it all together.  It was neat to go back through various huge stacks of old drawings, some on the back of school assignments, and scan them at print resolution.  I also had fun with the marginal notes.  I’m really excited to finally have it in print, and I’m looking forward to seeing people and signing copies at the release events this weekend.  I’m also excited about getting back to work on some other projects which have been on hold for a bit, at least one of which will involve lakes and a recently-acquired Arduino.

P.S. Thank you so much for the help with the phone this weekend.  Through a bizarre set of exploits, I’ve gotten it unlocked without losing any data (for details, see the edits to the previous post).