Trochee Chart

Here’s something I made as I drew today’s comic.  It’s a chart of Google results for “X Y” (in quotes) where X and Y are words from the first panel of the strip.  The first word is on the top, the second down the side (the opposite of the intuitive way, of course).

"Doctor Doctor" and "Jesus Jesus" are highest. The highest non-repeating combo is "Pirate Captain", followed by "Robot Monkey" and "Penguin Zombie".

I generated this using a Google API variable search tool developed by Eviltwin on #xkcd (I’m not linking to the tool so as to avoid potentially getting his API key revoked) Edit: He now offers the source and says it can be run without a key, and is happy to let people use it until Google does something. Not only is the API helpful in making these kinds of charts (which I spend more time doing than I care to admit), it also gives a roughly accurate count of results—in contrast to the Google search page.

The “number of results” count that Google gives when you search is clearly fabricated.  This is clear for a few reasons.  When Google says this:

Excellent!  That's a lot!

You can tell that it’s wrong first by scrolling to the end of the results.  When you get to page 32, it suddenly becomes:

I learned in AP Calculus that 316 is WAY less than 190,000.

This doesn’t usually matter, since nobody looks much past the first few pages of results, but it’s annoying if you’re trying to use the number of results as a measure of something.  When I was making the Numbers comic, I didn’t use the API, and there were a few graphs I had to throw out, crop, or put on an unnecessary log scale; otherwise, Google’s clumsy number-fudging made the graphs look nonsensical.  I can’t find a good example now (perhaps they’ve smoothed it out a bit) but when searching for things like “I was born in <X>”, the results for successive years would look something like this:

… 150 : 200 : 250 : 300 : 350 : 117,000 : 450 : 251,000 : 500 : 550 : 312,000 : 320,000 : 390,000 : 425,000 …

If you scrolled to the last page for each, you’d find that the smaller counts were roughly accurate, but the counts in the hundreds of thousands had no more actual results than their neighbors.

I suppose it’s remotely possible that these numbers are correct, there are no years with an in-between number of hits, and for some reason they’re just not showing you most of the promised pages when you try to flip through them.  But making this even less likely is the fact that the search API (which is apparently being deprecated and replaced right now) doesn’t return these bad numbers—it gives reasonable-looking results which seem to be roughly consistent with the number you come up with by navigating to the last search page.

So it really looks like there’s a certain threshold of result volume beyond which Google apparently says “screw it” and throws out a gigantic number.  I imagine this is probably due to incompetence rather than intentional deception; I’m sure it’s hard to generate pages quickly from many sources, and maybe for searches with a lot of results they don’t have time to get it all synced up.  So they fudge the numbers.  The fact that this makes it look like they have way more results than they do is presumably just an unintended bonus.

All in all, this isn’t a big deal and I don’t think there’s anything particularly evil about it. It does make it hard to use Google hits as an accurate gauge of anything, but I suppose if you’re trying to study something by seriously analyzing Google result counts, you have bigger methodological problems to worry about.

Edit: As Mankoff observes, it looks like the API sometimes *underestimates* the number of results, too.  For example, it still reports 0 results for “narwhal zombie”, when a regular search shows quite a few. Now, I notice, scrolling through them, that most either have some minor character/text in between the two words, or are related to the comic I just posted.  But at least one seems to date back to last year.

141 thoughts on “Trochee Chart

  1. I suppose it’s remotely possible that these numbers are correct, there are no years with an in-between number of hits!Very interesting!

    Like

  2. A great article written with great hard work…i must say….a great work of your which shows…I like your site its quite informative and i would like to come here again as i get some time from my studies. And I will share it with my friends.

    Like

  3. Pingback: Google, Apple, Microsoft: Who’s bad? | Erbloggtes

  4. A great article written with great hard work…i must say….a great work of your which shows…I like your site its quite informative and i would like to come here again as i get some time from my studies. And I will share it with my friends.good

    Like

  5. Pingback: This is how quickly we give up on resolutions

  6. Pingback: Attack of the narwhal zombies! » Aaron at the Internet Archive

  7. the number may be accurate. whenever you get to the last page it always tells you some were omitted because of duplication/similarity/etc

    Like

  8. Pingback: The infographics of xkcd | halfblog.net

  9. ay is, perhaps, the best western movie about Budhism.
    He is born, lives and dies everyday, over and over untill ove

    Like

  10. It was an interesting discussion. Starting this debate so thank you., I had not known about the plot. Much I thank you for sharing this.

    Like

  11. Good post. I learn something totally new and challenging on websites I stumbleupon every day. It will always be useful to read articles from other writers and use something from other sites

    Like

  12. Thank you for presenting your points and providing this information. I have learned something about this topic.

    Like

  13. This excellent article assisted me very much! Saved your site, extremely great categories everywhere that I read here! I really appreciate the info.

    Like

  14. Thank you for a wonderful blog. Your suggestions are so helpful. I’ve been journaling for quite some time and it helps me to write down my prayer for the day and helps me focus.

    Like

  15. This is the most amazing article that I’ve ever read in my whole life. I am really impressed by the work you’ve done on this topic. I want to learn more about this topic.

    Like

Comments are closed.