Trochee Chart

Posted byRandall 144 Comments on Trochee Chart

Here’s something I made as I drew today’s comic.  It’s a chart of Google results for “X Y” (in quotes) where X and Y are words from the first panel of the strip.  The first word is on the top, the second down the side (the opposite of the intuitive way, of course).

"Doctor Doctor" and "Jesus Jesus" are highest. The highest non-repeating combo is "Pirate Captain", followed by "Robot Monkey" and "Penguin Zombie".

I generated this using a Google API variable search tool developed by Eviltwin on #xkcd (I’m not linking to the tool so as to avoid potentially getting his API key revoked) Edit: He now offers the source and says it can be run without a key, and is happy to let people use it until Google does something. Not only is the API helpful in making these kinds of charts (which I spend more time doing than I care to admit), it also gives a roughly accurate count of results—in contrast to the Google search page.

The “number of results” count that Google gives when you search is clearly fabricated.  This is clear for a few reasons.  When Google says this:

Excellent! That's a lot!

You can tell that it’s wrong first by scrolling to the end of the results.  When you get to page 32, it suddenly becomes:

I learned in AP Calculus that 316 is WAY less than 190,000.

This doesn’t usually matter, since nobody looks much past the first few pages of results, but it’s annoying if you’re trying to use the number of results as a measure of something.  When I was making the Numbers comic, I didn’t use the API, and there were a few graphs I had to throw out, crop, or put on an unnecessary log scale; otherwise, Google’s clumsy number-fudging made the graphs look nonsensical.  I can’t find a good example now (perhaps they’ve smoothed it out a bit) but when searching for things like “I was born in <X>”, the results for successive years would look something like this:

… 150 : 200 : 250 : 300 : 350 : 117,000 : 450 : 251,000 : 500 : 550 : 312,000 : 320,000 : 390,000 : 425,000 …

If you scrolled to the last page for each, you’d find that the smaller counts were roughly accurate, but the counts in the hundreds of thousands had no more actual results than their neighbors.

I suppose it’s remotely possible that these numbers are correct, there are no years with an in-between number of hits, and for some reason they’re just not showing you most of the promised pages when you try to flip through them.  But making this even less likely is the fact that the search API (which is apparently being deprecated and replaced right now) doesn’t return these bad numbers—it gives reasonable-looking results which seem to be roughly consistent with the number you come up with by navigating to the last search page.

So it really looks like there’s a certain threshold of result volume beyond which Google apparently says “screw it” and throws out a gigantic number.  I imagine this is probably due to incompetence rather than intentional deception; I’m sure it’s hard to generate pages quickly from many sources, and maybe for searches with a lot of results they don’t have time to get it all synced up.  So they fudge the numbers.  The fact that this makes it look like they have way more results than they do is presumably just an unintended bonus.

All in all, this isn’t a big deal and I don’t think there’s anything particularly evil about it. It does make it hard to use Google hits as an accurate gauge of anything, but I suppose if you’re trying to study something by seriously analyzing Google result counts, you have bigger methodological problems to worry about.

Edit: As Mankoff observes, it looks like the API sometimes *underestimates* the number of results, too.  For example, it still reports 0 results for “narwhal zombie”, when a regular search shows quite a few. Now, I notice, scrolling through them, that most either have some minor character/text in between the two words, or are related to the comic I just posted.  But at least one seems to date back to last year.

144 replies on “Trochee Chart”

  3. @gormster

    Just substitute the trochees into We Didn’t Start the Fire.

    Works like a charm.

    Like

  7. A great article written with great hard work…i must say….a great work of your which shows…I like your site its quite informative and i would like to come here again as i get some time from my studies. And I will share it with my friends.

    Like

  9. Pingback: Google, Apple, Microsoft: Who’s bad? | Erbloggtes

  10. A great article written with great hard work…i must say….a great work of your which shows…I like your site its quite informative and i would like to come here again as i get some time from my studies. And I will share it with my friends.good

    Like

  14. Pingback: This is how quickly we give up on resolutions
  17. Pingback: Attack of the narwhal zombies! » Aaron at the Internet Archive

  18. the number may be accurate. whenever you get to the last page it always tells you some were omitted because of duplication/similarity/etc

    Like

  20. Pingback: The infographics of xkcd | halfblog.net

  28. Good post. I learn something totally new and challenging on websites I stumbleupon every day. It will always be useful to read articles from other writers and use something from other sites

    Like

  37. Thank you for a wonderful blog. Your suggestions are so helpful. I’ve been journaling for quite some time and it helps me to write down my prayer for the day and helps me focus.

    Like

  43. This is the most amazing article that I’ve ever read in my whole life. I am really impressed by the work you’ve done on this topic. I want to learn more about this topic.

    Like

Comments are closed.