Edit 2: Oh God Oh God 4chan has Robot9000. soup /r9k/. Have fun with the bot and do one last barrel roll for me.
Edit: As expected, with the huge flood of new traffic after this post went up, the channel is full of new folks coming in and playing with the bot. This is unavoidable and expected for these first few days, and ROBOT9000 is actually controlling the noise pretty well. Still, #xkcd-signal is a social channel — if you just want to play games with the moderator/concept, please use #moderator-sandbox. Thanks!
#xkcd has had about 250 chatters these days. Large communities suck. This problem is hard to solve, but we’ve come up with a fun attack on it — enforced originality (in a very narrow sense). My friend zigdon and I have put together an auto-moderation system in an experimental channel, #xkcd-signal, and it seems to work well, so we invite you all to take part.
When social communities grow past a certain point (Dunbar’s Number?), they start to suck. Be they sororities or IRC channels, there’s a point where they get big enough that nobody knows everybody anymore. The community becomes overwhelmed with noise from various small cliques and floods of obnoxious people and the signal-to-noise ratio eventually drops to near-zero — no signal, just noise. This has happened to every channel I’ve been on that started small and slowly got big.
There are a couple of standard ways to deal with this, and each one has problems. Here’s an outline of the major approaches (skip down if you just want to read about ROBOT9000):
- Strict entry requirements: This is the secret club/sorority approach. You can vet every new person before they’re allowed to speak. This sucks. It reminds me of Feynman’s comment on resigning the National Academy of Sciences — he said that he saw no point in belonging to an organization that spent most of its time deciding who to let in. The problems are apparent during sorority rush week on college campuses. Not only is the question of who does the vetting (and how) difficult, but the drama reaches horrifying levels as bitter counter-cliques rise up and do battle.
- Moderators: This is the approach IRC channels and forums usually take. You designate a few ‘good’ people who can deal with noise as it happens, by muting, kicking, banning, or editing content as need be. There are a couple problems here — the circle of moderators has to grow with the community. It eventually becomes fairly large, with complicated dynamics of its own, and the process of choosing moderators leads to sorority/NSF-esque drama and general obnoxiousness. I don’t like the elitism that inevitably develops, and prefer more egalitarian systems.
- Running peer-moderation: When it’s possible, this is a good approach. It’s used to great effect on comment threads, with Slashdot pioneering the whole thing and sites like reddit stripping it down to an effective core. But it doesn’t work very well for live time-dependent things like IRC channels.
- Splinter communities: This has happened on most IRC channels I’ve been on — small invite-only side channels sprout up with particular focuses. Often, the older core members of the community go off to create their own high-signal channel, which is generally kept quiet. But this is limited — it lacks the open mixing of the internet that often makes online communities work.
I was trying to decide what made a channel consistently enjoyable. A common factor in my favorite hangouts seemed to be a focus on original and unpredictable content on each line. It didn’t necessarily need to be useful, just interesting. I started trying to think of ways to encourage this.
And then I had an idea — what if you were only allowed to say sentences that had never been said before, ever? A bot with access to the full channel logs could kick you out when you repeated something that had already been said. There would be no “all your base are belong to us”, no “lol”, no “asl”, no “there are no girls on the internet”. No “I know rite”, no “hi everyone”, no “morning sucks.” Just thoughtful, full sentences.
There are a few obvious questions/objections, and I think each of them has been answered by experiment:
Q: Can’t you just tack a random set of letters on the end to ensure your line is unique (or misspell things, add in gibberish, etc)?
A: Of course. The moderator has plenty of holes if you’re acting in bad faith. But if you’re doing that, why are you in the channel at all? Folks who persist in doing this anyway earn (like any spammers) a prompt manual ban.
Q: Won’t it get harder and harder to chat as lines get “used up”?
A: You underestimate the number of possible sentences. We’ve been working off two years (2 million) lines of logs, and it’s not very hard at all — I expect the channel will be able to run for at least a decade before it becomes a problem, and probably long past that.
Q: What about common parts of conversation, like “yeah” and the like?
A: Surprisingly, it doesn’t seem to be a huge problem. In some cases, they can be done without entirely, and in others, you’re just forced to elaborate a little bit on what you’re agreeing with and why.
I talked it over with zigdon, a Perl guru, and he coded it up. We called the project ROBOT9000 (the most generic, unoriginal name for a bot that we could think of). Then we started a sister channel to #xkcd and put the bot in it. #xkcd-signal has been running for the last couple weeks (using the last two years of #xkcd logs) with about 60 reasonably active chatters, and it’s working beautifully — good, solid chat between relative strangers, with very little noise. (We’ll see how it handles the influx of people as we announce the experiment to the wider net.)
In zig’s implementation, the moderator bot mutes (-v) chatters for a period after every violation. The mute time starts at two seconds and quadruples with each subsequent violation, so you have five or six tries to get the hang of it. Your mute-time decays by half every six hours (we’re still tweaking the parameters). When looking for matches, the bot ignores punctuation, case, and nicks.
The big problem we ran into, actually, was meta-discussion overwhelming the channel. Every new person wanted to speculate about the rules and their effect, and every violation was followed by a long postmortem. At first, we had a scoreboard showing who was the best at talking without violation, but this quickly turned into a competition, destroying actual chat. When we took down the scoreboard and banished meta-discussion of the channel to #meta-discussion, everything worked out nicely. (And, of course, for discussion of the concept of #meta-discussion people had to go to #meta-meta-discussion, and for chat about how silly that whole idea was, we created #meta-meta-meta-discussion …)
You’re welcome to come hang out with us. The moderator bot is running in #xkcd-signal on Foonetic (irc.foonetic.net or irc.xkcd.com). But again, it’s a social channel; take discussion of the concept to #meta-discussion.
If you’d like to run this bot in your own channel, zig has published an initial version of the code here:
http://irc.peeron.com/xkcd/ROBOT9000.html (Perl bot, SQL skeleton, Changelog)
xkcd 
YOU SHOULD REALLY CALL IT #XKCD-ZIGNAL! BECAUSE YOU’RE AN ADORABLE TEAM
LikeLike
I can’t express how brilliant this is. I wish this could be applied to non-electronic communications/conversations/lectures/publications. I may have to learn to use IRC.
LikeLike
This is very interesting – one wonders if it should be applied to comments…
Would it work in a group devoted to a particular topic, though, instead of just general chat?
LikeLike
To expand slightly on my previous comment about how this might come into trouble with topic-specific chat…
If you had a chat about “Canadian Law”, wouldn’t there be major precedent-setting cases that would continually be brought up, as they would be relevant to many discussions? Or sections of the Canadian Charter being quoted?
In the case of topic specific chat, it seems like there would be a sort of “standard toolbox” of things that would be totally relevant in many circumstances; but talking about things within this “toolbox” would get harder and harder.
I suppose you could designate sentences and specific elements within this “toolbox” as safe – but now you’re back at your original problem (who determines what is allowed? Groups and Counter-Groups clawing at each other, stating that something should or should not be allowed into the “toolbox”)
LikeLike
Lol.
LikeLike
Nice idea…!
Have you seen the stupidfilter project? Inspired by the same frustration, aims to rid the world of stupid comments on forums and blogs!
http://stupidfilter.org/
..hmm…seems to be down… I was really hoping it was for real… 😦
LikeLike
With all due respect, ROBOT2000 is a much more generic and unoriginal name for a bot than ROBOT9000. Of course it’s possible that I’m just showing my age.
LikeLike
I would love to see this applied to 4chan.
LikeLike
This is a fascinating post. I am really interested in methods of increasing the signal to noise ratio without losing the positives of variability and mixing of ideas. If you’re generally interested in the topic as well, you may want to check out the “ephemeral interest groups” implemented at Bellcore (http://portal.acm.org/citation.cfm?id=143457.143465&coll=portal&dl=ACM).
LikeLike
If it works good for two year’s log, you can just continue – not scan the whole archive, but only last two years. So repeating of meaningfull stuff will never be a problem. And a channel can stand one “lol” every 2 years…
LikeLike
genius, freaking genius! i’m there!
LikeLike
Thought I should point out that Dunbar’s Number is a theoretical average limit on your direct social connections, not the sub-networks in it.
The biggest problem here is that sub-groups exist even in relatively small IRC communities, due to various factors. Time of day that a user regularly visits is the most predominate sub-group predictor in communities with less than 500 total individuals (this comes out to around 170 individuals on-line at any given time). With more than 500 individuals, at-least in the groups where I managed to collect the data, geography was a better predictor.
/Totally studied IRC social dynamics for my high-school Advanced Sociology class.
//Personally read over 15 million lines of chat logs to classify the sub-group relationships.
///57% of the studied communities had an average public “u wanna cyber?” rate higher than 1 per day.
LikeLike
Love the idea. Would love it even more if I didn’t see the moderation messages. The channel description says “Use /ignore for less status-change noise”, but then I’m told “IGNORE Unknown command”. I’m using Opera.
LikeLike
Your mom jokes are still a problem. If someone manages to successfully say something, then adding “your mom” into it would likely be successful as well. Of course, this would force the creation of “original” “your mom” jokes, but “your mom” jokes are more noisy than signaltastic, even when they are original.
LikeLike
You should name the bot “Anna”. http://en.wikipedia.org/wiki/Boten_Anna
LikeLike
This is pretty fascinating. For a case study, look at the subforum subcultures of the Somethingawful Forums, FYAD and BYOB in particular.
LikeLike
Now as a clickable URI: irc://irc.xkcd.com/xkcd-signal
LikeLike
I still think there’s a problem. I think a lot of culture, and nerd culture in particular is based on repetition. Yes, tropes get old and dated (“there are no girls on the internet”, “hugahlugah”) but what this will do is stifle new tropes.
Look, if I want meaningful, thoughtful discussion I’ve got a campus full of fellow professors to talk with about damn-near anything. I come to the intarwobs for something other than port and walnuts in the sitting room, and I don’t think I’m alone.
LikeLike
John, speaking as someone with experience in both #xkcd and #xkcd-port-walnuts, I can assure you that the atmosphere in the latter is neither prim nor academic, most of the time. I’m able to keep up a pretty constant stream of dick jokes with only minor repercussions, mostly xkcd’s hurt feelings and diminishing professional respect.
Tropes, cliches, “memes”, and other shared cultural touchpoints don’t develop from char-for-char repetition, anyway. It’s precisely that attitude towards jokes–Just Repeat Them, Corpsefucker–that makes jokey communities so hard to take. As you can see from the good parts of YTMND and 4chan and SA and anything else you care to name, the reason these jokes stay alive is that funny people make an effort to put a new twist on them every time they’re retold. And ROBOT9000 doesn’t ban you for puns.
Maybe it should, though.
LikeLike
I’ve found that splinter communities work well – extremely high signal to noise ratio. The problem becomes attracting new members when all they see initially is the high-noise public-facing community.
Otherwise, creating a splinter community actually give cohesion to both – essentially splitting each into groups that fall below dunbar’s number, which on the internet is much smaller than in real life (there are far fewer distinguishing characteristics when all you know of a person is the text they produce).
Oh and the meta discussion eventually dies down once the ‘Oooh Shiny’ wears off.
LikeLike
“But [peer-moderation] doesn?t work very well for live time-dependent things like IRC channels.”
If I gather correctly, Zed Shaw’s Utu project has this exact aim.
http://savingtheinternetwithhate.com/
LikeLike
I agree with John Armstrong. I’ve been at the root of a few running jokes in my time, most of which were not Little Britain-style “corpsefucks” because their very format encouraged experimentation. Either way, their humour derived from context-based char-for-char repetition (many of them were similar to That’s What She Said jokes) and banning that would kill the very things that I find help me bond with people online.
LikeLike
/ignore needs a username, e.g. “/ignore moderatorbot” to ignore messages from username “moderatorbot”
LikeLike
yeah
LikeLike
I like this. The possibilites of people actually having to spontaniously communicate with a thought-out response is great. Only thing else I can think of like this is this podcast where they let people sign up to be random strangers or provide questions and then the moderators call the random strangers and provide them with a random question.. Makes for a interesting on the spot response. Check it out.
http://dialastranger.com
LikeLike
Seems like the bot needs to make special cases for IRC commands (like /bye)
LikeLike
Anyone who is or was part of the Youtube community is aware of a similar occurence. The noise level is so high hence the community no longer appeals as it used to.
LikeLike
I have a question about the implementation. Is there a scanning function for TAKE OFF EVERY ZIG recycling a partial word or phrase? It seems to me as MY FRIEND ROBERT SAYS HE SAW A NINJA TOTALLY UPPERCUT SOME KID though you could embed a repeated word or phrase in some random noise OMGWTFBBQLOLPXPLZKTHXL8R or even parseable language. On the other PS3XBOXWIIRTEHSUXORZ hand, maybe that would actually increase the VELOCIRAPTORS signal-to-noise ratio.
LikeLike
Brilliant. A suggestion:
Add a karma calculation scaling factor that biases the muting function. The goal is to blunt (to some extent) the immediate application of chat mute delays for frequent chatters that are never muted much, if at all. The Karma scaling factor could be a product of the chatter’s submission frequency * chatter’s learned probability of contributing an acceptable comment. This way, the most valuable contributors are cut some slack when they otherwise wouldn’t get it. If a chatter with high +karma starts making a lot of mutable comments, they start spending their karma quickly. All new chatters start with -0- karma. Excellent chatters earn their way toward a higher positive karma. Bad chatters move into negative karma quickly. Negative karma means the muting algorithm is more aggressive. High positive karma means less aggressive muting. And don’t let human’s rewared subjective karma points to other chatters – keep it machine oriented.
Like training dogs… 🙂
LikeLike
I love the new approach to this problem.
Also, thanks to you and Zig for publishing the source code. Many people will find that extremely useful.
LikeLike
Please apply it to forum here at XKCD, where people leave ridiculous comments like “that was funny” after every comic.
LikeLike
You’d get way less meta-discussion if you put the functionality in your IRC server rather than a bot: just don’t broadcast the identical messages, and have a bot message the individual informing them of their violation. That way everyone else in the chat room doesn’t have to watch the newbies learning.
Your current system depends on punishment & learning in order to increase signal, which is an interesting experiment, but the punishment might be too painful for most participants. Make the reward structure more like Quake and less like Counterstrike.
Of course, my suggestion might mean that people just retype their stale messages differently, while making them pause encourages them to rethink whether it should be said at all.
LikeLike
I managed to get muted by a operator before the bot. What does this mean?
LikeLike
Hey – park from LJ – uh – don’t 1337 5P33K and misspellings completely doom this effort? I can become the human spam-engineer of my own comments quite trivially:
lol walrus
lol eskimo
lol peanut winkle
|o|
y34h.
uniqueness++, content–
LikeLike
I was going to post something but I couldn’t think of anything that hadn’t been said. It looks like moderator is making me post more thoughtfully even outside #xkcd-signal.
LikeLike
wait, what about “hi” and “brb”? and other statements whose intent is to inform about a current state/event, not so much… provide entertainment or interesting information or something…
Or is “the current state” of one individual not important enough? 😛
LikeLike
All you need now are rules for hesitation and deviation.
[If you didn’t get that, go here: http://www.bbc.co.uk/radio4/comedy/justaminute.shtml and select “listen again” from the panel on the left. You won’t regret it.]
LikeLike
Interesting about ROBOT9000 is the way it handles weblinks.
You know… when news of another natural disaster, another american attack or whatnot hits the air links of the news will be all over irc. People will want others to know..
[08:21] JohnDoe: http://blag.xkcd.com/2008/01/14/robot9000-and-xkcd-signal-attacking-noise-in-chat/
[01:42] JaneDoe: http://blag.xkcd.com/2008/01/14/robot9000-and-xkcd-signal-attacking-noise-in-chat/
Jane would receive a -v for her efforts of trying to educate people.
While this will definately cut down on the crap links people post it will also hinder informative linking. Or am I missing something here?
LikeLike
Couple thoughts occur to me. First, a white-list of common short phrases (or maybe not so short) might be a good idea. This would allow for simple responses like “Yes”, “That’s correct”, “I agree”, etc. as well as not entirely negating common phrases that are often useful to new users such as “Please read the channel topic before commenting”. The point is, there are many phrases that either through their short (simple, KISS) nature are difficult to express without resorting to linguistic contortions that are really rather pointless.
Another nice feature would be the ability to vet a sentence with the bot before actually saying it. A simple implementation would be something like “/t check ” and it could reply whether that phrase had been used in the past. Of course this also opens up some room for abuse, but could also help prevent unnecessary mutings caused by ignorance.
Also as someone pointed out, this could destroy certain meme and running jokes that often become part of the “flavor” of a particular group (such as slashdots many memes like “you must be new here”). Admittedly some of these would tend to get old rather quickly, but at the same time having a common in-joke often helps with building a sense of community. Of course, given the white-list idea from early this could easily be gotten around by simply adding select phrases to the whitelist.
LikeLike
Just realized the commenting software ate part of my previous post. The bot check command was supposed to be “/t <botname> check <phrase>”.
(I hope this posts correctly this time)
LikeLike
I heard about a better mild form of punishment in discussion groups: The Disemvoweler. It removes all vowels from your message if it find the message offensive.
Isn’t better to disemvowel messages or users which are unoriginal than kick/ban users?
LikeLike
Interesting, but that just limits what people can say. Your real problem are the people in the channel. When I used to idle there I just saw constant flaming and constant crap-talking aswell as all the childish speak. Those same people will still be there.
The only difference is they will start to say the same old rubbish, but using different sentences.
LikeLike
@Someguy:
In a static context such as a forum that might be possible, particularly when combined with some sort of preference system that can allow you to never even see the disemvoweled posts in the first place, but it’s much less effective for a real-time system. Essentially the issue is one of eliminating noise, and all the disemvoweler does is to increase the noise level in an already noisy post.
The approach suggested here on the other hand is attempting to create a negative feedback system for generating noise. That is, it attempts to make is undesirable to generate noise, rather then attempting to remove or filter the noise in the first place. This is arguably a better approach as concerns the community because it attempts to shape the behavior of the members into a more positive direction. It of course does nothing for those out to game or otherwise disrupt the system, which are the ones the disemvoweler is meant to deal with. In the context of IRC however there is already something more effective than the disemvoweler for dealing with people attempting to disrupt the community which is the voice and ban systems.
LikeLike
Can you design a bot that detects tautologies and applies suitable punishment, as well?
LikeLike