Comment Spam and the IMD Blog

Sorry for the recent trouble commenting on the weblog! Let me explain what happened -

We use Movable Type to manage the student/class/department weblogs. It's great software because it let's us give dozens of people access to edit and publish dozens of weblogs. And, for each entry on each of those weblogs, people have the ability to leave comments.

Movable Type versions before 3 were based on a trusted model of web annotation - anyone could type in a URL, email address and name, and leave their remarks on one of the articles. This brought a fascinating range of commentary to humble little weblogs. A number of folks saw an opportunity here, and they began running scripts to post unsolicited commercial advertisements on weblogs. By adding links to their sites selling Viagra, Cialis, Teen Sex, Texas Holdem, Diet Pills, whatever, they were driving up their pageranks in Google. (Pfizer, for one says they're not happy about Viagra spam). The more incoming links to a page, the higher it comes up in a search result. (Some folks think Google could fix that).

So by going to a wide range of weblogs, and posting links to their sites, these spammers were hoping that their free-rx-canada-pharmacy-400 site would come up on top. I imagine summer camps in Eastern Europe where hundreds of kids are being trained to write scripts, fill out comment forms and infiltrate the free and open web with make-money-fast-from-home schemes. Maybe they're giving these people a direct share of the profits? Either way, sites like this one, with hundreds of pages on dozens of blogs extending over years into the archives became huge targets. We were a giant sponge, soaking up all this muddy crud.

Like most folks running old installations of Movable Type, we installed MT-Blacklist, a program that allowed us to catch and filter out words like "viagra" and "erection" and "sluts." But each week there seems to be a new name for another erectile disfunction drug. And the spammers perform creative domain registration: block a URL like "how-to-play-hold-em-poker-4-you.info" and two days later they're using "free-texas-hold-em-poker-best-winning-chances.info." With a blog like this one, touching on gameplay, we can't exactly blacklist the word "poker."

So even with MT-Blacklist we were totally inundated. Dozens of spams a day, across these various weblogs. Scott asked me to look into stemming that tide. He suggested turning off comments after a certain time period - so that one month, or two months later, a script would turn comments off. This is a good idea when you have thousands of posts that might not be touched on. But with Google being the English-speaking web-world brain, you want to keep comments turned on for at least a few months, because it takes a little while for the page to be indexed and distributed out to other searchers online. When is a topic dead?

Rather than trying to pin down the right ratio of time between spam and insightful comments, and artificially limiting the period of discourse, I proposed that we move to an authentication system. SixApart, the company that makes Movable Type, realized they had a massive problem on their hands with hard-working spammers. They created a user registration system for comments: TypeKey. TypeKey is a free user authentic registration system for Movable Type weblogs; it uses the same login information as TypePad, the paid weblog hosting service (which is unfortunately not the same login students and faculty have here on the IMD site). TypeKey does allow you to login once, and keep that name and password saved on your computer so you can comment on any site using that kind of authentication.

Besides helping out on the IMD blog, I perform tech support for Game Girl Advance, a site with thousands of comments on hundreds of entries. There was an unstoppable flow of spam there too, and like this site, we turned on MT-Blacklist and comment approval. But what a pain in the butt! Every day I had to spend twenty minutes or a half an hour weeding out fifty spam comments to approve a single useful comment. Humph - I could have spent that time actually writing posts.

So I moved Game Girl Advance to comment authorization through TypeKey. Now, whenever there's a new comment, it was written by someone with an identity, something of a virtual stake in their presentation of self and their chosen remarks. I wonder how long it will take the spammers to crack TypeKey? And I wonder what is lost by preventing anonymous insights.

My hope for the IMD weblog is to see the maintenance time go down. I don't want a dozen people each spending 10-30 minutes a day chasing flies. When people have a new comment on their site, I want that to be a source of joy, because people expect it will be informed, invested commentary. It's definitely an experiment - the first TypeKey setup here was slightly broken (I mis-pasted a TypeKey code into the configuration). It required some upgrades to the templates here. Now it seems to be up and running - I hope it works for the group.