A small page about spam

Spam is the same thing, lots of times

Introduction

I seem to have got involved in writing spam filters and the like, as well as articles about spam and filtering. All this got too big for my main web page, so here's another one, totally dedicated to spam.

Infamy

Kuro5hin's kind users voted my spam filtering article onto their front page. It's a comparison of filtering methods with some links to free software to filter spam, both for Unix and Windows.

Filters and the like

I wrote a brief HOWTO on filtering using the DCC and Exim on Linux. At some point I'll update it to cover my how I do it for mail I pull down from Gradwell (who host noctua.org.uk).
For you lovely Windows users, James Farmer's SpamPal is a filtering program which can check mail against various blacklists. While blacklists have the problem that they will sometimes classify legitimate email as spam, with Spampal's auto-whitelisting feature, it should be OK for personal use. As some blacklists will list for spam support (i.e. hosting spammers' websites), I've written a plugin for SpamPal which will filter on blacklisted websites mentioned the message body.

General anti-spam things

I hang out on news.admin.net-abuse.email in the hope that someone will say something interesting. My main interest there is in filtering and blocking technologies, although winding up the odd spammer passes the time, too.
A while back, I was interested in finding a fast algorithm for finding similar Nilsimsa codes in a database. Nilsimsa codes are message digest codes which are the opposite of hash functions: they're designed to change a little when the message changes a little. If you ran an email system which kept track of how many times it had seen a particular code, you'd have a means of detecting bulk email. This is not a new idea: the DCC does this, but it uses conventional hash functions and does ad hoc stuff to make them fuzzy (that's not to disparage the DCC though, as the ad hoc stuff works).
After doing some timing measurements, I believe using Nilsimsa in a bulk mail detection scheme would be practical on a machine with lots of memory (about half a gigabyte should do it).
The problem with this idea is that Nilsimsa's statistical properties when fed English text (or text in any other language, for that matter) aren't the Gaussian approximation to a binomial which you'd really like them to be able to work out useful measurements of how two texts differ. This effectively kills it as a fuzzy checksum, alas. This information is left here purely for historical reasons: I'm no longer looking at Nilsimsa myself.

Back to the index.