Many years ago, I signed up for Bloglines. It’s a service which aggregates the feeds from various blogging sites, so you can read them in one place without having to do the rounds of your favourite sites looking for updates. (On LiveJournal, your friends page serves the same function, and you can add the feeds of external sites if you’re a paying customer).
I left Bloglines for Google Reader when Bloglines became unreliable. Google Reader is nice: it looks clean, and there’s an app for it for my Android phone. I recommend it over LiveJournal, which is dying of spam; and Bloglines, for the reasons I’ll now get into.
A while back, Bloglines was taken over by a company called MerchantCircle. They sent me an email to say they were the new owners, which is fair enough. As far as I remember, I hadn’t logged into Bloglines since I moved to the superior Google Reader service, so I just ignored it.
Yesterday I got an unsolicited bulk email (spam) from MerchantCircle advertising a service not related to Bloglines. Worse, the link they offered to unsubscribe from their mailing list didn’t work, as it required a login and password (first mistake: removal links from mailing lists should authenticate the user sufficiently to get off the list). Worse still, giving the email address to which MerchantCircle sent spam to the “forgot password” box gave an error saying that the address was not known: MerchchantCircle don’t even know who they’re spamming. Logging back into Bloglines doesn’t give an “unsubscribe” option either.
I consider Bloglines/MerchantCircle to have gone rogue. I’ve removed the “subscribe with Bloglines” buttons from my blog, and advise anyone else who still has those buttons to do the same. Use Google Reader instead: Google don’t spam.
Edited to add: MerchantCircle have emailed back to apologise, saying they had a “weird glitch” in their email system which caused some Bloglines users to get MerchantCircle emails. In recognition of this, I’m downgrading them from “rogue” to “incompetent”.
LiveJournal coughs to their crimes, sort of
So, LiveJournal finally sort of owned up to getting blacklisted for helping spammers, as mentioned previously. This posting is their response to the situation. They say they’re doing the right things, although you do have to wonder what took them so long.
They didn’t name Spamhaus or properly explain why they’d been blacklisted, so I explained in the comments.
The spice must flow
Notifications are coming through now because LJ have changed the IP address of their outgoing mail server from 184.108.40.206 (the address of www.livejournal.com) to 220.127.116.11 (which calls itself mail.livejournal.net, but isn’t accepting inbound mail). The blacklisting for the old address is still in place. The spammy journals specifically mentioned in the SBL listing seem to have been suspended, though.
It’s not clear if this change of IP address is part of some agreement between Spamhaus and LJ or whether LJ think they can avoid the blacklist and continue to ignore complaints. If it’s the latter, I’m fetching popcorn. It’s the work of a few keystrokes for Spamhaus to block LJ’s entire address range, and I vaguely recall they’ve been happy to do that in the past for people who’ve taken the piss.
(Disclaimer: I’m not Spamhaus, I just used to hang out on news.admin.net-abuse.email in the 1990s, when it was cool).
Postings in news have been a bit cagey about what’s going on with comment notification emails. They’ve mentioned that there’s a “third party” involved. It turns out that LiveJournal have got themselves blacklisted by the Spamhaus Block List for providing spam support services, in this case, hosting websites for spammers.
This is why comment notifications aren’t getting through: the SBL is a widely respected and widely used email blacklist. They’re not saying LJ are spammers or indeed sending spam email, they are saying that LJ aren’t taking down journals set up by spammers, so they’re effectively helping the spammers to spam. Most email spam directs the mark to a website, so providing those websites is a serious matter to Spamhaus.
This is worrying: it means LJ probably aren’t responding to complaints about hosting the spammers’ sites. I think Spamhaus would have tried sending email to abuse@lj, though possibly not under their own names, as you want to be sure that reports from ordinary users are handled correctly, same way as restaurant reviewers don’t book saying “I’m Jones from the Times“. The detailed information from Spamhaus lists a huge number of spammy journals, and at least a couple of them were still there when I tried them. This doesn’t bode well for LJ’s future, to my mind.
livredor brought this to my attention. There’s a thread on a news posting discussing the problem. azurelunatic (who is head of anti-spam for Dreamwidth) has more here, and I’ve commented on their posting.
I look up potential interviewees on Facebook (as well as Google, obviously). Unlike the proctors at Oxfrod, I don’t care whether you’ve been photographed covered in flour or shaving cream, as long as you look like someone who’s smart, and gets things done.
livredor recently posted an entry in which she talks about online privacy, linking to Charlie Stross’s essay on the subject. I think Stross has this article on teenagers and online privacy in mind when he talks of a generation growing up with the idea that you have no privacy online and it doesn’t matter anyway. livredor is coming to the conclusion (which I share, see my replies in the comments) that she “should just make everything open and take care never to post anything that I could be ashamed or embarrassed about”.
As the comments on her posting point out, the problem is working out what you could be embarrassed about. The problems mentioned in the Times article are partly the result of a generation gap between people who aren’t surprised that some of their peers have put their lives online, warts and all, and the staid elders who are shocked to learn stuff that proctors, employers and parents didn’t previously find out about. I suspect that absence of evidence of shaving cream was never really evidence of absence, but it’s going to take a while for the elders to work that out. It seems sensible for the younger people to be a little circumspect in the meantime, so it’s not surprising that many existing Facebook users are tightening up their privacy options. Relying on privacy settings is another risk, because you’re trusting your e-friends and the site you’re using, but at least you’re keeping your embarrassing university antics out of sight of indexers and archivers, and you’re not assuming that the elders cannot join the site you’re using.
livredor also mentioned the possible problems which might be caused by people migrating away from email to the messaging systems offered by sites like Facebook. Gervase Markham has some thoughts on the subject. Conventional email is a lot less slick than, say, Facebook’s internal messages, and faces a greater spam problem, in part because email is distributed but Facebook has centralised control. These proprietary systems have their downsides too, of course: balkanisation, and a single point of failure when Facebook gets shut down by a law suit.
I think there’s some mileage in building an email system which is a bit more like Facebook’s walled garden. When I say spam in its current form is a solved problem, what I mean is that you can solve it by only accepting messages from well-behaved parts of the Internet. What I mean by well-behaved is stuff like not being in space given to cable modems and the like (Spamhaus PBL, checks on the presence of reverse DNS and that the hostname does not contain some variant of the IP address), not being a known baddie (Spamhaus SBL and XBL or your own email providers local list of scumbags), and not sending bulk email except by prior arrangement (DCC with whitelisting for mailing lists).
Alas, not all badly-behaved emailers are spammers, some of them are just managed by incompetents. Sometimes these incompetents work for large companies who aren’t going to change, so you have to start making holes in your garden wall to keep your users happy. However, an inbound email gateway for a hugely popular site like Facebook could enforce these restrictions by fiat without losing anything, since their users are using the internal system to send each other messages anyway, so anything else is a bonus (you could also make a nice interface for whitelisting legitimate bulk senders by requiring them to produce a Facebook application, say). If Facebook does take over the world, it needn’t mean the death of email. It might just bring the incompetents into line, we can but hope.
For a while now, I’ve been getting comments on my LiveJournal which apparently aren’t spam, but rather are questions which are totally out of context. For instance, I got one the other day which said “Hi. I find forum about work and travel. Where can I to see it?”
I recently got some more comment spam advertising something called XRumer, a clever and nasty program for spamming bulletin boards and other forums (like LJ), which is brought to us by some evil Russians (“No Meester Bond, I expect you to die”). One of the things the authors claim it can do is a crude form of astroturfing. They say you can configure it to post a comment asking about something, and response apparently from another user mentioning the site you actually want to advertise. It looks like this feature doesn’t quite work, and that the questions I’ve been seeing are examples of it misfiring. Mystery solved.
The spammers seem to favour certain entries of mine, so I’m screening anonymous comments on those entries (and on this one too, since I imagine it might attract undesirables). I don’t want to do that for my entire journal, as I get comments from people who aren’t on LJ but who say worthwhile things. In an ideal world, the way round this would be OpenID, but that’s not in widespread use yet, possibly because people who have an OpenID often don’t know they do. [Attention LJ users: you have an OpenID. Congrats. You’ve got a Jabber instant messaging account, too. See how good bradfitz is to you?]
A system which allows easy communication between two people who have no previous connection to each other is susceptible to spam. The trick is to keep this desirable feature while not being buried in junk (you could go the other way and remove this feature, of course, as many some IM users have, or make a virtue of it with social networking sites, but that’s not really an option for public blogs). Anything an ordinary user might to do create an identity, a spammer can do too, so cryptographic certificates aren’t a magical solution. Legislation doesn’t help, because the police don’t care and anyhow, spammers are in Wild West states like China or Russia, or at least run front operations there.
Most spam is still sent via email. Email spammers have been subject to an evolutionary arms race. The remaining effective spammers are bright and totally amoral. They’ll hijack millions of other people’s computers to send their spam or even to host the website they’re advertising, making it hard for blacklists to keep up (and they’ll use these computers to flood centralised blacklist sites with traffic in an attempt to knock them off the net). They’ll vary the text they use, to defeat schemes which detect the same posting lots of times. They’ll use images rather than text, or simply links to those images, to defeat textual analysis. You can bet that blog spammers will learn from this (some of them are probably email spammers too).
What’s working for email spam, and will similar ideas work for blog spam?
- Banning mail sent directly from consumer ISP connections is the single most effective thing I do (you can do this with the Spamhaus PBL and with a few checks for generic rDNS to catch what the PBL misses). You can’t do that with blog comments, as spam or not, they almost all come from consumer ISP connections.
- Banning mail sent from IPs which are known sources of spam is also effective. You can do that with blog comments, but you either need to be big enough to generate your own list (as LJ might be) or have the resources to run a centralised list like Spamhaus (which will be attacked by spammers). There are currently no IP blacklists devoted to blog spamming, as far as I know, although some spam comments I’ve seen came from IPs which were in the Spamhaus XBL.
- Filtering on ways in which spamming programs differ from legitimate SMTP clients (greylisting, greet pause) is currently effective, but only as long as these methods don’t become so widespread that it’s worth the spammers’ while to look more like a legitimate sender. Still, this doesn’t seem that likely. Incompetent admins aren’t in short supply, and I don’t have to outrun the bear, only outrun them. This sounds promising against blog spammers. Apparently simple minded schemes are pretty effective.
What else can we do with a website that we can’t do on email?
- CAPTCHAs are popular, but a bit of a bugger if you’re blind. The evil Russians claim to have defeated most of the deployed ones which use obscured letters, though that still leaves the “click on the picture of a cat” variant.
- Proof-of-work or hashcash schemes are currently very effective, suggesting that blog spammers don’t yet have the huge amounts of stolen computing resources available to email spammers, or that they don’t have the knowledge to implement the hashcash algorithm in their spamming software. By using proof-of-work, we can at least drive the weak blog spammers to the wall.
Background: news.admin.net-abuse.sightings is a newsgroup for posting copies of spam, so that the domains and servers involved become public record. Gradwell, run by the eponymous Peter, currently host noctua.org.uk. Peter Gradwell objected to my posting copies of spam to the newsgroup because his machines appear in the headers of all my email. <lj-cut text=”Now read on…”>
Date: Tue, 28 Sep 2004 00:11:16 +0100 From: Paul Wright <-$P-Wemail@example.com> Cc: Peter Gradwell <firstname.lastname@example.org> Newsgroups: news.admin.net-abuse.email, uk.net Subject: Gradwell.net policy on .sightings postings (was Re: spam reports in usenet) On Mon, 27 Sep 2004, Peter Gradwell wrote: > You have reported a spam in news.admin.net-abuse.sightings. In fact, I have reported several hundred, I should think, if not more. I've not counted then all. > Unfortunately, when customers do this it tends to backfire horribly as > the mail is seen to pass through our mail forwarding system and we > then get accused of spamming. Accused by whom? news.admin.net-abuse.sightings is for posting copies of spam with full headers. I tend to obscure my own email addresses so that it won't be picked up, but many other posters don't. > It takes a lot of time and resources to persuade our server colocation > providers that we are not spamming. Usually we are only able to enter > into these discussions after we have had our servers unplugged. The spam I have posted clearly does not originate at your servers, as the headers show it mostly comes via pobox.com's forwarding service, and originates at open proxy servers (usually in the Far East). If your colocation providers are really so clueless, I would be concerned for the reliablity of your internet services, in any case. > I must therefore insist that you immediately > > - post a followup message to your posting pointing out that > we have nothing to do with the origination of this spam and > that you did not intend to cause us to be associated with it. For the avoidance of doubt, I make it clear that Gradwell's machines did not originate any of the spam which I have posted to news.admin.net-abuse.sightings. I shall post a copy of this email to both news.admin.net-abuse.email and uk.net. I hope that will suffice. > - cease posting spam reports of this nature that include our > mail servers in the headers. I shall certainly do so. > If we receive another spam report originating from yourself that > detriments our good standing in the community we will be forced to > terminate your account without further notice. Coo. I believe your policy is misguided but be assured I shall abide by it for as long as you continue to host my domain. -- Paul Wright | http://pobox.com/~pw201 | http://blog.noctua.org.uk/ Reply address is valid but discards anything which isn't plain text
Now, I would have been OK had he not made that threat at the very end. We could have come to an arrangement, or discussed the problem. As it is, come renewal time, I’m gone. I hear Black Cat Networks are cheaper. 😉
Update: of course, I’m still there as I’m far too busy to work out how to transfer the domain. Still not particularly impressed, OTOH, they’re technically competant and have a nice interface.
There is a confusing multitude of spam filters out there. I once wrote an article listing all the ways of filtering spam I could think of. If you’re confused by all this, here’s what I do, along with ways of doing the same thing on both Unix and Windows systems.
<lj-cut> My first line of defence is a bunch of blacklists. These don’t work on the From address of the spam, which is usually forged, but rather on the IP address of the machine sending the email. There are a multitude of blacklists available, too. They differ in their listing criteria from narrow listings of machines which have sent spam, to broad listings of entire networks, intended to help you boycott ISPs which support spam. Getting legitimate email is more important to me than filtering all the spam, so I choose narrowly focussed blacklists. I use:
- The Spamhaus Blocklist, a manually edited list of the worst corners of the Internet. These days, spammers tend to host their websites in these places and exploit other people’s machines to actually send their spam. Which is why I also use…
- The Spamhaus Exploits Blocklist, an automatically compiled list of machines which have been taken over by spammers, probably without their owners’ knowledge. Windows users with cable modems, usually.
- The Open Relay Database, another list of machines which are exploitable in a different way (mostly not a way which is used by spammers these days, but it occasionally catches something).
If you want to filter your email using these blacklists, and you’re on Windows, you could try Spampal. It is completely free and very stable. It will work for you if you collect your mail using something like Thunderbird or Outlook Express (but don’t use OE unless you want to become one of the aforementioned exploited Windows owners). It works by sitting between your mail server and your mail program and marking suspect mail as it goes by. You then configure a filtering rule in your mail program to move the suspect mail into a separate folder. If you pare down the blacklists Spampal uses to just those listed above, it shouldn’t slow your mail downloads too much.
If you’re on Unix and you run your own mail server, receiving mail directly from the Internet, that server will probably have support for using these blacklists. If you pull mail from elsewhere, using fetchmail, say, so that your mail server doesn’t see the IP address of the machine which originated the mail, there’s a little Perl script called rblfilter which will help. It doesn’t seem to be maintained anymore, so I’ve put a copy here. You’ll need to work out how to tie it into your email system and edit the script according to the instructions in the comments.
The next line of defence is the Distributed Checksum Clearinghouse. The DCC works by sharing information about how many other copies of a particular email are floating around the Internet. If there are a lot of copies, it’s either something like a mailing list, or it’s spam. To use the DCC, you tell it where you expect to get legitimate bulk email from. Everything else you get which is bulk is therefore spam. The DCC is designed for Unix, so the web pages and Google will tell you how to get it set up there. There is a plugin for Spampal which will also let Windows people use the DCC. It’s beta software, that is, released to the public for testing, so it may contain some bugs: I’ve no idea how stable it is (despite getting a credit on that page, I didn’t actually write it).
If someone else manages your email for you, and you read it via a web interface, for example, then you should have a look a the spam filtering options you have available. I’ve just noticed that Pobox.com, who provide a forwarding address for me, now let people configure their service to reject mail based on those blacklists.
Fight the pink menace!
Kevin S. Wilson writes in NANAE:
You just don’t get it, do you? WE ARE PISSED, VENGEFUL, AND UNSYMPATHETIC. You helped to create the mess that e-mail has become, invading the privacy of millions of people and generally making an annoyance of yourself on a GLOBAL scale. Ultimately, you may have helped to render e-mail unuseable. You think anyone cares that you can’t find hosting for a vanity domain? Instead of looking for sympathy here, you ought to be thanking your lucky stars that someone sick of your spam hasn’t hunted you down and broken your arms, or worse.
I’m sure we all feel that way some days. (If ASR is the scary devil monastery, what does that make NANAE, I wonder?)
In other good news, Microsoft, AOL, Earthlink and Yahoo are going after some of the most prolific spammers. A quick look at the example emails in the lawsuit documents shows that many of the obvious suspects are in the frame. They’re filed against “John Doe” (the US legal equivalent of “John Smith”) as this allows the plaintiffs to get ISPs and other organisations to disclose the identities of the people behind the spam, but the targets here are well chosen, so I think the plaintiffs know who they expect to end up bankrupting. The mills of justice grind slowly, but we may hope they grind exceeding small.
There is a conspiracy theory which says this is just large commercial interests getting the porn’n’pills people out of the way, leaving the field clear for mainsleaze. Even if this is the plan of people like Microsoft, there are strategies in place for dealing with spamming from mainstream companies. Such companies can’t afford to use the deceptive and criminal tactics of the worst spammers, so blacklists and bulk email detectors like the DCC should see them off.
When you get your shiny new cable modem, you usually configure your mail program to send email via your ISP’s server at smtp.ntlhellworld.com (or whatever). smtp.ntlhellworld.com then sends on your mail to the destination at its leisure (or in NTL‘s case, doesn’t). There was no particular reason why a clever enough computer couldn’t just connect to the destination directly, especially if it’s a computer which is left on most of the time, so that if the destination is down or busy, it can try again later. This is what my computer did. But now lots of servers are blocking mail sent by my machine. This is because of spam.
Know, O King, that the modern porn’n’pills spammer uses open proxies to send email advertising his website. His website is hosted in China or Brazil (Spammy himself is actually a resident of Florida, and the mail originates from his machines in China, but the trail goes cold at the proxy, so it’s hard to prove this). Most of these open proxies are on machines connected to cable modems. Sometimes the proxy has been installed without the owner’s knowledge, perhaps by one of these “virus” things you Outlook users are so keen on. Sometimes, the owner installed the proxy themselves to share their cable connection with a local network, but misconfigured it. Misconfiguration is easy when your chosen software is insecure by design. Marc Thompson, author of the AnalogX proxy, must surely be a prime candidate for first trials of makali and jwz‘s famed audio-cock technology.
But, anyway, the solution adopted by some servers is to block any cable modem (or technically, any machine with a dynamic IP address) from sending them mail directly. That’s why my mail bounces: my IP address is on a list of dynamically allocated IPs. I can advocate that the admins use the Spamhaus XBL instead, since that only lists the addresses of insecure machines. But then someone will point out that my address is right next door to someone who is compromised, and, being a dynamic address space, I could get that address tomorrow.
So, I’m going to start using Gradwell‘s machines to relay my mail (they’ll let me do this as they also host my domain and incoming mail). They’re a lot more clued up than NTL, so their relay machine will probably be up most of the time and will probably ensure my email reaches its destination. But still, it’s a shame. It takes that little bit of control away, as I can only tell when something has left here, not when it’s been finally received. And it breaks something that wouldn’t need to be broken, were it not for those pesky spammers.