I’ve updated the little script I wrote to keep track of which comments are new on LJ and Dreamwidth (LJ now does this automatically in its default style, DW doesn’t, by the looks of it). Thanks to various people for telling me it was broken for HTTPS sites, which LJ and DW both default to these days.
Userscripts.org is long dead, so I’m now hosting it on my site.
I’ve updated the little script I wrote to keep track of which comments are new on LJ and Dreamwidth (LJ now does this automatically in its default style, DW doesn’t, by the looks of it). Thanks to sally_maria for alerting me to both the problem and the solution.
Userscripts.org is long dead, so I’m now hosting it on my site.
I’m mostly writing this down so I remember it, but maybe it’ll also come in useful to other people. This is how I converted from LJ to WordPress.
Getting data out of LJ
WordPress’s LiveJournal importer is buggy and doesn’t do a bunch of stuff I want (such as re-writing links to my own posts so that they now point to the new blog). Luckily, jwz has been here before, and wrote a Perl script to download a journal and output WordPress’s XML import/export format. This does better, but needed a bit of hacking to suit my obsessive need to avoid information loss in the transfer to WP. I’ve stuck my own version here: the comment at the top describes what I changed. You’ll need LJ::GetCookieSession. Like all Perl scripts, this one is configured by global variables near the top, so you’ll need to change those too. You then say
ljgrabber.pl -v --wordpress --comments > wordpress.xml
and then upload wordpress.xml to the WordPress importer (Tools, Import on the WordPress dashboard).
Note that I haven’t used any of the other advertised options (to re-write bits of your LJ so they point to the new blog) in my modified version of the script, so damned if I know whether they work, crash, or delete your journal. Probably best to try it on a spare journal first, I’d’ve thought.
I ran through WordPress’s famous 5 minute install having stuck the untarred WP download in the right place on my site. Excitingly, this left wp-config.php (which has stuff like the database password in it) with both public read and public write permissions (assuming it was the installer and I wasn’t immediately pwned by something before anyone had seen the blog). So, you might want to watch for that.
Plugins you will want
- Akismet: WordPress blogs attract a lot of spam comments. Akismet kills them all. Possibly there’s something I can do about this to make my blog less obviously a WordPress one, but I haven’t worked out what they’re using to identify it yet.
- Avalicious will grab user pictures from LJ if your commenters specify a LiveJournal as their website URL. Since jwz’s Perl script produces such comments, installing this gets you the familiar looking icons for everyone. Note that you will want to apply jwz’s patch or it’ll kill your performance on pages containing comments from people who deleted their journals.
- Live Comment Preview: cos it’s handy.
- Subscribe to Comments: nearest thing I’ve found to LJ’s email functionality. I’m not sure whether it’s actually emailing you replies to your comment or just any new comments. Probably should check that.
- LiveJournal Crossposter: does what it says on the tin. Note that if you go back and edit imported posts, it seems to want to post them again (presumably because the imported posts don’t have whatever magic it uses to tell that they’re already posted to LJ), but for posts which it has cross-posted for you, it’s clever enough to apply subsequent edits back to LJ, too. Note that there’s a setting which controls whether it just posts excerpts or the whole entry. For now, I’ve set it to the whole thing, even if it does mean the Russian mafia are getting advertising revenue from my writing.
- Updraft Plus Backup/Restore: backs up the database and files to Google Drive, which I wasn’t using for anything else.
- WP Super Cache: Crimefighting Jesus told me to, and he runs the hosting company, so he should know.
I expect I’ll tart it up a bit at some point but the default theme seems reasonable enough for now. Any other top tips welcome, I guess.
So, I’ve been looking into ways of running a “proper” blog, and I’ve come down to PyBlosxom or WordPress. In either case, I’ll get my own hosting for it.
Advantages of PyBlosxom over WordPress:
- Keeps entries in text files. I fear databases.
- Seems to have a better security record than WordPress.
- In Python, so hackable and I’d feel I’d have some hope of understanding what it’s doing (WordPress is in PHP).
Advantages of WordPress over PyBlosxom:
- Very active developer community, so lots of nice plugins. (PyBlosxom isn’t abandoned but doesn’t have so many people working on it).
- More themes, some of which are pretty (PyBlosxom has a few themes in their repository, none of which are that pretty).
Anyone who’s used either of those care to comment?
A thing I found while investigating how to get journal backups going again in the wake of LJ’s most recent debacle:
A while back, geeks kept saying that LiveJournal should be Usenet news, that is, instead of mucking about with all the tedious web forum stuff, it’d be nice to have a program which let you read comments and entries, kept track of threading and which comments you’d already read, and so on (remembering what you’ve read on LJ was the motivation for my LJ New Comments script, but that doesn’t avoid LJ’s clunky interface).
This was tricky as there was no obvious way to get all the comments from an entry. There was the old comment export thing, but that only works on your own journal. You could “screen scrape” with a program that tried to pull the comments from the human-readable versions of LJ’s pages, but that’s considered rude because of the load it’d put on LJ’s server, and it’s fragile as it might break if LJ changes the human-readable output.
Luckily, LJ added a bunch of new stuff to its existing interface for “clients” (programs which access LJ, like Semagic). This includes the getcomments method, which allows you to get all the comments on any entry you can see.
Add this to the existing machine-readable stuff (Atom feeds, getfriendspage) and you could probably write either a client specific for LJ (the iPhone client is the reason LJ added the getcomments method, by the looks of it) or a proxy to turn the whole thing into NNTP and let you use conventional Usenet clients. Who’s first?
(Personally, I still plan to be off once I can actually back up this journal, including the comments of my esteemed readers. But I won’t stop reading, so this would be a nifty toy even for me.)
Edit: another thing this allows is third parties offering comment feeds of your journal: someone could write a thing which turned the comments from an LJ entry into an Atom feed. Real blogs have these, so LJ could too.
The latest code release onto LiveJournal has introduced a problem where people are randomly getting logged into the wrong journals. This exposes friends locked and filtered entries belonging to those journals to those random people.
There’s no indication that this used to read the locked entries of a specific, targeted user, but there’s no analysis of the problem available, so we don’t know that it can’t be, either. Edit: It looks like this was a problem with caching. If that’s true, it’s unlikely that it could have been used to read posts from a specific user. More here from cahwyguy.
More information is available here.
This has been going on since at least yesterday morning,
yet LJ still hasn’t responded officially to reports of the problem or warned users that their private data is at risk. Edit: LJ has posted about the problem, however, they don’t seem to have some details right. For instance, they’re claiming it was only a problem for a few minutes, when people were noticing it all day on Thursday.
This is the second time that LJ has dealt with a major security incident with staggering incompetence. It illustrates that they apparently don’t have a test server, i.e. they’re a bunch of coyboys. My vague plans to move this blog just got a lot less vague.
LiveJournal coughs to their crimes, sort of
So, LiveJournal finally sort of owned up to getting blacklisted for helping spammers, as mentioned previously. This posting is their response to the situation. They say they’re doing the right things, although you do have to wonder what took them so long.
They didn’t name Spamhaus or properly explain why they’d been blacklisted, so I explained in the comments.
The spice must flow
Notifications are coming through now because LJ have changed the IP address of their outgoing mail server from 18.104.22.168 (the address of www.livejournal.com) to 22.214.171.124 (which calls itself mail.livejournal.net, but isn’t accepting inbound mail). The blacklisting for the old address is still in place. The spammy journals specifically mentioned in the SBL listing seem to have been suspended, though.
It’s not clear if this change of IP address is part of some agreement between Spamhaus and LJ or whether LJ think they can avoid the blacklist and continue to ignore complaints. If it’s the latter, I’m fetching popcorn. It’s the work of a few keystrokes for Spamhaus to block LJ’s entire address range, and I vaguely recall they’ve been happy to do that in the past for people who’ve taken the piss.
(Disclaimer: I’m not Spamhaus, I just used to hang out on news.admin.net-abuse.email in the 1990s, when it was cool).
Postings in news have been a bit cagey about what’s going on with comment notification emails. They’ve mentioned that there’s a “third party” involved. It turns out that LiveJournal have got themselves blacklisted by the Spamhaus Block List for providing spam support services, in this case, hosting websites for spammers.
This is why comment notifications aren’t getting through: the SBL is a widely respected and widely used email blacklist. They’re not saying LJ are spammers or indeed sending spam email, they are saying that LJ aren’t taking down journals set up by spammers, so they’re effectively helping the spammers to spam. Most email spam directs the mark to a website, so providing those websites is a serious matter to Spamhaus.
This is worrying: it means LJ probably aren’t responding to complaints about hosting the spammers’ sites. I think Spamhaus would have tried sending email to abuse@lj, though possibly not under their own names, as you want to be sure that reports from ordinary users are handled correctly, same way as restaurant reviewers don’t book saying “I’m Jones from the Times“. The detailed information from Spamhaus lists a huge number of spammy journals, and at least a couple of them were still there when I tried them. This doesn’t bode well for LJ’s future, to my mind.
livredor brought this to my attention. There’s a thread on a news posting discussing the problem. azurelunatic (who is head of anti-spam for Dreamwidth) has more here, and I’ve commented on their posting.
Here are what I think are my best posts of 2010:
- The bad arguments series: theist guff about worldviews (including the “atheism is a religion” line which seems to have cropped up a lot lately), atheists’ embarrassing errors about the Bible and gays or “Why President Bartlet was wrong”, and theist guff about science. I realised I promised more on science at the end of the last one, and some day I may get round to writing it.
- The Special Edition of Bishops Gone Wild, when the Lords Spiritual got their way with the Equality Bill.
- My post on Georges Rey’s Meta-atheism: Religious Avowal as Self-Deception drew together some previous thoughts on how much believers actually believe.
- Is the Creator good? drew together Hume, William Lane Craig and Ptolemy the Gnostic to show that, even if there are such things as personal Creators, we don’t seem to have much reason to call them good. No-one commented on the article, but I assume that was because my argument was so devastating as to admit no rebuttal.
- Bigots, Brown, Justice, Laws is mostly about the ruling on Gary McFarlane, a relationship counsellor who was sacked by Relate for refusing to give therapy to homosexual couples.
- Q: When is a person like a rock? A: When there’s no God got the most comments this year, although all of them were either by me or Comrade. If you can’t be bothered to read the entire lot, I think my summary is a reasonable one.
- Reform and the Interminable Anglican Sex Kerfuffle, in which the Guardian‘s Andrew Brown discovers complementarianism. Contains a picture of me dressed as a bishop, if that’s your thing.
- A great cloud of witlessness: the Not Ashamed campaign criticises people who think they’re living in a Frank Peretti novel.
As I hinted at in that last post of mine, this has been a difficult year for me, culminating in my ongoing divorce proceedings after my then wife unexpectedly told me that she considered that whole “forsaking all others” thing as less of a solemn vow and more of a guideline. I’ve taken a long look at my priorities as a result, and resolved to spent less time arguing with idiots on the Internet (so, if you see me back on the Premier Christian Radio forums, remind me of my resolution) and more time going out dancing. There’ll probably be fewer posts of substance from me in 2011; however, I’m perfectly happy to argue with sane and sensible people, and I doubt I’ll be able to resist that urge entirely.
Thanks to my friends and family for all their support, and to the strangers who wrote to ask where I’d gone during the hiatus in my postings. May 2011 be a better year for us all.