livejournal

Mindful of the recent LJ drama about breastfeeding icons, I propose we take LiveJournal’s codebase (which is open source) and start a site where it is compulsory to display breasts in your default icon. That’ll show the Patriarchy! Start using those tools to back up your livejournal now, because titsorleave.com will be live soon.

Another controversial LJ Abuse policy which people are complaining about is that LJ don’t care about comments made in journals which are actually RSS feeds of blogs off-site, because no-one owns the journal. There’s one LJ user whose life’s purpose appears to be to post the word “BOOBIES!!” as a comment to every single entry on popular feeds. apod, a feed of NASA’s Astronomy Picture of the Day, also has the_duke, who always posts “I’ve been there” in response to all the pretty pictures of galaxies and suchlike. The other day, he got the best reply evar.

It would be possible to adapt the web 2.0 technologyTM used by my LJ New Comments script so that you could killfile comments from particular users, I suppose, but I think there’s little motivation to do so while the trolls’ immaturity is less objectionable than the complainants’ huge sense of entitlement.

Just uploaded another version of the LJ New Comments script. This one never displays negative numbers of new comments (“-3 new”) next to your friends’ entries. I also noticed that Firefox was getting fat and slow after much LJ browsing. Turns out that Firefox leaks memory like a sieve when you do the thing that the script does to let you click from one new comment to the next. This is Firefox’s fault, but luckily a way to get around this was discussed on the Greasemonkey mailing list recently. So I did that. All seems well so far. FF1.5 is so leaky that it’ll probably get fat for some other reason, but with the amount of LJ I read I’ve probably helped myself a bit.

It occurs to me that the same sort of tricks LJ New Comments uses to find comments could probably also be used to make a version of the LJ Thread Unfolder which worked with more than just the default comment style. I might do that if I get around to it.

The recent change to LJ’s URL formats seems to be part of an attempt to defend against one or more attacks which allow the attacker to steal another LJ user’s credentials, gaining the ability to impersonate that user. The theft occurs when the victim visits a page on LiveJournal which contains some malicious Javascript inserted by the attacker (more technical details below for those that care).

What’s been happening?

Slashdot linked to an article with some more details on the attacks. This article includes details supplied by the Bantown group (who live at bantown.com, a site you probably want to visit using lynx). Bantown have use these attacks to pwn LiveJournal quite comprehensively: the comments on the news entry contained comments from tens of different users with the same demand from Bantown. It’s likely that these users all had their credentials stolen by Bantown.

I found a comment quoting an explanation of the vulnerability in an entry on lj_dev, but that entry has now been deleted. The quoted explanation is about a vulnerability which only applies to browsers based on Mozilla (so, Mozilla, Firefox and Netscape). The Bantowners claim that this is not the vulnerability they were using, as they have a vulnerability which affects all browsers. LJ recently patched a vulnerability which would do the job for all browsers, but it’s possible there are other, similar, vulnerabilities in LJ’s code. Or it’s possible that the Bantown people are lying.

Is it fixed?

LJ went down for a while on Friday afternoon, and seems to have invalidated all existing cookies. However, bradfitz is keeping quieter than I’d like about whether the risks still exist and about what workarounds users can use while LJ’s crack programmers are working on a fix. bradfitz‘s use of “soon” suggests that the URL change was part of further changes. These further changes aren’t in place as I write this, which I think means that it’s still possible to use whatever attack the Bantowners have been using to steal credentials, although it’s not possible for an attacker to use an old set of credentials from logins before this afternoon.

Edited: LJ has now fixed this, so it’s safe to turn Javascript on again.

What can we do about it?

For now, I’m running with No Script turned on, and using that to disable Javascript for all but trusted sites, of which LJ obviously isn’t one. LJ’s lack of communication about the risks to user data, and about possible workarounds, displays a worrying incompetence, as I’ve said elsewhere.

The Science Part

LJ uses cookies, small pieces of data stored by your web browser, as your credentials. When you log in to LJ, you get a cookie. From then on, your browser presents the cookie whenever it requests a page from LJ. LJ trusts you because you have the cookie, and lets you do things that only you should be able to do. The cookie can persist just until you close your browser, or longer if you’ve ticked the “remember me” option when you log in.

The attacks on LJ are cross-site scripting or XSS attacks. A Javascript running on a particular page can access the cookies for that page. Currently, any Javascript running on an LJ page can see your cookie, because the same cookie applies to the entire site. If an attacker can cause their own Javascript to run on a page supplied by LJ, they can steal that cookie and send it to a remote server that they own.

How might the attacker get their script onto LJ’s pages? Well, LJ lets you submit HTML as entries, comments, and as your own styles, and then displays it on its pages. LJ attempts to sanitise the HTML you supply it, but if it doesn’t do this correctly, the attacker has a way in. They can put their Javascript on the page, and visiting that page would then send your cookie to their server. Also, browsers based on Mozilla (such as Netscape and Firefox) allow stylesheet authors to embed Javascript in a CSS stylesheet, so the way LJ lets users reference their own external stylesheet is another security hole (although as I said above, it’s possibly not the one that the Bantown people are using).

There’s some more discussion of how this works (in amongst a lot of sarcasm) in this thread on jameth‘s journal.

The LJ New Comments script now copes better with the bewildering variety of journal styles that are out there. I also stopped it from giving up in disgust if a style allows it to see the comments but doesn’t provide a permanent link to each comment, as the “n” and “p” keys will still work in these styles (q.v. peacerose‘s journal, for example).

I’m now using scrollIntoView to move each new comment to the top as you click or press keys, so you don’t get a new history entry for each comment you visit (I was annoyed with having to hit the “Back” button multiple times to leave the entry). The docs for Greasemonkey allege that scrollIntoView doesn’t work within Greasemonkey unless you do special stuff, but I seem to be getting away with it. Possibly I’ve broken the script for people not using Firefox 1.5, but such people need to feel the white heat of technology, anyway.

Ph34r my sk1llz!

ETA: Except that I broke it again trying to make it handle all the extra ways of denoting comments. v0.4, now on the userscripts.org site, seems to be working.

I’ve finally got around to writing the Greasemonkey script which I’ve long been threatening.

What it does

The script remembers which comments you’ve seen on LJ (or Dreamwidth) and helps you navigate to new comments. That’s right, I’m finally dragging LiveJournal kicking and screaming into the 1980s.

If you’re on an entry page, pressing “n” skips you to the next new comment, and “p” skips to the previous one. If the style has an “Expand” link, moving to an unexpanded comment with these keys will also expand the thread. If the style has a permanent link or a reply link for each comment in that comment’s header or footer, the script inserts another link next to it, labelled “NEW”. That link shows you that the comment is new at a glance. Clicking the “NEW” link selects the comment so that pressing “n” will go to the next comment from there. On some styles, the currently selected comment will be outlined with a dotted line.

On a journal or friends page, the script will also add the number of new comments to the link text, so that, say, “15 comments” becomes “15 comments (10 new)”, and enable the “n” and “p” keys to move between entries which have new comments, and the “Enter” key to view the selected entry. This only works if you’re looking at a journal which adds “nc=N” to entry links to say there are N comments on an entry (LJ can do this as a trick to confuse your browser’s history function into thinking you’ve not visited that entry whenever there are new comments). If you want to turn this on for your journal then ensure you’re logged in, visit this page, check the box which says “Add &nc=xx to comment URLs” and hit the “Save” button.

How it works

You don’t need to understand this section to use the script. If you don’t care about programming, skip to the next part.

<lj-cut text=”Gory details”> LJ makes it a total pig to do this sort of thing: there’s so little uniformity in journal styles that getting a script like this to work for all of them is impossible. It’s fair enough that LJ allows people to customise their journal’s appearance, but there aren’t even standardised CSS class names for stuff. Not that I’m bitter. So, what the script does is look for anchor tags of the form <a name="tNNNN"> or elements with an id attribute of ljcmtNNNN or tNNNN. NNNN is the comment number, which seems to be unique for each comment on a given user’s journal. It then looks for the permanent link to that comment, which is usually to be found in the header of the comment (or footer, in my current style), and adds a “New” link after that. So, new comments are marked with a link to the next new comment.

The upshot of all this is that if you’re reading a journal with a style which doesn’t use either anchor tags or elements with the given id for all comments, the script won’t work correctly. If the style doesn’t provide each comment with a permanent link in the comment’s header, the comment won’t be marked with a “New” link. Such is life. Please don’t ask me for special case changes to make it work with LJ’s many horribly customised journals. Pick a sensible style of your own and learn to use “style=mine” instead. There’s even another Greasemonkey userscript which will help. On the other hand, if there’s a large class of the standard styles for which it doesn’t work, tell me and I’ll have a look at it.

Using it

If you want to use it, you will need:

  • Firefox, the web browser, version 1.5 or later.
  • Greasemonkey, the extension which lets people write little bits of Javascript to run on certain pages.
  • LJ New Comments, which is what I’ve imaginatively entitled my script. If the userscripts site is down again, you can find a copy on my site.
  • Your flask of weak lemon drink.

After you’ve installed all of the above, visit an entry on LJ and marvel at the “NEW” links on all the new comments (which will be all of them at this point, as the script wasn’t around previously to remember which ones you’d seen before). See above for operating instructions.

Privacy

Note that the script stores a Firefox preference key for each journal entry you visit, listing the IDs of the comments it finds there. The script doesn’t let the database grow without limit: when the script has seen 500 entries, it starts to drop the history for the entries you’ve not visited recently.

Clearing the browser’s history doesn’t affect the script’s list of visited entries. Thus your visits to polybdsmfurries will be recorded for posterity, even if you clear the browser’s history. You can wipe the entire history by using the “Manage User Scripts” entry on the Tools menu to delete the script and its associated preferences (you can re-install it afterwards, but you must clear out the preferences for it to delete the history).

The script does not record the contents of any entry or comment. The script does not transmit any information to LJ or any other website, it merely acts on what it sees when you request journal entries.

Your questions

I’ve given this entry as the homepage for the script on Userscripts.org. That means this entry is intended to serve as a repository for questions about the script, so if you’ve got a question, comment here. I prefer this to commenting on my other entries or to emailing me, unless you already know me. Ta.

To keep up to date with new releases of my greasemonkey scripts, track the tag “greasemonkey” on my journal. This link should enable you to subscribe to that tag and get notified when I post a new entry about greasemonkey scripts.

Revision history

2006-01-02, version 0.1: First version.

2006-01-03, version 0.2: Added the “p” key. Used javascript to move between comments so doing so does not pollute the browser’s history. Coped with the id=ljcmtNNNN way of marking comments. Made “n” and “p” keys work even in the absence of permalinks on each comment.

2006-01-04, version 0.3: Apparently you can have id=tNNNN, too.

2006-01-04, version 0.4: Broke 0.3, fixed it again. I hope.

2006-01-19, version 0.5: Updated to cope with LJ’s new URL formats. Changed how comments are stored internally so that the database does not grow without limit: the script now remembers comments for the last 500 entries you visited, and forgets the entries you’ve visited least. Also added “New” marker based on reply link as well as thread link, for styles which don’t have a thread link for every comment.

2006-01-19, version 0.6: Convert dashes I find in URLs to underscores internally, to preserve access to history from older versions of the script before LJ’s URL change.

2006-02-09, version 0.7: Work around the fact that Firefox leaks memory like a sieve. Never display negative number of new comments. Change licence to MIT as GPL is overkill for this script.

2006-02-09, version 0.8: There was a bug in the workaround code I got off the Greasemonkey mailing list. Fixed that.

2006-06-04, version 0.9: Enabled the “n” and “p” keys on the friends/journal view. Added the box around the current comment.

2007-02-20, version 1.0, baby: Try harder to draw a box around the current new comment. Applied legolas‘s fix for pressing CTRL at same time as the N or P keys (see comments).

2008-03-31, version 1.1: Make it work faster on entries with lots of comments. Altered behaviour of “NEW” link so it now selects the comment you’re clicking on, as that makes more sense.

2008-09-24, version 1.2: Support Russian keyboards thanks to mumi_0, make threads expand.

2009-01-27, version 1.3: Support for independentminds journals.

2009-05-04, version 1.4: Support for Dreamwidth.

2009-09-22, version 1.5: Amend support for Dreamwidth.

2010-08-09, version 1.6: Made syndicated journals work.

Finally got Mozex going on Firefox with Mac OS X. This means I can edit my comments on LiveJournal with Vim rather than messing about with LJ’s comment posting box and the less powerful editing facilities from my browser. I can also use Danny O’Brien’s marvellous Google linkification script. Which is nice. It’d be even nicer if Firefox’s process creation API worked properly on Mac OS X, though.

As a result on all this mucking about, I’ve not had time to respond to comments on the God Hates Hair entry. I’ll get around to it sooner or later, though.

Greasemonkey is an extension for the Firefox browser which lets you write little programs to change how websites appear. For example, ilishin has created a script which lets you expand collapsed LJ comment threads in place (that is, on the same page, rather than on a new one). It only seems to work with the standard comment layout at the moment, but I hope the author will fix that soon (if not, it doesn’t look so hard that I couldn’t do it myself).

I noticed that the later versions of Greasemonkey support a key/value database which persists when you shut down and restart your browser. This means that it’s probably possible to write something which remembers how many comments there are for an entry and will highlight items (on your Friends list, say) which have new comments. It might even be possible to highlight the new comments themselves, although it’s not clear how good the database is, so you’d want to avoid overloading it, I suppose. I was vaguely aware of Greasemonkey, but I don’t think I’d realised just how much it can do. Greasemonkey may be the thing which makes me switch from Safari to Firefox (it’s just a shame nobody has sorted out Mozex for the Mac, as that’d certainly clinch it for me, too: I miss being able to edit LJ comments in a proper text editor).

Think I’d better dance now.

Chiark is a Unix box on which a large number of Cambridge geeks have accounts (I’m not one of them, as it happens, but I know some of them by name and a few of them by sight). It runs some local newsgroups, which are only accessible to people with accounts. They’ve recently added a journals newsgroup, to which some people are publishing their LJs (it’s a one way street at the moment, by the sounds of it: entries and comments go from LJ to the newsgroup, but not vice-versa). This has caused some excitement on my friends and friends-of-friends lists. Of particular note are atriec‘s posting on what LJ’s are for, emperor‘s own views (I’m not sure why Chiark is “cabal” there, but it’s the same thing being discussed), and mobbsy‘s comparison of LJ and newsgroups. There are a couple of coupled problems here: LJ’s interface is not useful for having discussions (as opposed to simply pontificating) and some people don’t actually want to have discussions anyway.

LJ’s limitations do annoy me. As I said to livredor recently, I’m here for the people, not the interface. Compared to sites like Google or Flickr, LJ hasn’t done very well at making its stuff accessible by computer programs which might do useful things with it, such as re-presenting it in a way which is easier to to read, remembering what I’ve already seen and alerting me to new stuff, and so on. OK, so there’s RSS, but that’s no good for comments. OpenID is a step in the right direction, but largely solves the opposite problem, namely letting non-LJers put their stuff here. The client protocol is, again, designed to let people put stuff on LJ, not to take it out. LJ explicitly says that they don’t like screen scraping (that is, programs which extract information from the LJ pages which are designed to be read by humans) as lots of programs doing this will request lots of pages very rapidly and put more strain on their server more than they’d like.

LJ slowly getting better as a discussion forum, but the pace of change is slow. Tags are useful, OpenID is pretty cool, but on the whole LJ’s developers also seem to spend a lot of time on making it look pretty (a worthy goal, since newsgroups are pretty ugly by comparison, but probably not worth all that much time from the developers, who could just provide the users with the tools to do it themselves). That’s probably down to their target audience, I suppose: a few refreshes of the random journal link shows that LJ is largely populated by teenage girls (and by Russians, for some reason). See also the large number of people saying “actually, we want more user icons, not this OpenID thing” on the OpenID announcement.

There’s also the question of what a LiveJournal is for. livredor‘s posting on manners on LJ made the point that nobody is very sure what the etiquette is for making comments on other people’s postings. Having been brought up on newsgroups, I assume that anything I can see and which has comments enabled is fair game, although in deference to the fact that I’m entering someone’s personal space, I’ll usually introduce myself before diving in. But I suppose I could still end up horribly offending someone. It’s possible that most LJ users don’t want to have long discussions on their journals, in which case LJ would be wasting their time by making that easier, and I should just find somewhere better suited to that, which supports OpenID.

What would be the ideal, for me? The distribution system of Usenet (the network of servers which provides access to the public newsgroups) means that you can’t really recall postings once you’ve made them, and also makes it hard to make the equivalent of friends-only postings (you could do it, but it’d be hard to conceal the fact that you’d at least made a posting that someone else couldn’t see). So, I don’t object to LiveJournal’s centralisation in itself, because it helps me keep control (and now OpenID means I can entrust non-LJ people with my friends-only stuff, if I want). On the other hand, the interface sucks when you want to follow a discussion.

I’d like to see more machine readable stuff (especially comments) and a better API for clients to use to pull out comments and so on. I suppose I’d really like to see LJ run an NNTP (newsgroup) server which wouldn’t distribute stuff, but which would allow the same restricted amount of HTML that LJ itself does. A journal would be a group, an article would start a new thread, and the comments would be followups. Stuff that you weren’t meant to see just wouldn’t show up in the group, because you’d need to log in to the server to see it. I like this idea, although I can’t really see LJ implementing it. Maybe we should start a meme to campaign for it? We could call ourselves the Campaign for Real News.

After much prodding, LiveJournal has finally introduced tags, a way of categorising entries and of retrieving entries which have a particular tag. I’ve spent an entertaining hour going back through my old entries and tagging them. So, for example, you can see all my posts on religion or all the posts where I mention what I’ve been up to lately (I’ve nicked livredor‘s “quotidian” tag to describe my daily life). Hopefully they’ll do something similar to Flickr and allow you to search other people’s journals for particular tags, or get a feed which displays all posts with a particular tag.

Speaking of what I’ve been up to lately, I had an excellent time at S’s Graduation Dinner the other night (although the name is a misnomer as they’ve not graduated yet). It was at St John’s, who produced some of the best food I’ve had at a Cambridge college. My favourite photo is this one, as the Three Musketeers seem to be enjoying themselves.

Adam Kay and Suman Biswas, medics themselves, have joined to form Amateur Transplants, a beat combo. They are reminiscent of Flanders and Swann or Tom Lehrer, but with gratuitous use of the word “fuck”. You might have heard their seminal London Underground a while back, but it turns out there’s a whole album, entitled Fitness to Practice. Our favourites are Paracetamoxyfrusebendroneomycin and Snippets, for the excellent parodies of Coldplay’s Yellow and Phil Collins’s Against All Odds. Some MP3s are here, but sadly, physical copies of the entire album seem to have sold out. They should charge to download the remaining MP3s or something: it’s for charidee.

S and I want to go on holiday somewhere scenic, not too hot during the summer (anything over the high 20s in Celsius is too hot, in my book), and not monumentally expensive. Any suggestions?