Software "releases"

There's an interesting post over at Neil's blog about Methods vs Discovery (I started off adding this as a comment there but decided it was too long and ranty):

One of my pet issues in bioinformatics is that of methodology versus discovery. In other words - is your new, cute and clever piece of software any use if biologists are not using it to discover interesting things?

Reading it reminded me of being frustrated in my efforts to find software relating to chromatin structure for a project (not the S/MARs thing, but related) a wee while back. In the last year there've been two potentially really useful tools "released" which could have helped speed my work along. When I say "released", though, I mean that papers describing them were published; neither piece of software was ever actually made available for download (as you'll be aware if you've written an application note recently, editors "strongly suggest" that the software is released under an open source licence or made publically available, but it's just a suggestion).

This isn't too bad; I mean, sometimes code is messy or it's got some peculiar prerequisites or there's some other reason that you want to deal with potential downloaders in person. So I emailed the relevant authors to ask nicely if there was any possibility of obtaining the software, it'd be very useful, all due credit to be given etc.

I never got an answer from one group; the other responded that I'd have to wait until they'd published results from their own analyses with the software in question (presumably in case I had the cheek to try and use their software to discover something significant enough to be publishable before them).

Part of me understands the whole "I did the work, why should you reap the benefits?" idea but frankly it's a bit sneaky to publish a pointless software application note to pad out your CV with when the software is of no use to anybody (though not for lack of them trying). What's the point of a paper announcing that you've got this piece of software that, realistically, nobody else is ever going to use and you know it?

Isn't it short-sighted to keep a piece of software back so that you can get a few more papers out of it when the field is moving so quickly that your code will be obsolete within the next twelve months? Wouldn't it be better for everybody concerned - other scientists, who don't have to reinvent the wheel and implement their own tools, you, who will get cited more frequently and pick up more collaborations and the journal, who won't be publishing vanity pieces any longer - if we all shared?

There are sometimes other complications, of course. There's the possibility that the software in question is actually a bit crap; which will become obvious the moment anybody applies it to something other than the carefully selected test set in the paper (roll on transparent reviewing and reader comments). Perhaps you're using some sort of commercial or proprietary database.

Or perhaps the university technology transfer people got to you...