Nature Methods on software availability

Nature Methods has a new editorial clarifying its position on making the software used in papers available to readers (about time a journal did this):

The minimum level of disclosure that Nature Methods requires depends on how central the software is to the paper. If a software program is the focus of the report, we expect the programming code to be made available. Without the code, the software—and thus the paper—would become a black box of little use to the scientific community. In many papers, however, the software is only an ancillary part of the method, and the focus is on the methodological approach or an insight gained from it.

In these cases, releasing the code may not be a requirement for publication, but such custom-developed software will often be as important for the replication of the procedure as plasmids or mutant cell lines. We therefore insist that software or algorithms be made available to readers in a usable form. The guiding principle is that enough information must be provided so that users can reproduce the procedure and use the method in their own research at reasonable cost—both monetary and in terms of labor.

I think it's quite a well thought out piece. The editors recognize, for instance, that some short programs and algorithms are better made available as pseudocode (well, they say 'a small set of equations', but I know which one I'd prefer).

I'm not sure it goes far enough, though. For example: if the software runs as a web service, is making that service public enough to satisfy the journal's requirements? Can you host any code releases on your own server?

The problem with answering either of those questions with a 'yes' is that there's no guarantee that the software is still going to be available after a year or two (something most bioinformaticians are acutely aware of): postdocs and grad students move on, server accounts (and labs) get closed, bugs crop up and there's nobody willing to fix them, websites get redeveloped... etc.

What happens when we read an older paper, the software isn't around any more and we report it to an editor?

When we ask authors to make sequences available we require them to be deposited in GenBank. Should we require software authors to deposit their code on Sourceforge, Google Code or some other (more) permanent repository (in which case, what about the executable only software or software that has a restrictive licence)?

There are open comment threads at both Methagora - the Nature Methods blog - and Nautilus, which covers the whole spread of Nature journals. I urge you to go forth and help shape journal policy (perhaps).