No one talks much about Usenet these days, but people are still posting plenty of stuff there, including a large volume of binary files – images, executables, music, and video. If you want an easy way to skim some of that bounty for yourself, check out URD, an application that lets you download binaries from Usenet newsgroups through a web interface.
URD lets you subscribe to newsgroups, then automatically analyzes articles to form downloadable sets of all the files belonging to a specific upload. It displays the sets and lets you point and click to specify what you want to download. While some similar utilities need external NZB files to function, URD does not, though it can import them for downloads and allows you to export them for others to use.
Architecturally, URD is a daemon process that runs continuously in the background on Linux, *BSD, or Mac OS X systems, supported by a database like MySQL or PostgreSQL.
“URD is probably the most versatile Usenet client that is entirely web-based,” says Dutch developer Gavin Spearhead, “and it’s very easy to extend. For instance, I just added support to download subtitles through a subdownloader. And it has many nifty capabilities, such as automatic detection of RAR encryption and scheduling of downloads. In fact URD has so many options nowadays that half the forum posts consist of ‘can you make it do this or that,’ and I always have to tell them it already does.”
Spearhead recommends users take advantage of the program’s ability to search and block terms, “so interesting stuff will be marked automatically and be on the top of the browse page. For instance if you like Ubuntu, just put it in the search terms and once something passes by that matches “ubuntu” it will be highlighted. And if you couldn’t care less about Kubuntu, put that in the block terms and you won’t see it. And if things slow down, tweak your database to fit large datasets, with a utility like MySQLTuner; the performance impact may be huge.”
Spearhead began putting together the software in August 2008, coding in PHP with Vim and taking advantage of other open source external tools. “I didn’t see any purpose in keeping it closed source, as others might benefit from it. A commercial project didn’t seem likely, and wouldn’t have been interesting to me.”
In future versions, Spearhead hopes to improve performance with huge data sets. A new user interface has been in the planning stages for months. And eventually he hopes to make URD a complete Usenet client, with the ability to post and read non-binary articles.
If any those ideas appeal to the developer in you, Spearhead wants to hear from you. “Stuff I can’t do includes translating it to other languages, especially Spanish, Italian, German, Norwegian, and Danish. And we need packagers for systems other than Debian, such as CentOS/Red Hat/Fedora, Mac OS X, and *BSD.” The easiest way to get in touch is through the project’s forums, or via e-mail.