Project of the Month, March 2004

GNU Mailman

Collaboration on the Internet has a single 'killer app', the mailing list. SourceForge.net itself hosts over 35,000 mailing lists, that allow people to communicate via email all around the world. How does SF.net manage all the lists, email subscriptions, passwords, setup, etc? We use GNU Mailman, an Open Source, powerful application sporting an elegant web-based frontend that makes managing any number of lists (from three to tens of thousands) a breeze. GNU Mailman has been with SF.net since the very beginning. SF.net now has (as of March 2004) over 75,000 projects; Mailman was registered when the site had just 102. The SF.net team is proud to make GNU Mailman the March 2004 SourceForge.net Project of the Month. We couldn't run the site without it.

Project Name: GNU Mailman
Founded / Started:
1996

URL: http://mailman.sourceforge.net/
Project page: https://sourceforge.net/projects/mailman/

Description of project:
Mailman is a list server. You can create both discussion lists and one-way (announce) lists, and you can manage them all via the Web. Mailman's features include NNTP gatewaying, archiving (both built-in and to optional external archivers), MIME and plain text digests, auto-responses, sophisticated bounce processing, some spam defenses (with experimental hooks to Spambayes, another great Python SourceForge project), and more.

What most people are initially struck by is Mailman's ability to manage all your mailing lists through the Web. The fact that Web access is so well integrated was what made it unique in its early days. With Mailman 2.1.4 we support about two dozen non-English natural languages out of the box. For developers, Mailman is really structured as a framework, which makes it easier to extend and modify Mailman, as well as integrate it with existing Web sites. The Web and command line interfaces are mostly veneers put on top of the core engine, which makes it fairly easy to control Mailman programmatically from Python.

Trove info:

Mailman runs on any Unix-like operating system. It is almost completely written in Python, with a little bit of C code for security. Other than Python, it requires a Web server and a mail server to make it go. Mailman 2.1.4 (the most current stable release as of 2004-03-05) requires Python 2.3.3 (its latest stable release as of this writing).

How did you get started?
John Viega was the original inventor of Mailman. John, Ken Manheimer (Mailman's savior), and Barry (Mailman's yappy guard dog) were the most prominent developers early on, although the cabal now officially includes Thomas Wouters (Mailman's Dutch treat), Harald Meland (Norse Mailman), and Scott Cotton (Cookie Monster).

Ken and I were Webmastering python.org back in our CNRI days and we obviously needed discussion lists. The Python community had several SIGs and other related mailing lists and we were running all of them on a crufty pile of old Majordomo code we had hacked together. We found it increasingly difficult to make Majordomo do what we wanted. Besides, it just wouldn't do to run the Python Web site on a Perl list server! When John released Mailman, it seemed like the perfect code base to run python.org on. In 1999, Ken left CNRI to join Digital Creations (now Zope Corp.) and I became the project leader. I now work at Zope Corp. too, so Ken and I are at our third job together!

What is the intended audience?
There are a few, really. Most important to me is the end user -- in other words, the member of a mailing list. These folks can run the gamut from highly technical, such as the members of most of the python.org lists, to non-technical, such as my mom, who is a member of my band's mailing list. To me, these are the most important people to help because they should have a pleasant, or at least non-frustrating, experience with Mailman. They should be able to easily subscribe and unsubscribe, get email they can read in any mail reader, and be able to configure their subscriptions with the minimal amount of pain. Juggling this with security and anti-spam defenses is the biggest trick.

Then there are the list administrators. These folks are usually a bit more technically minded, but they needn't be. They have to interact with a larger portion of the Mailman interface, to perform tasks such as bulk uploads of new members or configuring the bounce processor. Mailman has a lot of features, and making them accessible to list managers without overwhelming them is a big challenge.

Finally, there are the site administrators. These folks are usually pretty savvy, and they need to know stuff like how to configure mail and Web servers. I try to make it as easy as possible to install, configure, and babysit Mailman, but there's probably more pain there then there needs to be.

How many people do you believe are using your software?
There are millions of subscribers to Mailman mailing lists. I once did a Google search for unique features on the Mailman "listinfo" overview page (a page that almost every Mailman site would have) and found on the order of 5,000-7,000 sites.

What gave you an indication that your project was becoming successful?
When I started getting way way more email than I could answer. Oh, that and the groupies.

What has been your biggest surprise?
Most of my groupies are geeks. Actually, the most pleasant surprise has been the really great people I've met from all around the world, both electronically and in the real world.

What has been your biggest challenge?
Probably like most open source projects, it's finding the time to do everything I want to do. I don't get paid to work on Mailman, so I have to fit it in between work, family, and music.

What are you most proud of?
That people use and (generally) like Mailman. It's very gratifying to know that something you helped create is useful to other people. It's a lot like writing music (something else I do). The most rewarding aspect of it, though, is when I hear from people who are using Mailman for good social purposes, such as promoting democracy here in the US, or helping to connect dissidents in other countries. There are lots of non-profits using Mailman and I've had lots of very positive messages from people who wouldn't be able to afford commercial alternatives.

Where do you see your project going?
There's definitely room for improvement. There has been a lot of planning for Mailman 3 and I've even started to do some experimentation. Mailman 3 will probably be a very significant (or total) rewrite, but it also has the risk of second system syndrome. We're going to be having a Mailman 3 sprint at the upcoming Python conference later this month in DC, so I encourage anyone who wants to contribute to come and join us. Given the limited resources to work on it, there probably will not be a Mailman 2.2 release, but there will continue to be patch releases in the Mailman 2.1 series.

How can others contribute?
Translators can contribute by working on new, or existing language, support. I'd really like someone to fix the Chinese translations, for example. Also, the archiver is a major subsystem that doesn't get as much actual coding attention as it should -- that would be a great area for someone to step up and own. Folks wishing to participate should join the mailman-developers@python.org mailing list, at http://mail.python.org/mailman/listinfo/mailman-developers.

Do you work on the Open Source project full-time, or do you have another job?
In a sense, I do work on open source at my day job, although not on Mailman. I work for Zope Corp., makers of the open source content management system called Zope. For a while I was working on Zope 3 and ZODB technology, and those are all open source, but lately I've been working on a non-open customer project, developing a system that is an add-on to Zope.

How do you coordinate the project?
The project is almost exclusively coordinated through the mailing lists. Since I'm the primary developer of the source code, there isn't much coordination I need to do with other programmers, but I rely very heavily on the experience, wisdom, and suggestions of others on the developers list. I don't participate as much as I'd like on the users list, though occasionally topics there will migrate over.

Mailman 2.1 has a limited unit test suite, but I'm a big fan of unit and functional testing. I heard it said that you build up a "tech debt" when you neglect stuff like your unit tests, and in that respect Mailman 2.1 has quite a tech debt. Part of that's historical, since Mailman 2.1 was in development before Python had a standard unit test story, but mostly it's just lack of time. I intend to pay down that debt with Mailman 2.2 and Mailman 3.

In the meantime, I test changes in the patch releases pretty carefully, and I usually run the code base on python.org for a few weeks before I release it. So far that strategy has helped avoid most major embarrassments (knocking on his wooden head).

What is your development environment like?
I'm Windows-free both at home and work, so my primary development environment right now is Red Hat 9 on a Dell box of a few years ago. I use Python 2.3 exclusively now and I'm a total XEmacs-head (although I've been regressing lately by using Ximian Evolution as my mail client). I have a couple of Mac OS X boxes that I also occasionally test with. Since Mailman is almost 100% Python, I live in pdb, and specifically pdbtrack, an add-on to python-mode.el for XEmacs that makes debugging Python programs a joy. I'm not much of an IDE kind of guy.

If you could change one thing about the project, what would it be?
To have a team of the sharpest five Python programmers in the world backed by $10 million dollars of VC money. Okay, maybe that's two things.

Seriously, I'd like to make life easier for translators by providing more on-line tools for creating new language packs and for keeping existing ones up to date. Some of our thoughts are in the Translation Web Service vision statement for Zope 3. This isn't just a Zope thing -- while the TWS would be run on Zope, my dream would be to really use the same infrastructure for any interested open source project.

What's on your project wish list?
We're just now starting to put together the wish list for Mailman 2.2, but nothing's set in stone. You can see where we're heading (and participate!) in the Mailman 2.2 wiki.

Mailman 3 is more ambitious, and it too has its own wiki.

Briefly, high on my list is creating a unified user database so that Mailman's current list-centric approach to membership is turned inside out. I want a list member's experience to be centered around them, so that there's one login, one password, one place to change their options and manage their subscriptions, etc. That's a big enough change so that it won't happen until Mailman 3.

Milestones
We released Mailman 2.1.4 on 2003-12-31. While primarily a bug fix release, we included four new languages. Mailman 2.1.5 is expected to be released shortly.


More projects of the month


Project Name: GNU Mailman

Background of leaders:

Barry Warsaw

Age: 42
Employer: Senior software engineer, Zope Corp.

Key Developers:
A full list is in the ACKNOWLEDGMENTS file in the source distribution. Juan Carlos Rey Anaya and Victoriano Giralt contributed the bulk of the internationalization code that made it into Mailman 2.1, while others who have contributed code, documentation, and translations to the FSF (the holder of Mailman's copyrights) include Richard Barrett, Norbert Bollow, Ben Gertzfield, Mads Kiilerich, The Dragon De Monsyne, Les Niles, Terri Oda, and Simone Piunno. Scores of others have also contributed, either with code or with crucial ideas and insights.

Quote about SF.net?
Mailman uses SF.net and SF.net uses Mailman -- it's a great symbiotic relationship!

Why did you place the project on SF.net?
SourceForge was the only way to manage such a geographically diverse project. For years, I managed Python's development infrastructure, but after we added the first few non-Pythonlabs committers, it was clear that this would turn into a full-time job of its own. At one of the Python conferences a few years back, SourceForge was just getting going and a few Pythoneers were raving about it. I decided to use Mailman as a guinea pig project for moving to SF. It worked out well enough that we eventually moved the Python project there too.

How has SF.net helped you?
It provides a robust suite of tools that any harried and time-constrained project leader will welcome.

The number one benefit of using SourceForge.net is:
The Trackers. They're not perfect (what software is?) but they don't suck. I really like working with them and they are a much better bug and patch tracking system than my inbox.