Menu
▾
▴
popfile-announce — Subscribe to this list to receive announcements about the progress of POPFile
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(2) |
Nov
(6) |
Dec
(3) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
(2) |
Jun
(1) |
Jul
(1) |
Aug
|
Sep
|
Oct
(1) |
Nov
(1) |
Dec
|
| 2004 |
Jan
|
Feb
|
Mar
(4) |
Apr
|
May
|
Jun
|
Jul
|
Aug
(2) |
Sep
(1) |
Oct
(1) |
Nov
(1) |
Dec
(1) |
| 2005 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2006 |
Jan
|
Feb
(1) |
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
| 2008 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Brian S. <bd...@su...> - 2008-05-26 18:44:41
|
Welcome to POPFile v1.0.1 POPFile is an email classification tool with a Naive Bayes classifier, POP3, SMTP, NNTP proxies and IMAP filter and a web interface. It runs on most platforms and with most email clients. This version is a bug fix and minor feature release. It's also the first release to have been built by Brian Smith. WHAT'S CHANGED SINCE v1.0.0 1. Bug fixes When POPFile starts up it no longer complains about a missing IMAP file if IMAP has not been installed. Fixed a bug in the IMAP module that allowed for unintentional reclassification to the unclassified bucket. Fixed a bug that causes segmentation faults when displaying messages which have very long To, Cc or From headers in the POPFile UI. Fixed a bug that causes a few warnings during the corpus upgrade process. Add code to check for unexpected NULL characters in email messages (to avoid SQLite problems). 2. POPFile improvements POPFile's Brazilian Portuguese language file has been renamed. The new name does not include any spaces. The simplified and traditional Chinese language translations have been updated. POPFile's log file now includes entries showing when POPFile starts up and shuts down. This makes it easier to read the log file. The startup messages in the log file have been made easier to read by splitting the long entries into shorter lines. The IMAP module now issues a log message when it fails to connect to the server. Deletion of messages from the message history has been made faster. 3. Installer improvements When upgrading an existing installation the installer now recommends upgrading the existing SSL support files. The installer and all of the NSIS-based programs have been built using the latest NSIS compiler and newer versions of the plugins used to download files from the internet. These NSIS-related improvements should reduce the number of false alarms from various anti-virus packages. The latest version of the SQLite command-line utility is now included. 4. UI improvements The links to the wiki have been improved. 5. Other changes The (outdated) manual is no longer included in the download. All documentation for POPFile is online and can be found at http://getpopfile.org/docs/ WHERE TO DOWNLOAD http://getpopfile.org/download/ GETTING STARTED WITH POPFILE An introduction to installing and using POPFile can be found in the QuickStart guide: http://getpopfile.org/docs/QuickStart SSL SUPPORT IN WINDOWS SSL Support is offered as one of the optional components by the installer. If the SSL Support option is selected the installer will download the necessary files during installation. If SSL support is not selected when installing (or upgrading) POPFile or if the installer was unable to download all of the SSL files then POPFile's "Add/Remove Programs" entry can be used to add SSL support to an existing installation. I AM USING THE CROSS PLATFORM VERSION POPFile requires a number of Perl modules that are available from CPAN. You will need: Date::Parse HTML::Template HTML::Tagset DBD::SQLite2 DBI TimeDate You can install all the required POPFile modules by getting the Bundle::POPFile bundle from CPAN. Please refer to the installation instructions on the POPFile wiki: http://getpopfile.org/docs/HowTos:CrossPlatformInstall Japanese users may need to install some extra programs and Perl modules, depending upon which Nihongo parser (wakachi-gaki program) they wish to use. For more information about how to install them, see the POPFile wiki: http://getpopfile.org/docs/JP:HowTos:CrossPlatformInstall CROSS PLATFORM VERSION KNOWN ISSUES The current version of SQLite (v3.x) is not compatible with POPFile. You must use DBD:SQLite2 to access the database. Users of SSL on non-Windows platforms should NOT use IO::Socket::SSL v0.97 or v0.99. They are known to be incompatible with POPFile; v1.13 is the most recent release of IO::Socket::SSL that works correctly. WINDOWS KNOWN ISSUES 1. THE TRAYICON ON WINDOWS CAN CAUSE POPFILE TO FREEZE. With certain setups, if you attempt to view the trayicon's tooltip or hover your cursor over it while checking or downloading mail POPFile may freeze. The exact cause is still unknown but it may be related to some video drivers. If this problem affects you turn off the trayicon. Due to this issue, we suggest users of the IMAP module disable the trayicon since a freeze may go unnoticed more easily than when using a mail client to make POP3 connections. 2. ON WINDOWS I WANT TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY. Because the time taken to start a new process on Windows is long under Perl there is an optimization for Windows that is present by default: when a new connection is made between your email program and POPFile, POPFile handles it in the 'parent' process. This means the connect happens fast and mail starts downloading very quickly, but it means that you can only download messages from one server at a time (up to 6 other connections will be queued up and dealt with in the order they arrive) and the UI is unavailable while downloading email. You can turn this behavior off (and get simultaneous UI/email access and as many email connections as you like) on the Configuration panel in the UI by making sure that "Allow concurrent POP3 connections:" is Yes, or by specifying --set pop3_force_fork=1 on the command line. The default behaviour (no concurrent POP3 connections) can cause email clients to time out if several accounts are being checked (because POPFile only handles one account at a time it can take a while to process all of the accounts). If SSL support is being used then the default setting (no concurrent POP3 connections) _MUST_ be used otherwise POPFile crashes. v1.0.0 RELEASE NOTES If you are upgrading from pre-v1.0.0 please read the v1.0.0 release notes for much more information: http://getpopfile.org/docs/ReleaseNotes:1.0.0 DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed their time through patches, feature requests, bug reports, user support and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 THANKS Big thanks to all who've contributed to POPFile. The POPFile Core Team |
|
From: John Graham-C. <jg...@jg...> - 2007-12-21 09:49:16
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Welcome to POPFile v1.0.0 POPFile is an email classification tool with a Naive Bayes classifier, POP3, SMTP, NNTP proxies and IMAP filter and a web interface. It runs on most platforms and with most email clients. This version is a bug fix and minor feature release. It's also the first release to have the version number 1. This is because we've finally decided to stop pretending that POPFile is 'beta' software :-) WHAT'S CHANGED SINCE v0.22.5 1. Changes to the IMAP module / Support for Gmail IMAP The IMAP module will now use only one connection to the IMAP server, instead of one connection for each watched folder. This means that when using IMAP mode, POPFile will be much lighter on the server and will even work with IMAP servers that are restrictively configured or if you have many buckets. As a consequence, you should now be able to let POPFile sort your email on Google's Gmail. Please refer to the setup instructions on the POPFile wiki: http://getpopfile.org/wiki/experimentalmodules:imapinstructions 2. Nihongo improvements Japanese (Nihongo) text does not use spaces between words so POPFile uses a 'Nihongo Parser' to split the text into words to allow the text to be analysed properly. Up until now POPFile has used the 'Kakasi' parser when analysing Japanese text. The 1.0.0 release offers a choice of three parsers (Kakasi, MeCab and internal). The new 'bayes_nihongo_parser' configuration parameter is used to specify the parser to be used (the default parser is Kakasi). The Windows installer lets the user select the Nihongo parser to be installed. All of the files needed for the Kakasi and internal parsers are included in the installer. The MeCab package size is about 13 MB (i.e. too big to include in the installer) so it will be downloaded from the Internet if the MeCab parser is selected when installing POPFile. Other Nihongo changes: fixed a bug that the history table in the UI was not being shown correctly when invalid characters were used in the From, To and Cc headers in Nihongo mode; fixed a bug in the shortening of Subjects in the history table. 3. Installer improvements To make it easier to add SSL support after installing POPFile the installer now creates an "Add/Remove Programs" entry which allows SSL support to be added by downloading and installing the necessary support files. This entry also allows the Nihongo parser to be changed and will download the MeCab package if necessary. The installer and uninstaller have been upgraded to work better when used on Vista, especially when run from a standard user account. Many of these improvements also apply to Windows XP. For example the uninstaller is now less likely to leave Start Menu shortcuts behind. 4. UI improvements Change the default skin choice to simplyblue. Add two new skins, coolmint and ocean. Drop less popular skins: klingon, prjbluegrey, and prjsteelbeach. Merge nearly duplicate tinydefault and lrclaptop with smalldefault. Add a little bit of polish to many of the others. Add a JavaScript Check All box for selecting messages to remove from history. 5. Other improvements Performance improvement in analyzing e-mails. Encoded attachment file names are now decoded. Fixed a bug that missed some e-mail addresses. WHERE TO DOWNLOAD http://getpopfile.org/wiki/download GETTING STARTED WITH POPFILE An introduction to installing and using POPFile can be found in the QuickStart guide: http://getpopfile.org/wiki/QuickStart SSL SUPPORT IN WINDOWS SSL Support is offered as one of the optional components by the installer. If the SSL Support option is selected the installer will download the necessary files during installation. If SSL support is not selected when installing (or upgrading) POPFile or if the installer was unable to download all of the SSL files then POPFile's "Add/Remove Programs" entry can be used to add SSL support to an existing installation. I AM USING THE CROSS PLATFORM VERSION POPFile requires a number of Perl modules that are available from CPAN. You will need: Date::Parse HTML::Template HTML::Tagset DBD::SQLite2 DBI TimeDate You can install all the required POPFile modules by getting the Bundle::POPFile bundle from CPAN. Please refer to the installation instructions on the POPFile wiki: http://getpopfile.org/wiki/HowTos:CrossPlatformInstall Japanese users may need to install some extra programs and Perl modules, depending upon which Nihongo parser (wakachi-gaki program) they wish to use. For more information about how to install them, see the POPFile wiki: http://getpopfile.org/wiki/JP:HowTos:CrossPlatformInstall CROSS PLATFORM VERSION KNOWN ISSUES The current version of SQLite (v3.x) is not compatible with POPFile. You must use DBD:SQLite2 to access the database. Users of SSL on non-Windows platforms should NOT use IO::Socket::SSL v0.97 or v0.99. They are known to be incompatible with POPFile; v1.07 is the most recent release of IO::Socket::SSL that works correctly. WINDOWS KNOWN ISSUES 1. THE TRAYICON ON WINDOWS CAN CAUSE POPFILE TO FREEZE. With certain setups, if you attempt to view the trayicon's tooltip or hover your cursor over it while checking or downloading mail POPFile may freeze. The exact cause is still unknown but it may be related to some video drivers. If this problem affects you turn off the trayicon. Due to this issue, we suggest users of the IMAP module disable the trayicon since a freeze may go unnoticed more easily than when using a mail client to make POP3 connections. 2. ON WINDOWS I WANT TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY. Because the time taken to start a new process on Windows is long under Perl there is an optimization for Windows that is present by default: when a new connection is made between your email program and POPFile, POPFile handles it in the 'parent' process. This means the connect happens fast and mail starts downloading very quickly, but is means that you can only downloaded messages from one server at a time (up to 6 other connections will be queued up and dealt with in the order they arrive) and the UI is unavailable while downloading email. You can turn this behavior off (and get simultaneous UI/email access and as many email connections as you like) on the Configuration panel in the UI and making sure that "Allow concurrent POP3 connections:" is Yes, or by specifying --set pop3_force_fork=1 on the command line. The default behaviour (no concurrent POP3 connections) can cause email clients to time out if several accounts are being checked (because POPFile only handles one account at a time it can take a while to process all of the accounts). If SSL support is being used then the default setting (no concurrent POP3 connections) _MUST_ be used otherwise POPFile crashes. v0.22.0 RELEASE NOTES If you are upgrading from pre-v0.22.0 please read the v0.22.0 release notes for much more information: http://getpopfile.org/wiki/ReleaseNotes DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed their time through patches, feature requests, bug reports, user support and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 THANKS Big thanks to all who've contributed to POPFile. John. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHa4v+Lphrp73n/hARAvB4AJwLubvJ3wygILyrExB/iRQqlITU5gCeJBwT McdDW1soQmTlFfQ7U5/42Gk= =GiCH -----END PGP SIGNATURE----- |
|
From: John Graham-C. <jg...@jg...> - 2007-07-14 12:08:25
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 POPFile has been nominated for a SourceForge Community Choice Award due to the efforts of many people. Voting closes on July 20 and there's strong competition in POPFile's category from the likes of Pidgin (formerly GAIM) and phpBB. If POPFile wins SourceForge will be making a donation to a charity that I picked: Doctors without Borders. If you feel like voting for POPFile, please vote in the Best Project for Communications category here: http://sourceforge.net/awards/cca/vote.php Cheers, John. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGmLyyLphrp73n/hARAiNYAJ9lO5+wa9rbxzWd6rxYykGEk2Ff4QCffbm0 nVUUhrGhWLgxYePEs81zZ1M= =Uz4D -----END PGP SIGNATURE----- |
|
From: John Graham-C. <jg...@jg...> - 2007-07-03 04:41:04
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Folks, POPFile has been nominated for a SourceForge Community Choice Award thanks to the efforts of many users. If love POPFile please vote for it here: http://sourceforge.net/awards/cca/vote.php It's in the Best Project for Communications category. John. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGidNcLphrp73n/hARAhkxAKDK6yRwKVu3vugSqlSoEzPICtB6qgCglXpx VH6C8Ws4ADIvAHfHS9RWdUg= =SbeU -----END PGP SIGNATURE----- |
|
From: John Graham-C. <jg...@jg...> - 2007-07-03 04:35:28
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Folks, POPFile has been nominated for a SourceForge Community Choice Award thanks to the efforts of many users. If love POPFile please vote for it here: http://sourceforge.net/awards/cca/vote.php It's in the Best Project for Communications category. John. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGidIKLphrp73n/hARAg+eAJ9Rrbgt9dnDoHlS+Woig//Osfd1YwCfT//f 5h++XcfGABKkeyWhWNZQpE8= =ZKjn -----END PGP SIGNATURE----- |
|
From: John Graham-C. <jg...@jg...> - 2007-06-18 09:42:46
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Welcome to POPFile v0.22.5 This version is a bug fix and minor feature release that's been over a year in the making (mostly due to me being preoccupied by other things). NOMINATING POPFILE FOR AN AWARD SourceForge has announced their 'Community Choice Awards' for 2007 and is looking for nominations. If you feel that POPFile deserves such an honour please visit the following link and nominate POPFile in the 'Best Project for Communications' category. POPFile requires multiple nominations (i.e. as many people as possible) to get into the list of finalists. http://sourceforge.net/awards/cca/nomination.php?group_id=63137 Thanks! WHAT'S CHANGED SINCE v0.22.4 1. POPFile now defaults to using SQLite2 (the Windows installer will convert existing installations to use SQLite2). 2. Various improvements to the handling of Japanese messages and improvements for the 'Nihongo' environment: Performance enhancement for converting character encoding by skipping conversion which was not needed. Fix a bug where the wrong character set was sometimes used when the charset was not defined in the mail header. Fix a bug where several HTML entities caused misclassification. Avoid a warning 'uninitialized value'. Fix a bug that the word's links in the bucket's detail page were not url-encoded. 3. Single Message View now has a link to 'download' the message from the message history folder. 4. Log file now indicates when an SSL connection is being made to the mail server. 5. A number of small bug fixes to the POPFile IMAP interface. 6. Installer improvements: Email reconfiguration no longer assumes Outlook Express is present. Add/Remove Programs entries for POPFile now show a more realistic estimate of the program and data size. Better support for proxies when downloading the SSL Support files. The SSL patches are no longer 'hard-coded', they are downloaded at install time. This will make it easier to respond to future changes to the SSL Support files. The Message Capture Utility now has a Start Menu shortcut to make it easier to use the utility. The minimal Perl has been updated to the latest version. The installer package has been updated to make it work better on Windows Vista (but further improvements are still required). WHERE TO DOWNLOAD http://getpopfile.org/wiki/download GETTING STARTED WITH POPFILE An introduction to installing and using POPFile can be found in the QuickStart guide: http://getpopfile.org/wiki/QuickStart SSL SUPPORT IN WINDOWS SSL Support is offered as one of the optional components by the installer. If the SSL Support option is selected the installer will download the necessary files during installation. If SSL support is not selected when installing (or upgrading) POPFile or if the installer was unable to download all of the SSL files then the command setup.exe /SSL can be used to run the installer again in a special mode which will only add SSL support to an existing installation. CROSS PLATFORM VERSION KNOWN ISSUES The current version of SQLite (v3.x) is not compatible with POPFile. You must use DBD:SQLite2 to access the database. Users of SSL on non-Windows platforms should NOT use IO::Socket::SSL v0.97 or v0.99. They are known to be incompatible with POPFile; v1.07 is the most recent release of IO::Socket::SSL that works correctly. v0.22.0 RELEASE NOTES If you are upgrading from pre-v0.22.0 please read the v0.22.0 release notes for much more information: http://getpopfile.org/wiki/ReleaseNotes DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed their time through patches, feature requests, bug reports, user support and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 THANKS Big thanks to all who've contributed to POPFile over the last year. John. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGdlG2Lphrp73n/hARAmybAJ9aA1HWoo3eESYIAluicCIFVD1nPQCeLLLW W/EDJX2xZt/6ldMjry7P594= =dGtF -----END PGP SIGNATURE----- |
|
From: John Graham-C. <jg...@jg...> - 2006-03-03 10:05:10
|
Folks, I don't often send to this mailing list (unless it's a POPFile release), but I've noticed recently a number of interesting commercial uses of POPFile and I would like to build a web page on the POPFile Wiki giving examples of ways people use POPFile. If you have an interesting example (e.g. I know of one large company who uses POPFile for automated customer service), please let me know via email. I am happy to keep the information confidential, or shout it from the rooftops if you are looking for publicity! In the meantime, here's an interesting blog entry that talks a little more about ways to use POPFile: http://www.loosewireblog.com/2006/02/how_to_make_mor.html John. |
|
From: John Graham-C. <jgr...@gm...> - 2006-02-22 12:48:13
|
Welcome to POPFile v0.22.4
This version is a bug fix release and contains an important security
related bug fix. It is a highly recommended upgrade because of the
problem described in #1 below.
WHAT'S CHANGED SINCE v0.22.3
1. A bug in the handling of the charset in an email could be exploited
to cause POPFile to crash when downloading messages.
2. Japanese users were unable to use the insert.pl script. This has
been corrected so the Nihongo language is now correctly recognized.
In addition the Nihongo language translation has been updated.
3. POPFile's license has changed (slightly). Prior to v0.22.4, POPFile
was released under the General Public License version 2, stating
that the user could choose that or any later version. With the
current uncertainty about GPLv3, POPFile has been restricted to
version 2 starting from v0.22.4.
Once GPLv3 has been finalized, we will review it and decide whether
to revert to the "version 2 or later" permission. Naturally, all
prior versions are unaffected.
4. The Installer's Brazilian Portuguese translations were updated.
The installer now uses a longer timeout value so it can better
monitor upgrading large SQLite databases. The SQLite Database Status
Check utility now displays the size of the database (in KB).
5. All the "Remove" buttons are now at the top of the History page as
well as the bottom.
6. It should be rare that anyone sees it, but we added a 404 page.
7. X-POPFile-Link improvements to let users change the link address
used in Stealth Mode. By default POPFile uses 127.0.0.1 for this
address, but at least one email client (Eudora) issues a security
warning for links with an IP address or 'localhost'. The wiki
explains how to avoid these warnings:
http://getpopfile.org/cgi-bin/wiki.pl?HowTos/SuppressXPLWarning
8. Prior to v0.22.0, "Remove Page" used to archive messages (when
archiving is active), that behavior has been restored.
9. Updated the UI to the latest Brazilian Portuguese translation.
GETTING STARTED WITH POPFILE
An introduction to installing and using POPFile can be found in the
QuickStart guide:
http://getpopfile.org/cgi-bin/wiki.pl?QuickStart
SSL SUPPORT IN WINDOWS
SSL Support is offered as one of the optional components by the
installer. If the SSL Support option is selected the installer will
download the necessary files during installation.
If SSL support is not selected when installing (or upgrading) POPFile
or if the installer was unable to download all of the SSL files then
the command
setup.exe /SSL
can be used to run the installer again in a special mode which will
only add SSL support to an existing installation.
CROSS PLATFORM VERSION KNOWN ISSUES
The current version of SQLite (v3.x) is not compatible with
POPFile. You must use DBD:SQLite2 to access the database.
Users of SSL on non-Windows platforms should NOT use IO::Socket::SSL
v0.97. It is known to be incompatible with POPFile; v0.96 is the most
recent release of IO::Socket::SSL that works correctly.
v0.22.0 RELEASE NOTES
If you are upgrading from pre-v0.22.0 please read the v0.22.0 release
notes for much more information:
http://sourceforge.net/project/shownotes.php?release_id=266477
DONATIONS
Thank you to everyone who has clicked the Donate! button and donated
their hard earned cash to me in support of POPFile. Thank you also to
the people who have contributed their time through patches, feature
requests, bug reports, user support and translations.
http://sourceforge.net/forum/forum.php?forum_id=213876
THANKS
Big thanks to Joseph Connors and Brian Smith for working on v0.22.4 and
to John Murray for the reporting and diagnosis of the charset bug.
John.
|
|
From: John Graham-C. <jg...@jg...> - 2005-10-28 12:47:58
|
Welcome to POPFile v0.22.3 This is a minor bug fix release prompted by POPFile developer Brian Smith who pointed out that we hadn't done a POPFile release for a while, that there was a much newer version of Perl available and that there were lots of changes in the installer that would help people. The new version of Perl enables Windows POPFile to use the latest SSL code (for connections to secure mail servers) and fixes the "3935 byte problem" that made POPFile's web server fail on some systems. In addition to Brian's installer a number of bug fixes and changes have been incorporated. We're still working on the major v0.23.0 release and fearless users can get it from CVS; for everyone else there's v0.22.3. Here's what's new in v0.22.3: 1. Lots of changes to the installer including the incorporation of the SSL download (was previously a separate executable). 2. Fix a problem with PRIMARY KEY errors on upgrading the database. 3. Minor skin update so that it's possible to click anywhere on a tab, and not just on the text within a tab. 4. When a user asks for an SSL connection from POPFile to a POP3 server with the :ssl option the port is defaulted to 995 (instead of 110). 5. Incorporate a new translation: Catalan. 6. Japanese users get support for Japanese base-64 messages, performance enhancements for Kakasi and the mutex code, and a small bug fix for the euc-jp encoding on the history page. 7. Fix minor UI bugs, and change the Buckets tab work more clearly (this is actually a backport of some of the UI changes coming in v0.23.0). 8. The Windows installer no longer installs the English manual (to save space and because the manually was getting very out of date). A far better introduction to using POPFile can be found in the QuickStart guide: http://getpopfile.org/cgi-bin/wiki.pl?QuickStart 9. A new utility is included with the Windows version that can be used to check the integrity of the POPFile database. The utility, pfidbstatus.exe, can be accessed via Start Menu -> Programs -> POPFile -> Support -> Check database status. CROSS PLATFORM VERSION KNOWN ISSUE The current version of SQLite (v3.x) is not compatible with POPFile. You must use DBD:SQLite2 to access the database. v0.22.0 RELEASE NOTES If you are upgrading from a pre-v0.22.0 version of POPFile please read these release notes: http://sf.net/projects/shownotes.php?release_id=266477 THANKS Big thanks to Brian, Manni, Naoki and Joseph for working on v0.22.3. John. |
|
From: John Graham-C. <jg...@jg...> - 2005-03-17 16:58:40
|
Folks, I realize that this list is used normally for just POPFile release announcements (and the occasional security advisory), but I'm making an exception to let you know that POPFile won a Productivity Award last night at the 2005 Jolt Awards at SD Expo in Santa Clara, CA. I know that everyone who works on POPFile is delighted to see the program get public recognition and I want to take this opportunity to thank everyone who has contributed to POPFile as a patcher, developer, documentation writer, Wiki updater, translator, technical supporter and supporter. Long-time POPFile developer, Sam Schinke was present at the awards last night and has written about the experience on his blog. You can read about that here: http://scrappy-do.blogspot.com/ or look for the June 2005 Software Development Magazine where we should get a mention. Many of you may be wondering where v0.23.0 is; it's coming... slowly. v0.23.0 represents a major change in POPFile with the addition of support for multiple, distinct users. Much of the work has already been done and can be found in CVS, but there's more to do. I'm not going to promise a release date, but when v0.23.0 it'll feature an improved UI (Joseph Connors is leading that) and multiple user handling which should improve POPFile in SOHO environments. John. |
|
From: John Graham-C. <jg...@jg...> - 2004-09-08 13:02:36
|
Welcome to POPFile v0.22.0
This version consists of a major update to v0.21.0 with many improvements
and bug fixes:
1. History in database
We've finally done away with the old method of keeping the message history
in a single directory full of files (which was frankly flaky and hard to
maintain). Messages are now placed in subdirectories with no more than 256
per directory. History is now moved it into the database along with all
the other information that POPFile uses.
This change brings two benefits: firstly it's a lot faster than previous
versions because we can rely on the SQL database to do searching and sorting
for us; secondly, it brings extra flexibility and we now have additional
columns available in the history (we now have two time values (the date/time
the message was received by POPFile and the date/time in the message
header),
the From, To, Cc and Subject headers and message size).
2. Switch UI to HTML templates
Back in the mists of POPFile time I grafted an HTML interface onto my little
Perl script and POPFile was born. Unfortunately that HTML interface had
grown into thousands of lines of ugly Perl and HTML interwined. This
release
completely separates out the HTML from the Perl.
More details are here:
http://popfile.sf.net/cgi-bin/wiki.pl?Template
This has a number of advantages: firstly, POPFile's code is easier to
maintain
because it's simpler; secondly, skinning POPFile can now take advantage
of the
full power of HTML and not just a limited set of classes and lastly it's
made
it easy to expand the UI, which brings us to...
3. UI improvements
The History page is redesigned to be cleaner and support multiple,
configurable
columns (From, To, Cc, Arrived Date, Message Date, Subject, etc.).
Filtering
options have been expanded. The page now shows the total number of
messages
and the line break showing when different download sessions occurred.
Many people will notice that the 'unmagnetized' filter has been removed;
this
is because we've added an 'Invert' option that allows inversion of any
search
or filter. Now it's possible to filter on 'not spam', for example. To
filter
on unmagnetized messages select magnetized and Invert.
Buckets page has been simplified to make it clearer. The confusing accuracy
graph has been removed and unclassified messages are now correctly counted
in total messages and total errors. The old color picker is replaced with
a simple drop down.
The Advanced page now shows which parameters have been changed from the
default. If a parameter has a non-default value then its name is bolded.
Many of the skins had minor improvements. And we have two new skins:
The first, 'osx' gives you a bit of the Mac feel.
As part of the UI improvement a new skin called 'oceanblue' has been added
that demonstrates how the UI for POPFile can be changed dramatically with
the new skinning scheme using only a few files.
4. Anti-spam improvements
The mail parsing engine has been updated to deal with the latest spammer
trickery: notably the Flex Hex trick which you can read about in The
Spammers' Compendium: http://www.jgc.org/tsc/
5. Support for SMTP and NNTP
For a long time we've had experimental support for two new protocols:
SMTP (often used for incoming email) and NNTP (most commonly used for
Usenet-style news). They are no longer experimental and are now distributed
with POPFile. The SMTP and NNTP modules are disabled by default
but can be enabled on the Advanced page.
The SMTP module is designed to act as a front-end for a real SMTP server
accepting messages and proxying them into a single SMTP server after
performing classification. The NNTP module acts as a proxy between your
News client and News server performing classification on articles.
6. Experimental support for IMAP
The new experimental IMAP module is not included in the default
distribution.
You can find the documentation and a download link here:
http://popfile.sf.net/cgi-bin/wiki.pl?ExperimentalModules/Imap
The IMAP module is not a proxy, but acts as a service which will regularily
check an IMAP server for new messages, classifies them and then moves
them into
IMAP folders that correspond to your buckets. Reclassification can be done
outside the POPFile user interface simply by moving messages between folders
on the IMAP server.
IMAP module users should watch out about upsetting their ISP if they have a
very large number of buckets. The IMAP module will keep a connection
open for
each folder you monitor. This has caused a complaint from at least one
ISP to
a user with an extrememly large number of buckets.
7. XML-RPC interface
For some time we've had an experimental external interface to POPFile
through
XML-RPC. For the first time we are shipping this XML-RPC component,
though
it is disabled by default. If it is enabled an external program written in
many different programming languages can communicate with POPFile to use its
services.
Full details of the API can be found here:
http://popfile.sf.net/cgi-bin/wiki.pl?ExperimentalModules/XmlRpc
8. Support for APOP, SSL and SOCKS
The POP3 proxy has new support for the APOP authentication method. To use
APOP authentication you change the POP3 server string in your email client
from 'host:username' to 'host:username:apop'.
SSL support is added for all proxies. POPFile can now establish secure
connections to servers. To use change the POP3 server string in your email
client from 'host:username' to 'host:username:ssl'. POPFile does not
accept SSL connections, but will establish connections to secure servers.
SOCKS proxying is added to all proxies. To configure the host and port
to use go to the Configuration tab and set it for the appropriate proxy.
Outbound proxy connections will go via the SOCKS server specified.
The Windows installer DOES NOT include the components necessary for SSL
support. If you need SSL support on Windows YOU MUST download the Perl
SSL components as follows:
IO::Socket::SSL
Net::SSLeay
OpenSSL
9. Transparent POP3 Proxying
v0.22.0 includes support for transparent proxying of POP3 connections where
the destination POP3 server is configured in POPFile and the mail client
talks to POPFile without using the special host:user syntax for the
hostname.
This is most useful in a small office environment with a dedicated POP3
server where POPFile can act as a front end for the real POP3 server and
by installing it on the same machine as the POP3 server there's no need to
change mail client settings. To use this mode set the host and port of the
real POP3 server on POPFile's Security tab in the SPA/AUTH settings.
ESSENTIAL READING IF YOU ARE UPGRADING TO v0.22.0
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
the entire POPFile directory somewhere. You can then safely install
POPFile v0.22.0 on top of your current installation; I just think a back up
is a sensible precaution.
On Windows you may want to check out Brian's Backup and Restore Utilities:
http://www.sugelan.co.uk/popfile/databru.html
2. ON WINDOWS POPFILE IS AN EXE. Windows users will now be able to see
POPFile running in the Task Manager with an executable called popfileXX.exe
where the XX is one of f, if, b, ib depending on configuration. POPFile
is started by running popfile.exe which chooses the appropriate
popfileXX.exe
This might cause your firewall to ask about giving popfileXX.exe
permissions,
in addition if you had allowed Perl permissions in your firewall they are
NO LONGER needed.
3. IGNORE "PRIMARY KEY" ERRORS. When the history is being upgraded you may
see some "PRIMARY KEY" errors which can safely be ignored. Note that the
date on entries in the upgraded history will be incorrect: it will appear
that all the messages arrived at the moment of upgrade. Subsequent messages
will be correctly time and date stamped.
KNOWN WINDOWS ISSUES
1. THE TRAYICON ON WINDOWS CAN CAUSE POPFILE TO FREEZE. With certain setups
if you attempt to view the trayicon's tooltip or hover your cursor over it
while checking or downloading mail POPFile may freeze. The exact cause is
still unknown but it may be related to some video drivers. If this problem
affects you turn off the trayicon. A solution is being looked into.
2. ON WINDOWS I WANT TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY.
Because the time taken to start a new process on Windows is long under Perl
there is an optimization for Windows that is present by default: when a new
connection is made between your email program and POPFile, POPFile
handles it
in the 'parent' process. This means that the connect happens fast and mail
starts downloading very quickly, but is means that you can only downloaded
messages from one server at a time (up to 6 other connections will be queued
up and dealt with in the order they arrive) and the UI is unavaiable while
downloading email.
You can turn this behavior off (and get simultaneous UI/email access and as
many email connections as you like) by going to the Configuration panel in
the UI and making sure that "Allow concurrent POP3 connections:" is set to
Yes, or by specifying --set pop3_force_fork=1 on the command line.
I AM USING THE CROSS PLATFORM VERSION
POPFile requires a number of Perl modules that are available from CPAN. New
in v0.22.0 are the need for the following:
Date::Parse
HTML::Template
HTML::Tagset
DBD::SQLite
DBI
Date::TimeDate
Please refer to the detailed installation instructions on the POPFile wiki
http://popfile.sf.net/cgi-bin/wiki.pl?HowTos/CrossPlatformInstall
I AM USING THE CROSS PLATFORM VERSION AND WANT TO USE SQLITE 3
Since there is currently a great deal of change in SQLite and the Perl
modules that interface with SQLite please read the following page for
a full explanation:
http://popfile.sf.net/cgi-bin/wiki.pl?SQLite3
(Thanks helphand for the detailed explanation of the issues.)
DOWNLOADING
You can obtain the latest releases of POPFile by visiting
http://sf.net/project/showfiles.php?group_id=63137
UPGRADING
Just install POPFile on top of the currently installed version. But did you
read the ESSENTIAL READING above first.
DONATIONS
Thank you to everyone who has clicked the Donate! button and donated their
hard earned cash to me in support of POPFile. Thank you also to the people
who have contributed patches, feature requests, bug reports and
translations.
http://sf.net/forum/forum.php?forum_id=213876
PRAISE
This release sees an enormous improvement in the Windows installer which
is the exclusive playground of Brian Smith. Brian has worked tirelessly to
make the Windows installer robust, clear and easy. Nice work, Brian!
A new POPFile developer is Manni Heumann who has made a number of
contributions
to the release and wiki, the most significant of which is the
implementation of
the experimental IMAP module.
Another new developer, Joseph Connors, has been very active in the HTML
skinning of POPFile and has made major contributions to the POPFile
documentation on the Wiki.
Amatubu is always working to translate and expand the Wiki into Japanese and
Yasuo Honda is spent a great deal of time reviewing the Japanese Wiki
pages.
CONCLUSION
Keep the ideas and bug reports coming. If you are interested in knowing
more about what's planned for future POPFile versions (or just learning
about POPFile's history) visit the POPFile Roadmap:
http://sf.net/docman/display_doc.php?docid=17906&group_id=63137
John.
|
|
From: John Graham-C. <jg...@jg...> - 2004-08-03 18:06:20
|
v0.21.2 RELEASE This is a release containing a fix for a security flaw found in v0.21.1 and is identical to that release apart from the fix. More details of the flaw can be found here: http://sf.net/mailarchive/forum.php?thread_id=5248725&forum_id=12356 John. |
|
From: John Graham-C. <jg...@jg...> - 2004-08-02 13:55:16
|
Folks, Just under one hour ago one of POPFile's developers reported to me that they had discovered a flaw in the way in which we handle the delivery of GIF/PNG/ICO/CSS/HTML files through the internal web browser. The good news is that people using the default stealth mode configuration are NOT affected, the bad news is that we have found a security flaw (the last (and first!) one was in May 2003). I already have a patch for this and will be testing it today for release tomorrow or Wednesday. This will be v0.21.2, but I did not want to delay getting this information out to you. WHO IS AFFECTED? This will only affect you if: 1. You are running version v0.21.0 or v0.21.1 AND 2. You have DISABLED stealth mode (the default configuration is stealth mode which prevents anyone but you from connecting to your machine). WHAT CAN I DO TO PROTECT MYSELF RIGHT NOW? Go to the Security page and enable Stealth Mode for the UI. WHAT EXACTLY COULD AN ATTACKER DO? They could, if they create a special URL, retrieve aribtrary files from your machine (that POPFile has access to) that end with the extensions: .gif, .png, .ico, .css or .html. In the case of .html they'd also have to be in a directory with manual/ in the path. They would have to know the full path to the file, they cannot get a file listing or go searching. NEXT Sorry to have let this happen and I'll get a patch out ASAP that fixed the problem. I guess we'd better start numbering the advisories, so this is officially PSA #2, you can still read about PSA #1 here: https://sourceforge.net/mailarchive/forum.php?thread_id=2563459&forum_id=12356 John. |
|
From: John Graham-C. <jg...@jg...> - 2004-03-20 20:10:32
|
Folks, As you may be aware I've decided that the best way to get the "nice installable version of POPFile for the Mac" is to actually have a Mac in my hands and so I launched the "BuyJohnAMac" campaign which you can read about here: http://popfile.sourceforge.net/cgi-bin/wiki.pl?BuyJohnAMac To get a head start on the project I've created a special tracker for Mac users. This tracker is a combined bug and feature database and I'd love Mac users of POPFile to start submitting reports there about specific issues or desires around POPFile on the Mac (obviously use the regular trackers for more general bugs or non-Mac specific features). Once the project is complete I'll get rid of this tracker and we'll just use the regular ones, but for the time being they make a convenient place to report things like problems accessing the UI with Safari, or reasons POPFile doesn't work well on Jaguar, etc. You can find that tracker here: http://sourceforge.net/tracker/?atid=639935&group_id=63137&func=browse Thanks, Mac users, for your assistance in educating me! John. |
|
From: John Graham-C. <jg...@jg...> - 2004-03-19 14:07:00
|
Changes in v0.21.1 This release was prompted by a number of users who saw very long corpus conversion times when upgrading to v0.21.0. This has been fixed along with a small number of others things: 1. Updated Arabic translation 2. Updated installer to handle corpus upgrade problems better 3. Logger always uses system specific line endings 4. A side effect of fixing the upgrade slowness is that reclassification will be faster. 5. Made FAQ, ID and (stealth mode) localizable 6. Make old bucket detection more robust 7. Fix problem on NextStep with directory creation 8. Fix slurp_ so that it doesn't break CRLF on slow links. All of the information in the v0.21.0 release notes still applies, you can read them here: http://sourceforge.net/forum/forum.php?forum_id=358366 Because of #8 above I'm recommending this as a good upgrade for everyone using v0.21.0 as well as users of older versions. John. |
|
From: John Graham-C. <jg...@jg...> - 2004-03-10 22:10:46
|
A number of people have reported ridiculously long times for the upgrade from per-v0.21.0 versions of POPFile. Brian Smith put together a simple test case to demonstrate the problem, and I've fixed it. So, I'm suggesting that if you have a corpus with GREATER THAN 30,000 unique words (you can figure this out from the Buckets page) please wait for v0.21.1. I am anticipating that we'll get the v0.21.1 out this weekend. Apologies for the problem. John. |
|
From: John Graham-C. <jg...@jg...> - 2004-03-09 14:55:53
|
Welcome to POPFile v0.21.0 This version consists of a major update to v0.20.1 with many improvements and bug fixes: 1. Multi-user phase 1 POPFile now recognizes two environment variables (POPFILE_ROOT and POPFILE_USER) which can be used to customize the location of POPFile and the location of its per-user data. POPFile *no longer* has to be run with the current working directory set to where popfile.pl is installed. POPFILE_ROOT: this is the full path to the popfile.pl file. Once set you can run POPFile from any directory and it will find its modules using POPFILE_ROOT POPFILE_USER: this is where POPFile will keep its per-user data (i.e. the database and history). This means that a single installation of POPFile can be used with multiple users. Just specify a different directory in POPFILE_USER for each user and run POPFile once per user. In a future release POPFile will support a single instance of POPFile with multiple users sharing the same database (this is known as Multi-user Phase 2 and you can read more in the POPFile Roadmap---see below for link) 2. Switch to SQL database In v0.20.0 POPFile switched from storing all its information in flat files to using BerkeleyDB. BerkeleyDB has proved to be not stable enough on Windows 98 in certain cases, for that reason and to prepare for Multi-user Phase 2, the underlying data has been moved into an SQL database. POPFile currently by default uses SQLite but can be successfully used with other SQL databases like MySQL. (SQLite can be found on the web at http://www.sqlite.com/ and is public domain software.) The schema for POPFile can be found in Classifier/popfile.sql for those that want to load it into their own SQL database. If you start using POPFile with a database we don't currently support please let me know (e.g. anyone want to try Oracle?). POPFile *should* work with any SQL database supported by Perl's DBI mechanism. 3. UI improvements The global options for Subject Modification, X-Text-Classification insertion and X-POPFile-Link insertion have been removed and replaced with individual options on a per bucket basis to give greater choice in configuring POPFile. IMPORTANT NOTE: Because of the elimination of the global parameters you will need to check and set/unset the individual bucket parameters for Subject Modification, X-Text-Classification and X-POPFile-Link. Visit the Buckets page to configure each bucket to your liking. The 'unclassified' bucket is now visible in the UI so that you can see how many messages were unclassified, and configure header modification. This also means that unclassified messages are counted in the accuracy statistics; previously they were not counted which could have skewed the accuracy statistics if there were unclassified messages. The history "page" bar has been simplified so that it uses a fixed amount of screen space, while making navigation easy. Filters and searches on the history are now persistent, for example you can click on the Buckets page and return to the History page without losing your filter or search settings. The UI password is now stored as a one-way hash and is no longer ever displayed or stored in plain text. The Buckets page has been modified to only show the 'distinct word' count per bucket and to (finally :-) show the total number of distinct words in the database. Previously we showed two counts with confusing titles: now we show the true number of words in the database, not the "word counts" (which was the number of times each word occurred). Another language has been added to the list of interface localizations: POPFile is now available in our second right-to-left language, Arabic. 4. Change to command line If you are using the command line to configure POPFile there has been a major change in the way it is parsed. The old style (for example, perl popfile.pl -pop3_port 110) has been deprecated and replaced with a proper getopt style command line. We now have a single -- parameter: --set which is used to set POPFile configuration options on the command line. For example 'perl popfile.pl -pop3_port 110' would be replaced with 'perl popfile.pl --set pop3_port=110'. Note that existing scripts will continue to work since the old style is merely deprecated, but they should be upgraded as support for the old style is not guaranteed for all future versions. 5. Enable/disable modules Each POPFile module can now be disabled or enabled with command line options: for example if you don't need the optional XMLRPC module loaded it's possible to specify --set xmlrpc_enabled=0 and it will be unloaded. 6. Anti-spam improvements We've recently seen spam start to use CSS to obscure messages and fool filters like POPFile; in response, this version of POPFile does analysis of CSS in HTML encoded messages. POPFile now correctly uses the SpamAssassin headers to make POPFile more efficient when used in conjunction with SpamAssassin. We now also look at TLDs (Top Level Domains) and store them as pseudowords (most useful for TLDs like .biz). It's possible that you might see a drop in accuracy as your corpus gets trained up on the new anti-spam features. This drop in accuracy will be corrected once you've retrained POPFile a little. 7. POPFile Documentation Project In addition since v0.20.1 a lot of work has gone into a set of web pages called the 'POPFile Documentation Project' that includes FAQs, setup guides, etc., please visit: http://popfile.sf.net/cgi-bin/wiki.pl ESSENTIAL READING IF YOU ARE UPGRADING TO v0.21.0 1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy the entire POPFile directory somewhere. You can then safely install POPFile v0.21.0 on top of your current installation. The installer will automatically create a backup of your existing corpus and configuration file; I just think a full back up is a sensible precaution. 2. IF YOU ARE RUNNING WINDOWS: Please read the section below I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY 3. ON WINDOWS POPFILE IS AN EXE. Windows users will now be able to see POPFile running in the Task Manager with an executable called popfileXX.exe where the XX is one of f, if, b, ib depending on configuration. POPFile is started by running runpopfile.exe which chooses the appropriate popfileXX.exe This might cause your firewall to ask about giving popfileXX.exe permissions, in addition if you had allowed Perl permissions in your firewall they are NO LONGER needed. I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY Because the time taken to start a new process on Windows is long under Perl there is an optimization for Windows that is active by default: when a new connection is made between your email program and POPFile, POPFile handles it in the 'parent' process. This means that the connect happens fast and mail starts downloading very quickly, but is means that you can only downloaded messages from one server at a time (up to 6 other connections will be queued up and dealt with in the order they arrive) and the UI is unavailable while downloading email. You can turn this behavior off (and get simultaneous UI/email access and as many email connections as you like) by going to the Configuration panel in the UI and making sure that "Allow concurrent POP3 connections:" is set to Yes, or by specifying --set pop3_force_fork=1 on the command line. SPECIAL NOTES FOR WINDOWS USERS POPFile works best if the POPFILE_ROOT and POPFILE_USER environment variables use lowercase short file name format. For example, instead of using C:\Program Files\POPFile you should use c:\progra~1\popfile. The use of lower case is *very important* if an existing flat file or BerkeleyDB corpus is to be converted. The short file name format is not very user friendly, so the Windows installer makes it easy to create suitable environment variables. The Windows installer assumes that each POPFile user will have different POPFILE_USER settings, and provides some simple utilities to make it easy to to create these new environment variables. When the installer is run, it automatically creates the necessary lowercase short file name format environment variables for the user running the installer. This data is stored in a user-specific part of the registry. The "Run POPFile" shortcuts created by the installer in the Start Menu (and, optionally, in the StartUp folder) start the 'runpopfile.exe' program which is responsible for ensuring that POPFILE_ROOT and POPFILE_USER have the correct values before calling the 'popfile.exe' program which starts POPFile. If 'runpopfile.exe' is run by a user who does not yet have any POPFile settings in the registry, the 'Add POPFile User' wizard will be started if it is available. [This wizard is only available if POPFile was installed by a user with 'Admin' rights] This wizard lets the user choose a location for their POPFile data and offers to reconfigure any suitable Outlook Express, Outlook or Eudora email accounts for use with POPFile. If any email settings are changed, the wizard will save the data needed to restore the original settings. After this, the wizard will start POPFile by calling 'popfile.exe'. Windows 9x supports system-wide environment variables which are defined in AUTOEXEC.BAT. Normally to change the value of these environment variables AUTOEXEC.BAT has to be updated and the computer rebooted. The 'runpopfile.exe' program avoids the need to reboot Windows 9x systems in order to switch between different sets of user data (users just need to log on and log off as normal). The 'Add POPFile User' wizard only offers some simple features at present, as it is still under development. For example there is very little error checking in the current version. I AM USING THE CROSS PLATFORM VERSION POPFile requires a number of Perl modules that are available from CPAN. New in v0.21.0 are the need for the following: DBI DBD::SQLite Digest::Base Digest::MD5 HTML::Tagset You may also need: BerkeleyDB (if you are upgrading from v0.20.x) Text::Kakasi (if you want Japanese language support) Encode (if you want Japanese language support) Notes on installing the cross platform version can be found here: http://popfile.sf.net/cgi-bin/wiki/pl?HowTos/CrossPlatformInstall I LIKE TO LIVE DANGEROUSLY In a future version POPFile will add official support for message classification through the SMTP and NNTP (Usenet news) protocols. There are currently proxy modules for these protocols that work with v0.21.0, but they have not been fully tested. If you are interested in getting them get them here: http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/SMTP.pm?rev=1.26 http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/NNTP.pm?rev=1.25 and place them in POPFile's Proxy/ directory. DOWNLOADING You can obtain the latest releases of POPFile by visiting http://sourceforge.net/project/showfiles.php?group_id=63137 UPGRADING Just install POPFile on top of the currently installed version. But did you read the ESSENTIAL READING above first. DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed patches, feature requests, bug reports and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 As well as the usual donations we have a specific "BuyJohnAMac" campaign that you can read about here: http://popfile.sf.net/cgi-bin/wiki.pl?BuyJohnAMac PRAISE This release sees an enormous improvement in the Windows installer which been the exclusive playground of Brian Smith. Brian has work tirelessly to make the Windows installer robust, clear and easy. Nice work, Brian! Sam continues to make contributions to POPFile's core, with the latest being his addition of the CSS parser. Naoki Iimura has done a huge amount of work converting the FAQ from English to Japanese for the benefit of the many users of POPFile in Japan. THE FUTURE The rough plan for the future of POPFile is the as follows: v0.22.0 Theme: Protocols/UI Add support for SMTP, NNTP, IMAP(?) New skinning system Support SSL and SOCKS for all proxies Move history into database v0.23.0 Theme: Multi-user Phase 2 Support for multiple users with logins Scalable to large organizations/ISPs History encryption v1.00.0 Theme: Stable Release First non v0.X release of POPFile The full roadmap can be found here: http://sourceforge.net/docman/display_doc.php?docid=17906&group_id=63137 CONCLUSION Keep the ideas and bug reports coming. If you are interested in knowing more about what's planned for future POPFile versions (or just learning about POPFile's history) visit the POPFile Roadmap: http://sourceforge.net/docman/display_doc.php?docid=17906&group_id=63137 John. |
|
From: John Graham-C. <jg...@jg...> - 2003-11-04 23:33:29
|
INTRODUCTION
v0.20.1 is a minor update on v0.20.0 to address the performance problems
that some
people were seeing with the introduction of the BerkeleyDB for storage
of the corpus.
v0.20.1 sets the BerkeleyDB cache size (i.e. the amount of the corpus
that is kept
in memory) to 64KB and it can be controlled through the bayes_db_cache_size
parameter if needed. The default should work for everyone, you can set
this to a
higher value to keep more of the database in memory at the cost of using
more
memory.
There are also a number of bug fixes in v0.20.1:
1. If you set a password and used X-POPFile-Link you were unable to get past
the password screen when clicking on an XPL. (bug #831472)
2. In some instances when the connection between the POP3 proxy and the mail
server was lost it was possible to get in a deadlock where POPFile was
waiting for a command from the client and the client was waiting for a
response from POPFile. (bug #829947)
3. It was not possible to add pseudowords (e.g. header:From) as ignored
words,
now it is. (bug #827785)
4. Magnets that included $ or @ could be incorrectly matched when the @
or $
was followed by __. (bug #826765)
ESSENTIAL READING IF YOU ARE UPGRADING TO v0.20.1
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
the entire POPFile directory somewhere. You can then safely install
POPFile v0.20.1 on top of your current installation; I just think a
back up
is a sensible precaution.
2. IF YOU ARE RUNNING WINDOWS: Please read the section below I AM RUNNING
WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY
3. ON WINDOWS POPFILE IS NOW AN EXE. Windows users will now be able to see
POPFile running in the Task Manager with an executable called
popfileXX.exe
where the XX is one of f, if, b, ib depending on configuration. POPFile
is started by running popfile.exe which chooses the appropriate
popfileXX.exe
This might cause your firewall to ask about giving popfileXX.exe
permissions,
in addition if you had allowed Perl permissions in your firewall
they are
NO LONGER needed.
4. The installer will cause POPFile to run in the foreground if the database
upgrade is required so that the upgrade process is evident to the user.
Once upgraded you can switch to background my going to the Configuration
tab and changing "Run POPFile in a console window?" to No.
I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS
SIMULTANEOUSLY
Because the time taken to start a new process on Windows is long under Perl
there is an optimization for Windows that is present by default: when a new
connection is made between your email program and POPFile, POPFile
handles it
in the 'parent' process. This means that the connect happens fast and mail
starts downloading very quickly, but is means that you can only downloaded
messages from one server at a time (up to 6 other connections will be queued
up and dealt with in the order they arrive) and the UI is unavaiable while
downloading email.
You can turn this behavior off (and get simultaneous UI/email access and as
many email connections as you like) by going to the Configuration panel in
the UI and making sure that "Allow concurrent POP3 connections:" is set to
Yes, or by specifying -pop3_force_fork 1 on the command line.
I AM USING THE CROSS PLATFORM VERSION
POPFile requires a number of Perl modules that are available from CPAN. New
in v0.20.x are the need for the following:
BerkeleyDB
Text::Kakasi (if you want Japanese language support)
Encode (if you want Japanese language support)
I LIKE TO LIVE DANGEROUSLY
In a future version POPFile will add official support for message
classification
through the SMTP and NNTP (Usenet news) protocols. There are currently
proxy
modules for these protocols that work with v0.20.1, but they have not been
fully tested. If you are interested in getting them get them here:
http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/SMTP.pm?rev=1.22
http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/NNTP.pm?rev=1.21
and place them in POPFile's Proxy/ directory.
DOWNLOADING
You can obtain the latest releases of POPFile by visiting
http://sourceforge.net/project/showfiles.php?group_id=63137
UPGRADING
Just install POPFile on top of the currently installed version. But did you
read the ESSENTIAL READING above first.
INTERNATIONALIZATION
This minor update includes localization into Turkish.
DONATIONS
Thank you to everyone who has clicked the Donate! button and donated their
hard earned cash to me in support of POPFile. Thank you also to the people
who have contributed patches, feature requests, bug reports and
translations.
http://sourceforge.net/forum/forum.php?forum_id=213876
CONCLUSION
Keep the ideas and bug reports coming. If you are interested in knowing
more about what's planned for future POPFile versions (or just learning
about POPFile's history) visit the POPFile Roadmap:
http://sourceforge.net/docman/display_doc.php?docid=17906&group_id=63137
John.
|
|
From: John Graham-C. <jg...@jg...> - 2003-10-17 02:41:52
|
INTRODUCTION v0.20.0 is a major update to POPFile with the focus being on performance. In addition POPFile makes another leap forward in support for non-English languages with many new UI translations (including our first right to left language) and full support for parsing of Japanese and Korean. v0.20.0 is intended to end-of-life v0.18.x and v0.19.x. All future development work will be occurring around the v0.20.0 line of code and no bugs will be fixed against previous versions. A lot of work has gone into making v0.20.0 the version of POPFile to have. To improve POPFile's performance the following changes have been made: 1. The corpus (where the word lists are stored) has been changed from a flat text file to a BerkeleyDB database. When you run POPFile v0.20.0 for the first time you will see your existing corpus get automatically upgraded to the BerkeleyDB. (See the 'license' file for details of BerkeleyDB's license) The use of the database both speeds up POPFile (especially the reclassification process which has slowed down in v0.19.x) and reduces its memory requirements. The time to load the corpus has now gone from minutes/seconds to close to zero. 2. The history data is cached between sessions. If you regularly start and stop POPFile (e.g. you start and stop your PC every day) then you'll notice another load-time speedup: the history data is being cached between POPFile sessions to make loading faster. 3. The history is progressively updated. As messages are downloaded from a server POPFile used to store all the messages for insertion into the history the next time the history was viewed. Now as messages are downloaded they are inserted in to the history progressively. 4. When viewing a colored message you'll notice a big speedup because there were previously two scans through the message (one for classification and one for coloring) this is now reduced to one. 5. When downloading messages we previously saved the message to disk and then reloaded it for classification. Now the mail parser has been modified so that the text of the message is streamed into it as it is read from the mail server and classification happens in line without the need to reread from disk. 6. On Windows the default configuration for the proxy is to no longer fork() the server. This means that downloading mail starts very quickly, but has the downside that only one email account can be checked at a time and the UI cannot be used during download. This new option is controlled by a configuration parameter (-pop3_force_fork) and through the UI. On non-Windows platforms POPFile will fork() each new connection. To improve POPFile's stability: There's been a huge effort to write a complete test suite for POPFile. Currently we have tests that cover 99% of POPFile's code (i.e. almost every line of code is exercised by a test) and the plan going forward is to try to keep it that way. The test suite exercises the UI as if it were a user clicking buttons and submitting forms, it includes a complete POP3 server and client so that the proxy functionality can be tested and contains hundreds of tests for mail parsing. Every module has an equivalent TestFOO.tst that tests it, if you are interested in running the test suite get the tests/ directory from CVS and run 'gmake test'. To improve POPFile's accuracy: A number of bugs have been fixed that sometimes caused POPFile internally to get the right classification and then insert the wrong headers. The mail parser has been updated with the latest spammers' tricks and new pseudowords and we've done an experiement with 'unsure' classifications and decided to ship with code that will mark a message as 'unclassified' if it isn't 100 times more certain it's in bucket A than bucket B. This should reduce the false positive rate a little at the expense of POPFile saying it's not sure. ESSENTIAL READING IF YOU ARE UPGRADING TO v0.20.0 1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy the entire POPFile directory somewhere. You can then safely install POPFile v0.20.0 on top of your current installation; I just think a back up is a sensible precaution. 2. IF YOU ARE RUNNING WINDOWS: Please read the section below I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY 3. ON WINDOWS POPFILE IS NOW AN EXE. Windows users will now be able to see POPFile running in the Task Manager with an executable called popfileXX.exe where the XX is one of f, if, b, ib depending on configuration. POPFile is started by running popfile.exe which chooses the appropriate popfileXX.exe This might cause your firewall to ask about giving popfileXX.exe permissions, in addition if you had allowed Perl permissions in your firewall they are NO LONGER needed. 4. The installer will cause POPFile to run in the foreground if the database upgrade is required so that the upgrade process is evident to the user. Once upgraded you can switch to background my going to the Configuration tab and changing "Run POPFile in a console window?" to No. I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY Because the time taken to start a new process on Windows is long under Perl there is an optimization for Windows that is present by default: when a new connection is made between your email program and POPFile, POPFile handles it in the 'parent' process. This means that the connect happens fast and mail starts downloading very quickly, but is means that you can only downloaded messages from one server at a time (up to 6 other connections will be queued up and dealt with in the order they arrive) and the UI is unavaiable while downloading email. You can turn this behavior off (and get simultaneous UI/email access and as many email connections as you like) by going to the Configuration panel in the UI and making sure that "Allow concurrent POP3 connections:" is set to Yes, or by specifying -pop3_force_fork 1 on the command line. I AM USING THE CROSS PLATFORM VERSION POPFile requires a number of Perl modules that are available from CPAN. New in v0.20.0 are the need for the following: BerkeleyDB Text::Kakasi (if you want Japanese language support) Encode (if you want Japanese language support) I LIKE TO LIVE DANGEROUSLY In a future version POPFile will add official support for message classification through the SMTP and NNTP (Usenet news) protocols. There are currently proxy modules for these protocols that work with v0.20.0, but they have not been fully tested. If you are interested in getting them get them here: http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/SMTP.p m?rev=1.22 http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/NNTP.p m?rev=1.21 and place them in POPFile's Proxy/ directory. DOWNLOADING You can obtain the latest releases of POPFile by visiting http://sourceforge.net/project/showfiles.php?group_id=63137 UPGRADING Just install POPFile on top of the currently installed version. But did you read the ESSENTIAL READING above first. INTERNATIONALIZATION POPFile's support for non-English languages has grown and the UI is now localized into 26 languages: Bulgarian Chinese (simplified) Chinese (traditional) NEW Czech Danish Dutch English English (UK) Finnish French German NEW Greek NEW Hebrew Hungarian NEW Italian NEW Japanese Korean Norwegian NEW Polish NEW Portugese (Iberian) Portugese (Brazilian) Russian Slovak Spanish Swedish Ukrainian Also added support for understanding Japanese and Korean and doing word splitting correctly. DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed patches, feature requests, bug reports and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 CONCLUSION Keep the ideas and bug reports coming. If you are interested in knowing more about what's planned for future POPFile versions (or just learning about POPFile's history) visit the POPFile Roadmap: http://sourceforge.net/docman/display_doc.php?docid=17906&group_id=63137 John. |
|
From: John Graham-C. <jg...@jg...> - 2003-07-15 21:53:02
|
INTRODUCTION
Today's release consists of two maintenance releases to v0.18.2 and
v0.19.0.
A special thank you goes out to Sam Schinke who continues to contribute
time and energy to POPFile.
I recommend v0.19.1 because of the many improvements, but v0.18.3 is the
STABLE release for those who want rock solid reliability and want others to
do the bug hunting on the new v0.19.x line :-)
ESSENTIAL READING IF YOU ARE UPGRADING TO v0.19.1 from v0.18.x
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
the entire POPFile directory somewhere. You can then safely install
POPFile v0.19.1 on top of your current installation; I just think a back
up
is a sensible precaution.
2. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in
the
mail parser it is possible that you might see accuracy drop initially and
you may find yourself reclassifying a few messages that used to work.
This
is unfortunate but necessary to make POPFile even more accurate than
before
and v0.19.1 incorporates changes that make POPFile's classification
accuracy better; however old corpuses might need a little retraining.
DOWNLOADING
You can obtain the latest releases of POPFile by visiting
http://sourceforge.net/project/showfiles.php?group_id=63137
UPGRADING
Just install POPFile on top of the currently installed version. But did you
read the ESSENTIAL READING above first if you are upgrading from a pre
v0.19.1
version?
FAQ
zonk3r has spent a great deal of time on a POPFile FAQ. Please check it out
as it covers many questions that you might have:
http://sourceforge.net/docman/display_doc.php?docid=14421&group_id=63137
THE GORY DETAILS FOR v0.18.3 and v0.19.1
1. Fixed a bug causing long message download to fail.
Reports: 753940 (mfichtner) (fixed by sschinke)
2. Provided a workaround for incompatibilties caused by odd HTTP headers
from
Norton Internet Security.
Reports: Many (fixed by jgrahamc)
3. Fixed a case that could prevent delivery of email
Reports: Via forums and email. (fixed by sschinke)
DONATIONS
Thank you to everyone who has clicked the Donate! button and donated their
hard earned cash to me in support of POPFile. Thank you also to the people
who have contributed patches, feature requests, bug reports and
translations.
http://sourceforge.net/forum/forum.php?forum_id=213876
CONCLUSION
Keep the ideas and bug reports coming.
John.
|
|
From: John Graham-C. <jg...@jg...> - 2003-06-11 15:10:44
|
INTRODUCTION
For the first time in POPFile history we are releasing two versions at the
same time. Please take a moment to decide which version is most appropriate
for you.
v0.18.2 is a minor bug fix update to the last v0.18.1 made in February and
contains no new features. It is intended for conservative users who are
happy with POPFile as it stands and would just like bug fixes and nothing
else.
v0.19.0 is a major update to v0.18.1 and include everything that is fixed
in v0.18.2 plus much more. Full details are below.
A special thank you goes out to Sam Schinke who has contributed a great
deal to this release particularly with his work fixing outstanding bugs and
porting them from v0.19.0 and v0.18.2. Nice work Sam.
Brian Smith also needs your applause for major updates to the Windows
installer
making it easier than ever to install and configure POPFile on Windows.
I recommend v0.19.0 because of the many improvements, but v0.18.2 is the
STABLE release for those who want rock solid reliability and want others to
do the bug hunting on the new v0.19.x line :-)
(Of course if you are really conservative you might like to download and use
v0.18.1 which has been in use by many people since February 27).
ESSENTIAL READING IF YOU ARE UPGRADING TO v0.19.0
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
the entire POPFile directory somewhere. You can then safely install
POPFile v0.19.0 on top of your current installation; I just think a back
up
is a sensible precaution.
2. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in
the
mail parser it is possible that you might see accuracy drop initially and
you may find yourself reclassifying a few messages that used to work.
This
is unfortunate but necessary to make POPFile even more accurate than
before
and v0.19.0 incorporates changes that make POPFile's classification
accuracy better; however old corpuses might need a little retraining.
DOWNLOADING
You can obtain the latest releases of POPFile by visiting
http://sourceforge.net/project/showfiles.php?group_id=63137
UPGRADING
Just install POPFile on top of the currently installed version. But did you
read the ESSENTIAL READING above first if you are upgrading from a pre
v0.19.0
version?
FAQ
zonk3r has spent a great deal of time on a POPFile FAQ. Please check it out
as it covers many questions that you might have:
http://sourceforge.net/docman/display_doc.php?docid=14421&group_id=63137
THE GORY DETAILS FOR v0.18.2
1. Fixed bug where -toptoo (now -pop3_toptoo) would cause duplicates in
the history.
Reports: 701390 () (fixed by sschinke)
2. Fixed a bug where -toptoo resulted in message overwriting and the wrong
message classification.
Reports: 705448, 694002 () (fixed by sschinke)
3. Fix warning on unclassified messages
Reports: 697278 (jdeifik)
4. Remove lines in headers consisting of only whitespace (Eudora has
trouble with them).
Patch: esniper
Reports: 701981 (esniper)
5. Fix colorized display of quoted-printable HTML
Reports: 699098 (mfichtner)
6. Recognize protocol-less href's
Reports: biljir
7. Re-work of header handling to handle multi-line headers. This will make
our handling of MIME messages much more robust.
Reports: 695565 (biljir), 702215 (gdvieira), 702316 (spf)
8. Fixed bug preventing decoding when headers are in an unexpected order
Reports: 729551 (fibrizo)
9. Allow "+" to be submitted encoded in a form
Reports: 719989 (thedonga)
10. Fixed unrequested History deletion, history doesn't go back far enough
Reports: 703364 (besonen), 708387 (beej) (and others)
11. Fixed XPL problems with filtered history view
Reports: 697046 (helphand), 692673 (nobody)
12. Restored main history form to POST
Reports: 690451 (nobody)
13. Fixed reporting of remaining child pid's
Reports: none
14. Make hostname decoding case-insensitive
Reports: none
15. Fix odd return value from classify_file for some messages:
"unclassified"
Reports: none
THE GORY DETAILS FOR v0.19.0
All of the above plus... (sschinke had a hand in lots of this too)
1. Large overhaul of the magnets system so that we add Cc magnets, make
magnets editable from the Magnets page, and a new feature called
QuickMagnets which allows you to create magnets from received emails
by selecting parts of the From, To, Cc and Subject lines.
Requests: 676341 (reason1000)
2. When an email is clicked on to see the colored version a new page
appears with complete coloring (including for the pseudowords which
now have popups on the colored areas), a new full dump of the
probabilities used in the calculation of the message classification.
Requests: lost track
3. On the Buckets page there are new statistics counting the number of
errors made by bucket. For each bucket we have the number of false
positives (i.e. mails that went in the bucket that should not have)
and the number of false negatives (i.e. mails that should have been
in that bucket but were not).
Requests: 692600 (nobody),
In addition when clicking on an individual bucket to look at the words
in it a new page appears with a clickable index for speed and words
are sorted by frequency so that you can see the most important words
in each bucket.
Requests: 691386 (philiplaw)
From the bucket page it is possible to remove all the words in a bucket
so that you can start retraining a bucket without having to delete
and reinsert it.
Requests: 675983 (nobody)
4. The Configuration page is now pluggable so that as new modules are
written their elements appear in the configuration page automatically.
Requests: none
There is a link to the current log file so that you can access it
from within the browser.
Requests:
5. The Security page is now pluggable so that as new modules are
written their elements appear in the configuration page automatically.
Requests: none
6. The Advanced tab no longer uses a hard coded set of ignored words,
they are now stored in a file and can be edited through the UI.
Requests: lost track
7. POPFile's internal structure has undergone a large change to make
use of Perl's object oriented features and there is now a common
base class for all POPFile modules (called POPFile::Module) and as
a result many of the POPFile modules have been simplified greatly.
An offshoot of this is that there are a number of experimental modules
that you can obtain only if you get the CVS version:
SMTP.pm - SMTP proxy that does SMTP mail classification
NNTP.pm - NNTP proxy that does NNTP Usenet news post
classification
XMLRPC.pm - Full access to POPFile's API (see Classifier::Bayes)
through XML-RPC
The XMLRPC available methods are (documentation in Classifier::Bayes):
classify_file
classify_and_modify
get_buckets
get_bucket_word_count
get_bucket_word_list
get_word_count
get_bucket_unique_count
get_bucket_color
set_bucket_color
get_bucket_parameter
set_bucket_parameter
get_html_colored_message
create_bucket
delete_bucket
rename_bucket
add_message_to_bucket
remove_message_from_bucket
get_buckets_with_magnets
get_magnet_types_in_bucket
clear_bucket
clear_magnets
get_magnets
create_magnet
get_magnet_types
delete_magnet
get_stop_word_list
add_stopword
remove_stopword
Requests: none
8. Numbers are now legal in bucket names.
Requests: 720681 (deaper)
9. The characters used around the classification in subject modification
can now be reconfigured with options -bayes_subject_mod_left and
-bayes_subject_mod_right and they default to [ and ]
Requests: 722837 (stanleyspanner)
10. All command line parameters have changed name. The old names all work
correctly and are upgraded automatically,
Old Parameter New Parameter
corpus bayes_corpus
unclassified_probability bayes_unclassified_probability
piddir config_piddir
debug GLOBAL_debug
ecount GLOBAL_ecount
mcount GLOBAL_mcount
msgdir GLOBAL_msgdir
subject GLOBAL_subject
imeout GLOBAL_timeout
xpl GLOBAL_xpl
xtc GLOBAL_xtc
download_count GLOBAL_download_count
logdir logger_logdir
localpop pop3_local
port pop3_port
sport pop3_secure_port
server pop3_secure_server
separator pop3_separator
toptoo pop3_toptoo
archive html_archive
archive_classes html_archive_classes
archive_dir html_archive_dir
history_days html_history_days
language html_language
last_reset html_last_reset
last_update_check html_last_update_check
localui html_local
page_size html_page_size
password html_password
send_stats html_send_stats
skin html_skin
test_language html_test_language
update_check html_update_check
Requests: none
11. There is now a Reclassify button at the top of the History page
as well as at the bottom.
Requests: 727834 (garowetz), 687138 (johnmccurdy)
12. If you do a search that results in no results there is now a search box
for you to search again.
Requests: 685327 (tomvoss)
13. The Windows installer now guides you in the creation of an initial set
of buckets.
Requests: 691348 (pkarlin)
14. Each time the history cache is reloaded we insert a marker line in the
UI,
these markers delimit the times the user reloaded the history with new
messages
and provide a useful marker for using POPFile throughout the day.
Everything
before the last marker is newly received.
Requests: none
15. POPFile now has 19 UI translations available in the following languages:
Bulgarian
Chinese (simplified)
Chinese (traditional)
Danish
Dutch
English
English (UK)
Finnish
French
German
Hungarian
Korean
Norwegian
Portugese (Brazilian)
Russian
Slovak
Spanish
Swedish
Ukrainian (http://zope.net.ua/POPFile)
Requests: none
16. To help in the fight against spam a number of new pseudowords have been
added to track spams that load images from across the web and web bugs,
the
complete list of pseudowords is now:
encoding:<mime encoding type>
header:<email header>
html:numericentity
html:td
html:imgremotesrc
html:imgwidth<pixels>
html:imgheight<pixels>
html:fontsize<size>
html:encodedurl
html:comment
html:authorization
trick:spaceout
trick:dottedwords
trick:invisibleink
Requests: none
17. Numerous modifications by Sam and I to the mail parser, MIME handling,
HTML
parser, colorizer, Base 64 and Quoted Printable decoder to improve
accuracy.
Requests: lost track
18. Windows users will discover that POPFile adds an icon to the system tray
with a popup menu (right click) that offers to take you to the POPFile
UI
or to shutdown POPFile. A double click on the icon navigates to the
POPFile
UI.
Requests: lost track
19. Sorting of columns in the History can be set to ascending or descending
by clicking on the column header. There's a new indicator that tells
you
which column is sorted and in what order.
Requests: none
20. Fixed security problem with the X-POPFile-Link functionality and
passwords.
Requests: none
21. Added new view of the last 10 items in the log on the configuration
page.
Requests: none
22. Added code to detect problem entries in the corpus when loading and
remove
them from the in memory copy. Currently only detects entries with
multiple
spaces.
Requests: none
DONATIONS
Thank you to everyone who has clicked the Donate! button and donated their
hard earned cash to me in support of POPFile. Thank you also to the people
who have contributed patches, feature requests, bug reports and
translations.
http://sourceforge.net/forum/forum.php?forum_id=213876
CONCLUSION
Keep the ideas and bug reports coming.
John.
|
|
From: John Graham-C. <jg...@jg...> - 2003-05-24 23:29:21
|
The password vulnerability I previously reported is being patched (by Sam) for a small v0.18.2 release, but to get the fix in the hands of people ASAP I am releasing a tiny patch that removes the problem. This patch applies to the v0.18.1 Windows and Cross Platform versions and consists of a replacement for the file HTML.pm which is found in the POPFile directory in the subdirectory called UI. 1. Locate the file HTML.pm on your computer. For Windows users this is usually in C:\Program Files\POPFile\UI. It is always in the UI subdirectory of the directory POPFile was installed in. 2. Obtain the patch here: http://popfile.sourceforge.net/HTML.pm and save it as HTML.pm on your computer. 3. Shutdown POPFile 4. Copy the HTML.pm you downloaded on top of the HTML.pm that was on your machine. 5. Start POPFile 6. Visit any page on the POPFile UI and verify that at the bottom it says v0.18.1 (PATCHED). The word (PATCHED) indicates that you have installed this security patch. To read details of the vulnerability go here: http://sourceforge.net/forum/forum.php?thread_id=872438&forum_id=213099. Note the MOST users DO NOT need this patch and can safely wait for v0.18.2 or v0.19.0. Please read the description of the problem to determine if you need it. John. |
|
From: John Graham-C. <jg...@jg...> - 2003-05-23 22:09:30
|
Folks, I've discovered a security problem in the way in which we handle the UI password which could affect you if you have altered POPFile's configuration from the standard "stealth mode" and are using the UI password. I have already made a fix and it is being tested and I will release a v0.18.2 very soon to patch the problem. But I didn't want to act like some big company and wait until we had the fix ready before warning you. The good news is that this will affect only a small percentage of POPFile users, the bad news is that there's a security problem at all. I hate that! WHEN THIS WOULD AFFECT YOU This is only a problem if: 1. You have DISABLED stealth mode for the UI. (By default stealth mode is always on, so by default users are not affected by this problem.) AND 2. You have SET a UI password. (By default the UI is not password protected). HOW THIS WOULD AFFECT YOU It is possible to create a URL that when sent to POPFile would reveal the internal session key which can then be used to bypass the password check and get access to POPFile's features as if you knew the password. If a malicious person figured out the special URL and then pointed their browser at your POPFile and went to http://your_machine/special_url they could obtain the session key. MY RECOMMENDATION If you are using non-stealth mode with a UI password then: 1. Do not expose that POPFile out on the public Internet, it would be vulnerable to someone getting access to your mail. 2. If you are not exposing on the Internet then make sure that you trust people on your internal LAN. AN APOLOGY And lastly an apology from me for letting this happen. This should not have been able to creep in and I am very sorry to have to report it. I am sending this advisory as soon as possible after discovering the problem because I want to be totally honest with users about this, the first ever, POPFile security problem. John. |
|
From: John Graham-C. <jg...@jg...> - 2003-02-28 04:02:59
|
INTRODUCTION v0.18.0 was a major update and inevitably some bugs crept in that we = didn't detect prior to shipping. Happily they have been fixed, and even more=20 happily the regression test suite now has tests for many of them so that = they will never reoccur. v0.18.1 fixes these bugs (see THE GORY DETAILS = below for the gory details) and also has two significant changes: the=20 caching code associated with the History page has been totally rewritten by me so that the History barely slows down as the number of messages in the History increases; the Windows installer can now configure Outlook and Outlook Express Internet email accounts for you (and undo the = changes if you uninstall POPFile). POPFile v0.18.1 is also the first version where the UI meets the Bobby = AA rating for accessibility. This means that not only does POPFile not=20 discriminate against particular operating system, or natural languages, or ways you want to sort your mail, but it is also accessible by = everyone regardless of the specific challenges they face. (FOr more details on this drop by http://bobby.watchfire.com/bobby/html/en/about.jsp) Aside: If you are young, fit, and computer literate you might wonder why = I=20 bothered to make Bobby-level accessibility a requirement. Early on in=20 POPFile's development a user told me that POPFile was a life changer. = For=20 him, sorting mail had been an arduous, frustrating task. ESSENTIAL READING IF YOU ARE UPGRADING FROM v0.17.x 1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just = copy the entire POPFile directory somewhere. You can then safely install POPFile v0.18.1 on top of your current installation; I just think a = back up is a sensible precaution. 2. YOUR HISTORY WILL CLEAR: I have changed the format of the History = files used in this version which means that the old History files will not = be read by POPFile and will in fact be deleted to save disk space. If = you need to do any reclassifications prior to installing v0.18.1 do them! POPFile does NOT clear buckets, statistics or anything else on = install of this version, just the History files. To refresh the History view = click the History tab at any time; POPFile will check disk for new = messages. 3. IF YOU HAD BROKEN MAGNETS: I have added automatic update of broken = magnets. If you had a magnet like [foo] which POPFile mistakenly changed to = \[foo\] it should get magically fixed and start working. 4. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made = in the mail parser it is possible that you might see accuracy drop initially = and you may find yourself reclassifying a few messages that used to work. = This is unfortunate but necessary to make POPFile even more accurate than = before and v0.18.1 incorporates changes that make POPFile's classification accuracy better; however old corpuses might need a little retraining. DOWNLOADING You can obtain the latest release of POPFile by visiting http://sourceforge.net/project/showfiles.php?group_id=3D63137 UPGRADING Just install POPFile on top of the currently installed version. But did = you read the ESSENTIAL READING above first if you are upgrading from a pre = v0.18.0 version? FAQ zonk3r has spent a great deal of time on a POPFile FAQ. Please check it = out as it covers many questions that you might have: = http://sourceforge.net/docman/display_doc.php?docid=3D14421&group_id=3D63= 137 THE GORY DETAILS 1. Fixed a serious bug that could cause POPFile to stop responding to = POP3 and HTTP requests in the middle of downloading mail when the mail = message containing a MIME encoding that used certain characters that were = special in a Perl regular expression. (Reported by ecarlseen, gilesjuk, = panther757, szkaroly and fixed by sschinke) =20 2. The skinning system continues to evole with kraelen, stanley_krute = and kinematics working madly to make everything skinnable, make = everything comply with HTML 4.01, CSS1 and Bobby AA guidelines. =20 3. The code that handled the History cache that was meant to speed the = loading of the History pages and that I originally wrote was pretty lame and=20 some people (including me!) were ending up with 1000 message in the = history and the load time sucked. So I fixed it. The new cache will load = ONCE and ONLY ONCE per file and caches everything in memory. When new messages = are received they get loaded once per new message as well. When a History = page needs to be generated everything can be loaded from the cache = without doing any disk access which means that you should see little = difference=20 between 10 messages and 1000. 4. To magnets would not work correctly if the To line in the email = header contained multiple lines. (Reported by drunin and fixed by = kinematics) 5. There was a nasty problem where sometimes the statistics would not = update correctly which typically happened when you downloaded a large number = of emails in one batch. This was caused by the fact that we weren't = flushing the pipe between the child POP3 process and the parent regularly, this = has been modified so that this can never happen and statistics update in = real=20 time even as mail is downloading. (Reported by burale, ct85711) 6. The Windows installer has been much improved with the assistance of=20 xuesheng to make it reconfigure Outlook and Outlook Express for you = and=20 include a number of new screens. =20 7. Quarantine had a couple of bugs where it would show the incorrect = date (reported by thejcab) and the wrong to address (reported by = dyoungmciwcom) under some circumtances. =20 8. When the From or Subject was encoded using base64 or quoted = printable=20 weird things would show up in the History. (Reported by goulduck and = fixed by williamxp) =20 9. spf and I went back and forth discussing line endings in the files = saved to disk so that all the MSG and CLS files could be loaded into an editor = on any platform without extraneous characters. =20 10. People who live on the Bleeding Edge were getting update warnings = from=20 POPFile even though they were on the latest version. This has been = fixed on the update server. =20 11. Magnets containing the & character could not be deleted. (Reported = by the infamous stanley_krute and fixed by helphand) =20 12. There was a bug associateds with invisible ink detection that could = cause POPFile's HTML engine to think invisible ink was in use when a font tag spanned a table. (Reported by mfichtner) =20 13. If POPFile didn't exit gracefully then statistics were not being = saved to disk and would not be up to date. (Reported by daemon72) =20 14. The Shutdown page had no CSS because POPFile was shutdown, we now = have a=20 simple SSI solution just for the shutdown page. (Fix and report by=20 helphand) =20 15. There was a problem with Subject Line Modification where if it was = turned off every subject line got an extra space in it. (Reported by and fixed by helphand) =20 16. There was a bug where you couldn't look up words that had # in them. = (Reported by adammc and fixed by helphand) =20 17. Viewing base64 encoded messages in the History resulted in little = useful output. (Reported by biljir and fixed by pkarlin) =20 18. sschinke whipped up a new -archive option that causes message = removed from the History to get saved away on disk. =20 19. sschinke made POPFile behave better when a POP3 server suddenly = stops=20 responding in the middle of a conversation. =20 DONATIONS Thank you to everyone who has clicked the Donate! button and donated = their hard earned cash to me in support of POPFile. Thank you also to the = people who have contributed patches, feature requests and bug reports and big = thanks to the two POPFile team members Stan and Sam for their continued = efforts. Keep the ideas and bug reports coming. John. |
|
From: John Graham-C. <jg...@jg...> - 2003-02-06 22:24:29
|
INTRODUCTION
It has been close to 2 months since the last POPFile release and boy have=
I
and others been busy. POPFile v0.18.0 is a *major* change from v0.17.9 a=
nd
you'd do well to read the section entitled ESSENTIAL READING before you
install it. This version is backwards compatible with previous versions
(even
autofixing some errors that the old versions created) but there are some
differences.
This is also the first release in which two additional developers
(additional
to me that is) really contributed a lot of code to POPFile. They are
Stanley
Krute and Sam Schinke. Stan and Sam worked on many of the changes to
POPFile
(which you can read all about in the CVS logs) including massive changes =
to
the mail parser, whole new skinning system, HTML 4.01 compliance, tons of
testing, lots of user interface changes. Guys, *you* made a big differen=
ce
to
POPFile, and I want to say a big "Thank you" in public.
This version of POPFile supports unlimited numbers of connections, multip=
le
natural languages, has a much improved UI, has many improvements to help
catch
spam, has a new Quarantine feature that lets POPFile automatically hide a=
ll
the mail in a specific bucket (each mail is replaced with a new mail stat=
ing
that POPFile quarantined a message with an option to see the original), a=
nd
incorporates features and bug fixes submitted by users.
ESSENTIAL READING
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just co=
py
the entire POPFile directory somewhere. You can then safely install
POPFile v0.18.0 on top of your current installation; I just think a ba=
ck
up
is a sensible precaution.
2. YOUR HISTORY WILL CLEAR: I have changed the format of the History file=
s
used in this version which means that the old History files will not b=
e
read by POPFile and will in fact be deleted to save disk space. If yo=
u
need to do any reclassifications prior to installing v0.18.0 do them!
POPFile does NOT clear buckets, statistics or anything else on install=
of
this version, just the History files. To refresh the History view cli=
ck
the History tab at any time; POPFile will check disk for new messages.
3. IF YOU HAD BROKEN MAGNETS: I have added automatic update of broken
magnets.
If you had a magnet like [foo] which POPFile mistakenly changed to
\[foo\]
it should get magically fixed and start working.
4. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in
the
mail parser it is possible that you might see accuracy drop initially =
and
you may find yourself reclassifying a few messages that used to work.
This
is unfortunate but necessary to make POPFile even more accurate than
before
and v0.18.0 incorporates changes that make POPFile's classification
accuracy better; however old corpuses might need a little retraining.
DOWNLOADING
You can obtain the latest release of POPFile by visiting
http://sourceforge.net/project/showfiles.php?group_id=3D63137
UPGRADING
Just install POPFile on top of the currently installed version. But did =
you
read the ESSENTIAL READING above first?
FAQ
zonk3r has spent a great deal of time on a POPFile FAQ. Please check it =
out
as it covers many questions that you might have:
http://sourceforge.net/docman/display_doc.php?docid=3D14421&group_id=3D=
63137
THE GORY DETAILS
1. Total rearchitecture of the POPFile code into a collection of loadabl=
e
modules to allow for future expansion of POPFile with different proxy
services and different UIs. The major modules are now:
- popfile.pl: controlling process that loads the modules that impleme=
nt
POPFile's main functionality;
- Proxy::POP3 the original POP3 proxying code from POPFile separated
into
its own module. In future we should expect to be able to have othe=
r
Proxy:: modules such as Proxy::IMAP.
- UI::HTML contains the current HTML user interface. I hope that we'=
ll
add other UI modules such as UI::XMLRPC or UI::SOAP in future.
- Classifier::Bayes is the module that does the Naive Bayes
classification. (Note this module also includes a method called
classify_and_modify that performs the classification and modificati=
on
of
email messages that may be of use to other module writers)
POPFile now supports two process types: the top level process round
robins
between the main modules listed above providing concurrent access to =
the
UI and POP3 proxy. The POP3 proxy itself (see below) forks to provid=
e
multiple simultaneous connections. (That last part was originally
requested by sschinke)
2. Introduction of a scheme for localization of the user interface using
message catalogs stored in the languages/ subfolder. Currently shippi=
ng
with languages: English, French, German, Danish, Dutch, Brazilian, an=
d
(for fun and for testing) H4X0R. Translators can take the English.msg
file
and translate the strings within to create their own language for the
user
interface. POPFile detects available languages on start up and they a=
re
available from the Configuration page. (Originally requested by
pfaelzerchen)
3. Rework of much of the user interface to make pages shorter and cleare=
r.
This is mostly limited to look and feel and no items have changed tab=
in
the UI.
4. Total rewrite of the history caching code for clarity with a change f=
rom
multiple arrays to a single hash containing all the cached informatio=
n.
5. Automatic update checking now defaults to off and there's a large
explanation of what it does in the user interface to allay the fears =
of
many people concerning the possibility that programs like POPFile are
spying on them. (Requested by quite a few people)
6. When creating the user interface password the password is now hidden
when
typing it. (Requested by kraelen, pfaelzerchen)
7. The Proxy::POP3 now forks after a new connection is accepted from a
valid
client and calls a special child() method that handles the connection
between a client and mail server via POPFile for the life of the clie=
nt
connection. This is the beginning of a version of POPFile that suppo=
rts
a
very large number of clients. (Requested by many people)
8. A new command line parameter -test_language can be set to 1 for UI
translators. The UI will show the language identifiers used for each
element rather than the strings associated with the identifier.
9. Rework of the History page to take into account ideas and suggestions
from
many people: Search now operates on From and Subject, From and Subjec=
t
have popup windows to show the full text of each, you can sort by ID,
From, Subject or Classification, you can filter by magnet classified
mail,
navigation is at the top and the bottom, state is remembered so that =
you
can sort/filter and remove pages and reclassify and stay in the same
place, many small improvements.
10. Major work on the MailParse.pm for dealing with improved mail parsing
and
the handling of spammers tricks (like S P A C E D out words etc.).
Switch
to using the Base64 and QuotedPrint modules from CPAN for improved ma=
il
cracking. (Work done mainly by sschinke)
11. Automated test suite now part of POPFile. (Suggested by aziegler)
12. Rework of all the HTML and CSS so that everything can be skinned and =
so
that the complete UI is HTML 4.01 compliant. (Work done mainly by
stanley_krute; there was a request from nobody for this)
13. You can now filter the History page by magnet filtered mails to look =
at
them or delete them. (Requested by jarmstro, tcdk, msc_buff, fitzsimo=
ns,
dyoungmcwicom)
14. The UI password no longer appears in URLs in plain text (requested by
gribnif)
15. You can do multiple classifications at a time by selecting multiple
messages and changing the drop down and hitting any Reclassify button.
(Requested by dave823; implemented by sschinke)
16. You can now sort the History page by message ID, From, Subject and
Classification (bucket). (Requested by jerometremblay, tomvoss, biobe=
ar)
17. When you remove a page the History no longer goes back to the start b=
ut
stays at the same place in the History if it can. (Requested by
ajweiner)
18. After you reclassify a message the History page stays at the same
position
(at the reclassified message). (Requested by harleythebrave, transit)
19. The installer now picks up old settings from a current POPFile instal=
l
to
fill in the options on the final dialog. (Requested by harleythebrave=
)
20. Added two new options -logdir and -msgdir to specify the location of =
the
log directory (default to ./) and the messages (where the history is
stored; defaults to messages/). (Requested by nobody)
21. Search in History now works on From as well as Subject. (Requested by
transit)
22. The navigation from page to page in the History is now at the top and=
at
the bottom of the page. (Requested by transit, nobody, nobody, nobody=
)
23. There are new [Remove] buttons on each message in the History to let =
you
remove messages one by one. (Requested by nobody)
24. The X-POPFile-Link: repositions the History page at the message selec=
ted
when you click the link. (Requested by zonk3r)
25. There is a new filter for unclassified messages. (Requested by nobody=
)
26. New -toptoo command line option enables classification on the POP3 TO=
P
command and is only recommended for people on fast connections.
(Requested
by jberkes)
27. When viewing a colorized message there is a blank line between the
header
and the body of the message. (Requested by whiteg)
28. Message IDs (and History filenames) are now globally unique per POPFi=
le
install. (Requested by helphand)
29. You can now control the inclusion of the X-Text-Classification and X-
POPFile-Link headers. (Requested by nobody)
30. Refreshing a page does not repeat the action any more. (Reported by
nobody)
31. Fixed a bug where magnets with certain characters in them (e.g. [ or =
<)
did not work and could not be deleted. (Reported by nobody, nobody,
nobody, alanmuk, boneill, bana-sea-)
32. Fixed a bug where you could make POPFile exit with a divide by zero
error
if you started it with no corpus and then tried to Lookup a word's
probability. (Reported by sunshine, gribnif)
33. Fixed a bug where the Shutdown link on the POPFile Control Center did
not
work if the UI password had been set. (Reported by actoews, nobody)
34. Fixed an embarassing bug where any POP server that contained the word
'stat' would not work with POPFile. (Reported by nobody, markabcxyz)
35. Fixed a bug where To magnets did not work correctly in the presence o=
f
other To style email headers. (Reported by dyoungmcwicom)
36. Fixed a bug where a specific email was causing the History page to lo=
ck
up
when you tried to view the colorized version. (Reported by ajweiner)
37. Fixed a bug where having pages in POPFile automatically refresh every
ten
minutes could cause POPFile to repeat an action (such as
Reclassification)
every 10 minutes. (Reported by mayfoev)
38. Fixed a bug where parts of a > attribute could appear in the From =
or
Subject lines on the History (Reported and fixed by sschinke)
39. Fixed a bug where some CLS and MSG files were not getting deleted fro=
m
the
History when they were no longer needed. (Reported by helphand)
TRANSLATIONS
Thanks to the following for helping to make POPFile a global piece of
software:
Brazilian Adriano RG Danish Nicki Br=F8chner Nielsen D=
utch
Jilles Oldenbeuving French Antony Boucher German
Matthias Deege Norweigian Jo Inge Fjellstad Russian
Alexander
Saltanov
Please translate the UI and the manual into your language.
DONATIONS
Thank you to everyone who as clicked the Feed Me! button and donated thei=
r
hard earned cash to me in support of POPFile. Thank you also to the peop=
le
who have contributed patches, feature requests and bug reports.
Keep the ideas and bug reports coming.
John.
|
4 messages has been excluded from this view by a project administrator.
