I can't access folders that have special characters in their names. How can I manage this situation? Bellow is the reised exeption.
13/06/2011 12:02:04 org.semanticdesktop.aperture.crawler.mail.AbstractJavaMailCrawler crawlMessages
AVISO: Exception while crawling message #1 in folder INBOX/Monitoração
javax.mail.MessagingException: A10 BAD Please select a mailbox first;
nested exception is:
com.sun.mail.iap.BadCommandException: A10 BAD Please select a mailbox first
at com.sun.mail.imap.IMAPMessage.loadBODYSTRUCTURE(IMAPMessage.java:1273)
Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This looks to me like a limitation of the underlying javamail implementation. Please try to see if you can connect to this mail store with pure javamail.
You can also try to rerun the imap crawler with a -Dmail.debug=true VM option. This should print the exact IMAP messages running between you and the server. You error feels like the name of the folder not being escaped properly in the underlying IMAP message.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I enabled the debug flag and bellow is the output. It seems that JavaMail can list folder contents at first but soon it get lost. I will try connect with pure javamail. What you think?
Thank you.
$ ./imapcrawler.sh -username user6 -password prognus -server expressodev -folder "INBOX/Coração"
DEBUG: JavaMail version 1.4.1ea-SNAPSHOT
DEBUG: not loading file: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.providers
DEBUG: java.io.FileNotFoundException: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.providers (No such file or directory)
DEBUG: URL jar:file:/home/edney/workspace/aperture/lib/mstor-0.9.13-SNAPSHOT-aperture13.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/home/edney/workspace/aperture/lib/mstor-0.9.13-SNAPSHOT-aperture13.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider, com.sun.mail.smtp.SMTPTransport=javax.mail.Provider, com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider, com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider, com.sun.mail.imap.IMAPStore=javax.mail.Provider, net.fortuna.mstor.MStorStore=javax.mail.Provider, com.sun.mail.pop3.POP3Store=javax.mail.Provider}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider, imap=javax.mail.Provider, smtps=javax.mail.Provider, mstor=javax.mail.Provider, pop3=javax.mail.Provider, pop3s=javax.mail.Provider, smtp=javax.mail.Provider}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: !anyLoaded
DEBUG: not loading resource: /META-INF/javamail.address.map
DEBUG: not loading file: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.address.map
DEBUG: java.io.FileNotFoundException: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.address.map (No such file or directory)
DEBUG: getProvider() returning javax.mail.Provider
DEBUG: mail.imap.fetchsize: 16384
* OK sith Cyrus IMAP4 v2.3.7-Invoca-RPM-2.3.7-7.el5_4.3 server ready
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH
A0 OK Completed
DEBUG: protocolConnect login, host=expressodev, user=user6, password=<non-null>
A1 LOGIN user6 prognus
A1 OK User logged in
A2 LIST "" INBOX/Cora&AOcA4w-o
* LIST (\HasNoChildren) "/" "INBOX/Cora&AOcA4w-o"
A2 OK Completed (0.000 secs 2 calls)
DEBUG: connection available - size: 1
A3 EXAMINE INBOX/Cora&AOcA4w-o
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
* OK
* 2 EXISTS
* 0 RECENT
* OK
* OK
* OK Sorry, modsequences have not been enabled on this mailbox
A3 OK Completed
A4 LIST "" INBOX/Cora&AOcA4w-o
* LIST (\HasNoChildren) "/" "INBOX/Cora&AOcA4w-o"
A4 OK Completed (0.000 secs 2 calls)
A5 SEARCH UNDELETED ALL
* SEARCH 1 2
A5 OK Completed (2 msgs in 0.000 secs)
A6 FETCH 1:2 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS UID)
* 1 FETCH (FLAGS (\Seen) UID 2 INTERNALDATE "10-May-2011 15:03:57 -0300" RFC822.SIZE 837 ENVELOPE ("Tue, 10 May 2011 15:03:57 -0300" "oi" (("User Six" NIL "user6" "prognus.org")) (("User Six" NIL "user6" "prognus.org")) (("User Six" NIL "user6" "prognus.org")) ((NIL NIL "user6" "prognus.org")) NIL NIL NIL "<e8783e8fd7a031a3b96f933934c1769d@expressodev.prognus.com.br>"))
* 2 FETCH (FLAGS (\Seen) UID 3 INTERNALDATE "27-Apr-2011 10:44:25 -0300" RFC822.SIZE 2128 ENVELOPE ("Wed, 27 Apr 2011 10:44:24 -0300" "=?iso-8859-1?Q?Evento_do_Calend=E1rio_-_Cancelado:_27/04/2011_-_09:00_tes?= =?iso-8859-1?Q?te_do_expresso?=" (("User Five" NIL "user5" "prognus.org")) (("User Five" NIL "user5" "prognus.org")) (("User Five" NIL "user5" "prognus.org")) ((NIL NIL "user6" "prognus.org")) NIL NIL NIL "<ade053aab793a4633135c5f9ad01ea94@expressodev.prognus.com.br>"))
A6 OK Completed (0.000 sec)
A7 EXAMINE INBOX/Cora��o
A7 NO Mailbox does not exist
A8 FETCH 1:2 (BODYSTRUCTURE BODY.PEEK)
A8 BAD Please select a mailbox first
A9 LIST "" "INBOX/Cora&AOcA4w-o/%"
A9 OK Completed (0.000 secs 1 calls)
A10 FETCH 1 (BODYSTRUCTURE)
A10 BAD Please select a mailbox first
IMAP DEBUG: IMAPProtocol noop
A11 NOOP
A11 OK Completed
A12 FETCH 2 (BODYSTRUCTURE)
A12 BAD Please select a mailbox first
IMAP DEBUG: IMAPProtocol noop
A13 NOOP
A13 OK Completed
A14 LIST "" "INBOX/Cora&AOcA4w-o/%"
A14 OK Completed (0.000 secs 1 calls)
A15 CLOSE
A15 BAD Please select a mailbox first
DEBUG: added an Authenticated connection - size: 1
A16 LOGOUT
* BYE LOGOUT received
A16 OK Completed
DEBUG: IMAPStore connection dead
DEBUG: IMAPStore cleanup, force false
DEBUG: IMAPStore cleanup done
Output discarded
Crawl report
Crawl started: Mon Jun 13 14:02:13 BRT 2011
Crawl stopped: Mon Jun 13 14:02:13 BRT 2011
Crawl time: 583ms
Exit code: fatal error
Fatal error cause:
org.semanticdesktop.aperture.datasource.DataSourceConfigurationException: MessagingException while crawling
at org.semanticdesktop.aperture.crawler.base.CrawlerBase.reportFatalErrorCause(CrawlerBase.java:437)
at org.semanticdesktop.aperture.crawler.imap.ImapCrawler.crawlObjects(ImapCrawler.java:209)
at org.semanticdesktop.aperture.crawler.base.CrawlerBase.crawl(CrawlerBase.java:216)
at org.semanticdesktop.aperture.examples.ExampleImapCrawler.crawl(ExampleImapCrawler.java:210)
at org.semanticdesktop.aperture.examples.ExampleImapCrawler.main(ExampleImapCrawler.java:299)
Caused by: javax.mail.MessagingException: A15 BAD Please select a mailbox first;
nested exception is:
com.sun.mail.iap.BadCommandException: A15 BAD Please select a mailbox first
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1128)
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1055)
at org.semanticdesktop.aperture.crawler.mail.AbstractJavaMailCrawler.crawlFolder(AbstractJavaMailCrawler.java:440)
at org.semanticdesktop.aperture.crawler.imap.ImapCrawler.crawlObjects(ImapCrawler.java:199)
… 3 more
Caused by: com.sun.mail.iap.BadCommandException: A15 BAD Please select a mailbox first
at com.sun.mail.iap.Protocol.handleResult(Protocol.java:343)
at com.sun.mail.iap.Protocol.simpleCommand(Protocol.java:363)
at com.sun.mail.imap.protocol.IMAPProtocol.close(IMAPProtocol.java:976)
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1125)
… 6 more
New objects: 1
Modified objects: 0
Unmodified objects: 0
Deleted objects: 0
New or modified objects with full text: 0
Total length of the extracted full text: 0
Exceptions while processing objects: 0
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I did the pure javamail code and it worked flawless with the mail.jar provided by aperture. I also used the latest javamail release and it also worked. Could it be aperture fault? Where should i look for a fix?
Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yup, this is definitely a bug. The code you pasted bypasses the the nice javamail APIs to string together our own IMAP request and obviously forgets to escape the name properly.
It seems that the testers were feeling much too English-centric, even though 99% of them are Dutch, German and Polish, all fine old languages with great diacritical marks.
Please confirm that the problem is away and close the issue.
Thanks a lot!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there.
I can't access folders that have special characters in their names. How can I manage this situation? Bellow is the reised exeption.
13/06/2011 12:02:04 org.semanticdesktop.aperture.crawler.mail.AbstractJavaMailCrawler crawlMessages
AVISO: Exception while crawling message #1 in folder INBOX/Monitoração
javax.mail.MessagingException: A10 BAD Please select a mailbox first;
nested exception is:
com.sun.mail.iap.BadCommandException: A10 BAD Please select a mailbox first
at com.sun.mail.imap.IMAPMessage.loadBODYSTRUCTURE(IMAPMessage.java:1273)
Thank you.
This looks to me like a limitation of the underlying javamail implementation. Please try to see if you can connect to this mail store with pure javamail.
You can also try to rerun the imap crawler with a -Dmail.debug=true VM option. This should print the exact IMAP messages running between you and the server. You error feels like the name of the folder not being escaped properly in the underlying IMAP message.
Hi there!
I enabled the debug flag and bellow is the output. It seems that JavaMail can list folder contents at first but soon it get lost. I will try connect with pure javamail. What you think?
Thank you.
$ ./imapcrawler.sh -username user6 -password prognus -server expressodev -folder "INBOX/Coração"
DEBUG: JavaMail version 1.4.1ea-SNAPSHOT
DEBUG: not loading file: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.providers
DEBUG: java.io.FileNotFoundException: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.providers (No such file or directory)
DEBUG: URL jar:file:/home/edney/workspace/aperture/lib/mstor-0.9.13-SNAPSHOT-aperture13.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/home/edney/workspace/aperture/lib/mstor-0.9.13-SNAPSHOT-aperture13.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider, com.sun.mail.smtp.SMTPTransport=javax.mail.Provider, com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider, com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider, com.sun.mail.imap.IMAPStore=javax.mail.Provider, net.fortuna.mstor.MStorStore=javax.mail.Provider, com.sun.mail.pop3.POP3Store=javax.mail.Provider}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider, imap=javax.mail.Provider, smtps=javax.mail.Provider, mstor=javax.mail.Provider, pop3=javax.mail.Provider, pop3s=javax.mail.Provider, smtp=javax.mail.Provider}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: !anyLoaded
DEBUG: not loading resource: /META-INF/javamail.address.map
DEBUG: not loading file: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.address.map
DEBUG: java.io.FileNotFoundException: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/javamail.address.map (No such file or directory)
DEBUG: getProvider() returning javax.mail.Provider
DEBUG: mail.imap.fetchsize: 16384
* OK sith Cyrus IMAP4 v2.3.7-Invoca-RPM-2.3.7-7.el5_4.3 server ready
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH
A0 OK Completed
DEBUG: protocolConnect login, host=expressodev, user=user6, password=<non-null>
A1 LOGIN user6 prognus
A1 OK User logged in
A2 LIST "" INBOX/Cora&AOcA4w-o
* LIST (\HasNoChildren) "/" "INBOX/Cora&AOcA4w-o"
A2 OK Completed (0.000 secs 2 calls)
DEBUG: connection available - size: 1
A3 EXAMINE INBOX/Cora&AOcA4w-o
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
* OK
* 2 EXISTS
* 0 RECENT
* OK
* OK
* OK Sorry, modsequences have not been enabled on this mailbox
A3 OK Completed
A4 LIST "" INBOX/Cora&AOcA4w-o
* LIST (\HasNoChildren) "/" "INBOX/Cora&AOcA4w-o"
A4 OK Completed (0.000 secs 2 calls)
A5 SEARCH UNDELETED ALL
* SEARCH 1 2
A5 OK Completed (2 msgs in 0.000 secs)
A6 FETCH 1:2 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS UID)
* 1 FETCH (FLAGS (\Seen) UID 2 INTERNALDATE "10-May-2011 15:03:57 -0300" RFC822.SIZE 837 ENVELOPE ("Tue, 10 May 2011 15:03:57 -0300" "oi" (("User Six" NIL "user6" "prognus.org")) (("User Six" NIL "user6" "prognus.org")) (("User Six" NIL "user6" "prognus.org")) ((NIL NIL "user6" "prognus.org")) NIL NIL NIL "<e8783e8fd7a031a3b96f933934c1769d@expressodev.prognus.com.br>"))
* 2 FETCH (FLAGS (\Seen) UID 3 INTERNALDATE "27-Apr-2011 10:44:25 -0300" RFC822.SIZE 2128 ENVELOPE ("Wed, 27 Apr 2011 10:44:24 -0300" "=?iso-8859-1?Q?Evento_do_Calend=E1rio_-_Cancelado:_27/04/2011_-_09:00_tes?= =?iso-8859-1?Q?te_do_expresso?=" (("User Five" NIL "user5" "prognus.org")) (("User Five" NIL "user5" "prognus.org")) (("User Five" NIL "user5" "prognus.org")) ((NIL NIL "user6" "prognus.org")) NIL NIL NIL "<ade053aab793a4633135c5f9ad01ea94@expressodev.prognus.com.br>"))
A6 OK Completed (0.000 sec)
A7 EXAMINE INBOX/Cora��o
A7 NO Mailbox does not exist
A8 FETCH 1:2 (BODYSTRUCTURE BODY.PEEK)
A8 BAD Please select a mailbox first
A9 LIST "" "INBOX/Cora&AOcA4w-o/%"
A9 OK Completed (0.000 secs 1 calls)
A10 FETCH 1 (BODYSTRUCTURE)
A10 BAD Please select a mailbox first
IMAP DEBUG: IMAPProtocol noop
A11 NOOP
A11 OK Completed
A12 FETCH 2 (BODYSTRUCTURE)
A12 BAD Please select a mailbox first
IMAP DEBUG: IMAPProtocol noop
A13 NOOP
A13 OK Completed
A14 LIST "" "INBOX/Cora&AOcA4w-o/%"
A14 OK Completed (0.000 secs 1 calls)
A15 CLOSE
A15 BAD Please select a mailbox first
DEBUG: added an Authenticated connection - size: 1
A16 LOGOUT
* BYE LOGOUT received
A16 OK Completed
DEBUG: IMAPStore connection dead
DEBUG: IMAPStore cleanup, force false
DEBUG: IMAPStore cleanup done
Output discarded
Crawl report
Crawl started: Mon Jun 13 14:02:13 BRT 2011
Crawl stopped: Mon Jun 13 14:02:13 BRT 2011
Crawl time: 583ms
Exit code: fatal error
Fatal error cause:
org.semanticdesktop.aperture.datasource.DataSourceConfigurationException: MessagingException while crawling
at org.semanticdesktop.aperture.crawler.base.CrawlerBase.reportFatalErrorCause(CrawlerBase.java:437)
at org.semanticdesktop.aperture.crawler.imap.ImapCrawler.crawlObjects(ImapCrawler.java:209)
at org.semanticdesktop.aperture.crawler.base.CrawlerBase.crawl(CrawlerBase.java:216)
at org.semanticdesktop.aperture.examples.ExampleImapCrawler.crawl(ExampleImapCrawler.java:210)
at org.semanticdesktop.aperture.examples.ExampleImapCrawler.main(ExampleImapCrawler.java:299)
Caused by: javax.mail.MessagingException: A15 BAD Please select a mailbox first;
nested exception is:
com.sun.mail.iap.BadCommandException: A15 BAD Please select a mailbox first
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1128)
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1055)
at org.semanticdesktop.aperture.crawler.mail.AbstractJavaMailCrawler.crawlFolder(AbstractJavaMailCrawler.java:440)
at org.semanticdesktop.aperture.crawler.imap.ImapCrawler.crawlObjects(ImapCrawler.java:199)
… 3 more
Caused by: com.sun.mail.iap.BadCommandException: A15 BAD Please select a mailbox first
at com.sun.mail.iap.Protocol.handleResult(Protocol.java:343)
at com.sun.mail.iap.Protocol.simpleCommand(Protocol.java:363)
at com.sun.mail.imap.protocol.IMAPProtocol.close(IMAPProtocol.java:976)
at com.sun.mail.imap.IMAPFolder.close(IMAPFolder.java:1125)
… 6 more
New objects: 1
Modified objects: 0
Unmodified objects: 0
Deleted objects: 0
New or modified objects with full text: 0
Total length of the extracted full text: 0
Exceptions while processing objects: 0
Hi there!
I did the pure javamail code and it worked flawless with the mail.jar provided by aperture. I also used the latest javamail release and it also worked. Could it be aperture fault? Where should i look for a fix?
Thank you.
Hi there!
This code
Boolean stickyness = (Boolean)((IMAPFolder)folder).doCommand(new IMAPFolder.ProtocolCommand() {
public Object doCommand(IMAPProtocol p) throws ProtocolException {
Response resps = p.command("EXAMINE " + ((IMAPFolder)folder).getFullName(), null);
if (resps == null) {
return Boolean.TRUE;
} else {
for (Response res : resps) {
String rest = res.getRest();
if (rest.contains("UIDNOTSTICKY")) {
return Boolean.FALSE;
}
}
return Boolean.TRUE;
}
}
});
makes IMAP outputs
A7 EXAMINE INBOX/Cora��o
A7 NO Mailbox does not exist
which i think it's the source of problems. Am i right? What can i do?
Hi there!
If I do like that
Boolean stickyness = (Boolean)((IMAPFolder)inbox).doCommand(new IMAPFolder.ProtocolCommand() {
public Object doCommand(IMAPProtocol p) throws ProtocolException {
String fullName = com.sun.mail.imap.protocol.BASE64MailboxEncoder.encode((((IMAPFolder)inbox).getFullName()));
Response resps = p.command("EXAMINE " + fullName, null);
if (resps == null) {
return Boolean.TRUE;
} else {
for (Response res : resps) {
String rest = res.getRest();
if (rest.contains("UIDNOTSTICKY")) {
return Boolean.FALSE;
}
}
return Boolean.TRUE;
}
}
});
it works!!!
Is it a bug?
Thank you!
Yup, this is definitely a bug. The code you pasted bypasses the the nice javamail APIs to string together our own IMAP request and obviously forgets to escape the name properly.
Created a bug
https://sourceforge.net/tracker/?func=detail&aid=3315958&group_id=150969&atid=779500
and fixed it in rev 2505.
It seems that the testers were feeling much too English-centric, even though 99% of them are Dutch, German and Polish, all fine old languages with great diacritical marks.
Please confirm that the problem is away and close the issue.
Thanks a lot!