Menu

#7 spider HTTPS sites not supported

weblech-0.0.3
open
Console (3)
5
2005-07-13
2005-07-13
scruf
No

The current code does not support protocols like https,
ftp, etc. Since I needed HTTPS, I looked for a simple
solution. Here is what I found:

1. insert into 138 of URLObject.java the following:
httpIdx = url.indexOf("https://");
if(httpIdx == 0)
{
url = url.substring(8);
}
then rebuild the classes.

2. populate a jvm keystore for trusted certificate
authorities using $JAVA_HOME/bin/keytool -import

3. alter the bin/run script instructing the jvm to use
the keystore by adding the parameter
-Djavax.net.ssl.trustStore="keyPath\.keystore"

Other protocols could be supported with simple changes
like listed in step one. Yet, I am sure a more elegant
method also exists.

Discussion


Log in to post a comment.