Menu

#13 I solve a spider exit jdbc bug

v1.0 (example)
closed-fixed
EJP
None
5
2021-07-14
2004-09-30
Wanjun Hu
No

I use the very 3.23MySQL database and Jdbc 2.0.14
java jdbc driver,but the spider can not work very well,it
exit when the html code is chinese in a english html file.

I go to the code of
net.javacoding.jspider.core.storage.jdbc.ResourceDAOImp
l ,I add three lines in

public synchronized void setError(URL url,
ResourceParsedErrorEvent event) throws
InvalidStateTransitionException {
ResourceInternal resource = getResource(url);
if(resource==null){
System.err.println("ERROR,setError");
return;
}
resource.setParseError();
save(resource);
}

the three follow codes is the add lines and the panic exit

if(resource==null){
System.err.println("ERROR,setError");
return;
}

I am a Spider developer,contact me email to
huwanjun at baidu.com

replace at to @

Discussion

  • EJP

    EJP - 2021-07-14
    • status: open --> closed-fixed
    • assigned_to: EJP
    • Group: --> v1.0 (example)
     
  • EJP

    EJP - 2021-07-14

    Fixed in both setError() methods in upcoming 1.0 release.

     

Log in to post a comment.

MongoDB Logo MongoDB