Menu

#1808 serialize htmlpage ConcurrentModificationException

2.22
open
nobody
None
1
2016-07-21
2016-07-20
aaa
No

whenever I try to serialize an Htmlpage an java.util.ConcurrentModificationException occur
it happen in this url :https://www.oxfordlearnersdictionaries.com/definition/english/good_1
but not this : https://www.oxfordlearnersdictionaries.com/definition/english/good
they are the same with a bit changes when i compare the html content
I have no idea how to fix it or how to make work another way.

java.util.ConcurrentModificationException
    at java.util.LinkedHashMap$LinkedHashIterator.nextNode(Unknown Source)
    at java.util.LinkedHashMap$LinkedKeyIterator.next(Unknown Source)
    at java.util.HashSet.writeObject(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
    at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
    at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
    at java.io.ObjectOutputStream.writeObject0(Unknown Source)
    at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
    at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
    at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
    at java.io.ObjectOutputStream.writeObject0(Unknown Source)
    at java.io.ObjectOutputStream.writeObject(Unknown Source)
    at Scraper.htmlunit.getPage(htmlunit.java:82)
    at Scraper.TranslateOxford.<init>(TranslateOxford.java:46)
    at Scraper.oxfordToCsv.<init>(oxfordToCsv.java:24)
    at Scraper.main.main(main.java:26)

I also tried to convert object to json by FasterXml/Jakson project,but it throws an exception caused by
"java.lang.UnsupportedOperationException: HtmlPage.getImplementation is not yet implemented."

i'm confused and I can't find a way somehow save htmlpage to disk

Discussion

  • Ahmed Ashour

    Ahmed Ashour - 2016-07-20

    Can you post your complete code?

     
  • aaa

    aaa - 2016-07-21
    public static void main(String[] args)
        {
    
            java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
    
            WebClient client = new WebClient(BrowserVersion.CHROME);
    
            String url = "https://www.oxfordlearnersdictionaries.com/definition/english/good_1";
            String filename = new File(url).getName().trim();
    
            client.getOptions().setJavaScriptEnabled(true);
            client.getOptions().setCssEnabled(false);
            client.getOptions().setDoNotTrackEnabled(true);
            client.getOptions().setThrowExceptionOnScriptError(false);
            client.getOptions().setThrowExceptionOnFailingStatusCode(false);
            client.getOptions().setTimeout(60000);
            client.getOptions().setAppletEnabled(false);
            client.getOptions().setRedirectEnabled(true);
            client.getOptions().setThrowExceptionOnFailingStatusCode(false);
            client.getOptions().setThrowExceptionOnScriptError(false);
            client.getOptions().setCssEnabled(false);
            client.getOptions().setUseInsecureSSL(true);
    
            try {
    
                WebRequest webRequest = new WebRequest(new java.net.URL(url));
                webRequest.setCharset("UTF-8");
    
                HtmlPage request = client.getPage(webRequest);
    
                //serialization
                FileOutputStream fos = new FileOutputStream(filename);
                ObjectOutputStream oos = new ObjectOutputStream(fos);
                oos.writeObject((Object)request);
                oos.close();
                client.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    
     

Log in to post a comment.