From: SourceForge.net <no...@so...> - 2010-06-01 08:23:37
|
Feature Requests item #3009113, was opened at 2010-05-30 06:57 Message generated for change (Comment added) made by mguillem You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=952181&aid=3009113&group_id=195122 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Priority: 5 Private: No Submitted By: shirley wilder (shirleyjwilder) >Assigned to: Marc Guillemot (mguillem) Summary: multi-thread support Initial Comment: Can add multi-thread support for nekohtml? ---------------------------------------------------------------------- >Comment By: Marc Guillemot (mguillem) Date: 2010-06-01 10:23 Message: You seem to speak from HtmlUnit, not NekoHTML. If you're able to fix your problem by changing HtmlUnit, then it should probably be changed in HtmlUnit, not in NekoHTML. I'm a bit tired of trying to guess what you want and therefore I'm closing this issue. Please reopen it when you're able to provide *precise* information. ---------------------------------------------------------------------- Comment By: shirley wilder (shirleyjwilder) Date: 2010-06-01 10:17 Message: I have modified the code of HTMLParser.java. So it can support for multi thread access. When I test it with 1000 threads.It blocks here. long start=System.currentTimeMillis(); super.parse(inputSource); System.out.println("parse time:"+(System.currentTimeMillis()-start)); The super.parse comes from nekohtml.So I think that is the problem. But I didn't read the code.So I really don't know the cause. ---------------------------------------------------------------------- Comment By: Marc Guillemot (mguillem) Date: 2010-06-01 08:43 Message: Can you provide more information? Which static methods are synchronized and causing problems? Looking at the source code of NekoHTML, I could only find one synchronized block (in ObjectFactory) and I'm quite sure that it can't be the cause of performance problems. ---------------------------------------------------------------------- Comment By: shirley wilder (shirleyjwilder) Date: 2010-05-31 18:56 Message: Err. And I have changed the htmlparse class to thread safe and remove all static method to normal.But it still blocks.Sometimes,One thread costs near several minutes to parse the html. I have tracked the code,and found the problem is nekohtml. I don't read the source code of nekohtml, I think the nekohtml or the dependence lib has used the synchronized static method which cost very long time.So it makes many threads block there. ---------------------------------------------------------------------- Comment By: Marc Guillemot (mguillem) Date: 2010-05-31 13:30 Message: sorry, but I still don't understand what the problem is. Can you explain more precisely? ---------------------------------------------------------------------- Comment By: shirley wilder (shirleyjwilder) Date: 2010-05-31 12:58 Message: I use one thread to parse a html,and it costs 5s. What I want to do is parses 1000 html using 1000 threads,and the using time is also 5s or a little more. Thanks! ---------------------------------------------------------------------- Comment By: Marc Guillemot (mguillem) Date: 2010-05-31 09:14 Message: Can you explain a bit what you want to do? For me it doesn't make sense to use more than one thread to parse a document but x threads can be used to parse x document. ---------------------------------------------------------------------- Comment By: shirley wilder (shirleyjwilder) Date: 2010-05-31 08:27 Message: I want to test my site with htmlunit with nearly 1000 threads.But it can only parse html with one thread using nekohtml.So the speed is slow.So I think it'll be wonderful if nekohtml supports multi-thead. Thanks! ---------------------------------------------------------------------- Comment By: Jacob Kjome (jacobk) Date: 2010-05-30 21:13 Message: Can you expand on what you mean and how it might be beneficial? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=952181&aid=3009113&group_id=195122 |