Nodelists containing nested elements loop infinitely.
HTML parser which can be used for screen-scraping applications
Brought to you by:
bhimsen92
Menu
▾
▴
#2 Nodelists containing nested elements loop infinitely.
If you create a dom containing nested elements, ala:
<html><body>
<div><p>Level One</p>
<div><p>Level Two</p></div>
</div>
</body></html>
And then try to iterate over a nodelist containing the nested elements, it loops infinitely
for item in dom.find( "div" ):
print item.html()
Discussion
Anonymous
In order to make an object iterator python requires us to implement "__iter__" and "next" function(for python 2.7) and "__next__"(for python 3.x). You must be testing this code in python 2.x interpreter. The current HtmlNodeList does not implement "next" function, that is why the above code is not working but will work fine in python 3.x interpreter.
I knew about this bug when i implemented the code(i am really ashamed to say this). I implemented the "next" function but it didn't work. The control was not even entering the "next" function. I spent hours on it but failed to identify the bug. So i just implemented "__next__" function to make it work on python 3.x interpreters.
But after seeing this ticket, i just gone through the entire code and to my horror i saw a "next" function defined(which returns siblings of the current node list) below "__next__" function. This new def was overriding the previous function def of "next", resulting in infinite loop.
This is one of those tricky bugs where you are really confident that some part of the code works correctly and you don't even want to take a look at it. And it is also one of the disadvantageous of dynamic languages as they don't complain even if they see multiple functions with the same name.
I will correct the code upload the new one as soon as possible.
Happy coding :)