Hi @scottwilson. Long time no speak! How are you? We have several issues reported quite a while ago, like this one (also reported at https://sourceforge.net/p/htmlcleaner/bugs/231/) or https://sourceforge.net/p/htmlcleaner/bugs/230/ and are wondering if we could expect some fixes. Anything we could do to help out? Thank you very much, you've always been very helpful to the XWiki project. -Vincent
Thanks Scott for fixing these bugs! :)
Returned DOM Document instance should not contain escaped characters for attribute values
Hmm... I don't understand why the user would need to specify and know in which version an issue would be fixed :) However what's important is to report in which version of CSS4J the problem occurred. That's not possible?
CSS4J strips HTML entities
I've gone ahead yesterday and applied the HC 2.22 upgrade with the change of tests. I'm crossing fingers now since it's hard to foresee what the behavior changes will lead to. Thx
For (1) do you mean that, where before you had a line feed character as a unicode entity, you are now getting a standard line feed character? Yes correct.
1) I'm also getting this now: assertHTML("<p>\n</p>", "<p>
</p>"); whereas before, I had: assertHTML("<p>
</p>", "<p>
</p>"); (
 is a line feed) Does that seem normal to you? 2) Should I open an issue for the overzealous quote escape? Thanks
AFAIR what we wanted was XHTML so it's normal to escape ampersand (since it's one of the 5 special characters in XML) but the quote doesn't need to be escaped in XHTML. So are you saying there's no way to escape only what's needed for XHTML and translateSpecialEntities=true will escape more than needed?
AFAIR what we wanted was XHTML so it's normal to escape ampersand (since it's one of the 5 special characters in XML) but the quote doesn't need to be escaped in XHTML. So are you saying there's no to escape only what's needed?
Yes thanks for that hint, that fixes Problem 1 (and Problem 2 for some reason). I now have Problem 3 + a new one: Problem 4 @Test public void specialCharacters() { // TODO: We still have a problem I think in that if there are characters such as "&" or quote in the source // text they are not escaped. This is because we have use "false" in DefaultHTMLCleaner here: // Document document = new JDomSerializer(this.cleanerProperties, false).createJDom(cleanedNode); // See the problem described here: http://sourceforge.net/forum/forum.php?thread_id=2243880&forum_id=637246...
Note that if I use DOMSerializer instead of XWikiDOMSerializer I also get these 3 problems (and 2 more failing tests related to CDATA).
And I believe you're confirming that the cleaner has changed behavior since 2.16 and that it's normal and wanted that it doesn't escape attribute values any more, correct?
Yes it helps,, thanks. I hadn't realized we were initializing the XWikiDOMSerialize with escapeXml=false (had forgotten). I tried putting it to true and indeed it fixes this issue, but breaks 3 other tests (which is why it was off before ;)): Problem 1 @Test public void verifyEntitiesAreNotBroken() { assertHTML("<p>É</p>", "É"); } Generates <html><head></head><body><p>É</p></body></html> instead of <html><head></head><body><p>É</p></body></html>. Problem 2 @Test public void specialCharacters()...
Ok I've done more tests: With HC 2.16 if I check the content of the cleaned node, I can see that the attribute has proper escaped values, i.e. aaa"bbb&ccc>ddd<eee With HC 2.22, the value is not escaped aaa"bbb&ccc>ddd<eee If I replace XWikiDOMSerializer by DOMSerializer with HC 2.22, I still get the problem and the test is not passing so the issue is not coming from XWikiDOMSerializer but from the cleaner it seems. Any idea? Thanks
Thanks a lot Scott for this code/diff :) I've tried it but it doesn't work though. I'll need to look at the commit 521 a bit later. Thx
Hi Scott! You're as fast as usal :) I would have thought about some XWiki-related thing but for the fact that keeping the same code and just changing the HC version in the POM changes the result so this clearly suggests a change in behavior. Maybe HC between 2.16 and 2.22 changed some default config values? Note that "setAllowHtmlInsideAttributes(false)" is the default and we're not setting it (I've set it to test and same result). FTR this is the config we have: https://github.com/xwiki/xwiki-commons/blob/d1657b0da41bc6fdca0cefd6e03b2a3705b17cec/xwiki-commons-core/xwiki-commons-xml/src/main/java/org/xwiki/xml/internal/html/DefaultHTMLCleaner.java#L211...
For reference this is out issue on the XWiki side: https://jira.xwiki.org/browse/XCOMMONS-1164
HC doesn't properly escape < and > characters anymore in attributes
Confirmed! The problem is on XWiki side... Seems I can't close this issue though... Thx and sorry again for the false positive.
Confired! The problem is on XWiki side... Seems I can't close this issue though... Thx and sorry again for the false positive.
hmmm hold on, I think the problem is comig from filters we have in XWiki. I'm debugging further and will close this issue if it's on our side... sorry about that.
Problems cleaning IFRAME
This is awesome, thanks! I've tried it and it works just fine ;) So the only difference...
ok I've retested and it's much better :) Remaining issues for XWiki: Using //instead...
Thanks for looking into this Scott!
ok Scott, so I've slowly examined all the XWiki unit tests related to CData inside...
Guillaume, what does IE10 gives (see https://sourceforge.net/p/htmlcleaner/bugs/...
ok so I guess we need to test your input above in IE. I've checked and XWiki supports...
So I think the consensus emerging here is that HC is correct on (1) and (3), but...
Would you also know if what is at https://github.com/xwiki/xwiki-commons/blob/a6976cfccfa4713dcb1ced7ee30cb28475d6f6fc/xwiki-commons-core/xwiki-commons-xml/src/main/java/org/htmlcleaner/XWikiDOMSerializer.java#L265-L265...
Hmm the probem is that I really don't remember how I coded the XWikiDOMSerializer...
Hi Scott. Thanks for working on those. I've upgraded to HC 2.19 and unfortunately...
Awesome Scott! I see you've fixed several issues I had raised thanks a lot for that....
Thanks Scott, that would be awesome to have all our CDATA test pass with htmlcleaner....
Ok after removing our custom XWikiDOMSerializer I get several failing tests. Just...
I've rebuilt and tested trunk without making any modification to xwiki (ie I still...
I've rebuilt and tested trunk without making any modification to xwiki (ie I still...
If the original content of the script tag is invalid, I'm not sure there's much you...
See http://stackoverflow.com/a/4227942
Oh I think I understand. The question is not whether to have CDATA or not. It's whether...
Oh I think I understand. The question is not whether to have CDATA or not. It's whether...
I probably don't understand the issue. At first glance, if you ask for useCdataFor="script,style"...
Regarding this last test case, you want to support it right (it's just not done ...
Thanks for looking into this Scott! Yes, I couldn't find another solution either...
cool!
Note: XWikiDOMSerialized has now been moved to the org.htmlcleaner package to circumvent...
Several issues with CDATA blocks
DomSerializer doesn't seem to take into account the namespacesAware configuration
Make it easier to extend DomSerializer
Thanks for answering. The generated jjMoveNfa_0 method is huge (over 4000+ lines)...
Error while instrumenting class com/steadystate/css/parser/SACParserCSS21TokenManager
great!
Still planned for tomorrow? :) Thanks!
ok, waiting for that date eagerly then :) Thanks!
Hi Scott, The XWiki project is waiting for HC 2.10 to be released in order to get...
The following works but it's probably not correct: tagInfo = new TagInfo("legend",...
Could someone help out on this since I don't fully understand the parameters to use...
So the regression was done by patmoore in: r203 | patmoore | 2013-02-26 05:53:49...
Seems it's just missing from DefaultTagProvider()... I see that it was added in rev...
ok found the problem, it's apparently an unknown tag: cp.setOmitUnknownTags(true);...
hmm actually if I modify it like this it passes so the problem is elsewhere... @Test...
Found a test for it in HtmlCleanerTest but this test has no asserts! @Test public...
Regression: <legend> tag is stripped
Thanks a lot Scott! :)
CleanerProperties defaultProperties = new CleanerProperties(); defaultProperties.setOmitUnknownTags(true);...
I've tested again with 2.7 and by upgrading to 2.8 and the attribute is still not...
I've tested again with 2.7 and by upgrading to 2.8 and the attribute is still not...
Hi Scott, Thanks for looking into this. The use case I had is described at https://sourceforge.net/p/htmlcleaner/bugs/99/#cfc2...
Getting ready for a 2.8 release? :) Too bad this is slipping from 2.8 since I consider...