htmlparser-user — The user mailing list for users of the htmlparser library

[Htmlparser-user] Need Help

[mani k. <maj...@gm...>]

Hi,

I am new to the HTML parser. I am trying to use the parse to grab all the
images referred in the page. can you please help me how to do that. ?

-- 
ALWAYS KEEP SMILING

 FOR U EVER,

     G.MANIKANDAN

Re: [Htmlparser-user] XML parsing and NULL children

[Derrick O. <der...@gm...>]

If you want these tags to contain children you have to do more work and make
them into composite tags:
see http://htmlparser.sourceforge.net/faq.html#composite

On Wed, Dec 15, 2010 at 11:58 AM, Francesco Fontana <
fra...@gm...> wrote:

> Thank you for the answer, I've tried, but nothing happens..
> The program finds all the right Nodes (with upper and with lower
> case), but everyone has node.getChildren()==NULL...
>
> > Try upper case tag names.
> > NodeFilter singleFieldFilter=new TagNameFilter("FIELD");
> > NodeFilter multipleFieldFilter=new TagNameFilter("FIELD_LIST");
>
>
> Thanks a lot, any more suggestion?
>
> Francesco
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>

Re: [Htmlparser-user] XML parsing and NULL children

[Francesco F. <fra...@gm...>]

Thank you for the answer, I've tried, but nothing happens..
The program finds all the right Nodes (with upper and with lower
case), but everyone has node.getChildren()==NULL...

> Try upper case tag names.
> NodeFilter singleFieldFilter=new TagNameFilter("FIELD");
> NodeFilter multipleFieldFilter=new TagNameFilter("FIELD_LIST");


Thanks a lot, any more suggestion?

Francesco

Re: [Htmlparser-user] XML parsing and NULL children

[Derrick O. <der...@gm...>]

Try upper case tag names.
        NodeFilter singleFieldFilter=new TagNameFilter("FIELD");
        NodeFilter multipleFieldFilter=new TagNameFilter("FIELD_LIST");


On Tue, Dec 14, 2010 at 11:18 AM, Francesco Fontana <
fra...@gm...> wrote:

> Hi,
> I'm trying to parse an xml file, but I receive the message "WARNING: URL
> [filename] does not contain text"...
> When I put a watch on a filter, every node found has id and attributes, but
> the children still null...
>
> The code is really simple, the xml is well formed and have a dtd. Someone
> knows what I'm doing wrong?
> Thank you very much,
> Francesco
>
> ------------------------- java code --------------------------
>      public FilterSet setBaseValues(String siteXMLName) throws
> ParserException {
>         NodeFilter singleFieldFilter=new TagNameFilter("field");
>         NodeFilter multipleFieldFilter=new TagNameFilter("field_list");
>         NodeList singleFieldList=new NodeList();
>         NodeList multipleFieldList=new NodeList();
>
>         Parser parser=new Parser("./"+siteXMLName);
>         for (NodeIterator e = parser.elements(); e.hasMoreNodes();) {
>             Node node = e.nextNode();
>             node.collectInto(singleFieldList, singleFieldFilter);
>             node.collectInto(multipleFieldList, multipleFieldFilter);
>         }
> }
> ------------------------------------------------------------------
>
> ------------------------- try.xml content ----------------------
> <?xml version="1.0"?>
> <!DOCTYPE site SYSTEM "hsh.dtd">
> <site>
>     <field id="image" type="attribute">
>         <caption>a caption</caption>
>         <attribute>src</attribute>
>     </field>
>     <field id="description" type="text">
>         <caption>text text</caption>
>     </field>
>     <field_list list="parent_name" type="text">
>         <caption>another caption</caption>
>         <names>
>             <name id="field1">Field 1</name>
>             <name id="field2">Field 2</name>
>         </names>
>     </field_list>
> </site>
> ----------------------------------------------------------------
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] XML parsing and NULL children

[Francesco F. <fra...@gm...>]

Hi,
I'm trying to parse an xml file, but I receive the message "WARNING: URL
[filename] does not contain text"...
When I put a watch on a filter, every node found has id and attributes, but
the children still null...

The code is really simple, the xml is well formed and have a dtd. Someone
knows what I'm doing wrong?
Thank you very much,
Francesco

------------------------- java code --------------------------
     public FilterSet setBaseValues(String siteXMLName) throws
ParserException {
        NodeFilter singleFieldFilter=new TagNameFilter("field");
        NodeFilter multipleFieldFilter=new TagNameFilter("field_list");
        NodeList singleFieldList=new NodeList();
        NodeList multipleFieldList=new NodeList();

        Parser parser=new Parser("./"+siteXMLName);
        for (NodeIterator e = parser.elements(); e.hasMoreNodes();) {
            Node node = e.nextNode();
            node.collectInto(singleFieldList, singleFieldFilter);
            node.collectInto(multipleFieldList, multipleFieldFilter);
        }
}
------------------------------------------------------------------

------------------------- try.xml content ----------------------
<?xml version="1.0"?>
<!DOCTYPE site SYSTEM "hsh.dtd">
<site>
    <field id="image" type="attribute">
        <caption>a caption</caption>
        <attribute>src</attribute>
    </field>
    <field id="description" type="text">
        <caption>text text</caption>
    </field>
    <field_list list="parent_name" type="text">
        <caption>another caption</caption>
        <names>
            <name id="field1">Field 1</name>
            <name id="field2">Field 2</name>
        </names>
    </field_list>
</site>
----------------------------------------------------------------

Re: [Htmlparser-user] Excluding some tags

[Derrick O. <der...@gm...>]

Tgat's not valid HTML. You'll want  to turn strict script scanning off then.

On Wed, Nov 17, 2010 at 7:44 AM, Manish Kashyap <ma...@we...>wrote:

> Thanks for the revert Derrick. So, here's the real problem -
> I do want to retain the script tag. At the same time, I want to override
> all the links in the page. The parser doesn't play nice. Consider the
> scenario underneath for an html
>
> <script>
>>  document.write("<a href='/jslink'>JS Link</a>")
>> </script>
>> <a href="/somelink">Some link</a>
>>
>
> To me the string literal inside script tag above is not a link at all.
> However, when I try to fetch all the <a> using the parser it would give me
> both of the above. Is there a way to not get the <a>s which are not in the
> <script> tag?
>
> Thanks
> Manish
>
> On Tue, Nov 16, 2010 at 11:39 PM, Derrick Oswald <der...@gm...
> > wrote:
>
>> Although the filter is correct, the tag enclosing the <script> tag is
>> accepted, and with it it's child tags - including the <script> tag.
>> Maybe a way to do it is to override the ScriptTag class with MyScriptTag
>> so that it prints nothing in the toHtml () call.
>> Add the overridden class to the PrototypicalNodeFactory as described
>> here: http://htmlparser.sourceforge.net/faq.html#composite, and then get
>> all tags and print the whole thing with System.out.println (this.parser.parse(null).toHtml
>> ());
>>
>> On Tue, Nov 16, 2010 at 8:19 AM, Manish Kashyap <ma...@we...>wrote:
>>
>>> This indeed is a newbie question. I could not find a work around to
>>> exclude some tags (<script> in my case) while parsing.
>>>
>>> I tried using the NotFilter as underneath, but it didn't work as I got
>>> all the <script> tags in my NodeList -
>>>
>>>> NotFilter noScriptFilter = new NotFilter();
>>>> noScriptFilter.setPredicate(new NodeFilter(){
>>>>   public boolean accept(Node currNode){
>>>>     if(currNode instanceof TagNode){
>>>>
>>>> if(((TagNode)currNode).getRawTagName().equalsIgnoreCase("script")){
>>>>          return true;
>>>>       }
>>>>     }
>>>>     return false;
>>>>   }
>>>> });
>>>> NodeList allNodes = this.parser.parse(noScriptFilter);
>>>>
>>>
>>> Would appreciate if someone can guide me throgh this.
>>>
>>> Thanks
>>> Manish
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Beautiful is writing same markup. Internet Explorer 9 supports
>>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>>> Spend less time writing and  rewriting code and more time creating great
>>> experiences on the web. Be a part of the beta today
>>> http://p.sf.net/sfu/msIE9-sfdev2dev
>>> _______________________________________________
>>> Htmlparser-user mailing list
>>> Htm...@li...
>>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9 supports
>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>> Spend less time writing and  rewriting code and more time creating great
>> experiences on the web. Be a part of the beta today
>> http://p.sf.net/sfu/msIE9-sfdev2dev
>> _______________________________________________
>> Htmlparser-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>
>>
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today
> http://p.sf.net/sfu/msIE9-sfdev2dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

Re: [Htmlparser-user] Excluding some tags

[Manish K. <ma...@we...>]

Sorry i modify my question ignore the previous one.

Is there a way to get the <a>s which are not in the <script> tag?

Thanks,
MAnish

On Wed, Nov 17, 2010 at 12:14 PM, Manish Kashyap <ma...@we...>wrote:

> Thanks for the revert Derrick. So, here's the real problem -
> I do want to retain the script tag. At the same time, I want to override
> all the links in the page. The parser doesn't play nice. Consider the
> scenario underneath for an html
>
> <script>
>>  document.write("<a href='/jslink'>JS Link</a>")
>> </script>
>> <a href="/somelink">Some link</a>
>>
>
> To me the string literal inside script tag above is not a link at all.
> However, when I try to fetch all the <a> using the parser it would give me
> both of the above. Is there a way to not get the <a>s which are not in the
> <script> tag?
>
> Thanks
> Manish
>
>
> On Tue, Nov 16, 2010 at 11:39 PM, Derrick Oswald <der...@gm...
> > wrote:
>
>> Although the filter is correct, the tag enclosing the <script> tag is
>> accepted, and with it it's child tags - including the <script> tag.
>> Maybe a way to do it is to override the ScriptTag class with MyScriptTag
>> so that it prints nothing in the toHtml () call.
>> Add the overridden class to the PrototypicalNodeFactory as described
>> here: http://htmlparser.sourceforge.net/faq.html#composite, and then get
>> all tags and print the whole thing with System.out.println (this.parser.parse(null).toHtml
>> ());
>>
>> On Tue, Nov 16, 2010 at 8:19 AM, Manish Kashyap <ma...@we...>wrote:
>>
>>> This indeed is a newbie question. I could not find a work around to
>>> exclude some tags (<script> in my case) while parsing.
>>>
>>> I tried using the NotFilter as underneath, but it didn't work as I got
>>> all the <script> tags in my NodeList -
>>>
>>>> NotFilter noScriptFilter = new NotFilter();
>>>> noScriptFilter.setPredicate(new NodeFilter(){
>>>>   public boolean accept(Node currNode){
>>>>     if(currNode instanceof TagNode){
>>>>
>>>> if(((TagNode)currNode).getRawTagName().equalsIgnoreCase("script")){
>>>>          return true;
>>>>       }
>>>>     }
>>>>     return false;
>>>>   }
>>>> });
>>>> NodeList allNodes = this.parser.parse(noScriptFilter);
>>>>
>>>
>>> Would appreciate if someone can guide me throgh this.
>>>
>>> Thanks
>>> Manish
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Beautiful is writing same markup. Internet Explorer 9 supports
>>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>>> Spend less time writing and  rewriting code and more time creating great
>>> experiences on the web. Be a part of the beta today
>>> http://p.sf.net/sfu/msIE9-sfdev2dev
>>> _______________________________________________
>>> Htmlparser-user mailing list
>>> Htm...@li...
>>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9 supports
>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>> Spend less time writing and  rewriting code and more time creating great
>> experiences on the web. Be a part of the beta today
>> http://p.sf.net/sfu/msIE9-sfdev2dev
>> _______________________________________________
>> Htmlparser-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>
>>
>

Re: [Htmlparser-user] Excluding some tags

[Manish K. <ma...@we...>]

Thanks for the revert Derrick. So, here's the real problem -
I do want to retain the script tag. At the same time, I want to override all
the links in the page. The parser doesn't play nice. Consider the scenario
underneath for an html

<script>
>  document.write("<a href='/jslink'>JS Link</a>")
> </script>
> <a href="/somelink">Some link</a>
>

To me the string literal inside script tag above is not a link at all.
However, when I try to fetch all the <a> using the parser it would give me
both of the above. Is there a way to not get the <a>s which are not in the
<script> tag?

Thanks
Manish

On Tue, Nov 16, 2010 at 11:39 PM, Derrick Oswald
<der...@gm...>wrote:

> Although the filter is correct, the tag enclosing the <script> tag is
> accepted, and with it it's child tags - including the <script> tag.
> Maybe a way to do it is to override the ScriptTag class with MyScriptTag so
> that it prints nothing in the toHtml () call.
> Add the overridden class to the PrototypicalNodeFactory as described
> here: http://htmlparser.sourceforge.net/faq.html#composite, and then get
> all tags and print the whole thing with System.out.println (this.parser.parse(null).toHtml
> ());
>
> On Tue, Nov 16, 2010 at 8:19 AM, Manish Kashyap <ma...@we...>wrote:
>
>> This indeed is a newbie question. I could not find a work around to
>> exclude some tags (<script> in my case) while parsing.
>>
>> I tried using the NotFilter as underneath, but it didn't work as I got all
>> the <script> tags in my NodeList -
>>
>>> NotFilter noScriptFilter = new NotFilter();
>>> noScriptFilter.setPredicate(new NodeFilter(){
>>>   public boolean accept(Node currNode){
>>>     if(currNode instanceof TagNode){
>>>       if(((TagNode)currNode).getRawTagName().equalsIgnoreCase("script")){
>>>          return true;
>>>       }
>>>     }
>>>     return false;
>>>   }
>>> });
>>> NodeList allNodes = this.parser.parse(noScriptFilter);
>>>
>>
>> Would appreciate if someone can guide me throgh this.
>>
>> Thanks
>> Manish
>>
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9 supports
>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>> Spend less time writing and  rewriting code and more time creating great
>> experiences on the web. Be a part of the beta today
>> http://p.sf.net/sfu/msIE9-sfdev2dev
>> _______________________________________________
>> Htmlparser-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>
>>
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today
> http://p.sf.net/sfu/msIE9-sfdev2dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

Re: [Htmlparser-user] Excluding some tags

[Derrick O. <der...@gm...>]

Although the filter is correct, the tag enclosing the <script> tag is
accepted, and with it it's child tags - including the <script> tag.
Maybe a way to do it is to override the ScriptTag class with MyScriptTag so
that it prints nothing in the toHtml () call.
Add the overridden class to the PrototypicalNodeFactory as described here:
http://htmlparser.sourceforge.net/faq.html#composite, and then get all tags
and print the whole thing with System.out.println
(this.parser.parse(null).toHtml
());

On Tue, Nov 16, 2010 at 8:19 AM, Manish Kashyap <ma...@we...>wrote:

> This indeed is a newbie question. I could not find a work around to exclude
> some tags (<script> in my case) while parsing.
>
> I tried using the NotFilter as underneath, but it didn't work as I got all
> the <script> tags in my NodeList -
>
>> NotFilter noScriptFilter = new NotFilter();
>> noScriptFilter.setPredicate(new NodeFilter(){
>>   public boolean accept(Node currNode){
>>     if(currNode instanceof TagNode){
>>       if(((TagNode)currNode).getRawTagName().equalsIgnoreCase("script")){
>>          return true;
>>       }
>>     }
>>     return false;
>>   }
>> });
>> NodeList allNodes = this.parser.parse(noScriptFilter);
>>
>
> Would appreciate if someone can guide me throgh this.
>
> Thanks
> Manish
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today
> http://p.sf.net/sfu/msIE9-sfdev2dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] Excluding some tags

[Manish K. <ma...@we...>]

This indeed is a newbie question. I could not find a work around to exclude
some tags (<script> in my case) while parsing.

I tried using the NotFilter as underneath, but it didn't work as I got all
the <script> tags in my NodeList -

> NotFilter noScriptFilter = new NotFilter();
> noScriptFilter.setPredicate(new NodeFilter(){
>   public boolean accept(Node currNode){
>     if(currNode instanceof TagNode){
>       if(((TagNode)currNode).getRawTagName().equalsIgnoreCase("script")){
>          return true;
>       }
>     }
>     return false;
>   }
> });
> NodeList allNodes = this.parser.parse(noScriptFilter);
>

Would appreciate if someone can guide me throgh this.

Thanks
Manish

Re: [Htmlparser-user] Best way to extract all the links from a HTML page

[Derrick O. <der...@gm...>]

If you set the document base href on the page (see how BaseHrefTag handles
it in doSemanticAction, basically page.setBaseUrl (base)), then the links
you get back can be 'canonized' as you call it by using the
page getAbsoluteURL (String link, boolean strict) method.

On Tue, Oct 12, 2010 at 10:50 PM, Santiago Basulto <
san...@gm...> wrote:

> Hello people.
>
> I'm starting with HTMLParser. It seems a great library. I've doing
> some benchmarking and runs really fast.
>
> Now i'm trying to improve it a little bit.
>
> In my software, i use something like this to extract all links:
>
> public class LinkVisitor extends NodeVisitor {
>        private Set<String> links = new HashSet<String>(100);
>        public LinkVisitor(){
>        }
>        public void visitTag(Tag tag) {
>                String name = tag.getTagName();
>                if ("a".equalsIgnoreCase(name)){
>                        String hrefValue = tag.getAttribute("href");
>                        links.add(tag.getAttribute("href"));
>                }
>        }
>        public Set<String> getLinks(){
>                return this.urls;
>        }
>
> }
>
> But, reading a little bit i found other classes that may help, but
> don't know how to use them. Can anyone help me out?
>
> The idea is to extract all the links from a String (that contains an
> HTML page already read from an URLConnection). Is there anyway to
> "Canonize" them? I mean, if the href says "/food/fruits/2" convert it
> to "http://www.foodsite.com/home/fruits/2"?
>
>
> Thanks a lot!
>
> --
> Santiago Basulto.-
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
> Spend less time writing and  rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>

Re: [Htmlparser-user] Best way to extract all the links from a HTML page

[Stanislav O. <orl...@gm...>]

Hi
You may try to use filters (org.htmlparser.filters). In this way you'll
get all link tags from the page:

Parser parser = parserMain.getParser(parseURL);

NodeList links = null;
try {
    links = parser.parse(new TagNameFilter("a"));
} catch (ParserException ex) {
    logger.error(null, ex);
}

for (SimpleNodeIterator sni = links.element(); sni.hasMoreNodes();) {
    Node node = sni.nextNode();
    if (node instanceof LinkTag) {
        LinkTag lt = (LinkTag) node;
        // link text - lt.getLinkText()
        // link href  - lt.getLink()
    }
}



On Tue, 2010-10-12 at 17:50 -0300, Santiago Basulto wrote:

> Hello people.
> 
> I'm starting with HTMLParser. It seems a great library. I've doing
> some benchmarking and runs really fast.
> 
> Now i'm trying to improve it a little bit.
> 
> In my software, i use something like this to extract all links:
> 
> public class LinkVisitor extends NodeVisitor {
>         private Set<String> links = new HashSet<String>(100);
> 	public LinkVisitor(){
> 	}
> 	public void visitTag(Tag tag) {
> 		String name = tag.getTagName();
> 		if ("a".equalsIgnoreCase(name)){
> 			String hrefValue = tag.getAttribute("href");
> 			links.add(tag.getAttribute("href"));
> 		}
> 	}
> 	public Set<String> getLinks(){
> 		return this.urls;
> 	}
> 	
> }
> 
> But, reading a little bit i found other classes that may help, but
> don't know how to use them. Can anyone help me out?
> 
> The idea is to extract all the links from a String (that contains an
> HTML page already read from an URLConnection). Is there anyway to
> "Canonize" them? I mean, if the href says "/food/fruits/2" convert it
> to "http://www.foodsite.com/home/fruits/2"?
> 
> 
> Thanks a lot!
> 

[Htmlparser-user] Best way to extract all the links from a HTML page

[Santiago B. <san...@gm...>]

Hello people.

I'm starting with HTMLParser. It seems a great library. I've doing
some benchmarking and runs really fast.

Now i'm trying to improve it a little bit.

In my software, i use something like this to extract all links:

public class LinkVisitor extends NodeVisitor {
        private Set<String> links = new HashSet<String>(100);
	public LinkVisitor(){
	}
	public void visitTag(Tag tag) {
		String name = tag.getTagName();
		if ("a".equalsIgnoreCase(name)){
			String hrefValue = tag.getAttribute("href");
			links.add(tag.getAttribute("href"));
		}
	}
	public Set<String> getLinks(){
		return this.urls;
	}
	
}

But, reading a little bit i found other classes that may help, but
don't know how to use them. Can anyone help me out?

The idea is to extract all the links from a String (that contains an
HTML page already read from an URLConnection). Is there anyway to
"Canonize" them? I mean, if the href says "/food/fruits/2" convert it
to "http://www.foodsite.com/home/fruits/2"?


Thanks a lot!

-- 
Santiago Basulto.-

[Htmlparser-user] Hey htmlparser-user 80% OFF. Lifeguard

[TopPfizer's P. <htm...@li...>]

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<title>Newsletter</title>
</head>
<body>
<table width="620" cellpadding="0" cellspacing="0" align="center">
	<tr>
		<td>
		<div style="text-align: center">
			<font size="1" face="arial"><a href="http://as.portfinger.ru/?election=Ff7b614C252">View as a web
			page</a></font><br><br>
			<a href="http://on.portfinger.ru/?Law=27C91524D4">
			<img alt="Unable to view this image? Click here" src="http://Gazette.portfinger.ru/at.gif" style="border-width: 0px"></a><br>
			<a href="http://writers.Europe.com/that/parallel.php?Nigeria=3Dc8FFAeD3">
			<img alt="" src="http://his.flag.com/Dentistry/picture.gif" style="border-width: 0px"></a><br>
			<a href="http://the.or.com/was/English.php?industrial=620aD72ef63">
			<img alt="" src="http://aircraft.related.com/Cuban/The.gif" style="border-width: 0px"></a><br>
			<a href="http://December.is.com/of/the.php?uninhabited=4A7fA3c34E1">
			<img alt="" src="http://same.in.com/scavenged/for.gif" style="border-width: 0px"></a><br>
			<a href="http://the.the.com/political/homes.php?median=575CF17a1A">
			<img alt="" src="http://Saxon.overall.com/by/ed.gif" style="border-width: 0px"></a><br>
			<a href="http://for.Germanic.com/use/Great.php?in=F4bFd7783f9">
			<img alt="" src="http://being.MSA.com/an/customs.gif" style="border-width: 0px"></a><br>
			<span style="color: #EEE2E2; font-size: xx-small; font-family: Arial, Helvetica, sans-serif">
			The latter include fish-canning and meat-processing plants in the northern regions, as well as about 25 factories in the Mogadishu area, which manufacture pasta, mineral water, confections, plastic bags, fabric, hides and skins, detergent and soap, aluminum, foam mattresses and pillows, fishing boats, carry out packaging, and stone processing.<br>
			Bureau of Democracy, Human Rights, and Labor (2006-09-15).<br>
			Alfred Knopf retired in 1972, becoming chairman emeritus of the firm until his death in 1984.<br>
			The war was the largest and most destructive in human history, with 60million dead across the world.<br>
			<img alt="" src="http://George.students.com/May/laws.gif" style="border-width: 0px">
			Luce and His Empire (1972), outdated popular history.<br>
			Film industry has largely been based in and around Hollywood, California.<br>
			Displaying available languages on a multilingual website or software.<br>
			<img alt="" src="http://Channel.David.com/which/for.gif" style="border-width: 0px">
			Guillaume du Bellay, writer and general.<br>
			The current Constitution of Florida was ratified on November 5, 1968.<br>
			Census Bureau, Population Division.<br>
			President of the Executive Council.<br>
			Journalistic accounts and televised footage of the daily deprivation and indignities suffered by southern blacks, and of segregationist violence and harassment of civil rights workers and marchers, produced a wave of sympathetic public opinion that convinced the majority of Americans that the Civil Rights Movement was the most important issue in American politics in the early 1960s.<br>
			<img alt="" src="http://The.In.com/to/flags.gif" style="border-width: 0px">
			It requires a cadmium atom to capture sufficient neutrons and then undergo Beta decay.<br>
			King believed that organized, nonviolent protest against the system of southern segregation known as Jim Crow laws would lead to extensive media coverage of the struggle for black equality and voting rights.<br>
			New York Film Critics Circle Award for Best Actress.<br>
			Had children under the age of 18 living with them, 36.<br>
			</span>
		</div>
		<hr></td>
	</tr>
	<tr>
		<td><font size="1" face="arial">This e-mail message was sent to:
		htm...@li... <p>
		<a href="http://were.portfinger.ru/?her=cE0896f323A">Unsubsribe</a></p>
	<p>
	(c) 2007 of Lawrence A <a href="http://Eight.portfinger.ru/?considered=90ae2ECfa83">Privacy Statement</a>.<br>
	All rights reserved.</p>
	</font>
 </body>
</HTML>

[Htmlparser-user] The secret of Macho Men

[<aje...@as...>]

Love her with all your heart and might, and also every inch of your rod.
http://www.pooldeal.ru/

Re: [Htmlparser-user] How to parser HTML string

[Derrick O. <der...@gm...>]

The same constructor for the Parser that takes a string [Parser (String
resource) and Parser (String resource, ParserFeedback feedback)] checks for
a string that starts with an angle bracket ('<') and if so it assumes it is
HTML - otherwise it is assumes to be some sort of URL.

If you already have a parser you can use setResource and pass it the HTML,
since this is the same mechanism the constructor uses...

    /**
     * Set the html, a url, or a file.
     * @param resource The resource to use.
     * @exception IllegalArgumentException if <code>resource</code> is
<code>null</code>.
     * @exception ParserException if a problem occurs in connecting.
     */
    public void setResource (String resource)


On Sun, Aug 1, 2010 at 8:57 AM, Mohammad Waqar <waq...@gm...>wrote:

> how can i parse an HTML string stored in a variable?
>
> Vakar
>
>
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] Friend htmlparser-user, today new Sale starts. which

[AmericanViagra on-l. <htm...@li...>]

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>no made against portability Sir of in Newsletter</title>
</head>
<body>
	<table style="width: 700px;" align="center" cellspacing="0" cellpadding="0">
		<tr>
			<td style="font-family: Arial, Helvetica, sans-serif; font-size: x-small; text-align: center;">
			If you are unable to see the message below, <a href="http://diimpulsion.co.cc/chalk36.html">
			click here</a> to view.</td>
		</tr>
		<tr>
			<td style="text-align: center">
			<br />
			<a href="http://diimpulsion.co.cc/chalk36.html"><img src="http://diimpulsion.co.cc/chalk36.jpg" style="border:0px" alt="Click here to see graphics" /></a></td>
		</tr>
		<tr>
			<td style="font-size: x-small; color: #F0F0F0">
			<img src="http://Stephanie.a.com/instead/Oct.jpg" style="border:0px" alt="" />
			<br />
The Istrian Y 
highway is being built in 
two phases.
Quisling, 
as minister president, later formed a collaborationist 
government under German control.
The Northern Territory was founded in 1911 when it was 
excised from South 
Australia.<strong>Mother Angelica</strong> Awarded Top Honor by Pope Benedict XVI.
See a complete list of group 
identifiers.
Com, struck a deal with Col Needham 
and other principal shareholders to buy IMDb outright and attach it to Amazon as a subsidiary, private 
company.
Melbourne, The Macquarie Library Pty Ltd.
Television stations in North Platte.Department of 
the Environment and Heritage, Australian Government.Its diverse geography ranges <center>from</center> the mountainous regions of the Ozarks and the Ouachita Mountains, which make up the U.
May 
2, 
2005, Media Matters for America.
Paris is de 
facto capital 
of France (seat of the 
Presidency, the Government, the National Assembly and the Senate), 
but the parliament holds its joint congresses in Versailles.
Jim Mecir <center>- Major</center> League Baseball player.
Web Site Design 
and Hosting by 
LogicalSolutions.Theoretical models of human development.The 4th 
District is currently represented 
by Republican Steve Buyer.It should be pronounced in three (3) syllables, with the final "s" silent, the "a" in each 
syllable with the Italian sound, and the 
accent on the first and last syllables.
The second district lies in north-central part of the state and 
includes all of LaPorte, St.
This disambiguation page lists articles associated with the same title.
The activity is intrinsically rewarding, so there is an effortlessness of action.
The west coast of southern Norway and the coast of northern Norway present 
some of the most visually impressive coastal 
sceneries 
in the world.
Redistricted from the 10th district, Died.
Olav at the Nidaros shrine, and with them, much of 
the contact with cultural 
and economic life in the rest of Europe.
English language Broadcast television networks in the United States.
Though for 
a time as a Kansas 
City team, the "A"s wore "Kansas City" on 
their road jerseys and an interlocking "KC" 
on the cap, upon moving to 
Oakland the 
"A" cap emblem was 
restored, although in 1970 an "apostrophe-s" was added to 
the cap 
and uniform emblem to reflect the fact 
that then-team owner 
Charles O.
The population density of the state is 51.
He also supported a constitutional amendment requiring a balanced federal budget.
The first wave of Australian feature film production"".
As the Danish kingdom found itself on 
the losing side 
in 1814, 
it was forced, under terms of the Treaty of Kiel, 
to cede Norway to the king of Sweden, while the old 
Danish-Norwegian provinces of Iceland, Greenland and the Faroe Islands remained 
with the Danish crown.
Jeremy Wall - Founding pianist of the 
Jazz Fusion Band Spyro Gyra.
A former United States 
Representative, Hindman commanded Confederate forces at the Battle of Cane Hill and Battle of Prairie Grove.
Australia has a free-market economy with high GDP 
per capita 
and low rate of poverty.
As such, 
Norway is fundamentally structured as a representative democracy.
This United States Congress -related article is a stub.
In 
unitary states, "administrative center" or other similar 
terms 
are typically used.
Rondeslottet in Rondane National Park, Eastern Norway.
			</td>
		</tr>
		<tr>
			<td style="font-family: Arial, Helvetica, sans-serif; font-size: small">
			<br />
			© 2009 Macintyre Inc. All rights reserved.<br />
			<br />
			<a href="http://diimpulsion.co.cc/chalk36.html">Unsubscribe</a></td>
		</tr>
	</table>
	<br />
</body>
</html>

[Htmlparser-user] How to parser HTML string

[Mohammad W. <waq...@gm...>]

how can i parse an HTML string stored in a variable?

Vakar

[Htmlparser-user] ConnectionTimeout after reconnect, caching?

[Johann H. <h.h...@ic...>]

Hello community,
I am writing a website parser with htmlparser and I think it's a great 
library.
My problem is, the website I'm parsing shows me a captcha after a 
certain number of crawls.
As a workaround I wrote a redial routine to reconnect my router and get 
a new ip.
That is working quite well, but my problem is, that my jvm seems to 
cache DNS.
I read this post http://forum.vis.ethz.ch/showthread.php?t=13457 and 
applied everything which is supposed there,
but still I can't continue parsing after a reconnect and I get a 
ConnectionTimeoutException from htmlparser.
It seems, that there might still be some kind of cache.
Could anybody tell me, how I can get the new instance of Parser to 
connect after a reconnect.

Thank you.
Hans.

[Htmlparser-user] Reminder: geeraza wants to add you to his friends list on Netlog

[geeraza <nor...@ne...>]

Hi,

Since Tuesday 20 July 2010, you have been invited by 1 of your contacts to join Netlog, 
the social community for over 49 million young people.

[---- Invitation from geeraza ---- ]
34 yrs - male - Baden-Wurttemberg (Germany)
Connect with geeraza:
http://en.netlog.com/go/mailurl/-bT05ODQwODIyNjImbD0xJmdtPTE2JnU9JTJGZ28lMkZyZWdpc3RlciUyRmlkJTNEMjEwNTI2NTY0NSUyNnIlM0R0MQ__
	

On Netlog you can:

- Create your own web page
- Extend your social network
- Publish your music playlists- Share pictures and videos- Post blogs
- And much more ... ....

http://en.netlog.com/go/mailurl/-bT05ODQwODIyNjImbD0xJmdtPTE2JnU9JTJGZ28lMkZyZWdpc3RlciUyRmlkJTNEMjEwNTI2NTY0NSUyNnIlM0R0OTg0MDgyMjYyMQ__

----------------------------------------------------------------
Don't want to receive invitations from your friends anymore?
http://en.netlog.com/go/mailurl/-bT05ODQwODIyNjImbD0zJmdtPTE2JnU9JTJGZ28lMkZub21haWxzJTJGaW52aXRlJTJGZW1haWwlM0QtYUhSdGJIQmhjbk5sY2kxMWMyVnlRR3hwYzNSekxuTnZkWEpqWldadmNtZGxMbTVsZEFfXyUyNmNvZGUlM0QxMTc1MjQxNSUyNmlkJTNEMjEwNTI2NTY0NQ__



----------------------------------------------------------------
Netlog NV/SA. E. Braunplein 18. B-9000 Gent. Belgium BE0859635972. abu...@ne...

[Htmlparser-user] Visit my Netlog profile

[geeraza <nor...@ne...>]

Hey,

I have created a Netlog profile with my pictures, videos, blogs and events and I want to add you as a friend so you can see it. You first need to register on Netlog! When you log in, you can create your own profile.

Take a look:
http://en.netlog.com/go/mailurl/-bT05Njg2NzYwNDMmbD0xJmdtPTM3JnU9JTJGZ28lMkZyZWdpc3RlciUyRmlkJTNEMjEwNTI2NTY0NSUyNmklM0R0OTE_

Cheers,
geeraza

----------------------------------------------------------------
Don't want to receive invitations from your friends anymore?
http://en.netlog.com/go/mailurl/-bT05Njg2NzYwNDMmbD0yJmdtPTM3JnU9JTJGZ28lMkZub21haWxzJTJGaW52aXRlJTJGZW1haWwlM0QtYUhSdGJIQmhjbk5sY2kxMWMyVnlRR3hwYzNSekxuTnZkWEpqWldadmNtZGxMbTVsZEFfXyUyNmNvZGUlM0QxMTc1MjQxNSUyNmlkJTNEMjEwNTI2NTY0NSUyNmklM0R0OTI_

Re: [Htmlparser-user] HTML parser parsing script incorrectly

[Derrick O. <der...@gm...>]

Did you set STRICT false:

http://htmlparser.sourceforge.net/javadoc/org/htmlparser/scanners/ScriptScanner.html



On Wed, Jul 7, 2010 at 9:48 PM, Niket Arora <nik...@ex...>wrote:

>  I m parsing a page
> http://www.healthline.com/search?q1=how+to+improve+prostate+blood+levels using
> htmlparser api and I m getting content inside a script tag in some other tag
> and reason for this is html tags are present in a string inside javascript
> tags and are not escaped …. so htmlparser api is closing on those tags.
>
>
>
>
>
>
> ================================================================================================================================================================================================
>
>
>
> <div id="myHealthlineHeader">
>
>         <script>
>
>               if(isLoggedIn()) {
>
>                 document.write("<a href=\"/action/LogOutServlet\">Sign
> Off</a> | <a rel=\"nofollow\" href=\"/myhealthline/account_overview.jsp\">My
> Healthline</a> | Welcome, <strong>" + getNickname() + "</strong>");
>
>                 document.getElementById("myHealthlineHeader").className =
> "hl_state_top_signed_in";
>
>               } else {
>
>
>
>                 document.write("<div
> style=\"float:right;text-align:right;padding:0 5px 0
> 0;\">&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a class=\"underlineless\"
> rel=\"nofollow\"
> href=\"/yourfeedback.jsp?url=%2Fsearch%3Fq1%3Dhow%2Bto%2Bimprove%2Bprostate%2Bblood%2Blevels\">Feedback</a></div>");
>
>                 document.write("<div style=\"float:right\"><a
> class=\"underlineless\" rel=\"nofollow\" href=\"/signin.jsp\">Sign
> in</a>&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a class=\"underlineless\"
> rel=\"nofollow\" href=\"/registration.jsp\">Join Now</a>&nbsp;</div>")
>
>                 document.getElementById("myHealthlineHeader").className =
> "hl_state_top";
>
>               }
>
>         </script>
>
> </div>
>
>
>
>
> ================================================================================================================================================================================================
>
>
>
> Is there anyway to fix this issue?
>
>
>
> Regards
>
> Niket
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] HTML parser parsing script incorrectly

[Niket A. <nik...@ex...>]

I m parsing a page http://www.healthline.com/search?q1=how+to+improve+prostate+blood+levels using htmlparser api and I m getting content inside a script tag in some other tag and reason for this is html tags are present in a string inside javascript tags and are not escaped .... so htmlparser api is closing on those tags.


================================================================================================================================================================================================

<div id="myHealthlineHeader">
        <script>
              if(isLoggedIn()) {
                document.write("<a href=\"/action/LogOutServlet\">Sign Off</a> | <a rel=\"nofollow\" href=\"/myhealthline/account_overview.jsp\">My Healthline</a> | Welcome, <strong>" + getNickname() + "</strong>");
                document.getElementById("myHealthlineHeader").className = "hl_state_top_signed_in";
              } else {

                document.write("<div style=\"float:right;text-align:right;padding:0 5px 0 0;\">&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a class=\"underlineless\" rel=\"nofollow\" href=\"/yourfeedback.jsp?url=%2Fsearch%3Fq1%3Dhow%2Bto%2Bimprove%2Bprostate%2Bblood%2Blevels\">Feedback</a></div>");
                document.write("<div style=\"float:right\"><a class=\"underlineless\" rel=\"nofollow\" href=\"/signin.jsp\">Sign in</a>&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;<a class=\"underlineless\" rel=\"nofollow\" href=\"/registration.jsp\">Join Now</a>&nbsp;</div>")
                document.getElementById("myHealthlineHeader").className = "hl_state_top";
              }
        </script>
</div>

================================================================================================================================================================================================

Is there anyway to fix this issue?

Regards
Niket

[Htmlparser-user] Extract HTML Body and output as (X)HTML standards

[Oliver S. <oli...@gm...>]

Hi,

I need to read arbitrary HTML (HTML 4 transitional, XHTML 1.0 strict, ...) extract the body as a fragment and output it again as another (XHTML standard).

Reading the file is simple enough:

		Parser p = new Parser(resource);
		NodeFilter f = new NodeClassFilter(BodyTag.class);
		NodeList listOfBodies = p.extractAllNodesThatMatch(f);
		Node firstBody = listOfBodies.elementAt(0);
		NodeList bodyChildren = firstBody.getChildren();
		System.out.println(bodyChildren.toHtml());

From this hpw can I output either valid HTML 4.0 code or valid XHTML 1.0 code?

Best regards
Oliver 

[Htmlparser-user] hows it going?

[<sem...@ya...>]

http://Werger2yfjh.servepics.com?Major-Blakelock