htmlparser-user — The user mailing list for users of the htmlparser library

Re: [Htmlparser-user] Getting Started

[Sandeep K. G. <san...@gm...>]

Thanks Derrick...
I shifted to HTMLParser in Python in the mean time.
So now, i make an invocation to the python code and then redirect its output
back to my java file . :)

Sandeep.

On Mon, Jun 14, 2010 at 10:16 PM, Derrick Oswald
<der...@gm...>wrote:

> Add also the htmllexer.jar to your classpath.
>
> On Mon, Jun 14, 2010 at 8:12 AM, Sandeep Kumar Gupta <
> san...@gm...> wrote:
>
>> Hello Everyone,
>> I need to parse JSP file so I am starting with an HTML parser, but it
>> seems the binary distribution that i downloaded from Sourceforge does not
>> have all the classes( i put htmlparser.jar in classpath of my eclipse
>> project) as mentioned in the JavaDoc. ? What is it that I am doing wrong ..
>>
>> And are there initial snippets that would get me started with the HTML
>> parser ?
>>
>> --
>> Sandeep
>>
>>
>> ------------------------------------------------------------------------------
>> ThinkGeek and WIRED's GeekDad team up for the Ultimate
>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
>> lucky parental unit.  See the prize list and enter to win:
>> http://p.sf.net/sfu/thinkgeek-promo
>> _______________________________________________
>> Htmlparser-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>>
>>
>
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>


-- 
Sandeep

Re: [Htmlparser-user] Getting Started

[Derrick O. <der...@gm...>]

Add also the htmllexer.jar to your classpath.

On Mon, Jun 14, 2010 at 8:12 AM, Sandeep Kumar Gupta <san...@gm...
> wrote:

> Hello Everyone,
> I need to parse JSP file so I am starting with an HTML parser, but it seems
> the binary distribution that i downloaded from Sourceforge does not have all
> the classes( i put htmlparser.jar in classpath of my eclipse project) as
> mentioned in the JavaDoc. ? What is it that I am doing wrong ..
>
> And are there initial snippets that would get me started with the HTML
> parser ?
>
> --
> Sandeep
>
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] Getting Started

[Sandeep K. G. <san...@gm...>]

Hello Everyone,
I need to parse JSP file so I am starting with an HTML parser, but it seems
the binary distribution that i downloaded from Sourceforge does not have all
the classes( i put htmlparser.jar in classpath of my eclipse project) as
mentioned in the JavaDoc. ? What is it that I am doing wrong ..

And are there initial snippets that would get me started with the HTML
parser ?

-- 
Sandeep

Re: [Htmlparser-user] problem in text extraction

[Derrick O. <der...@gm...>]

Is it null or empty.
If it's null, it may be because that textInPage variable is local to that
block.

On Mon, May 31, 2010 at 12:08 PM, karanjit cheema
<kar...@gm...>wrote:

> hi
> i tried extracting text using the following code:
>
>
>            try {
>                Parser parser = new Parser (urlConnection);
>                TextExtractingVisitor visitor = new TextExtractingVisitor();
>                parser.visitAllNodesWith(visitor);
>                String textInPage = visitor.getExtractedText();
>               }
>            catch (ParserException pe)
>            {
>                pe.printStackTrace ();
>            }
>
>
> the field textInPage is always returning to be empty. can any one tell
> what the problem is?
>
>
> warm regards
> Karanjit Cheema
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>

[Htmlparser-user] problem in text extraction

[karanjit c. <kar...@gm...>]

hi
i tried extracting text using the following code:


            try {
                Parser parser = new Parser (urlConnection);
                TextExtractingVisitor visitor = new TextExtractingVisitor();
                parser.visitAllNodesWith(visitor);
                String textInPage = visitor.getExtractedText();
               }
            catch (ParserException pe)
            {
                pe.printStackTrace ();
            }


the field textInPage is always returning to be empty. can any one tell
what the problem is?


warm regards
Karanjit Cheema

Re: [Htmlparser-user] htmlparser2.0 in maven2

[Derrick O. <der...@gm...>]

It should be found under HTMLParser-2.0-SNAPSHOT.

On Fri, May 21, 2010 at 7:15 AM, Akihiko M <ams...@gm...> wrote:
> Hi
>
> Htmlparser is a wonderful library. It always uses it.
> By the way,I want to use htmlparser2.0 by maven2.
> But it is not registered to maven2 central
> repository(http://repo2.maven.org/maven2/).
>
> I hope that htmlparser2.0 or 2.x will regist soon.
>
> When it will be registed?
>
> Thanks,
>
> Akihiko
>
> --
> --------------------------------
>
> Akihiko
> ams...@gm...
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>

[Htmlparser-user] htmlparser2.0 in maven2

[Akihiko M <ams...@gm...>]

Hi

Htmlparser is a wonderful library. It always uses it.
By the way,I want to use htmlparser2.0 by maven2.
But it is not registered to maven2 central
repository(http://repo2.maven.org/maven2/).

I hope that htmlparser2.0 or 2.x will regist soon.

When it will be registed?

Thanks,

Akihiko

-- 
--------------------------------

Akihiko
ams...@gm...

[Htmlparser-user] (no subject)

[Akihiko M <ams...@gm...>]

[Htmlparser-user] HTMLParser with dom4j for Xpath

[Misha K. <mis...@gm...>]

Dear All:

I have successfully gotten the SAX parser of HTML Parser to work with dom4j so that I can use Xpath expressions with HTMLParser.

I like HTMLParser as it is quite faster than other frameworks.

However, I have two issues:
1) Is it possible, in any way, to change the SAX parser to report _lowercase_ tag names?
2) For some reason, although I get a valid tree in dom4j, I cannot seem to find elements that I can see exist by browsing the tree
using XPath? (for example a TD element with a certain classpath, no matter whether or not I capitalize TD). Anyone used HTMLParser with dom4j
successfully?

Thank you
Misha

[Htmlparser-user] HTMLParser with XOM - followup

[Misha K. <mis...@gm...>]

     [java] nu.xom.XMLException: org.htmlparser.sax.XMLReader does not support the entity resolution features XOM requires.

Any ideas?

Has anyone tried with dom4j?

I would love to have XPath support with HTMLParser, as it is quite fast.

Thank you!
Misha

[Htmlparser-user] Possible to use HTMLParser with XOM?

[Misha K. <mis...@gm...>]

Dear All:

I am currently using TagSoup with XOM to get XPath support as described here:
http://nicklothian.com/blog/2006/09/11/using-xpath-on-real-world-html-documents/
seems to work well except the following namespace problem:
http://www.supermind.org/blog/613/dom4j-xpath-tagsoup-namespaces-sweet

I noticed HTMLParser is, in my test, the fastest available, and has SAX Parser support:
http://htmlparser.sourceforge.net/javadoc/org/htmlparser/sax/package-summary.html

Has anyone used this with XOM? Any luck? Is it better/worse (i.e., slower/faster) than Tagsoup or other alternatives?

Thank you
Misha

[Htmlparser-user] Doreen Pritzel

[Pony N. <nth...@gm...>]

http://secure-power.org/home/index.php

-- 
Pony Onthusitse Nthatsi
P O Box 26496
Game City
GABORONE
+267 3133832
+267 71467530

[Htmlparser-user] problem with unicode numbered entity on linux

[Kesavanarayanan, R. <Ram...@Pe...>]

Hi all,

  I have an XML file which we parse using htmlParser in conjunction with
CSRF. It seems that the following line in the XMl file 

&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;I.<span class="math">x^2
+ x - 2 </span><br />

Is being displayed as junk characters and not translated into spaces.

This is happening only in my LINUX box and not in my WINDOWS box.

I am using FF3.6 / IE7 / IE8 etc.,

Let me know if I need to do anything special to ensure that the Unicode
characters are displayed correctly.



Regards |  Ramesh Kesavanarayanan  |    319-354-9200 ext 215785 / 215972
(O) |  /  319-621-7641 (M)   | ram...@pe...

Re: [Htmlparser-user] runtime exception web9033

[Derrick O. <der...@gm...>]

Also include the lexer.jar.

On 3/23/10, Gazihan Işıldak <gaz...@gm...> wrote:
> hi,
>
> i'm developing a rest web service in java.
>
> i'm using htmlparser library on it.
>
> but when i try to run service i'm getting this exception. i can build it
> successfully. and org.htmlparser.beans.StringBean class exists in project.
>
>> exception
>>
>> javax.servlet.ServletException: java.lang.RuntimeException: WEB9033:
>> Unable to load class with name [org.htmlparser.beans.StringBean],
>> reason:java
>> .lang.NoClassDefFoundError: org/htmlparser/visitors/NodeVisitor
>>
>> root cause
>>
>> java.lang.RuntimeException: WEB9033: Unable to load class with name [org.
>> htmlparser.beans.StringBean], reason: java.lang.NoClassDefFoundError: org/
>> htmlparser/visitors/NodeVisitor
>>
>> root cause
>>
>> java.lang.NoClassDefFoundError: org/htmlparser/visitors/NodeVisitor
>>
>> root cause
>>
>> java.lang.ClassNotFoundException: org.htmlparser.visitors.NodeVisitor
>>
>
> i checked that htmlparser.jar exists in server.
>
> what should i do to achieve this?
>

[Htmlparser-user] runtime exception web9033

[Gazihan I. <gaz...@gm...>]

hi,

i'm developing a rest web service in java.

i'm using htmlparser library on it.

but when i try to run service i'm getting this exception. i can build it
successfully. and org.htmlparser.beans.StringBean class exists in project.

> exception
>
> javax.servlet.ServletException: java.lang.RuntimeException: WEB9033:
> Unable to load class with name [org.htmlparser.beans.StringBean], reason:java
> .lang.NoClassDefFoundError: org/htmlparser/visitors/NodeVisitor
>
> root cause
>
> java.lang.RuntimeException: WEB9033: Unable to load class with name [org.
> htmlparser.beans.StringBean], reason: java.lang.NoClassDefFoundError: org/
> htmlparser/visitors/NodeVisitor
>
> root cause
>
> java.lang.NoClassDefFoundError: org/htmlparser/visitors/NodeVisitor
>
> root cause
>
> java.lang.ClassNotFoundException: org.htmlparser.visitors.NodeVisitor
>

i checked that htmlparser.jar exists in server.

what should i do to achieve this?

Re: [Htmlparser-user] end tag for known subclasses of TagNode?

[Derrick O. <der...@gm...>]

Not in the node list.
But there is a
    Tag getEndTag ();
method to get it.

On Thu, Feb 18, 2010 at 11:08 AM, Rajorshi Biswas <raj...@in...> wrote:

> Hi,
> For 'known' tags, it seems that HTMLParser does not visit the 'end' tags
> (e.g. "P" tag). But for tags that arent directly supported, such as "STRONG"
> tag, the parser does return the end tag in the nodelist.
>
> Is there a way to ask the parser to retrieve the "end" tags for known
> classes as well?
>
> Thanks,
> Raj
>
>
> Dear *htmlparser-user !* Get Yourself a cool, short *@in.com* Email ID
> now!<http://mail.in.com/mails/new_reg.php?utm_source=invite&utm_medium=outgoing>
>
>
> ------------------------------------------------------------------------------
> Download Intel&reg; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] end tag for known subclasses of TagNode?

[Rajorshi B. <raj...@in...>]

Hi, For 'known' tags, it seems that HTMLParser does not visit the 'end' tags (e.g. "P" tag). But for tags that arent directly supported, such as "STRONG" tag, the parser does return the end tag in the nodelist.Is there a way to ask the parser to retrieve the "end" tags for known classes as well?Thanks,RajDear htmlparseruser ! Get Yourself a cool, short @in.com Email ID now!

Re: [Htmlparser-user] query on how to read

[Rajorshi B. <raj...@in...>]

 Thanks much! TextNode is something I missed out on. (I didnt realize text inside a node was modeled as a Node  silly me) Original message From:Derrick Oswald< der...@gm... >Date: 18 Feb 10 11:04:51Subject:Re: [Htmlparseruser] query on how to read "data" for a particular TagNodeTo: Rajorshi Biswas , htmlparser user list If you have the div tag, then since it is a composite node, "
foo
" will be the first child:divtag.getChildren ()[0]On Thu, Feb 18, 2010 at 5:16 AM, Rajorshi Biswaswrote: Hello, I am new to htmlparser, so please forgive me if this is a naive question. I have an HTML fragment for which I need to determine if the first visible text is in bold or not.For this, I am trying to get the 'first' text content of the fragment. Suppose the fragment is of the following form: foo something something else My question is: how do I get the "data" portion of the 'div'. That is, when I arrive at the "div" node (Div object), I wish to retrieve the content of the div WITHOUT its children elements  I wish to retrieve "foo" in this case.I could not find an API in the Node/TagNode classes for this. Could anyone please help me out here?Thanks in advance!Raj Dear htmlparseruser! Get Yourself a cool, short @in.com Email ID now! Download Intel&reg; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and finetune appli
 cations for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intelswdev Htmlparseruser mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlparseruser 

Re: [Htmlparser-user] query on how to read "data" for a particular TagNode

[Derrick O. <der...@gm...>]

If you have the div tag, then since it is a composite node, "\nfoo\n" will
be the first child:

divtag.getChildren ()[0]

On Thu, Feb 18, 2010 at 5:16 AM, Rajorshi Biswas <raj...@in...> wrote:

> Hello,
> I am new to htmlparser, so please forgive me if this is a naive question. I
> have an HTML fragment for which I need to determine if the first visible
> text is in bold or not.
>
> For this, I am trying to get the 'first' text content of the fragment.
> Suppose the fragment is of the following form:
>
> <div>
> foo
> <p>something</p>
> <span>something else</span>
> </div>
>
> My question is: how do I get the "data" portion of the 'div'. That is, when
> I arrive at the "div" node (Div object), I wish to retrieve the content of
> the div WITHOUT its children elements - I wish to retrieve "foo" in this
> case.
>
> I could not find an API in the Node/TagNode classes for this. Could anyone
> please help me out here?
>
>
> Thanks in advance!
> Raj
>
>
> Dear *htmlparser-user!* Get Yourself a cool, short *@in.com* Email ID now!<http://mail.in.com/mails/new_reg.php?utm_source=invite&utm_medium=outgoin+g>
>
>
> ------------------------------------------------------------------------------
> Download Intel&reg; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] query on how to read "data" for a particular TagNode

[Rajorshi B. <raj...@in...>]

Hello, I am new to htmlparser, so please forgive me if this is a naive question. I have an HTML fragment for which I need to determine if the first visible text is in bold or not. For this, I am trying to get the 'first' text content of the fragment. Suppose the fragment is of the following form: foo something something elseMy question is: how do I get the "data" portion of the 'div'. That is, when I arrive at the "div" node (Div object), I wish to retrieve the content of the div WITHOUT its children elements  I wish to retrieve "foo" in this case. I could not find an API in the Node/TagNode classes for this. Could anyone please help me out here?Thanks in advance!RajDear htmlparseruser! Get Yourself a cool, short @in.com Email ID now!

Re: [Htmlparser-user] href rewrite

[Derrick O. <der...@gm...>]

Hava a look at the site capturer code.
It basically does what you want I think.

2010/2/17 Wagner Montalvão Camarão <wag...@gm...>

> Hello everyone,
>
> I'm new here and I'd like to get some directions about the htmlparser
> usage. I need to go through a html code and update each link (href content)
> with a new one. Like: <a href="www.google.com"> will become <a href="
> www.mysite.com/click?link=www.google.com">
>
> First I tried using the javax.xml.parsers and org.w3c.dom but the problem
> is I get an exception if I try to parse an invalid xhtml by w3c. I can't
> work like this because some users may post html codes generated by designer
> tools which formatting may not be valid by w3c.
>
> I could write some regex to do this but I heard about htmlparser and I
> would like to know if it can help me with this.
>
> Any suggestion will be appreciated.
>
> Thank you
>
> Wagner Montalvão Camarão
>
>
> ------------------------------------------------------------------------------
> SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
> Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
> http://p.sf.net/sfu/solaris-dev2dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>

[Htmlparser-user] href rewrite

[Wagner M. C. <wag...@gm...>]

Hello everyone,

I'm new here and I'd like to get some directions about the htmlparser usage.
I need to go through a html code and update each link (href content) with a
new one. Like: <a href="www.google.com"> will become <a href="
www.mysite.com/click?link=www.google.com">

First I tried using the javax.xml.parsers and org.w3c.dom but the problem is
I get an exception if I try to parse an invalid xhtml by w3c. I can't work
like this because some users may post html codes generated by designer tools
which formatting may not be valid by w3c.

I could write some regex to do this but I heard about htmlparser and I would
like to know if it can help me with this.

Any suggestion will be appreciated.

Thank you

Wagner Montalvão Camarão

Re: [Htmlparser-user] StringExtractor

[Joshua K. <jo...@in...>]

On Sat, Dec 12, 2009 at 10:02 AM, Derrick Oswald
<der...@gm...>wrote:

> This has been replaced by the main program in
> org.htmlparser.beans.StringBean.
>

I never did get that name - StringBean.  It extracts strings but it isn't
called a StringExtractor.  Hmmmm...

best
jk

Re: [Htmlparser-user] StringExtractor

[David P. C. <dav...@gm...>]

Great, thanks for the answer! (I just saw it now)

The library seems great!
One question, it seems that it does not handle the div elements correctly.
A div element is a block
element<http://www.webdesignfromscratch.com/html-css/css-block-and-inline.php>
(by
default), and thus it should render a new line.

For example, with this html file:
++++++++++++++++++
<html>
<body>
test1
test2
<div>test3</div>
test4
<span>test5</span>
<span>test6</span>
</body>
</html>
++++++++++++++++++

if should produce:
++++++++++++++++++
test1 test2
test3
test4 test5 test6
++++++++++++++++++

note the new line between test3 and test4.

However, StringBean produces the following:
++++++++++++++++++
test1 test2
test3 test4 test5 test6
++++++++++++++++++

It handles correctly the new lines for text and span nodes, but not for
divs.

Is that the intended effect? if so, is it possible to override this (add a
new line for block elements)?


Regards,
David Portabella


On Sat, Dec 12, 2009 at 10:02 AM, Derrick Oswald
<der...@gm...>wrote:

> This has been replaced by the main program in
> org.htmlparser.beans.StringBean.
>
> Sorry for the misdirection
>
> On Wed, Dec 9, 2009 at 11:18 PM, David Portabella Clotet <
> dav...@gm...> wrote:
>
>> Hello,
>>
>> In the website: http://htmlparser.sourceforge.net/samples.html
>> there is info about the "StringExtractor" example:
>> ++++++++++++++++++
>> String Extractor
>> Extract text from a web page.
>> org.htmlparser.parserapplications.StringExtractor
>> bin/stringextractor http://website_url
>> ++++++++++++++++++
>>
>> However, I did not find this example in any of this two downloads:
>> HTMLParser-2.0-SNAPSHOT-src.zip
>> HTMLParser-2.0-SNAPSHOT-bin.zip
>>
>> Can you please tell me where to find the StringExtractor example?
>>
>>
>> Best regards,
>> DAvid Portabella
>>
>>

Re: [Htmlparser-user] Bold Tag doesn't have children

[Derrick O. <der...@gm...>]

Silly question, but did you change mIds to be "B" ?
It has to be upper case.

On Wed, Feb 10, 2010 at 12:55 AM, Ben Rose <be...@in...> wrote:

> Greetings,
>
> I would like to use bold tags ("b") with org.htmlparser, but have been
> unable to get them to work as expected.
>
> I added and registered a BoldTag class that is an exact copy of
> org.htmlparser.tags.ParagraphTag (with the "B" added to mEnders) but
> have not been able to make it work as expected.
>
> When looking at the children of the body node from the following html
> string, there are 3 children (<b>, <a>, </b>) instead of the single <b>
> as I would expect.
>
> "<html><body><b><a href='test.com'>Test</a></b></body></html>"
>
> Am I missing something here?
>
> -Ben Rose
>
>
>
> ------------------------------------------------------------------------------
> SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
> Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
> http://p.sf.net/sfu/solaris-dev2dev
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>