htmlparser-user — The user mailing list for users of the htmlparser library

Re: [Htmlparser-user] java based

[Somik R. <so...@ya...>]

Hi Mats,
    Pls find attached the class that does what you want. Dont ask me about
the junk GUI code - that was made by VAJ.
    I only wrote processURL() and run(). As you can see, its quite simple to
integrate the parser. HTH.

Regards,
Somik
----- Original Message -----
From: "Sodergren, M.G." <mg...@le...>
To: <htm...@li...>
Sent: Wednesday, April 24, 2002 12:16 AM
Subject: [Htmlparser-user] java based


Hello,
Have you still got this applet (reply to the previous question) so i can run
it on my machine?
Also, have you had a chance yet to have a look at my previous questions
about extracting links using my program i attached.
Thanks for your time and effort.
Mats

_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user

[Htmlparser-user] Any way to modify the image tag location?

[Doyle, A. <Ann...@au...>]

Is there any way to modify the image tag locations? I want to find the
image tags in a document and then modify the location to a local
location. I will  download the images from the original location and
save locally. I also want to then write out the resulting html to a file
locally. The resulting document I write out will have the new local
location for the images.
=20
=20
Thanks-
Annette

Re: [Htmlparser-user] java based

[Somik R. <so...@ya...>]

Hi Mats
>Have you still got this applet (reply to the previous question) so i can
run it on my machine?
>Also, have you had a chance yet to have a look at my previous questions
about extracting links using my
>program i attached.
>Thanks for your time and effort
    Actually I was looking into it right now - finally had some time. Will
let you know very soon...

Cheers,
Somik

[Htmlparser-user] java based

[Sodergren, M.G. <mg...@le...>]

Hello,
Have you still got this applet (reply to the previous question) so i can =
run it on my machine?
Also, have you had a chance yet to have a look at my previous questions =
about extracting links using my program i attached.
Thanks for your time and effort.
Mats

Re: [Htmlparser-user] Java version of html parser

[Somik R. <so...@ya...>]

Hi Annette,
    Yes - HTML Parser works with Java 1.1. I had tested it a few months =
back in an applet..
   =20
Regards,
Somik
----- Original Message -----=20
  From: Doyle, Annette=20
  To: htm...@li...=20
  Sent: Tuesday, April 23, 2002 9:57 PM
  Subject: [Htmlparser-user] Java version of html parser


  What is the Java version of the html parser ? (Is it Java 1.1 =
compatible?)

  =20

  Annette

[Htmlparser-user] Java version of html parser

[Doyle, A. <Ann...@au...>]

What is the Java version of the html parser ? (Is it Java 1.1
compatible?)
=20
Annette

Re: [Htmlparser-user] Unsubscribe me

[Somik R. <so...@ya...>]

Go to https://lists.sourceforge.net/lists/listinfo/htmlparser-user to do the
same (this link is also in the footer).

Cheers,
Somik
----- Original Message -----
From: "sultan" <ma...@ho...>
To: <htm...@li...>
Sent: Monday, April 22, 2002 1:52 PM
Subject: [Htmlparser-user] Unsubscribe me


>
>
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user

[Htmlparser-user] Unsubscribe me

[sultan <ma...@ho...>]

[Htmlparser-user] applet based reply

[Sodergren, M.G. <mg...@le...>]

Thanks for your reply. I really appriciate if you get the time when you =
get back to code and send back WebViewer to me as i have to finish this =
by the end of next week.
Thanks again,
Mats Sodergren

Re: [Htmlparser-user] applet based

[Somik R. <so...@ya...>]

Hi Mats
    Unfortunately this weekend I am going to be out of station. Will try to
help you when I get back.

Cheers,
Somik

[Htmlparser-user] applet based

[Sodergren, M.G. <mg...@le...>]

Hello,
I was wondering if it is possible that you could help me with the =
following. If you have not got time to do this i understand. I have been =
trying it code this but cannot make it work and i am streesed for time =
to complete it.

I want my WebViewer program (attached) to be run under emacs in windows. =
This will display a window where i can enter a url at the top. Once =
enter is pressed i want WebViewer to work with htmlparser to extract the =
links of that HTML page and return them below in the window as a result. =
The way in which WebViewer works now is that it only shows the HTML page =
of the url entered. So i need Webviewer to be changed in order to work =
with htmlparser and i need the classes used from the html parser to be =
sent to me (as all of them will not be used for just extracting links). =
So i want all my programs just to be in one directory when i get them =
and run them so i dont have any package com.kizna....... import =
com.kizna........ etc.. in the programs. Please run WebViewer and you =
will understand more clearly what i mean.

Thanks for your time,
Mats

[Htmlparser-user] HTMLParser 1.1 released

[Somik R. <so...@ya...>]

Hi Folks,
    HTMLParser 1.1 has just been released. This is a production release =
- HTMLParser finally moves out of the beta stage.=20
    A whole lot of bug fixes, architecture modifications, and intense =
testing has been done.=20
    You can get it from http://htmlparser.sourceforge.net

Regards,
Somik

Re: [Htmlparser-user] the parser

[Somik R. <so...@ya...>]

Hi Mats,
    HTMLParser.elements returns an Enumeration. So you can enumerate through
a list of nodes. This is actually the Iterator design pattern.
    HTMLNode is the interface that represents just about any kind of html
element. The element might be a string node, a remark node, a tag, or end
tag. If it is a tag - then there are several types of tags - and that forms
another hierarchy.
    All this is explained with class diagrams at :
http://htmlparser.sourceforge.net/design/index.html
http://htmlparser.sourceforge.net/design/tags.html (this shows the HTMLNode
particularly).

    To use the parser is quite simple - from the user perspective, you only
need a loop -
HTMLNode node;
for (Enumeration e = parser.elements();e.hasMoreElements();) {
    node = (HTMLNode)e.nextElement();
    // Now you have an object of type HTMLNode.
    // This is however of a type which implements HTMLNode. So you can use
instanceof if you are interested
    // in particular types. Or you can use reflections to find out
information about the object itself. The former is usually
    // what is used by most folks.

    // Suppose you want to only print strings, you will want to take action
if the node is a HTMLStringNode
    if (node instanceof HTMLStringNode) {
        // Yes, now we can downcast it to HTMLStringNode
        HTMLStringNode stringNode = (HTMLStringNode)node;
        // Print the contents of the string node
        System.out.println(stringNode.getText());
    }
}

HTH. Pls feel free to ask any questions that you have.

Regards,
Somik
----- Original Message -----
From: "Sodergren, M.G." <mg...@le...>
To: <htm...@li...>
Sent: Tuesday, April 16, 2002 6:26 PM
Subject: [Htmlparser-user] the parser


I have a problem with the following:

node = (HTMLNode)e.nextElement();

Please tell me what is the content and return type of
HTMLParser.elements(), and how the HTMLNode is defined.

_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user

[Htmlparser-user] the parser

[Sodergren, M.G. <mg...@le...>]

I have a problem with the following:=20

node =3D (HTMLNode)e.nextElement();=20

Please tell me what is the content and return type of=20
HTMLParser.elements(), and how the HTMLNode is defined.=20

[Htmlparser-user] Major bug fix - spaces in links and images are now handled

[Somik R. <so...@ya...>]

Hi Folks,
    Thanks to Sam Joseph (creator of Neurogrid). Sam is using the parser =
in the neurogrid project, and has pointed out a bug that slipped our =
attention. If links or image urls contain spaces, those spaces were =
being absorbed. That is incorrect behaviour, especially if you have =
something of the form :

http://myservlet.com/someservlet?name=3DSam Joseph&age=3D22

The same goes for images like
http://www.kizna.com/images/kizna corp.jpg

Also - previously, newline character were being converted to spaces. =
This has been modified - new line characters are left as is. The =
responsibility to deal with them is now with the appropriate scanner. =
So, the link and image scanners specifically filter out the newline =
characters, whereas jsp tags which might have jsp code - would like to =
preserve the new line chars.

Over 73 testcases now in the htmlparser, and all passing..
I think we're ready for release 1.1 now, unless I get any more bug =
reports this week.

You can check out the latest code from CVS.

Regards,
Somik

Re: [Htmlparser-user] HTML parser 1.1

[Somik R. <so...@ya...>]

Hi Raghav
> when would be this HTMLparser 1.1 out?
As soon as I can wrap it up. Technically, the code is ready and already
checked into CVS. I need to do the process of creating a release - make some
documentation, check everything is ok, ..
If I had some help I could wrap it up sooner.

> I am not sure, but to me the way htmlparser parses is it gives me the tag
> parameter of the first line in the above snippet of html code, when I do
> Hashtable table = tag.parseParameters();
> it is looking for parameters inside <FORM ..... >, but not <FORM
> .....</FORM>

Yes - parseParameters() will give you the stuff inside the FORM tag. That is
what I call "microscopic" parsing. But to get the remaining tags - till you
encounter </FORM> you need to do "macroscopic" parsing. This is not hard-
check HTMLAppletScanner as an example.

In a nutshell - concept is very simple. The scan method provides you with a
reader. So you are to use that reader to read ahead and get the next tags.
This is simple bcos the reader will automatically identify the correct tags,
and the mechanism is very similar to using the parser to get the tags you
want. The HTMLLinkScanner among others, also works on the same principle.

Bytway - I think we should take this discussion to the Developer list.

Regards,
Somik



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: [Htmlparser-user] extract links

[Somik R. <so...@ya...>]

Mats wrote :
> So can i not some how get the htmlparser (just the extract link programs)
to work with my WebPage Viewer so that i enter a > search like before but
this time the HTML page shown in my window is just the extracted links? Can
I do this? If so how?

Yes you can do this.
Look at the program MailRipper.java in the com.kizna.html.parserapplications
package. This program will print only the email addresses on a page (which
are also link tags). Its even simpler to simply filter out all the tags
except the link tags.

Try this :
HTMLParser parser = new HTMLParser("http://myurl.com");
HTMLNode node;
for (Enumeration e = parser.elements();e.hasMoreElements();) {
    node = (HTMLNode)e.nextElement();   // Get the next html node
    if (node instanceof HTMLLinkTag) {
        // Yes - this is a web link. Now you can downcast and do what you
want with it
        HTMLLinkTag linkTag = (HTMLLinkTag)node;
        System.out.println("link to "+linkTag.getLink()+", link text =
"+linkTag.getLinkText());
    }
}

This should give you what you want.

Regards,
Somik


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

[Htmlparser-user] HTML parser 1.1

[Raghavender S. <kin...@ho...>]

Hi Somik,
when would be this HTMLparser 1.1 out?
one more question. to parse the FORM tags, I have a small question.
let us say this is a form tag

<FORM NAME="LoginForm" METHOD=POST ACTION="urltoInvoke">
<P>User name:
<INPUT TYPE="text" NAME="userName" SIZE="10">
<P>Password:
<INPUT TYPE="password" NAME="password" SIZE="12">
<P><INPUT TYPE="submit" VALUE="Log in">
<INPUT TYPE="button" VALUE="Cancel" onClick="window.close()">
</FORM>

I am not sure, but to me the way htmlparser parses is it gives me the tag 
parameter of the first line in the above snippet of html code, when I do
Hashtable table = tag.parseParameters();
it is looking for parameters inside <FORM ..... >, but not <FORM 
.....</FORM>

could you suggest me how to go ahead with this.
Raghav


to extract the INPUT tag parameters





_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx

[Htmlparser-user] extract links

[Sodergren, M.G. <mg...@le...>]

Hello again,
I think I have figured out how to do it. Its as follows:
- I have written two programs called WebPage (which is a static utility =
class that provides operations for downloading web pages) and =
WebPageViewer (contains utilities to download and display HTML pages). =
So if i run WebPageViewer i just enter my search at the top (which makes =
the yahoo search url with my search) and it displays the yahoo search =
page for this search because the beginning URL is the standard yahoo =
search url. So can i not some how get the htmlparser (just the extract =
link programs) to work with my WebPage Viewer so that i enter a search =
like before but this time the HTML page shown in my window is just the =
extracted links? Can I do this? If so how? This would mean that I dont =
need a servlet or something else. I have attached my two programs so you =
can have a look at them and run them if you want. They are quite basic =
and really i want them to contain a text field where i can enter my =
search without the basic yahoo search url been shown and then have a =
search button i can press to perform the search instead of pressing =
enter.=20
Thanks,
Mats

[Htmlparser-user] Dynamic page parsing bug fixed (ready for release 1.1)

[Somik R. <so...@ya...>]

Hi Folks,
    The dynamic page parsing bug is fixed, and as far as I've tested, I =
am able to parse correctly pages like =
http://search.yahoo.com/bin/search?p=3Ddogs=20
which Mats had posted earlier.

    We are now ready for release 1.1. I'd be grateful if I had some help =
in testing the parser - and see if there are any showstopper bugs for =
this release. (Get the latest code from CVS)

Regards,
Somik

[Htmlparser-user] Bug fixed in HTMLTag - report from Raj Sharma

[Somik R. <so...@ya...>]

Hi Folks,
    An important bug has been pointed out by Raj Sharma, which would =
halt the parser if a page contained a link spread over two lines. This =
was a bug in HTMLTag, and I was able to find it quickly, thanks to the =
refactoring done earlier with the help of Arnaud.
    Also - HTMLLinkScanner and HTMLImageScanner have some small changes =
in connection with the fix.
    Please get the latest code from CVS.
   =20
Regards,
Somik
   =20

Re: [Htmlparser-user] link extractor

[Somik R. <so...@ya...>]

Of course you can use the library in your servlet. The library isnt meant to
be used thru the command line - that is only provided as a quick demo and
for testing. Putting it in an applet is also an interesting idea. I have
kept the  parser JDK 1.1 compatible - so you should be able to get it into
an applet on IE.

Regards,
Somik
----- Original Message -----
From: "Sodergren, M.G." <mg...@le...>
To: "Somik Raha" <so...@ya...>
Sent: Friday, April 05, 2002 12:50 AM
Subject: RE: [Htmlparser-user] link extractor


What i meant was if i have an address where i can store my web page
(actually where i will run my programs from) is it easiest to make my
programs into a applet, servlet, etc... or do i even have to do this at
all. I am going to adapt the program so i can enter a url in a box
instead of on the command line.
Thanks
Mats

-----Original Message-----
From: Somik Raha [mailto:so...@ya...]
Sent: 04 April 2002 16:47
To: Sodergren, M.G.
Subject: Re: [Htmlparser-user] link extractor


Hi Mats,
>What is the easiest way to make the htmlparser (link extractor)
internet
>based?

    Sorry, didnt understand this question.
Regards,
Somik



_________________________________________________________

Do You Yahoo!?

Get your free @yahoo.com address at http://mail.yahoo.com




_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

[Htmlparser-user] extracting links

[Sodergren, M.G. <mg...@le...>]

How come when you use the parser on most sites to extract links it works
fine but when you use it on search engine i.e.=20
http://search.yahoo.com/bin/search?p=3Ddogs which is a page with search
results for dogs, it does not work?
Thanks
Mats

[Htmlparser-user] link extractor

[Sodergren, M.G. <mg...@le...>]

What is the easiest way to make the htmlparser (link extractor) internet
based?
Mats

Re: [Htmlparser-user] Swing integration

[Somik R. <so...@ya...>]

Hi Craig,
    Thanks a lot for the post. Pls go ahead with your analysis. I will try
to catch up this weekend.
Regards,
Somik
----- Original Message -----
From: "Craig Raw" <cr...@qu...>
To: "'Somik Raha'" <so...@ya...>
Sent: Tuesday, April 02, 2002 3:32 PM
Subject: RE: [Htmlparser-user] Swing integration


> Hi Somik,
>
> A quick excerpt from javax.swing.text.html.HTMLEditorKit javadoc - which
> is the driver behind JEditorPane's reading and writing HTML
> capabilities.
>
> ---
> Extendable/Scalable
>
> To maximize the usefulness of this kit, a great deal of effort has gone
> into making it extendable. These are some of the features.
> The parser is replaceable. The default parser is the Hot Java parser
> which is DTD based. A different DTD can be used, or an entirely
> different parser can be used. To change the parser, reimplement the
> getParser method. The default parser is dynamically loaded when first
> asked for, so the class files will never be loaded if an alternative
> parser is used. The default parser is in a separate package called
> parser below this package.
>
> The parser drives the ParserCallback, which is provided by HTMLDocument.
> To change the callback, subclass HTMLDocument and reimplement the
> createDefaultDocument method to return document that produces a
> different reader. The reader controls how the document is structured.
> Although the Document provides HTML support by default, there is nothing
> preventing support of non-HTML tags that result in alternative element
> structures.
> ---
>
> I may find some time to look into this as well, although I am not sure
> how much it would fix JEditorPane's somewhat buggy HTML rendering
> capabilities....
>
> -craig
>
>
> -----Original Message-----
> From: htm...@li...
> [mailto:htm...@li...] On Behalf Of Somik
> Raha
> Sent: 01 April 2002 05:28 PM
> To: HTMLParser User List
> Cc: HTMLParser Developer List
> Subject: Re: [Htmlparser-user] Swing integration
>
> Hi Craig
>     Wow! Thats a great question.
>     Actually, I doubt if I could replace Sun Microsystems' code with
> mine. I
> dont think Java is that open (or is it ?)
> However, we could think of writing our own adapter for the html parser
> that
> might plugin in some way...
>      I have never used Sun's html parser (If I had, I might not have
> started
> this project).
>      I will need to study Sun's parser before I can answer your
> question..
>     But there does seem to be some interesting possibilities.
>
> Regards
> Somik
> ----- Original Message -----
> From: "Craig Raw" <cr...@qu...>
> To: <htm...@li...>
> Sent: Monday, April 01, 2002 10:20 PM
> Subject: [Htmlparser-user] Swing integration
>
>
> > Has the HTML Parser been integrated into Swing's HTMLEditorKit to
> > provide a better implementation of JEditorPane's HTML viewing
> > capabilities? HTML Parser would need to replace
> > javax.swing.text.html.parser.Parser, which is currently somewhat
> buggy.
> > Anyone tried this?
> >
> > -craig
> >
> >
> >
> >
> > _______________________________________________
> > Htmlparser-user mailing list
> > Htm...@li...
> > https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com