htmlparser-user

[Htmlparser-user] cookie

[<xia...@gm...>]

I want to parse a web page that need to log in.so I use the wiki example bu=
t
can not work. the cookie expired when the browser shut down.
Can you tell me how to handle this situation.

--
Best Regards.

Xiaodong Han
MSN:hx...@ho...

Re: [Htmlparser-user] cookie

[Derrick O. <Der...@Ro...>]

You may need to 'POST' to the login form using the ConnectionManager 
with your credentials.
See the doc-comments for src/org/htmlparser/tests/ParserTest.testPOST() 
for an example.

?? wrote:

> I want to parse a web page that need to log in.so I use the wiki 
> example but can not work. the cookie expired when the browser shut down.
> Can you tell me how to handle this situation.
>
> -- 
> Best Regards.
>
> Xiaodong Han
> MSN:hx...@ho... <mailto:MSN:hx...@ho...>

Re: [Htmlparser-user] cookie

[<xia...@gm...>]

I tryed that but It still can not work, I research the web page flow and
find that
when you log in ,then server redirect you to another page.
does the testPost() can handle this ?


On 3/29/06, Derrick Oswald <Der...@ro...> wrote:
>
> You may need to 'POST' to the login form using the ConnectionManager
> with your credentials.
> See the doc-comments for src/org/htmlparser/tests/ParserTest.testPOST()
> for an example.
>
> ?? wrote:
>
> > I want to parse a web page that need to log in.so I use the wiki
> > example but can not work. the cookie expired when the browser shut down=
.
> > Can you tell me how to handle this situation.
> >
> > --
> > Best Regards.
> >
> > Xiaodong Han
> > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>



--
Best Regards.

Xiaodong Han
MSN:hx...@ho...

Re: [Htmlparser-user] cookie

[<xia...@gm...>]

TXkgY29kZSBpcyBzb21ldGhpbmcgbGlrZSB0aGlzOgoKICAgICAgICAgICAgdXJsID0gbmV3IFVS
TCAoIgpodHRwOi8vd3d3LnRqZncuY29tLzItaGFuZC9pbmNsdWRlL2NoZWNrbG9naW4uYXNwIik7
CiAgICAgICAgICAgIGNvbm5lY3Rpb24gPSAoSHR0cFVSTENvbm5lY3Rpb24pdXJsLm9wZW5Db25u
ZWN0aW9uICgpOwogICAgICAgICAgICBjb25uZWN0aW9uLnNldFJlcXVlc3RNZXRob2QgKCJQT1NU
Iik7CgogICAgICAgICAgICBjb25uZWN0aW9uLnNldERvT3V0cHV0ICh0cnVlKTsKICAgICAgICAg
ICAgY29ubmVjdGlvbi5zZXREb0lucHV0ICh0cnVlKTsKICAgICAgICAgICAgY29ubmVjdGlvbi5z
ZXRVc2VDYWNoZXMgKGZhbHNlKTsKCiAgICAgICAgICAgY29ubmVjdGlvbi5zZXRSZXF1ZXN0UHJv
cGVydHkoIlJlZmVyZXIiLCIKaHR0cDovL3d3dy50amZ3LmNvbS9yZWcvbG9naW4uYXNwIik7Cgog
ICAgICAgICAgIGJ1ZmZlciA9IG5ldyBTdHJpbmdCdWZmZXIgKDEwMjQpOwoKCiAgICAgICAgICAg
YnVmZmVyLmFwcGVuZCgiYmFja1VybD0iKTsKICAgICAgICAgICAvL2J1ZmZlci5hcHBlbmQoImh0
dHA6Ly93d3cudGpmdy5jb20vcmVnL2xvZ2luLmFzcCIpOwogICAgICAgICAgIGJ1ZmZlci5hcHBl
bmQoIiYiKTsKICAgICAgICAgICAgYnVmZmVyLmFwcGVuZCAoInVzZXJuYW1lPSIpOwoKCiAgICAg
ICAgICAgIGJ1ZmZlci5hcHBlbmQgKCImIik7CiAgICAgICAgICAgIGJ1ZmZlci5hcHBlbmQoInBh
c3N3b3JkPSIpOwogICAgICAgICAgIG91dCA9IG5ldyBQcmludFdyaXRlciAoY29ubmVjdGlvbi5n
ZXRPdXRwdXRTdHJlYW0gKCkpOwogICAgICAgICAgICBvdXQucHJpbnQgKGJ1ZmZlcik7CiAgICAg
ICAgICAgIG91dC5jbG9zZSAoKTsKCiAgICAgICAgICAgIFBhcnNlciBwYXJzZXI9bmV3IFBhcnNl
cihjb25uZWN0aW9uKTsKCk9uIDMvMzAvMDYsIN/L38sgPHhpYW9kb25nLmhhbkBnbWFpbC5jb20+
IHdyb3RlOgo+Cj4gSSB0cnllZCB0aGF0IGJ1dCBJdCBzdGlsbCBjYW4gbm90IHdvcmssIEkgcmVz
ZWFyY2ggdGhlIHdlYiBwYWdlIGZsb3cgYW5kCj4gZmluZCB0aGF0Cj4gd2hlbiB5b3UgbG9nIGlu
ICx0aGVuIHNlcnZlciByZWRpcmVjdCB5b3UgdG8gYW5vdGhlciBwYWdlLgo+IGRvZXMgdGhlIHRl
c3RQb3N0KCkgY2FuIGhhbmRsZSB0aGlzID8KPgo+Cj4KPiBPbiAzLzI5LzA2LCBEZXJyaWNrIE9z
d2FsZCA8RGVycmlja09zd2FsZEByb2dlcnMuY29tPiB3cm90ZToKPiA+Cj4gPiBZb3UgbWF5IG5l
ZWQgdG8gJ1BPU1QnIHRvIHRoZSBsb2dpbiBmb3JtIHVzaW5nIHRoZSBDb25uZWN0aW9uTWFuYWdl
cgo+ID4gd2l0aCB5b3VyIGNyZWRlbnRpYWxzLgo+ID4gU2VlIHRoZSBkb2MtY29tbWVudHMgZm9y
IHNyYy9vcmcvaHRtbHBhcnNlci90ZXN0cy9QYXJzZXJUZXN0LnRlc3RQT1NUKCkKPiA+IGZvciBh
biBleGFtcGxlLgo+ID4KPiA+ID8/IHdyb3RlOgo+ID4KPiA+ID4gSSB3YW50IHRvIHBhcnNlIGEg
d2ViIHBhZ2UgdGhhdCBuZWVkIHRvIGxvZyBpbi5zbyBJIHVzZSB0aGUgd2lraQo+ID4gPiBleGFt
cGxlIGJ1dCBjYW4gbm90IHdvcmsuIHRoZSBjb29raWUgZXhwaXJlZCB3aGVuIHRoZSBicm93c2Vy
IHNodXQKPiA+IGRvd24uCj4gPiA+IENhbiB5b3UgdGVsbCBtZSBob3cgdG8gaGFuZGxlIHRoaXMg
c2l0dWF0aW9uLgo+ID4gPgo+ID4gPiAtLQo+ID4gPiBCZXN0IFJlZ2FyZHMuCj4gPiA+Cj4gPiA+
IFhpYW9kb25nIEhhbgo+ID4gPiBNU046aHhkaGFuQGhvdG1haWwuY29tIDxtYWlsdG86TVNOOmh4
ZGhhbkBob3RtYWlsLmNvbT4KPiA+Cj4gPgo+ID4KPiA+Cj4gPiAtLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gPiBUaGlzIFNGLk5ldCBlbWFp
bCBpcyBzcG9uc29yZWQgYnkgeFBNTCwgYSBncm91bmRicmVha2luZyBzY3JpcHRpbmcKPiA+IGxh
bmd1YWdlCj4gPiB0aGF0IGV4dGVuZHMgYXBwbGljYXRpb25zIGludG8gd2ViIGFuZCBtb2JpbGUg
bWVkaWEuIEF0dGVuZCB0aGUgbGl2ZQo+ID4gd2ViY2FzdAo+ID4gYW5kIGpvaW4gdGhlIHByaW1l
IGRldmVsb3BlciBncm91cCBicmVha2luZyBpbnRvIHRoaXMgbmV3IGNvZGluZwo+ID4gdGVycml0
b3J5IQo+ID4gaHR0cDovL3NlbC5hcy11cy5mYWxrYWcubmV0L3NlbD9jbWQ9bG5rJmtpZD0xMTA5
NDQmYmlkPTI0MTcyMCZkYXQ9MTIxNjQyCj4gPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fXwo+ID4gSHRtbHBhcnNlci11c2VyIG1haWxpbmcgbGlzdAo+ID4g
SHRtbHBhcnNlci11c2VyQGxpc3RzLnNvdXJjZWZvcmdlLm5ldAo+ID4gaHR0cHM6Ly9saXN0cy5z
b3VyY2Vmb3JnZS5uZXQvbGlzdHMvbGlzdGluZm8vaHRtbHBhcnNlci11c2VyCj4gPgo+Cj4KPgo+
IC0tCj4gQmVzdCBSZWdhcmRzLgo+Cj4gWGlhb2RvbmcgSGFuCj4gTVNOOmh4ZGhhbkBob3RtYWls
LmNvbQo+CgoKCi0tCkJlc3QgUmVnYXJkcy4KClhpYW9kb25nIEhhbgpNU046aHhkaGFuQGhvdG1h
aWwuY29tCg==

Re: [Htmlparser-user] cookie

[Derrick O. <Der...@Ro...>]

No, there is a 'FollowRedirects' flag that automatically follows it.
See the discussion in RFE #1436082 Follow redirections with cookie 
processing

http://sourceforge.net/tracker/index.php?func=detail&aid=1436082&group_id=24399&atid=381402


?? wrote:

> I tryed that but It still can not work, I research the web page flow 
> and find that
> when you log in ,then server redirect you to another page.
> does the testPost() can handle this ?
>
>
> On 3/29/06, *Derrick Oswald* <Der...@ro... 
> <mailto:Der...@ro...>> wrote:
>
>     You may need to 'POST' to the login form using the ConnectionManager
>     with your credentials.
>     See the doc-comments for
>     src/org/htmlparser/tests/ParserTest.testPOST()
>     for an example.
>
>     ?? wrote:
>
>     > I want to parse a web page that need to log in.so I use the wiki
>     > example but can not work. the cookie expired when the browser
>     shut down.
>     > Can you tell me how to handle this situation.
>     >
>     > --
>     > Best Regards.
>     >
>     > Xiaodong Han
>     > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
>     <mailto:MSN <mailto:MSN>:hx...@ho...
>     <mailto:hx...@ho...>>
>
>
>
>
>     -------------------------------------------------------
>     This SF.Net email is sponsored by xPML, a groundbreaking scripting
>     language
>     that extends applications into web and mobile media. Attend the
>     live webcast
>     and join the prime developer group breaking into this new coding
>     territory!
>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>
>     _______________________________________________
>     Htmlparser-user mailing list
>     Htm...@li...
>     <mailto:Htm...@li...>
>     https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>     <https://lists.sourceforge.net/lists/listinfo/htmlparser-user>
>
>
>
>
> -- 
> Best Regards.
>
> Xiaodong Han
> MSN:hx...@ho... <mailto:MSN:hx...@ho...>

Re: [Htmlparser-user] cookie

[<xia...@gm...>]

in this topic ,you said the connectionManager can handle the cookie and
redirect.
but how to use this object set cookie for parse another page.
because parge another page need cookie to be set.
On 3/30/06, Derrick Oswald <Der...@ro...> wrote:
>
>
> No, there is a 'FollowRedirects' flag that automatically follows it.
> See the discussion in RFE #1436082 Follow redirections with cookie
> processing
>
>
> http://sourceforge.net/tracker/index.php?func=3Ddetail&aid=3D1436082&grou=
p_id=3D24399&atid=3D381402
>
>
> ?? wrote:
>
> > I tryed that but It still can not work, I research the web page flow
> > and find that
> > when you log in ,then server redirect you to another page.
> > does the testPost() can handle this ?
> >
> >
> > On 3/29/06, *Derrick Oswald* <Der...@ro...
> > <mailto:Der...@ro...>> wrote:
> >
> >     You may need to 'POST' to the login form using the ConnectionManage=
r
> >     with your credentials.
> >     See the doc-comments for
> >     src/org/htmlparser/tests/ParserTest.testPOST()
> >     for an example.
> >
> >     ?? wrote:
> >
> >     > I want to parse a web page that need to log in.so I use the wiki
> >     > example but can not work. the cookie expired when the browser
> >     shut down.
> >     > Can you tell me how to handle this situation.
> >     >
> >     > --
> >     > Best Regards.
> >     >
> >     > Xiaodong Han
> >     > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
> >     <mailto:MSN <mailto:MSN>:hx...@ho...
> >     <mailto:hx...@ho...>>
> >
> >
> >
> >
> >     -------------------------------------------------------
> >     This SF.Net email is sponsored by xPML, a groundbreaking scripting
> >     language
> >     that extends applications into web and mobile media. Attend the
> >     live webcast
> >     and join the prime developer group breaking into this new coding
> >     territory!
> >
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> >     <
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642>
> >     _______________________________________________
> >     Htmlparser-user mailing list
> >     Htm...@li...
> >     <mailto:Htm...@li...>
> >     https://lists.sourceforge.net/lists/listinfo/htmlparser-user
> >     <https://lists.sourceforge.net/lists/listinfo/htmlparser-user>
> >
> >
> >
> >
> > --
> > Best Regards.
> >
> > Xiaodong Han
> > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>



--
Best Regards.

Xiaodong Han
MSN:hx...@ho...

Re: [Htmlparser-user] cookie

[Derrick O. <Der...@Ro...>]

The only documentation is in the http package javadocs:

http://htmlparser.sourceforge.net/javadoc/org/htmlparser/http/package-summary.html

?? wrote:

> in this topic ,you said the connectionManager can handle the cookie 
> and redirect.
> but how to use this object set cookie for parse another page.
> because parge another page need cookie to be set.
> On 3/30/06, *Derrick Oswald* <Der...@ro... 
> <mailto:Der...@ro...>> wrote:
>
>
>     No, there is a 'FollowRedirects' flag that automatically follows it.
>     See the discussion in RFE #1436082 Follow redirections with cookie
>     processing
>
>     http://sourceforge.net/tracker/index.php?func=detail&aid=1436082&group_id=24399&atid=381402
>     <http://sourceforge.net/tracker/index.php?func=detail&aid=1436082&group_id=24399&atid=381402>
>
>
>     ?? wrote:
>
>     > I tryed that but It still can not work, I research the web page flow
>     > and find that
>     > when you log in ,then server redirect you to another page.
>     > does the testPost() can handle this ?
>     >
>     >
>     > On 3/29/06, *Derrick Oswald* <Der...@ro...
>     <mailto:Der...@ro...>
>     > <mailto:Der...@ro...
>     <mailto:Der...@ro...>>> wrote:
>     >
>     > You may need to 'POST' to the login form using the ConnectionManager
>     > with your credentials.
>     > See the doc-comments for
>     > src/org/htmlparser/tests/ParserTest.testPOST()
>     > for an example.
>     >
>     > ?? wrote:
>     >
>     > > I want to parse a web page that need to log in.so I use the wiki
>     > > example but can not work. the cookie expired when the browser
>     > shut down.
>     > > Can you tell me how to handle this situation.
>     > >
>     > > --
>     > > Best Regards.
>     > >
>     > > Xiaodong Han
>     > > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
>     <mailto:MSN <mailto:MSN>: hx...@ho...
>     <mailto:hx...@ho...>>
>     > <mailto:MSN <mailto:MSN> <mailto:MSN
>     <mailto:MSN>>:hx...@ho... <mailto:hx...@ho...>
>     > <mailto: hx...@ho... <mailto:hx...@ho...>>>
>     >
>     >
>     >
>     >
>     > -------------------------------------------------------
>     > This SF.Net email is sponsored by xPML, a groundbreaking scripting
>     > language
>     > that extends applications into web and mobile media. Attend the
>     > live webcast
>     > and join the prime developer group breaking into this new coding
>     > territory!
>     >
>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>
>     >
>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>>
>     > _______________________________________________
>     > Htmlparser-user mailing list
>     > Htm...@li...
>     <mailto:Htm...@li...>
>     > <mailto:Htm...@li...
>     <mailto:Htm...@li...>>
>     > https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>     <https://lists.sourceforge.net/lists/listinfo/htmlparser-user>
>     > <https://lists.sourceforge.net/lists/listinfo/htmlparser-user>
>     >
>     >
>     >
>     >
>     > --
>     > Best Regards.
>     >
>     > Xiaodong Han
>     > MSN:hx...@ho... <mailto:MSN:hx...@ho...>
>     <mailto:MSN <mailto:MSN>:hx...@ho...
>     <mailto:hx...@ho...>>
>
>
>
>
>     -------------------------------------------------------
>     This SF.Net email is sponsored by xPML, a groundbreaking scripting
>     language
>     that extends applications into web and mobile media. Attend the
>     live webcast
>     and join the prime developer group breaking into this new coding
>     territory!
>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642>
>     _______________________________________________
>     Htmlparser-user mailing list
>     Htm...@li...
>     <mailto:Htm...@li...>
>     https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>
>
>
>
> -- 
> Best Regards.
>
> Xiaodong Han
> MSN:hx...@ho... <mailto:MSN:hx...@ho...>

Re: [Htmlparser-user] cookie

[Vishal M. <mon...@ho...>]

Hi Everybody,

  I have used this htmlparser in my project and the next week I have to give 
presentation for this. I have few question about htmlparser.

1) What kind of parsing technique is used to develop it. Top down parsing or 
Bottom up parsing.
2) How one can figure it out that it is is using top down parsing / bottom 
up parsing technique is used.

Thanks in advance.

Regards,
Vishal Monpara

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: [Htmlparser-user] cookie

[Derrick O. <Der...@Ro...>]

I guess it would be considered top down parsing. The lexer is under the 
direction of a scanner for each tag, which knows the 'production rule'.

To figure it out I think you look for 'push' or 'pull'.  Is something 
pulling (see IteratorImpl.nextNode()) or is it event driven and 
something is reacting to lower level lexeme recognition.

Vishal Monpara wrote:

> Hi Everybody,
>
>  I have used this htmlparser in my project and the next week I have to 
> give presentation for this. I have few question about htmlparser.
>
> 1) What kind of parsing technique is used to develop it. Top down 
> parsing or Bottom up parsing.
> 2) How one can figure it out that it is is using top down parsing / 
> bottom up parsing technique is used.
>
> Thanks in advance.
>
> Regards,
> Vishal Monpara
>

Re: [Htmlparser-user] cookie

[<xia...@gm...>]

I try to use connectionManager but it can not hold the connection to web
server.
that is to say when you post your credential to server ,you cookie message
could not
be hold for second connection.

I change httpclient to hold cookie message to solve this .

On 4/3/06, Derrick Oswald <Der...@ro...> wrote:
>
>
> I guess it would be considered top down parsing. The lexer is under the
> direction of a scanner for each tag, which knows the 'production rule'.
>
> To figure it out I think you look for 'push' or 'pull'.  Is something
> pulling (see IteratorImpl.nextNode()) or is it event driven and
> something is reacting to lower level lexeme recognition.
>
> Vishal Monpara wrote:
>
> > Hi Everybody,
> >
> >  I have used this htmlparser in my project and the next week I have to
> > give presentation for this. I have few question about htmlparser.
> >
> > 1) What kind of parsing technique is used to develop it. Top down
> > parsing or Bottom up parsing.
> > 2) How one can figure it out that it is is using top down parsing /
> > bottom up parsing technique is used.
> >
> > Thanks in advance.
> >
> > Regards,
> > Vishal Monpara
> >
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=
=3D121642
> _______________________________________________
> Htmlparser-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlparser-user
>



--
Best Regards.

Xiaodong Han
MSN:hx...@ho...