Re: [Htmlparser-user] Character Encoding

[Derrick O. <der...@ro...>]

That <meta> tag doesn't look like the problem.

If you use the built in ConnectionMonitor on the parser, you can see the header:

C:>java -classpath parser\target\htmlparser.jar;lexer\target\htmllexer.jar org.htmlparser.Parser http://cbc.ca
INFO: GET http://cbc.ca HTTP/1.1
Accept-Encoding: gzip, deflate
User-Agent: HTMLParser/2.0

INFO: HTTP/1.1 301 Moved Permanently
Date: Tue, 13 May 2008 01:12:31 GMT
Server: Apache/2.0.59 (Linux/SuSE) mod_jk/1.2.6-dev
Location: http://www.cbc.ca/
Cache-Control: max-age=120
Expires: Tue, 13 May 2008 01:14:31 GMT
Content-Length: 226
Keep-Alive: timeout=15, max=150
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

INFO: GET http://www.cbc.ca/ HTTP/1.1
Accept-Encoding: gzip, deflate
User-Agent: HTMLParser/2.0

INFO: HTTP/1.1 200 OK
Server: Apache/2.0.59 (Linux/SuSE) mod_jk/1.2.6-dev
Accept-Ranges: bytes
Content-Type: text/html
Cache-Control: max-age=61
Expires: Tue, 13 May 2008 01:13:32 GMT
Date: Tue, 13 May 2008 01:12:31 GMT
Content-Length: 28625
Connection: keep-alive


----- Original Message ----
From: "bo...@ti..." <bo...@ti...>
To: htm...@li...
Sent: Monday, May 12, 2008 7:55:56 AM
Subject: [Htmlparser-user] Character Encoding

Thanks Derrick,

The page in question includes the following tags:

<META http-equiv=Content-Type content="text/html; charset=utf-8">
<META http-equiv=content-type>

I don't understand why the second one is there but it really is. With 
that information can you suggest a resolution? I am not entirely sure 
how to verify your point (1).

Best Regards

Brian
-----------------------------------------------------------------------------

There are two possibilities.

1) The HTTP server is/is not serving up content type meta information 
in the HTTP header like so:
text/html; charset=utf-8

2) The source HTML does/does not contain a meta tag like so:

<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
You need to determine which one so the appropriate 'fix' can be 
applied.


----- Original Message ----
From: "bo...@ti..." <bo...@ti...>
To: htm...@li...
Sent: Monday, May 12, 2008 7:31:39 AM
Subject: [Htmlparser-user] Character Encoding

Hi,

I have a strange problem and I can’t get my head around it. Hopefully 
someone can point me in the right direction. I’m using the following 
code with HTMLParser 1.6 to retrieve web pages:

                parser                      = new Parser
(URL);              
                ThePage                  = parser.parse
(null);          
                MyPage                    = ThePage.toHtml();
                

On some pages (not all…) if the HTML page contains:

£10 Free

“My Page” contains “?10 Free” on other pages it works fine.

I guess it has something to do with character encoding? Can someone 
suggest what I add where to get this to work correctly (I would like 
to 
keep the “£10 Free”)

Thanks in advance

Brian



_______________________________
How can you protect children online?  Find out - http://www.tiscali.co.uk/protection


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Htmlparser-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlparser-user