[htdig] 8-bit and/or encoded characters

[Mac OS X S. A. <mac...@gm...>]

Hi,

I've gone through the list archives, but most of the promising results
(e.g. http://www.geocrawler.com/archives/3/8822/2002/2/0/7814312/)
return 404s from SourceForge...

Any accented characters are parsed as an empty box (IE/Win) or a
question mark (in a diamond in Safari) in search results pages.

Should this be handled via set_locale? If so, then how does one handle
German- or Spanish-accented characters on an otherwise English site
(e.g. proper nouns), which has an en_US locale set? We do use the span
tag with a language attribute in the XHTML; is there any way to hook
that into ht://Dig?

HTML entities show up as plain text -- m-dashes, curly-quotes, curly
apostrophes, etc. render as source code, not as m-dashes, curly-quotes
and curly apostrophes.

Are these separate problems, and what can be done?