Re: [Python-markdown-discuss] Unicode errors with 1.6b rc2.

[Yuri T. <qar...@gm...>]

It seems that you are loading your file as if it was ASCII (i.e., with
just plain open()) even thought it's not (because of the smart quotes
saved as UTF8).  When you do this, the non-ascii characters sometimes
do not cause problems and sometimes they do.  So, I am not sure why
this disabling removeBOM makes this problem go away and wouldn't rely
on this.

Instead, you should open your file with using:

input_file = codecs.open(input, mode="r", encoding="utf8")

(Or make it ASCII-compliant.)

  - yuri

On 5/15/07, Aaron Gyes <fl...@sh...> wrote:
> It seems to be "all of them". It oddly enough doesn't happen when I
> give them to it running ./markdown.py at a CLI.
>
> One such file would be http://aarongyes.com/pages/home.txt, another
> http://aarongyes.com/pages/resume.txt. Those files should be
> completely identical, they are in the exact same location as they
> really are being used at, not even copied.
>
> I have wedged it into a home-made templating system, and am calling
> it with .convert(string).
>
> You can look at my code here:
> http://aarongyes.com/sitestruc.py_
> http://aarongyes.com/site.py_
>
> (the latter being relevant).
>
> Temporarily making removeBOM do nothing but return text has it
> hobbling along.
>
> On May 15, 2007, at 7:18 PM, Yuri Takhteyev wrote:
>
> > Can you send a file that causes the error?  Also, how are you
> > calling it?
> >
> >  - yuri
> >
> > On 5/15/07, Aaron Gyes <fl...@sh...> wrote:
> >> Not entirely sure what is going on, it was just fine with 1.6a.
> >>
> >> File "/www/htdocs/floam.sh.nu/html/markdown.py", line 1611, in
> >> convert
> >> \n    self.source = removeBOM(self.source, self.encoding)
> >> File "/www/htdocs/floam.sh.nu/html/markdown.py", line 74, in
> >> removeBOM
> >> \n    if text.startswith(bom):
> >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
> >> 253: ordinal not in range(128)
> >>
> >> Aaron Gyes
> >
> >
> > --
> > Yuri Takhteyev
> > UC Berkeley School of Information
> > http://www.freewisdom.org/
>


-- 
Yuri Takhteyev
UC Berkeley School of Information
http://www.freewisdom.org/