python-markdown-discuss — A general purpose discussion list

Re: [Python-markdown-discuss] Access Python variables from Markdown?

[Waylan L. <wa...@gm...>]

Ivan,

This is way out of scope for Markdown. That said, I don't see why an
extension wouldn't be able to do something like that.  I don't really
recommend this approach, but I suppose an inline pattern would do the
trick.

Another approach, and one I know some have used successfully, is to
pass the markdown text through a template system first (like Jinja) -
which replaces the variables - then pass it through the markdown
parser. Or maybe they passed it through markdown first, then the
template system. I'm sure one or the other could work depending upon
the needs of the specific project. Personally, this would be my
approach - rather than an extension to markdown.


On Sat, Mar 23, 2013 at 6:34 PM, Ivan Mirić <ph...@gm...> wrote:
> Hi,
>
> Is there a chance of adding the ability to access Python variables from
> Markdown?
>
> As seen here (https://github.com/ipython/ipython/issues/2958), the `{{}}`
> syntax would be great.
>
> If this is not in the pipeline as a built-in Python-Markdown feature, how
> difficult would it be to write an extension for it? If someone points me in
> the right direction, I might attempt it myself.
>
> Thanks,
>
> Ivan
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] Access Python variables from Markdown?

[Ivan M. <ph...@gm...>]

Hi,

Is there a chance of adding the ability to access Python variables from
Markdown?

As seen here (https://github.com/ipython/ipython/issues/2958), the `{{}}`
syntax would be great.

If this is not in the pipeline as a built-in Python-Markdown feature, how
difficult would it be to write an extension for it? If someone points me in
the right direction, I might attempt it myself.

Thanks,

Ivan

Re: [Python-markdown-discuss] sectionalize extension

[Waylan L. <wa...@gm...>]

Alex,

There are already a couple extensions which do something similar
listed on the wiki [1]. You might want to start there. Or perhaps
those will fill your need.

[1]: https://github.com/waylan/Python-Markdown/wiki/Third-Party-Extensions



On Mon, Mar 4, 2013 at 3:51 PM, AL <new...@al...> wrote:
> Hi,
>
> I'm thinking about writing an extension that would wrap content into
> html5 section tags. It would split the content according to "8<" markers
> (or similar), turning this:
>
>      Some content
>
>      8<
>
>      # Some Header
>
>      Some more content
>
>      8<
>
>      # Another Header
>
>      etc.
>
> into:
>
>      <p>Some content</p>
>
>      <section>
>        <h1>Some Header</h1>
>        <p>Some more content</p>
>      </section>
>
>      <section>
>        <h1>Another Header</h1>
>        <p>etc.</p>
>      </section>
>
> I'd like to be able to use the attr_list extension, and something like
> the headerid one. I'm wondering what would be the best way to approach
> this. Do you have any advices?
>
> Thanks,
>
> Alex
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss



--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] sectionalize extension

[AL <new...@al...>]

Hi,

I'm thinking about writing an extension that would wrap content into 
html5 section tags. It would split the content according to "8<" markers 
(or similar), turning this:

     Some content

     8<

     # Some Header

     Some more content

     8<

     # Another Header

     etc.

into:

     <p>Some content</p>

     <section>
       <h1>Some Header</h1>
       <p>Some more content</p>
     </section>

     <section>
       <h1>Another Header</h1>
       <p>etc.</p>
     </section>

I'd like to be able to use the attr_list extension, and something like 
the headerid one. I'm wondering what would be the best way to approach 
this. Do you have any advices?

Thanks,

Alex

Re: [Python-markdown-discuss] Oddity?

[Waylan L. <wa...@gm...>]

I can't replicate this with the source you provide, but...

On Wed, Dec 12, 2012 at 9:22 AM, Dave Pawson <dav...@gm...> wrote:
> 1. All she had to do was write 25 lines of poetry
> 2. she wrote two sonnets, which have 14 lines each
> 3. ∴ Nancy finished all her homework
>
> The third line is split out
>
> <ol>
> <li>All she had to do was write 25 lines of poetry</li>
> <li>she wrote two sonnets, which have 14 lines each</li>
> <li>
> <p>∴ Nancy finished all her homework</p>
> <p>She wrote two sonnets of 14 lines each. (dependency is 'of 14 lines
> each')</p>
> </li>
> </ol>

I do notice that you have a fourth line in your output. Is that fourth
line perhaps indented in your source? If so, then it is seen as a
child of the last list item and all children of that list item are
separate paragraphs in that list item.

This is all spelled out in the syntax documentation (scroll down to
the part about multiple paragraphs):
http://daringfireball.net/projects/markdown/syntax#list

Therefore, I'm assuming that your source input actually looks like
this (as opposed to what you provided above):

~~~
1. All she had to do was write 25 lines of poetry
2. she wrote two sonnets, which have 14 lines each
3. ∴ Nancy finished all her homework

    She wrote two sonnets of 14 lines each. (dependency is 'of 14 lines each')
~~~

In which case the output is exactly correct. I would suggest testing
these sorts of things out with Babelmark2:
http://johnmacfarlane.net/babelmark2/. It is a great way to determine
whether there is a bug in the one parser you happen to be using, or if
you have some bad input (when all/most parsers agree).

--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] Oddity?

[Dave P. <dav...@gm...>]

1. All she had to do was write 25 lines of poetry
2. she wrote two sonnets, which have 14 lines each
3. ∴ Nancy finished all her homework

The third line is split out

<ol>
<li>All she had to do was write 25 lines of poetry</li>
<li>she wrote two sonnets, which have 14 lines each</li>
<li>
<p>∴ Nancy finished all her homework</p>
<p>She wrote two sonnets of 14 lines each. (dependency is 'of 14 lines
each')</p>
</li>
</ol>

with a p wrapper? Any idea why please?


regards

-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Waylan L. <wa...@gm...>]

On Mon, Dec 10, 2012 at 1:31 PM, Dave Pawson <dav...@gm...> wrote:
>
> Now working,

Glad to hear

> ps. Why is it called xhtml1 output, when there is no <html> and <head/> markup?
> It's clearly wrong.

Python-Markdown has two goals:

1. Implement markdown in Python as closely as reasonably possible to
the original perl implementation.
2. Provide an API which allows any of the behavior to be overriden.

The answer to your question (which I believe is a follow-up to an
earlier question you asked and I failed to respond to) relates to the
first goal. Simply put, that is the way markdown.pl does it. It
outputs xhtml style syntax (for example `<br />` rather than `<br>`)
so we do too. Of course, unlike markdown.pl we also provide html style
output as an option (and even allow the serializer to be overridden
via python subclassing).

But that's not the only reason. Python-Markdown is a library first and
a commandline script second (the commandline script is a light wrapper
around the library). Most users are using the library from within
python based web server backends (like Django) and pass the output to
various template systems. A oversimplfied `<html>` wrapper would be
useless in such settings.

Even when I use it from the commandline, I always want something more
complex than I could justify including in a default implementation. In
fact, whenever I use other tools which do wrap their output in
`<html>` tags, I always find myself writing code to strip those
wrappers and then passing the contents of the body to my own
templates. Therefore, in Python-Markdown it is left for the user to
implement however s/he wishes.

Of course, given goal #2, it wouldn't be too much trouble to write an
extension which  wrapped the output. Although, if you're using the
commandline, a shell script certainly makes more sense.
--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 18:44, Brian Neal <bg...@gm...> wrote:
> On Mon, Dec 10, 2012 at 12:18 PM, Dave Pawson <dav...@gm...> wrote:
>> Is there a Pythonic way to install from github please?
>
> $ pip install git+https://github.com/waylan/Python-Markdown.git
>
> I highly recommend using pip + virtualenv.
>
> http://www.pip-installer.org/en/latest/index.html
> http://www.virtualenv.org/en/latest/

.... OK pip installed.
I now have the 2.3 version installed.... but seemingly only within
this virtual environment?

$ . myenv/bin/activate
(myenv)[dpawson@homer python]$ python -m markdown --version
__main__.py 2.3.dev

But

[dpawson@homer ~]$ python -m markdown --version
__main__.py 2.2.0

So I guess I need to move pymarkdown from virtual to 'real'?
1. How to get rid of 2.2.0 please ? Installed via easy_install
2. How to replace it with 2.3?


TIA


-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Brian N. <bg...@gm...>]

On Mon, Dec 10, 2012 at 12:18 PM, Dave Pawson <dav...@gm...> wrote:
> Is there a Pythonic way to install from github please?

$ pip install git+https://github.com/waylan/Python-Markdown.git

I highly recommend using pip + virtualenv.

http://www.pip-installer.org/en/latest/index.html
http://www.virtualenv.org/en/latest/

Best,
BN

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 18:05, Waylan Limberg <wa...@gm...> wrote:
> Nope, I still can't replicate the problem. Just to be clear, I am not
> using the latest release available on pypi, but the most recent code
> in the git repo on GitHub [1] (which contains a patch for this kind of
> issue). In order words, I believe the bug has been fixed - with the
> fix becoming available in the next release. If however, you are using
> the latest from Github and still seeing the problem, all I can suggest
> is to set PYTHONENCODING (or avoid using pipes).
>
> [1]: https://github.com/waylan/Python-Markdown

Installed (the hard way!).
Now handles utf-8 as default, includeing redirection.

Now working, shows
]$ python -m markdown --version
__main__.py 2.2.0

Thanks.

ps. Why is it called xhtml1 output, when there is no <html> and <head/> markup?
It's clearly wrong.

regards



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 18:05, Waylan Limberg <wa...@gm...> wrote:
> Nope, I still can't replicate the problem. Just to be clear, I am not
> using the latest release available on pypi, but the most recent code
> in the git repo on GitHub [1] (which contains a patch for this kind of
> issue). In order words, I believe the bug has been fixed - with the
> fix becoming available in the next release. If however, you are using
> the latest from Github and still seeing the problem, all I can suggest
> is to set PYTHONENCODING (or avoid using pipes).
>
> [1]: https://github.com/waylan/Python-Markdown

Is there a Pythonic way to install from github please?

regards


>
> On Mon, Dec 10, 2012 at 12:29 PM, Dave Pawson <dav...@gm...> wrote:
>> On 10 December 2012 17:05, Waylan Limberg <wa...@gm...> wrote:
>>> On Mon, Dec 10, 2012 at 11:30 AM, Dave Pawson <dav...@gm...> wrote:
>>>> I think the error I was getting shows that didn't happen in my case?
>>>> 0x2234 as the character
>>>> Python 2.7
>>>> Emacs encoding set to utf-8
>>>>
>>>> cmd line either defaulted or -e utf-8 (if that is right?)
>>>> markdown >> file.ext
>>>>
>>>> I received an error.
>>>>
>>>
>>> And I can't seem duplicate that error on my system with the latest
>>> code. Which makes me wonder if perhaps the problem is with your
>>> script, but I can't say for sure.
>>
>> Script as in bash script?
>>
>> #!/bin/bash
>> source ~/bin/dpFunctions.sh
>> # Run python markdown with following extensions
>> # abbr, def_list
>> # See /installation/python/pymarkdown
>>
>> bn=`nameonly $1`
>> op=${bn}.html
>> rmif tmp.tmp
>>
>> echo -e """
>> <!DOCTYPE html>
>> <html>
>>  <head>
>>    <title> $1 </title>
>>    <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />
>>     <link rel=\"stylesheet\" href=\"/styles/css/md.css\" type=\"text/css\" />
>>
>>  </head>
>> <body>""" >$op
>> python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy -f
>> tmp.tmp ${1}
>> cat  tmp.tmp >>${op}
>> echo """
>> </body>
>> </html>
>> """ >> $op
>>
>> Now corrected.
>>
>> If you run that more simply
>>
>> echo -e "<html>" >op.html
>> python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy >>op.html
>> echo -e "</html>" >>op.html
>>
>> with a md file containing 0x2234 you should see the error.
>>
>> Running directly or with -f sidesteps this one.
>>
>> HTH
>>
>>
>>
>> --
>> Dave Pawson
>> XSLT XSL-FO FAQ.
>> Docbook FAQ.
>> http://www.dpawson.co.uk
>>
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>> http://p.sf.net/sfu/logmein_12329d2d
>> _______________________________________________
>> Python-markdown-discuss mailing list
>> Pyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
>
>
>
> --
> ----
> \X/ /-\ `/ |_ /-\ |\|
> Waylan Limberg



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Waylan L. <wa...@gm...>]

Nope, I still can't replicate the problem. Just to be clear, I am not
using the latest release available on pypi, but the most recent code
in the git repo on GitHub [1] (which contains a patch for this kind of
issue). In order words, I believe the bug has been fixed - with the
fix becoming available in the next release. If however, you are using
the latest from Github and still seeing the problem, all I can suggest
is to set PYTHONENCODING (or avoid using pipes).

[1]: https://github.com/waylan/Python-Markdown

On Mon, Dec 10, 2012 at 12:29 PM, Dave Pawson <dav...@gm...> wrote:
> On 10 December 2012 17:05, Waylan Limberg <wa...@gm...> wrote:
>> On Mon, Dec 10, 2012 at 11:30 AM, Dave Pawson <dav...@gm...> wrote:
>>> I think the error I was getting shows that didn't happen in my case?
>>> 0x2234 as the character
>>> Python 2.7
>>> Emacs encoding set to utf-8
>>>
>>> cmd line either defaulted or -e utf-8 (if that is right?)
>>> markdown >> file.ext
>>>
>>> I received an error.
>>>
>>
>> And I can't seem duplicate that error on my system with the latest
>> code. Which makes me wonder if perhaps the problem is with your
>> script, but I can't say for sure.
>
> Script as in bash script?
>
> #!/bin/bash
> source ~/bin/dpFunctions.sh
> # Run python markdown with following extensions
> # abbr, def_list
> # See /installation/python/pymarkdown
>
> bn=`nameonly $1`
> op=${bn}.html
> rmif tmp.tmp
>
> echo -e """
> <!DOCTYPE html>
> <html>
>  <head>
>    <title> $1 </title>
>    <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />
>     <link rel=\"stylesheet\" href=\"/styles/css/md.css\" type=\"text/css\" />
>
>  </head>
> <body>""" >$op
> python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy -f
> tmp.tmp ${1}
> cat  tmp.tmp >>${op}
> echo """
> </body>
> </html>
> """ >> $op
>
> Now corrected.
>
> If you run that more simply
>
> echo -e "<html>" >op.html
> python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy >>op.html
> echo -e "</html>" >>op.html
>
> with a md file containing 0x2234 you should see the error.
>
> Running directly or with -f sidesteps this one.
>
> HTH
>
>
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> Docbook FAQ.
> http://www.dpawson.co.uk
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 17:05, Waylan Limberg <wa...@gm...> wrote:
> On Mon, Dec 10, 2012 at 11:30 AM, Dave Pawson <dav...@gm...> wrote:
>> I think the error I was getting shows that didn't happen in my case?
>> 0x2234 as the character
>> Python 2.7
>> Emacs encoding set to utf-8
>>
>> cmd line either defaulted or -e utf-8 (if that is right?)
>> markdown >> file.ext
>>
>> I received an error.
>>
>
> And I can't seem duplicate that error on my system with the latest
> code. Which makes me wonder if perhaps the problem is with your
> script, but I can't say for sure.

Script as in bash script?

#!/bin/bash
source ~/bin/dpFunctions.sh
# Run python markdown with following extensions
# abbr, def_list
# See /installation/python/pymarkdown

bn=`nameonly $1`
op=${bn}.html
rmif tmp.tmp

echo -e """
<!DOCTYPE html>
<html>
 <head>
   <title> $1 </title>
   <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />
    <link rel=\"stylesheet\" href=\"/styles/css/md.css\" type=\"text/css\" />

 </head>
<body>""" >$op
python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy -f
tmp.tmp ${1}
cat  tmp.tmp >>${op}
echo """
</body>
</html>
""" >> $op

Now corrected.

If you run that more simply

echo -e "<html>" >op.html
python -m markdown -v -e utf-8 -o xhtml1  -x extra -x toc --noisy >>op.html
echo -e "</html>" >>op.html

with a md file containing 0x2234 you should see the error.

Running directly or with -f sidesteps this one.

HTH



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Waylan L. <wa...@gm...>]

On Mon, Dec 10, 2012 at 11:30 AM, Dave Pawson <dav...@gm...> wrote:
> I think the error I was getting shows that didn't happen in my case?
> 0x2234 as the character
> Python 2.7
> Emacs encoding set to utf-8
>
> cmd line either defaulted or -e utf-8 (if that is right?)
> markdown >> file.ext
>
> I received an error.
>

And I can't seem duplicate that error on my system with the latest
code. Which makes me wonder if perhaps the problem is with your
script, but I can't say for sure.

-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

I think the error I was getting shows that didn't happen in my case?
0x2234 as the character
Python 2.7
Emacs encoding set to utf-8

cmd line either defaulted or -e utf-8 (if that is right?)
markdown >> file.ext

I received an error.

HTH Dave


On 10 December 2012 16:18, Waylan Limberg <wa...@gm...> wrote:
> On Mon, Dec 10, 2012 at 10:48 AM, Brian Neal <bg...@gm...> wrote:
>> On Mon, Dec 10, 2012 at 9:16 AM, Brian Neal <bg...@gm...> wrote:
>>> You can also set the environment variable PYTHONIOENCODING if you find
>>> yourself needing to do a lot of redirection or piping.
>>
>> Having said that, I went back to the script that I wrote that was
>> having this problem. I fixed it by using unicode everywhere but then
>> picking an encoding on output, something like:
>>
>> print my_string.encode('utf-8')
>>
>> It does look to me like Markdown is trying to do something like this
>> if you run it as a module. Perhaps Waylan can chime in here.
>>
>
> Yes, I have striven to have the code do the same thing regardless of
> what version of python is being used. Within the markdown.FromFile
> method, input is decoded to unicode (regardless of import source) and
> then forwarded to markdown, and the output is encoded (regardless of
> output method: file, stdout, etc) and written out as bytes (even in
> python3). In both instances (encoding and decoding) the character
> encoding used is the same user defined encoding (or defaults to uft8
> if not defined).
>
> There were a few edge cases where the code was failing to take the
> encoding into account (PYTHONENCODING would have made a difference),
> however, I don't think that is the case anymore. Although perhaps the
> fallback default should perhaps be set to PYTHONENCODING rather than
> hardcoded to utf8.
>
> The relevant code is here:
> https://github.com/waylan/Python-Markdown/blob/master/markdown/__init__.py#L324
>
> --
> ----
> \X/ /-\ `/ |_ /-\ |\|
> Waylan Limberg



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Waylan L. <wa...@gm...>]

On Mon, Dec 10, 2012 at 10:48 AM, Brian Neal <bg...@gm...> wrote:
> On Mon, Dec 10, 2012 at 9:16 AM, Brian Neal <bg...@gm...> wrote:
>> You can also set the environment variable PYTHONIOENCODING if you find
>> yourself needing to do a lot of redirection or piping.
>
> Having said that, I went back to the script that I wrote that was
> having this problem. I fixed it by using unicode everywhere but then
> picking an encoding on output, something like:
>
> print my_string.encode('utf-8')
>
> It does look to me like Markdown is trying to do something like this
> if you run it as a module. Perhaps Waylan can chime in here.
>

Yes, I have striven to have the code do the same thing regardless of
what version of python is being used. Within the markdown.FromFile
method, input is decoded to unicode (regardless of import source) and
then forwarded to markdown, and the output is encoded (regardless of
output method: file, stdout, etc) and written out as bytes (even in
python3). In both instances (encoding and decoding) the character
encoding used is the same user defined encoding (or defaults to uft8
if not defined).

There were a few edge cases where the code was failing to take the
encoding into account (PYTHONENCODING would have made a difference),
however, I don't think that is the case anymore. Although perhaps the
fallback default should perhaps be set to PYTHONENCODING rather than
hardcoded to utf8.

The relevant code is here:
https://github.com/waylan/Python-Markdown/blob/master/markdown/__init__.py#L324

--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 15:16, Brian Neal <bg...@gm...> wrote:
> On Mon, Dec 10, 2012 at 8:58 AM, Dave Pawson <dav...@gm...> wrote:
>>
>> I'm on Fedora, which uses Python heavily for scripts, so I'm reluctant
>> to install 3 ahead of Fedora.
>> My solution works which is enough for me.
>
>
> You can also set the environment variable PYTHONIOENCODING if you find
> yourself needing to do a lot of redirection or piping.
>

Useful to know...
http://docs.python.org/2/using/cmdline.html?highlight=pythonioencoding#PYTHONIOENCODING
presumable
export PYTHONIOENCODING=utf-8

I think that is worth adding to the user docs?
The values taken by the -e option are missing too.

regards



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Brian N. <bg...@gm...>]

On Mon, Dec 10, 2012 at 9:16 AM, Brian Neal <bg...@gm...> wrote:
> You can also set the environment variable PYTHONIOENCODING if you find
> yourself needing to do a lot of redirection or piping.

Having said that, I went back to the script that I wrote that was
having this problem. I fixed it by using unicode everywhere but then
picking an encoding on output, something like:

print my_string.encode('utf-8')

It does look to me like Markdown is trying to do something like this
if you run it as a module. Perhaps Waylan can chime in here.

-BN

Re: [Python-markdown-discuss] -e utf-8

[Brian N. <bg...@gm...>]

On Mon, Dec 10, 2012 at 8:58 AM, Dave Pawson <dav...@gm...> wrote:
>
> I'm on Fedora, which uses Python heavily for scripts, so I'm reluctant
> to install 3 ahead of Fedora.
> My solution works which is enough for me.


You can also set the environment variable PYTHONIOENCODING if you find
yourself needing to do a lot of redirection or piping.

Regards,
BN

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 10 December 2012 13:49, Brian Neal <bg...@gm...> wrote:
>
> On Mon, Dec 10, 2012 at 1:34 AM, Dave Pawson <dav...@gm...> wrote:
>>
>> Not that I was redirecting output to the output file? (The commented out
>> line)
>> This created the problem. It would seem to be a difference between
>> redirection and
>> cp and >>
>> The replacement line solves that problem.
>> It would seem redirected output is unable to utilise an appropriate
>> character set.
>
>
> I've run into this before. It isn't a problem with Markdown, more of an
> issue with Python 2.7 and redirection to a pipe. See
>
> http://stackoverflow.com/questions/4545661/unicodedecodeerror-when-redirecting-to-file
>
> In particular the answer by Mark Tolonen has helped me in the past.

"Python 3 defaults to 'utf8', but based on the OP's sample, he's using
Python 2.X, which defaults to 'ascii'. – Mark Tolonen Jan 5 '11 at
18:49"

That explains it.  Thanks Brian. Nice to know why.

I'm on Fedora, which uses Python heavily for scripts, so I'm reluctant
to install 3 ahead of Fedora.
My solution works which is enough for me.

regards




-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Brian N. <bg...@gm...>]

On Mon, Dec 10, 2012 at 1:34 AM, Dave Pawson <dav...@gm...> wrote:

> Not that I was redirecting output to the output file? (The commented out
> line)
> This created the problem. It would seem to be a difference between
> redirection and
> cp and >>
> The replacement line solves that problem.
> It would seem redirected output is unable to utilise an appropriate
> character set.
>

I've run into this before. It isn't a problem with Markdown, more of an
issue with Python 2.7 and redirection to a pipe. See

http://stackoverflow.com/questions/4545661/unicodedecodeerror-when-redirecting-to-file

In particular the answer by Mark Tolonen has helped me in the past.

Regards,
BN

Re: [Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

On 9 December 2012 15:57, Waylan Limberg <wa...@gm...> wrote:
> Dave,
>
> I'll need more info to be able to help you. What version of
> Python-Markdown are you using? What version of Python? What
> specifically are you trying to do? Can you provide a minimal document
> and command that replicates the problem?

python 2.7.3
md.py from couple of days ago, so guess latest.

I want to use utf-8 in output.

the problem is now resolved, but others may see it.
Solution below.

I want HTML output, not the body element contents?
I'm not sure why markdown.py doesn't do this.
Is there a good reason?


So I created a bash script

bn=`nameonly $1`
op=${bn}.html
rmif tmp.tmp

echo -e """
<!DOCTYPE html>
<html>
 <head>
   <title> $1 </title>
   <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />
    <link rel=\"stylesheet\" href=\"/styles/css/md.css\" type=\"text/css\" />

 </head>
<body>""" >$op
#python -m markdown -v -e utf-8 -o xhtml1  -x extra --noisy >> ${op}
python -m markdown -v -e utf-8 -o xhtml1  -x extra --noisy -f tmp.tmp ${1}
cat  tmp.tmp >>${op}
echo """
</body>
</html>
""" >> $op

Not that I was redirecting output to the output file? (The commented out line)
This created the problem. It would seem to be a difference between
redirection and
cp and >>
The replacement line solves that problem.
It would seem redirected output is unable to utilise an appropriate
character set.


The input which shows this error.


3. ∴  henrys car has four doors. (given)

(u2234) is the character.

HTH



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] -e utf-8

[Waylan L. <wa...@gm...>]

Dave,

I'll need more info to be able to help you. What version of
Python-Markdown are you using? What version of Python? What
specifically are you trying to do? Can you provide a minimal document
and command that replicates the problem?

Without that info, I can't say for sure, but it is possible that this
is the same bug I recently fixed here (the error message matches):
https://github.com/waylan/Python-Markdown/issues/158

If you apply the two commits referenced in that discussion, does that
fix your problem?

On Sat, Dec 8, 2012 at 6:43 AM, Dave Pawson <dav...@gm...> wrote:
> I'm having an issue with an encoding character.
>
> Called from the cmd line with
> -e utf-8
>
> I get
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u2234' in
> position 810: ordinal not in range(128)
>
> when I include 0x2234 in the .md file
>
> What am I getting wrong please?
> It seems the encoding options aren't documented anywhere?
>
>
> TIA
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> Docbook FAQ.
> http://www.dpawson.co.uk
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Python-markdown-discuss] -e utf-8

[Dave P. <dav...@gm...>]

I'm having an issue with an encoding character.

Called from the cmd line with
-e utf-8

I get
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2234' in
position 810: ordinal not in range(128)

when I include 0x2234 in the .md file

What am I getting wrong please?
It seems the encoding options aren't documented anywhere?


TIA

-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

Re: [Python-markdown-discuss] Shipping mathjax extension by default?

[Waylan L. <wa...@gm...>]

Dmitry,

I think this is something that belongs as a third party extension. If
it is not already, I would suggest listing it on the wiki [1] along
with a few other math extensions.

Why? Well, for one, I don't have a need for such an extension.
Therefore, I am not in a good position to determine which of the
existing extensions would be a good choice. Also, I am not interested
in maintaining it - that should be left to someone who uses and
understands the needs of its users. Finally, I'm trying to avoid
feature creep.

[1]: https://github.com/waylan/Python-Markdown/wiki/Third-Party-Extensions

Waylan

On Thu, Aug 30, 2012 at 10:34 AM, Dmitry Shachnev <mi...@gm...> wrote:
> Hi Waylan (and everybody subscribed)!
>
> There's a [wonderful extension][1] at GitHub that adds support for
> math formulas using [MathJax][2]. I've been using it myself for a
> while and added support for it to my [ReText editor][3] (will be
> available in 4.0 release).
>
> It's very small (less than 20 lines of code) and does its job well.
>
> Will it be possible to ship it by default in python-markdown? I can
> prepare a pull request if needed.
>
> [1]: https://github.com/mayoff/python-markdown-mathjax
> [2]: http://www.mathjax.org/
> [3]: http://retext.sourceforge.net/
>
> Cheers,
>
> --
> Dmitry Shachnev
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Python-markdown-discuss mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-markdown-discuss



-- 
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg