[Docutils-checkins] r6118 - in trunk/docutils: HISTORY.txt docutils/writers/html4css1/__init__.py docutils/writers/latex2e/__init__.py

[<milde@us...>]

Author: milde
Date: 2009-09-07 10:57:48 +0200 (Mon, 07 Sep 2009)
New Revision: 6118

Modified:
   trunk/docutils/HISTORY.txt
   trunk/docutils/docutils/writers/html4css1/__init__.py
   trunk/docutils/docutils/writers/latex2e/__init__.py
Log:
Fix [ 1692788 ] allow UTF-8 in style sheets.

Modified: trunk/docutils/HISTORY.txt
===================================================================
--- trunk/docutils/HISTORY.txt	2009-09-07 08:51:25 UTC (rev 6117)
+++ trunk/docutils/HISTORY.txt	2009-09-07 08:57:48 UTC (rev 6118)
@@ -27,7 +27,7 @@
     + minimize "types" module where possible.
     + Backwards-compatible changes to remove python2.6 -3 deprecation warnings
     + Text nodes now subclass unicode rather than UserString
-      (which is gone in python 3.0)
+      (which is gone in python 3.0).
     + 3.0 compatibility module docutils._compat
 
     + drop 2.2 compatibility workarounds
@@ -37,9 +37,10 @@
     core.
 
   - Apply [ 1719345 ] Galician translation
-  - Apply [ 1905741 ] polish-translation
+  - Apply [ 1905741 ] Polish translation
   - Apply [ 1878977 ] make_id(): deaccent characters.
-  - Apply [ 2029251 ] return nonzero when tests fail
+  - Apply [ 2029251 ] return nonzero when tests fail.
+  - Fix [ 1692788 ] allow UTF-8 in style sheets.
 
 * reStructuredText:
 

Modified: trunk/docutils/docutils/writers/html4css1/__init__.py
===================================================================
--- trunk/docutils/docutils/writers/html4css1/__init__.py	2009-09-07 08:51:25 UTC (rev 6117)
+++ trunk/docutils/docutils/writers/html4css1/__init__.py	2009-09-07 08:57:48 UTC (rev 6118)
@@ -261,7 +261,8 @@
                       for sheet in styles]
         if settings.embed_stylesheet:
             settings.record_dependencies.add(*styles)
-            self.stylesheet = [self.embedded_stylesheet % open(sheet).read()
+            self.stylesheet = [self.embedded_stylesheet %
+                               unicode(open(sheet).read(), 'utf-8')
                                for sheet in styles]
         else: # link to stylesheets
             self.stylesheet = [self.stylesheet_link % self.encode(stylesheet)

Modified: trunk/docutils/docutils/writers/latex2e/__init__.py
===================================================================
--- trunk/docutils/docutils/writers/latex2e/__init__.py	2009-09-07 08:51:25 UTC (rev 6117)
+++ trunk/docutils/docutils/writers/latex2e/__init__.py	2009-09-07 08:57:48 UTC (rev 6118)
@@ -214,7 +214,7 @@
         except IOError:
             file = open(os.path.join(os.path.dirname(__file__),
                                      self.document.settings.template), 'rb')
-        template = string.Template(file.read())
+        template = string.Template(unicode(file.read(), 'utf-8'))
         file.close()
         # fill template
         self.assemble_parts() # create dictionary of parts
@@ -982,7 +982,8 @@
                 else:
                     wrapper = '%% embedded stylesheet: %s\n%s'
                 settings.record_dependencies.add(sheet)
-                self.stylesheet.append(wrapper % (sheet, open(sheet).read()))
+                self.stylesheet.append(wrapper %
+                            (sheet, unicode(open(sheet).read(), 'utf-8')))
             else: # link to style sheet
                 if is_package:
                     self.stylesheet.append(r'\usepackage{%s}' % base)

[Docutils-checkins] r6117 - in trunk/docutils: HISTORY.txt docutils/writers/html4css1/__init__.py test/functional/expected/standalone_rst_html4css1.html test/functional/expected/standalone_rst_s5_html_1.html test/functional/expected/standalone_rst_s5_html_2.html test/test_writers/test_html4css1_parts.py

[<milde@us...>]

Author: milde
Date: 2009-09-07 10:51:25 +0200 (Mon, 07 Sep 2009)
New Revision: 6117

Modified:
   trunk/docutils/HISTORY.txt
   trunk/docutils/docutils/writers/html4css1/__init__.py
   trunk/docutils/test/functional/expected/standalone_rst_html4css1.html
   trunk/docutils/test/functional/expected/standalone_rst_s5_html_1.html
   trunk/docutils/test/functional/expected/standalone_rst_s5_html_2.html
   trunk/docutils/test/test_writers/test_html4css1_parts.py
Log:
Address [ 1938891 ] Inline literal text creates "pre" span only when needed 
to prevent inter-word line wraps.

Modified: trunk/docutils/HISTORY.txt
===================================================================
--- trunk/docutils/HISTORY.txt	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/HISTORY.txt	2009-09-07 08:51:25 UTC (rev 6117)
@@ -81,6 +81,8 @@
 
   - ``--stylesheet`` and ``--stylesheet-path`` options support a comma
     separated list of stylesheets.
+  - Address [ 1938891 ] Inline literal text creates "pre" span only when
+    needed to prevent inter-word line wraps.
 
 * docutils/writers/newlatex2e/__init__.py:
 

Modified: trunk/docutils/docutils/writers/html4css1/__init__.py
===================================================================
--- trunk/docutils/docutils/writers/html4css1/__init__.py	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/docutils/writers/html4css1/__init__.py	2009-09-07 08:51:25 UTC (rev 6117)
@@ -234,6 +234,7 @@
     stylesheet_link = '<link rel="stylesheet" href="%s" type="text/css" />\n'
     embedded_stylesheet = '<style type="text/css">\n\n%s\n</style>\n'
     words_and_spaces = re.compile(r'\S+| +|\n')
+    sollbruchstelle = re.compile(r'.+\W\W.+|[-?].+', re.U) # wrap point inside word
 
     def __init__(self, document):
         nodes.NodeVisitor.__init__(self, document)
@@ -1054,9 +1055,13 @@
         text = node.astext()
         for token in self.words_and_spaces.findall(text):
             if token.strip():
-                # Protect text like "--an-option" from bad line wrapping:
-                self.body.append('<span class="pre">%s</span>'
-                                 % self.encode(token))
+                # Protect text like "--an-option" and the regular expression
+                # ``[+]?(\d+(\.\d*)?|\.\d+)`` from bad line wrapping
+                if self.sollbruchstelle.search(token):
+                    self.body.append('<span class="pre">%s</span>'
+                                     % self.encode(token))
+                else:
+                    self.body.append(self.encode(token))
             elif token in ('\n', ' '):
                 # Allow breaks at whitespace:
                 self.body.append(token)

Modified: trunk/docutils/test/functional/expected/standalone_rst_html4css1.html
===================================================================
--- trunk/docutils/test/functional/expected/standalone_rst_html4css1.html	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/test/functional/expected/standalone_rst_html4css1.html	2009-09-07 08:51:25 UTC (rev 6117)
@@ -163,7 +163,7 @@
 <div class="section" id="inline-markup">
 <h3><a class="toc-backref" href="#id38">2.1.1&nbsp;&nbsp;&nbsp;Inline Markup</a></h3>
 <p>Paragraphs contain text and may contain inline markup: <em>emphasis</em>,
-<strong>strong emphasis</strong>, <tt class="docutils literal"><span class="pre">inline</span> <span class="pre">literals</span></tt>, standalone hyperlinks
+<strong>strong emphasis</strong>, <tt class="docutils literal">inline literals</tt>, standalone hyperlinks
 (<a class="reference external" href="http://www.python.org">http://www.python.org</a>;), external hyperlinks (<a class="reference external" href="http://www.python.org/">Python</a>; <a class="footnote-reference" href="#id25" id="id26">[5]</a>), internal
 cross-references (<a class="reference internal" href="#example">example</a>), external hyperlinks with embedded URIs
 (<a class="reference external" href="http://www.python.org">Python web site</a>), <a class="reference external" href="http://www.python.org/">anonymous hyperlink
@@ -172,21 +172,21 @@
 <a class="footnote-reference" href="#label" id="id3">[2]</a>, or symbolic <a class="footnote-reference" href="#id13" id="id4">[*]</a>), citation references (<a class="citation-reference" href="#cit2002" id="id5">[CIT2002]</a>),
 substitution references (<img alt="EXAMPLE" src="../../../docs/user/rst/images/biohazard.png" />), and <span class="target" id="inline-hyperlink-targets">inline hyperlink targets</span>
 (see <a class="reference internal" href="#targets">Targets</a> below for a reference back to here).  Character-level
-inline markup is also possible (although exceedingly ugly!) in <em>re</em><tt class="docutils literal"><span class="pre">Structured</span></tt><em>Text</em>.  Problems are indicated by <a href="#id23"><span class="problematic" id="id24">|problematic|</span></a> text
+inline markup is also possible (although exceedingly ugly!) in <em>re</em><tt class="docutils literal">Structured</tt><em>Text</em>.  Problems are indicated by <a href="#id23"><span class="problematic" id="id24">|problematic|</span></a> text
 (generated by processing errors; this one is intentional).  Here is a
 reference to the <a class="reference internal" href="#doctitle">doctitle</a> and the <a class="reference internal" href="#subtitle">subtitle</a>.</p>
 <p>The default role for interpreted text is <cite>Title Reference</cite>.  Here are
 some explicit interpreted text roles: a PEP reference (<a class="reference external" href="http://www.python.org/dev/peps/pep-0287">PEP 287</a>); an
 RFC reference (<a class="reference external" href="http://www.faqs.org/rfcs/rfc2822.html">RFC 2822</a>); a <sub>subscript</sub>; a <sup>superscript</sup>;
 and explicit roles for <em>standard</em> <strong>inline</strong>
-<tt class="docutils literal"><span class="pre">markup</span></tt>.</p>
+<tt class="docutils literal">markup</tt>.</p>
 <!-- DO NOT RE-WRAP THE FOLLOWING PARAGRAPH! -->
 <p>Let's test wrapping and whitespace significance in inline literals:
-<tt class="docutils literal"><span class="pre">This</span> <span class="pre">is</span> <span class="pre">an</span> <span class="pre">example</span> <span class="pre">of</span> <span class="pre">--inline-literal</span> <span class="pre">--text,</span> <span class="pre">--including</span> <span class="pre">some--</span>
+<tt class="docutils literal">This is an example of <span class="pre">--inline-literal</span> <span class="pre">--text,</span> <span class="pre">--including</span> <span class="pre">some--</span>
 <span class="pre">strangely--hyphenated-words.</span>&nbsp; <span class="pre">Adjust-the-width-of-your-browser-window</span>
-<span class="pre">to</span> <span class="pre">see</span> <span class="pre">how</span> <span class="pre">the</span> <span class="pre">text</span> <span class="pre">is</span> <span class="pre">wrapped.</span>&nbsp; <span class="pre">--</span> <span class="pre">----</span> <span class="pre">--------</span>&nbsp; <span class="pre">Now</span> <span class="pre">note</span>&nbsp;&nbsp;&nbsp; <span class="pre">the</span>
-<span class="pre">spacing</span>&nbsp;&nbsp;&nbsp; <span class="pre">between</span> <span class="pre">the</span>&nbsp;&nbsp;&nbsp; <span class="pre">words</span> <span class="pre">of</span>&nbsp;&nbsp;&nbsp; <span class="pre">this</span> <span class="pre">sentence</span>&nbsp;&nbsp;&nbsp; <span class="pre">(words</span>
-<span class="pre">should</span>&nbsp;&nbsp;&nbsp; <span class="pre">be</span> <span class="pre">grouped</span>&nbsp;&nbsp;&nbsp; <span class="pre">in</span> <span class="pre">pairs).</span></tt></p>
+to see how the text is wrapped.&nbsp; <span class="pre">--</span> <span class="pre">----</span> <span class="pre">--------</span>&nbsp; Now note&nbsp;&nbsp;&nbsp; the
+spacing&nbsp;&nbsp;&nbsp; between the&nbsp;&nbsp;&nbsp; words of&nbsp;&nbsp;&nbsp; this sentence&nbsp;&nbsp;&nbsp; (words
+should&nbsp;&nbsp;&nbsp; be grouped&nbsp;&nbsp;&nbsp; in pairs).</tt></p>
 <p>If the <tt class="docutils literal"><span class="pre">--pep-references</span></tt> option was supplied, there should be a
 live link to PEP 258 here.</p>
 </div>
@@ -798,8 +798,8 @@
 This is a parsed literal block.
     This line is indented.  The next line is blank.
 
-Inline markup is supported, e.g. <em>emphasis</em>, <strong>strong</strong>, <tt class="docutils literal"><span class="pre">literal</span>
-<span class="pre">text</span></tt>, footnotes <a class="footnote-reference" href="#id8" id="id22">[1]</a>, <span class="target" id="hyperlink-targets">hyperlink targets</span>, and <a class="reference external" href="http://www.python.org/">references</a>;.
+Inline markup is supported, e.g. <em>emphasis</em>, <strong>strong</strong>, <tt class="docutils literal">literal
+text</tt>, footnotes <a class="footnote-reference" href="#id8" id="id22">[1]</a>, <span class="target" id="hyperlink-targets">hyperlink targets</span>, and <a class="reference external" href="http://www.python.org/">references</a>;.
 </pre>
 </div>
 </div>
@@ -990,7 +990,7 @@
 <h2><a class="toc-backref" href="#id70">2.23&nbsp;&nbsp;&nbsp;Custom Roles</a></h2>
 <ul>
 <li><p class="first">A role based on an existing role.</p>
-<p><tt class="custom docutils literal"><span class="pre">one</span></tt> <tt class="custom docutils literal"><span class="pre">two</span></tt> <tt class="custom docutils literal"><span class="pre">three</span></tt></p>
+<p><tt class="custom docutils literal">one</tt> <tt class="custom docutils literal">two</tt> <tt class="custom docutils literal">three</tt></p>
 </li>
 <li><p class="first">A new role.</p>
 <p><span class="customnew">one two three</span></p>

Modified: trunk/docutils/test/functional/expected/standalone_rst_s5_html_1.html
===================================================================
--- trunk/docutils/test/functional/expected/standalone_rst_s5_html_1.html	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/test/functional/expected/standalone_rst_s5_html_1.html	2009-09-07 08:51:25 UTC (rev 6117)
@@ -103,7 +103,7 @@
 </li>
 <li>Themes<p class="handout">A variety of themes are available.</p>
 </li>
-<li><tt class="docutils literal"><span class="pre">rst2s5.py</span></tt><p class="handout">The front-end tool to generate S5 slide shows.</p>
+<li><tt class="docutils literal">rst2s5.py</tt><p class="handout">The front-end tool to generate S5 slide shows.</p>
 </li>
 </ul>
 </div>
@@ -139,7 +139,7 @@
 <h1>Checklist</h1>
 <ul class="simple">
 <li>The document title should be duplicated on each slide in the footer
-(except for the first slide, <tt class="docutils literal"><span class="pre">slide0</span></tt>, where the entire footer is
+(except for the first slide, <tt class="docutils literal">slide0</tt>, where the entire footer is
 disabled).</li>
 <li>The footer also contains a second line, &quot;Location • Date&quot;</li>
 <li>There's no table of contents on the first slide, although it does

Modified: trunk/docutils/test/functional/expected/standalone_rst_s5_html_2.html
===================================================================
--- trunk/docutils/test/functional/expected/standalone_rst_s5_html_2.html	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/test/functional/expected/standalone_rst_s5_html_2.html	2009-09-07 08:51:25 UTC (rev 6117)
@@ -99,7 +99,7 @@
 </li>
 <li>Themes<p class="handout">A variety of themes are available.</p>
 </li>
-<li><tt class="docutils literal"><span class="pre">rst2s5.py</span></tt><p class="handout">The front-end tool to generate S5 slide shows.</p>
+<li><tt class="docutils literal">rst2s5.py</tt><p class="handout">The front-end tool to generate S5 slide shows.</p>
 </li>
 </ul>
 </div>
@@ -135,7 +135,7 @@
 <h1>Checklist</h1>
 <ul class="simple">
 <li>The document title should be duplicated on each slide in the footer
-(except for the first slide, <tt class="docutils literal"><span class="pre">slide0</span></tt>, where the entire footer is
+(except for the first slide, <tt class="docutils literal">slide0</tt>, where the entire footer is
 disabled).</li>
 <li>The footer also contains a second line, &quot;Location • Date&quot;</li>
 <li>There's no table of contents on the first slide, although it does

Modified: trunk/docutils/test/test_writers/test_html4css1_parts.py
===================================================================
--- trunk/docutils/test/test_writers/test_html4css1_parts.py	2009-09-03 15:36:30 UTC (rev 6116)
+++ trunk/docutils/test/test_writers/test_html4css1_parts.py	2009-09-07 08:51:25 UTC (rev 6117)
@@ -49,9 +49,9 @@
 Simple String with an even simpler ``inline literal``
 """,
 """\
-{'fragment': '''<p>Simple String with an even simpler <tt class="docutils literal"><span class="pre">inline</span> <span class="pre">literal</span></tt></p>\\n''',
+{'fragment': '''<p>Simple String with an even simpler <tt class="docutils literal">inline literal</tt></p>\\n''',
  'html_body': '''<div class="document">
-<p>Simple String with an even simpler <tt class="docutils literal"><span class="pre">inline</span> <span class="pre">literal</span></tt></p>
+<p>Simple String with an even simpler <tt class="docutils literal">inline literal</tt></p>
 </div>\\n''',
  'html_head': '''...<title></title>\\n'''}
 """],
@@ -211,9 +211,9 @@
 Simple String with an even simpler ``inline literal``
 """,
 """\
-{'fragment': '''<p>Simple String with an even simpler <tt class="docutils literal"><span class="pre">inline</span> <span class="pre">literal</span></tt></p>\\n''',
+{'fragment': '''<p>Simple String with an even simpler <tt class="docutils literal">inline literal</tt></p>\\n''',
  'html_body': '''<div class="document">
-<p>Simple String with an even simpler <tt class="docutils literal"><span class="pre">inline</span> <span class="pre">literal</span></tt></p>
+<p>Simple String with an even simpler <tt class="docutils literal">inline literal</tt></p>
 </div>\\n''',
  'html_head': '''...<title></title>\\n'''}
 """],