rest2web-develop — A discussion list for rest2web development.

[Rest2web-develop] rest2web Presentation

[<fuz...@vo...>]

{ran_emo} Mike Muller of the `Python Acadamy <http://www.python-academy.de/>`_ has posted the note from his `rest2web <http://www.voidspace.org.uk/python/rest2web/>`_ presentation :

    `rest2web - dynamische Erstellung statischer Webseiten <http://www.python-academy.de/User-Group/vortraege/rest2web_vortrag/index.html>`_

If you would like a general introduction to **rest2web**, and you speak German, this could be very useful. {sm;:-)}

He also has some good ideas about turning **rest2web** into a more useful tool for making HTML presentations.

The Python Academy also sounds like an interesting place. Mike recently did an interview on the `Python 411 Podcast Series <http://www.awaretek.com/python/index.html>`_ that I haven't yet had a chance to listen to.

[Rest2web-develop] Re: rest2web

[Michael F. <fuz...@vo...>]

Robert Brewer wrote:
> Fuzzyman,
>
> Sorry it's taken me a while to get back, I got swamped for a bit
> and felt I should take a peek before replying.
>   

No problem. I'd be happier having this conversation on the 
rest2web-develop mailing list (you can sign up via sourceforge). It's 
quite low traffic, but then the discussion gets archived and other users 
can chip in if they want to.
> I am definitely willing to discuss how to get rest2web a little
> more toward my bent... :)  I got your second email on the subject
> about the uservalues.  I think that is on the right track, though I'd
> hope to go even further as I'll explain below.
>
> At the time I put the first cut of my website up I found that a couple
> tweaks to docutils and a makefile were enough to get me started.
> Obviously rest2web has plenty more features that I'd
> rather not even try to duplicate.
>
> Just for grins, I'll lay out a few things I would like to see in the
> "ultimate reST to website tool".  I'm not sure how these may fit
> with your vision of rest2web, but it can give us some start for discussion. :)
>
> 1. Super simple on the low end.  It'd be great to just put reST files
> in a directory hierarchy, press go, and have a website created
> with nice breadcrumbs, navbars, and all that stuff.  The tool
> would have defaults in this case, such as making the breadcrumbs
> follow directory names and page names or something like that.
>   
I've done a first cut of this - see rest2web-0.5.0alpha :

    http://www.voidspace.org.uk/rest2web-0.5.0alpha.zip

There are rough notes on some of the changes at :

    http://www.voidspace.org.uk/python/weblog/arch_d7_2006_04_22.shtml#e309

The thing to do is just give it a source directory of rest files and run 
``r2w.py --force`` at the command line. :-)

Please give me feedback. Currently you can't specify template files, but 
can modify the default ones supplied in the rest2web directory.

I'd like feedback on :

    Page crumbs
    Index crumbs
    Index titles
   

There should be a simple test directory included in the new zipfile 
which illustrates  the source and output...

> This may or may not work well in practice, but maybe it could even notice a
> directory full of .jpg files and create a simple gallery for that
> directory.  
That's harder but not impossible.

Again I'd need to supply a default template for the gallery pages *and* 
the gallery index page, and it would need to copy all the jpgs into the 
target directory.

The part that makes it fiddly is that the user might want control over 
any individual step in that process (supply an index page source file, 
but no page template - or the other way round) so it gets fiddly to 
allow full customisation.

It also means proper communication with the gallery plugin. At the 
moment the plugin system is very rough as the gallery is basically the 
only plugin.

I'm happy to get there (would be even happier if someone else was 
working on it to), but it may take a while to get right...

> In another case, if there is no index.txt or index.html
> in a directory, it may take it upon itself to create an index.txt (in reST,
>   
I'd rather just generate default HTML than generate a rest file as an 
intermediate step. Effectively all index pages will use the same source 
file, as there are standard functions for generating links to every page 
in a directory.

> subsequently translated to HTML) with a bulleted
> list of links to all files and subdirectories in that directory.  The user
> may specify some CSS and reST/HTML header/footer files that
> have global applicability for the generated website.
>   
Ok, something like this needs to be worked in. You're effectively 
describing a template page though - one that looks like :

<html>
<head><title><% title %></title>
... CSS links etc ...
</head>
<body>
<% body %>
</body>
</html>

You can now pass in a template file at the command line, or in force 
mode use the default one.

I don't see any need to split this up into header/footer chunks etc.

> 2. Customizable for the more advanced stuff.  The user can
> make config files to tweak these defaults, such as making
> a more user-friendly breadcrumb name than the directory name,
> or a page description for the index pages, etc.  Basically the
> tweaks you have in the restindex.
>
>   
Ok - we can work on this, maybe configure the defaults. Or are you 
asking for an external config file to specify crumb names for individual 
pages ?

> Number 1 basically says that I don't have to maintain any metadata
> if I don't want to.  For most of my website that would be fine.
Well - we've got something that behaves like that now.

I think it will take some work to get the auto-generated index pages 
looking good.

>   In my
> website directory I could create some symlinks to the documentation
> directories of the projects I want to publish.  
Interesting - do you know if ``os.path.isdir(pathname)`` returns 
``True`` for a symbolic link ? I wonder if I'd need to include any code 
to handle it ?

It could be a useful feature... I'll only add this if you promise to use 
it though. ;-)

> When I press go their latest
> reST files get converted and added into the main site with a seamless navbar
> and breadcrumb system.  I could easily add some pictures by adding
> symlinks to some of my photo directories.  This lets me add, rearrange,
> and remove stuff from the website with almost no effort. 
As mentioned before - certainly possible, details to sort.

Lets see where this takes us.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>  Just a few
> filesystem commands and then I press go to regenerate.
> As an example, the "projects" page on my site
> http://brewer123.home.comcast.net/projects/
> could be autogenerated as a bulleted list of links
> to each subdirectory.  Sure, it wouldn't have the text about
> "Here are some projects I've done and am releasing publicly"
> and it wouldn't have the little descriptions of some of the projects.
> But I might not even care about those things if it was autogenerated
> with "zero effort".  :)
>
> Some front pages of the website I may want to look a little nicer
> and be a little friendlier, so I use the config files to create some
> better metadata.  And if I have time and inclination, I can do that
> for the projects with the symlinks too, but I don't have to.
>
> So that's why I wasn't liking the having the restindex be part of
> the file, or even required at all for generating a page.  It's because
> I want to just plop a directory of reST files in my website directory and have
> it become part of my website.
>
> Well, this got a little longer and possibly more confusing than
> I expected.  Ask away if I'm not being clear.  I guess my personal
> desire is for a way to turn a directory hierarchy into a useful
> website with as little effort as possible.  Even with a hodge-podge
> of file types in the hierarchy.  And once the "zero-effort" website
> is created, it should be possible to spruce it up a bit by manually
> providing metadata to the tool and regenerating.
>
> By the way, the new look to your rest2web pages is very nice.
> I like it a lot.
>
> Thanks for opening the discussion, I'm looking forward to your
> thoughts...
>
> -Rob
>
> On 3/31/06, Michael Foord <fuz...@vo...> wrote:
>   
>> Hello Rob,
>>
>> I saw on your website that you considered rest2web, but couldn't use it
>> because of the restindex metadata stored in each page.
>>
>> I wondered if you wanted to explore a way round that with me ?
>>
>> If you are still interested we can bandy around a few ideas to come up
>> with something that works for you.
>>
>> Fuzzyman
>> (Michael Foord)
>> http://www.voidspace.org.uk/python/index.shtml
>>
>>     
>
>
> --
> Robert W. Brewer
>
>   

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[Michael F. <fuz...@vo...>]

Gaël Varoquaux wrote:
> On Mon, Apr 10, 2006 at 12:03:26AM +0100, Michael Foord wrote:
>   
>>>    I'll write a function to copy file from the source tree to the
>>> target tree that can be included in functions.py . I'll just wait until
>>> tomorrow in order to have a clearer idea of what options/features might
>>> be wanted for such function.
>>>       
>
>   
>>>    Suggestions welcomed.
>>>       
>
>   
>> Ok. The file keyword isn't actually one I use (currently), I guess being 
>> able to specify a list of source files and a single target directory 
>> (relative to the current target directory I guess).
>>     
>
>     OK, now that I am using Processor.target in my embedded scripts I
> can do what I want. But it would be nicer to have a function for do
> this. Something like :
>
> def add_file(source, targetdir=os.path.dirname(Processor.target+"/"+pagepath)
>   
I *haven't* yet added the target path to the standard namespace. I will 
do this very soon.

I have however made the namespace (and the 'uservalues') accessible to 
the functions (both as dictionaries).

>     Where source could be either a string or a tuple of strings.
>
>     Trying to implement this I realize I do not know how to access
> pagepath or Processor in functions.py . Maybe it would be nice if the
> namespace accessible to the embedded functions was also accessible in
> functions.py (note that I have no clue how to code this).
>
>     On a side note: it might be a good idea to have a config entry in
> the config file to add a user file where he could define his functions
> without modifying functions.py . For large project it could prove
> usefull.
>   
Hmm... I'll think about the best way of doing this. Currently you can 
import whatever you want at the start of the templates, but maybe a way 
to specify a list of modules to automatically import would be good too.

Fuzzyman
http://www.vidspace.org.uk/python/index.shtml
> 	Regards,
>
>     Gaël
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

Re: [Rest2web-develop] Nailed the File Keyword Bug

[Michael F. <fuz...@vo...>]

Andrew Ittner wrote:
> The verbosity option before the config file doesn't work:
>
> $ python ~/code/dev/rest2web/r2w.py -a ~/www/rp.ini
> Traceback (most recent call last):
>   File "/home/user/code/dev/rest2web/r2w.py", line 167, in ?
>     config = ConfigObj(cfg_file, file_error=True)
>   File "/home/user/code/dev/rest2web/pythonutils/configobj.py", line 1081,
> in __init__
>     raise IOError, 'Config file not found: "%s".' % self.filename
> IOError: Config file not found: "-a".
>
> Same result for -w and -v.
>   
This should be fixed now in the 0.5.0 alpha release. Probably other bugs 
though. :-(

Unfortunately rest2web is structured in a way that makes it very hard to 
add tests. Thinking about testing from the start would probably have 
resulted in a better structured system ! Some of the recent additions to 
rest2web are good though, and more to come.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Andrew
>
>   
>> -----Original Message-----
>> From: res...@li...
>> [mailto:res...@li...]On Behalf Of
>> Michael Foord
>> Sent: Monday, April 10, 2006 1:08 AM
>> To: res...@li...
>> Subject: Re: [Rest2web-develop] Nailed the File Keyword Bug
>>
>>
>> Andrew Ittner wrote:
>>     
>>> File keyword bug: fixed on Linux!  Thank you - that feature is
>>>       
>> a big help.
>>     
>>> However, given enough time, I'm sure I'll break it - but in a
>>>       
>> new and better
>>     
>>> way!
>>>
>>>       
>> Great (except the part about breaking it ;-)
>>     
>>> As for the verbosity settings, they appear to give the exact
>>>       
>> same output at
>>     
>>> the command line in Linux.  I see no difference between -v, -a,
>>>       
>> and -w.  I
>>     
>>> run r2w thusly:
>>>
>>> python ~/code/dev/rest2web/r2w.py ~/www/rp.ini -w
>>>
>>> Anything obviously wrong with that syntax?
>>>
>>>
>>>       
>> Try :
>>
>> python ~/code/dev/rest2web/r2w.py -w ~/www/rp.ini
>>     
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

[Rest2web-develop] Test Version of rest2web 0.5.0 alpha

[<fuz...@vo...>]

{ran_emo} There is now a new test version of `rest2web <http://www.voidspace.org.uk/python/rest2web/>`_ available :

   `rest2web 0.5.0 alpha <http://www.voidspace.org.uk/rest2web-0.5.0alpha.zip>`_

This has lots of new features available, but because of internet problems I haven't been able to update the subversion repository yet. There is still a lot of work to do (including documenting the features), but some of the new ideas need further development and I'm sure there will be bugs to fix.

Most of these features have been added in response to user requests. The main change is that **rest2web** can now build sites *without* needing restindexes, templates or index files. This allows you to generate a site just from a bunch of `ReST <http://docutils.sourceforge.net>`_ files. {sm;:!:}

The new features already added (since 0.4.0 alpha) include :

Can build a site with no index pages, no template and no restindexes.
(The ``force`` option.)

Allow passing of global uservalues at the command line and in the config file.
(Command line uservalues override config file ones.)

A ``--template-file=`` command line option. This overrides the top level ``template`` specified in the restindex and allows you to have different templates for the same site, e.g. for building different online versions and documentation versions.

The ``final_encoding`` should never be ``utf8`` (not recognised by browsers) - should be ``utf-8`` instead. (Now automatically handled.)

Added and documented ``initialheaderlevel``.

``file`` keyword - should only copy if the file has changed (checks timestamp and size, copies timestamp with file).

File keyword was broken if used outside an index file.

Gallery chokes on ``thumb.db`` and animated jpgs. (Now skips all non image files
and any image files it can't handle.)

Deleted ``urlpath`` from rest2web because it is now in pythonutils.

Implement levels of verbosity. These now work. {sm;:oops:}

A global set of uservalues from the config file. Overrideable in individual
pages. (Including ``__encoding__`` special value.)

Fix rendering of uservalues from ReST to HTML. Uservalues in ReST format should now use ``<* ... *>`` and ``<$ ... $>`` in pages instead of ``<% ... %>`` and ``<# ... #>``.

Add uservalues to the namespace.

Added ``modtimeiso``.

Added ``formattime`` standard function.

Made the namespace/uservalues available in the macros.

Made namespace/uservalues values available to the functions. This isn't yet used but will allow for some more interesting standard functions.

Removed the two ``<br />`` from ``listend`` in the standard function ``minibar``.

Added ``wrapper_class`` to ``print_details``.

Added ``os`` and ``sys`` to namespace.

The default crumb for index pages (if no ``page-title`` specified) is the filename,
minus the extension, turned to title case.

--------------

There are still various things to add before 0.5.0 becomes final, but **rest2web** is shaping up well.

You can see which command line options have been added by typing ``r2w.py -h`` at the command line.

[Rest2web-develop] problem with rest2web 0.4.0-alpha

[Stewart M. <mi...@ya...>]

the latest alpha version of rest2web appears to have broken documents
that previously were parsed correctly.  Specifically, macros.txt and
index.txt in Firedrop2

In macros.txt, the following bullet point causes rest2web to fail
with  the following error about a bulleted point not having a blank
line after it (the exceptions also make reference to apparently at
least two different installations of MovPy, but I'm running this
under Linux so you can be certain I don't have a g:\ drive :-)

Here's the guilty bullet point:

* If a macro name isn't found, it is left alone. 
  In fact, this piece of documentation exists as it is because of
this rule. Text like ``{knarf}`` would be replaced… if it were a
macro. 
  Since we didn't define a macro by that name, it is left alone and
shows up in text with curly braces and all. 

And here is the full text of the exception (sorry about the length):

macros.txt:93: (WARNING/2) Bullet list ends without a blank line;
unexpected unindent.
Traceback (most recent call last):
[err]   File "rest2web.py", line 168, in ?
[err]     count = main(config)
[err]   File "rest2web.py", line 104, in main
[err]     return processor.walk()
[err]   File
"/home/stewart/data/dev/python/firedrop2/firedrop2/rest2web/restprocessor.py",
line 254, in walk
[err]     subdir=subdir)
[err]   File
"/home/stewart/data/dev/python/firedrop2/firedrop2/rest2web/restprocessor.py",
line 672, in process
[err]     entry = html_parts(
[err]   File
"/home/stewart/data/dev/python/firedrop2/firedrop2/rest2web/restutils.py",
line 181, in html_parts
[err]     writer_name='html', settings_overrides=overrides)
[err]   File "G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\core.py",
line 431, in publish_parts
[err]   File "G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\core.py",
line 612, in publish_programmatically
[err]   File "G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\core.py",
line 204, in publish
[err]   File
"G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\readers\__init__.py",
line 71, in read
[err]   File
"G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\readers\__init__.py",
line 77, in parse
[err]   File
"/usr/lib/python2.4/site-packages/docutils/parsers/rst/__init__.py",
line 155, in parse
[err]     self.statemachine.run(inputlines, document,
inliner=self.inliner)
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 171, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 233, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 422, in check_line
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 2843, in text
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 309, in section
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 377, in new_subsection
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 267, in nested_parse
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 196, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 233, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 422, in check_line
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 2843, in text
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 309, in section
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 377, in new_subsection
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 267, in nested_parse
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 196, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 233, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 422, in check_line
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 2618, in underline
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 309, in section
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 377, in new_subsection
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 267, in nested_parse
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 196, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 233, in run
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/statemachine.py",
line 422, in check_line
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 1169, in bullet
[err]   File
"/Volumes/USB-1GB/movpy-1.0.1-py2.4.2/movpy/lib/firedrop2/../docutils/parsers/rst/states.py",
line 414, in unindent_warning
[err]   File "G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\utils.py",
line 202, in warning
[err]   File "G:\MOVPY-~2.2\MOVPY-~1.2\movpy\lib\docutils\utils.py",
line 175, in system_message
[err] SystemMessage: macros.txt:93: (WARNING/2) Bullet list ends
without a blank line; unexpected unindent.


After I remove the offending bullet point from macros.txt, all of the
files are processed correctly up to index.txt, which causes an
exception:

Processing indexfile.
Guessing encoding. We guessed: ISO8859-1
/usr/lib/python2.4/site-packages/pythonutils/odict.py:456:
DeprecationWarning: use of sequence attribute is deprecated. Use keys
method instead.
[err]   'instead.', DeprecationWarning)
Traceback (most recent call last):
[err]   File "rest2web.py", line 168, in ?
[err]     count = main(config)
[err]   File "rest2web.py", line 104, in main
[err]     return processor.walk()
[err]   File
"/home/stewart/data/dev/python/firedrop2/firedrop2/rest2web/restprocessor.py",
line 263, in walk
[err]     self.buildsection()
[err]   File
"/home/stewart/data/dev/python/firedrop2/firedrop2/rest2web/restprocessor.py",
line 1094, in buildsection
[err]     namespace['sections'].sequence = (namespace['sectionlist']
or
[err]   File "G:\MOVPY-~2.2\movpy\lib\pythonutils\odict.py", line
458, in __setattr__
[err]   File "G:\MOVPY-~2.2\movpy\lib\pythonutils\odict.py", line
769, in setkeys
[err] KeyError: 'Keylist is not the same as current keylist.'
[err]

Can you shed any light on this? 

I'll post all of updated Firedrop documentation .txt files to the svn
site where you can download them and inspect them at your leisure.
I'm also attaching the broken macros.txt. [err, no I'm not, because
Yahoo doesn't properly support Firefox!]

cheers
Stewart
 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[Rest2web-develop] Offline

[Michael F. <fuz...@vo...>]

Hello all,

Just to let you know that I move house tomorrow. That means I'll be 
offline until Thursday - that's if NTL are true to their word and 
connect us on Tuesday.

I will have email at work of course (new job starts Tuesday !), but 
won't be able to do much project work.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

RE: [Rest2web-develop] Nailed the File Keyword Bug

[Andrew I. <and...@us...>]

The verbosity option before the config file doesn't work:

$ python ~/code/dev/rest2web/r2w.py -a ~/www/rp.ini
Traceback (most recent call last):
  File "/home/user/code/dev/rest2web/r2w.py", line 167, in ?
    config = ConfigObj(cfg_file, file_error=True)
  File "/home/user/code/dev/rest2web/pythonutils/configobj.py", line 1081,
in __init__
    raise IOError, 'Config file not found: "%s".' % self.filename
IOError: Config file not found: "-a".

Same result for -w and -v.

Andrew

> -----Original Message-----
> From: res...@li...
> [mailto:res...@li...]On Behalf Of
> Michael Foord
> Sent: Monday, April 10, 2006 1:08 AM
> To: res...@li...
> Subject: Re: [Rest2web-develop] Nailed the File Keyword Bug
>
>
> Andrew Ittner wrote:
> > File keyword bug: fixed on Linux!  Thank you - that feature is
> a big help.
> > However, given enough time, I'm sure I'll break it - but in a
> new and better
> > way!
> >
>
> Great (except the part about breaking it ;-)
> > As for the verbosity settings, they appear to give the exact
> same output at
> > the command line in Linux.  I see no difference between -v, -a,
> and -w.  I
> > run r2w thusly:
> >
> > python ~/code/dev/rest2web/r2w.py ~/www/rp.ini -w
> >
> > Anything obviously wrong with that syntax?
> >
> >
>
> Try :
>
> python ~/code/dev/rest2web/r2w.py -w ~/www/rp.ini

[Rest2web-develop] rest2web Markup

[<fuz...@vo...>]

{ran_emo} In working on `rest2web <http://www.voidspace.org.uk/python/rest2web/>`_ I've realised that it would be *very* easy to support other markups.

The ones that spring to mind are :

* `markdown <http://daringfireball.net/projects/markdown/>`_
* `sextile <http://zephyrfalcon.org/labs/sextile_intro.html>`_
* `textile <http://textism.com/tools/textile/>`_
* `Wiki Markup <http://wiki.ittoolbox.com/index.php/ITtoolboxWiki:Wiki_markup>`_ [#]_

Would anyone find that useful ?

I've also discovered that the current system of `uservalues <http://www.voidspace.org.uk/python/rest2web/reference/uservalues.html>`_ is broken in lots of cases [#]_. To fix this I think I need to add a new syntax for including uservalues. I will probably go with :

    ``<* uservalue *>``

This allows uservalues to contain {acro;ReST;ReStructured Text} and still be turned into HTML.

.. [#] I `anticipated this <http://www.voidspace.org.uk/python/rest2web/reference/uservalues.html#id14>`_, but I didn't realise how serious it was. Now that I'm making *uservalues* more general I *have* to fix it.
.. [#] Although it seems that *wiki markup* is a nebulous concept, with every Wiki having its own variant. Anyone know of a *good* standalone Wiki-format-to-html module for Python ? Bonus points for not suggesting I extract the rendering engine from `MoinMoin <http://moinmoin.wikiwikiweb.de/>`_ but providing a link to the right module instead. {sm;:-p}

Re: [Rest2web-develop] Nailed the File Keyword Bug

[Michael F. <fuz...@vo...>]

Andrew Ittner wrote:
> File keyword bug: fixed on Linux!  Thank you - that feature is a big help.
> However, given enough time, I'm sure I'll break it - but in a new and better
> way!
>   

Great (except the part about breaking it ;-)
> As for the verbosity settings, they appear to give the exact same output at
> the command line in Linux.  I see no difference between -v, -a, and -w.  I
> run r2w thusly:
>
> python ~/code/dev/rest2web/r2w.py ~/www/rp.ini -w
>
> Anything obviously wrong with that syntax?
>
>   

Try :  

python ~/code/dev/rest2web/r2w.py -w ~/www/rp.ini



Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Andrew
>
>   
>> -----Original Message-----
>> From: res...@li...
>> [mailto:res...@li...]On Behalf Of
>> Michael Foord
>> Sent: Sunday, April 09, 2006 10:48 AM
>> To: res...@li...
>> Subject: [Rest2web-develop] Nailed the File Keyword Bug
>>
>>
>> Hello all,
>>
>> I think I've finally nailed the file keyword bug.
>>
>> The fix is now in subversion.
>>
>> I've tested this with normal pages, index pages, and an index page using
>> the 'index-file' keyword. For each of these I tested with a normal file
>> (in the same directory) and using an absolute file location.
>>
>> I would appreciate a test on Linux. I would also appreciate testing of
>> the new verbosity settings - I'm sure they work, so perhaps feedback is
>> a better word than testing. (Are the levels right - or would something
>> between ``-v`` and ``-a`` be useful ?)
>>     
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[V. <gae...@no...>]

On Mon, Apr 10, 2006 at 12:03:26AM +0100, Michael Foord wrote:
> >    I'll write a function to copy file from the source tree to the
> >target tree that can be included in functions.py . I'll just wait unti=
l
> >tomorrow in order to have a clearer idea of what options/features migh=
t
> >be wanted for such function.

> >    Suggestions welcomed.

> Ok. The file keyword isn't actually one I use (currently), I guess bein=
g=20
> able to specify a list of source files and a single target directory=20
> (relative to the current target directory I guess).

    OK, now that I am using Processor.target in my embedded scripts I
can do what I want. But it would be nicer to have a function for do
this. Something like :

def add_file(source, targetdir=3Dos.path.dirname(Processor.target+"/"+pag=
epath)

    Where source could be either a string or a tuple of strings.

    Trying to implement this I realize I do not know how to access
pagepath or Processor in functions.py . Maybe it would be nice if the
namespace accessible to the embedded functions was also accessible in
functions.py (note that I have no clue how to code this).

    On a side note: it might be a good idea to have a config entry in
the config file to add a user file where he could define his functions
without modifying functions.py . For large project it could prove
usefull.

	Regards,

    Ga=EBl

RE: [Rest2web-develop] Nailed the File Keyword Bug

[Andrew I. <and...@us...>]

File keyword bug: fixed on Linux!  Thank you - that feature is a big help.
However, given enough time, I'm sure I'll break it - but in a new and better
way!

As for the verbosity settings, they appear to give the exact same output at
the command line in Linux.  I see no difference between -v, -a, and -w.  I
run r2w thusly:

python ~/code/dev/rest2web/r2w.py ~/www/rp.ini -w

Anything obviously wrong with that syntax?

Andrew

> -----Original Message-----
> From: res...@li...
> [mailto:res...@li...]On Behalf Of
> Michael Foord
> Sent: Sunday, April 09, 2006 10:48 AM
> To: res...@li...
> Subject: [Rest2web-develop] Nailed the File Keyword Bug
>
>
> Hello all,
>
> I think I've finally nailed the file keyword bug.
>
> The fix is now in subversion.
>
> I've tested this with normal pages, index pages, and an index page using
> the 'index-file' keyword. For each of these I tested with a normal file
> (in the same directory) and using an absolute file location.
>
> I would appreciate a test on Linux. I would also appreciate testing of
> the new verbosity settings - I'm sure they work, so perhaps feedback is
> a better word than testing. (Are the levels right - or would something
> between ``-v`` and ``-a`` be useful ?)

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[Michael F. <fuz...@vo...>]

Gaël Varoquaux wrote:
> On Sun, Apr 09, 2006 at 11:51:25PM +0100, Michael Foord wrote:
>   
>>>    If you know a way of finding out the target directory or the root of
>>> the target tree from either functions.py or embedded code than I can
>>> code a appropriate function.
>>>       
>> You can find this using ``Processor.target``, which is kind of a hack - 
>> but why I included access to the Processor object.
>>     
>
>     Ha ! I should have investigator the processor object a bit more ( I
> had seen that it was in the the namespace)
>
>   
>> What values would you like (they are very easy to add into the namespace) ?
>>     
>
>     Well, you know as well as I do that it is hard to know in advance
> what a user might need. I think the root of the target tree would be
> enough. Do you confirm that os.getcwd will give me the root of the
> source tree or is this just a coincidence that it does on my tests.
>
>   
The current working directory is explicitly set at the start - so you 
can rely on this behaviour.

I'll probably add the target directory as an explicit value.

>     I'll write a function to copy file from the source tree to the
> target tree that can be included in functions.py . I'll just wait until
> tomorrow in order to have a clearer idea of what options/features might
> be wanted for such function.
>
>     Suggestions welcomed.
>   
Ok. The file keyword isn't actually one I use (currently), I guess being 
able to specify a list of source files and a single target directory 
(relative to the current target directory I guess).

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> 	Gaël
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[V. <gae...@no...>]

On Sun, Apr 09, 2006 at 11:51:25PM +0100, Michael Foord wrote:
> >    If you know a way of finding out the target directory or the root =
of
> >the target tree from either functions.py or embedded code than I can
> >code a appropriate function.

> You can find this using ``Processor.target``, which is kind of a hack -=
=20
> but why I included access to the Processor object.

    Ha ! I should have investigator the processor object a bit more ( I
had seen that it was in the the namespace)

> What values would you like (they are very easy to add into the namespac=
e) ?

    Well, you know as well as I do that it is hard to know in advance
what a user might need. I think the root of the target tree would be
enough. Do you confirm that os.getcwd will give me the root of the
source tree or is this just a coincidence that it does on my tests.

    I'll write a function to copy file from the source tree to the
target tree that can be included in functions.py . I'll just wait until
tomorrow in order to have a clearer idea of what options/features might
be wanted for such function.

    Suggestions welcomed.

	Ga=EBl

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[Michael F. <fuz...@vo...>]

Gaël Varoquaux wrote:
> On Sun, Apr 09, 2006 at 11:28:01PM +0100, Michael Foord wrote:
>   
>>>    You mean if the file keyword is "foo/bar" rest2web won't create a
>>> foo subdirectory in the target directory and copy bar there ? If so then
>>> I think it would be an interesting feature to add.
>>>       
>
>
>   
>> I *don't* do this so that you can copy relative files into the target 
>> directory using the ``file`` keyword.
>>     
>
>     Fare enough. You have a point.
>
>   
>> I think it would be easy enough to create a standard function that does 
>> this - so modifying the file keyword shouldn't be necessary.
>>     
>
>   
>> The full path to the page being generated is ``pagepath``.
>>     
>
>     Well unless I am wrong this is the relative path from the root of
> the web site. To be able to copy the file from the source tree to the
> target tree I also need to know their roots. The root of the source tree
> seems to be the current working dir of the script, that I can get with
> os.getcwd(), but I have not found a way of accessing the root of the
> target directory.
>
>   
>> You should find that ``os.path.dirname(pagepath)`` gives you the 
>> directory. Let me know how you get on with this.
>>     
>
>     I get the source directory with :
> fullpath=os.path.dirname(os.getcwd()+"/"+pagepath)
>
>     If you know a way of finding out the target directory or the root of
> the target tree from either functions.py or embedded code than I can
> code a appropriate function.
>
>   
Ahh.... good point, you need the target directory as well.

You can find this using ``Processor.target``, which is kind of a hack - 
but why I included access to the Processor object.

What values would you like (they are very easy to add into the namespace) ?

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>     Cheers,
>
> 	Gaël
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[V. <gae...@no...>]

On Sun, Apr 09, 2006 at 11:28:01PM +0100, Michael Foord wrote:
> >    You mean if the file keyword is "foo/bar" rest2web won't create a
> >foo subdirectory in the target directory and copy bar there ? If so th=
en
> >I think it would be an interesting feature to add.


> I *don't* do this so that you can copy relative files into the target=20
> directory using the ``file`` keyword.

    Fare enough. You have a point.

> I think it would be easy enough to create a standard function that does=
=20
> this - so modifying the file keyword shouldn't be necessary.

> The full path to the page being generated is ``pagepath``.

    Well unless I am wrong this is the relative path from the root of
the web site. To be able to copy the file from the source tree to the
target tree I also need to know their roots. The root of the source tree
seems to be the current working dir of the script, that I can get with
os.getcwd(), but I have not found a way of accessing the root of the
target directory.

> You should find that ``os.path.dirname(pagepath)`` gives you the=20
> directory. Let me know how you get on with this.

    I get the source directory with :
fullpath=3Dos.path.dirname(os.getcwd()+"/"+pagepath)

    If you know a way of finding out the target directory or the root of
the target tree from either functions.py or embedded code than I can
code a appropriate function.

    Cheers,

	Ga=EBl

Re: [Rest2web-develop] Re: Enhancement suggestion to rest2web

[Michael F. <fuz...@vo...>]

Gaël Varoquaux wrote:
> On Sun, Apr 09, 2006 at 10:42:26PM +0100, Michael Foord wrote:
>   
>>>    * A inlined function, such as <# print(2*2) #> should be able to add
>>>      files to the restindex['file'] list. That should probably be
>>>      rather easy to add.
>>>       
>
>
>   
>> There are several ways of doing this (including writing your own 
>> function and just importing it in the template).
>>     
>
>     I am reluctant to modify the rest2web files. But maybe I missed a
> way of doing this without modifying them. Currently I am only coding
> embedded code in the template.
>
>   
But you can import anything inside the templates - so keep it in an 
external file somewhere on ``sys.path`` (e.g. your ``site-packages`` 
folder) and import it.

If you wanted to make this easily available, the file to add it to is 
functions.py. If you write a general enough function I will add it to 
the distribution. Ah.. I can see below that you are aware of this file.
>> Where do you want to move files from and to ? Currently the 'file' 
>> keyword only moves files into the same directory as the target file 
>> being created. Is this what you want to do ?
>>     
>
>     You mean if the file keyword is "foo/bar" rest2web won't create a
> foo subdirectory in the target directory and copy bar there ? If so then
> I think it would be an interesting feature to add.
>
>   
I *don't* do this so that you can copy relative files into the target 
directory using the ``file`` keyword.

I think it would be easy enough to create a standard function that does 
this - so modifying the file keyword shouldn't be necessary.

>> I'm sure there is an easy function you could write to do this - I would 
>> be happy to include it in the set of standard functions if it was 
>> general enough.
>>     
> I have been trying to, but I cannot find a way of retrieving the
> target directory with embedded code. Are you suggesting that I modify
> function.py to add such a function and send you a patch ? I'll have a
> look at that.
>     If you can give me some advice on implementing that I am more then
> happy to do it.
>
>   
The full path to the page being generated is ``pagepath``.

You should find that ``os.path.dirname(pagepath)`` gives you the 
directory. Let me know how you get on with this.
>> I've been planning to add hashing for a while.
>>     
>> I am now busy implementing it - it jumped the queue !
>>     
>> It might still take a couple of weeks to get right, maybe less.
>>     
>     The more I think about it the more I think there is a certain amount
> of work. But I think it will be a very useful feature.
>
>     Thanks for such quick feedback (I joined the sourceforge mailing
> list and saw your mail).
>
>   
Well, I've started work on it. It might not be that easy because I need 
to pickle the data structures in use by rest2web. This means I need to 
check that there are no complicated cross references that won't be 
restored properly if I just copy data for individual pages from the pickle.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Rest2web-develop] Re: Enhancement suggestion to rest2web

[V. <Gae...@no...>]

On Sun, Apr 09, 2006 at 10:42:26PM +0100, Michael Foord wrote:
> >    * A inlined function, such as <# print(2*2) #> should be able to a=
dd
> >      files to the restindex['file'] list. That should probably be
> >      rather easy to add.


> There are several ways of doing this (including writing your own=20
> function and just importing it in the template).

    I am reluctant to modify the rest2web files. But maybe I missed a
way of doing this without modifying them. Currently I am only coding
embedded code in the template.

> Where do you want to move files from and to ? Currently the 'file'=20
> keyword only moves files into the same directory as the target file=20
> being created. Is this what you want to do ?

    You mean if the file keyword is "foo/bar" rest2web won't create a
foo subdirectory in the target directory and copy bar there ? If so then
I think it would be an interesting feature to add.

> I'm sure there is an easy function you could write to do this - I would=
=20
> be happy to include it in the set of standard functions if it was=20
> general enough.

    I have been trying to, but I cannot find a way of retrieving the
target directory with embedded code. Are you suggesting that I modify
function.py to add such a function and send you a patch ? I'll have a
look at that.
    If you can give me some advice on implementing that I am more then
happy to do it.

> I've been planning to add hashing for a while.

> I am now busy implementing it - it jumped the queue !

> It might still take a couple of weeks to get right, maybe less.

    The more I think about it the more I think there is a certain amount
of work. But I think it will be a very useful feature.

    Thanks for such quick feedback (I joined the sourceforge mailing
list and saw your mail).

--=20
    Ga=EBl

[Rest2web-develop] Page Caching

[Michael F. <fuz...@vo...>]

Hello all,

Thanks to an email from a new user, I've been inspired to implement 
'page-caching' a bit earlier than anticipated - it's jumped the queue. :-)

It's not trivial to implement, but not really that hard.

I've 'hashed out' a spec, see below. This pickles all page data as it 
builds and saves an MD5 hash for each page, as well as a hash of the 
template in use and the breadcrumbs.

When building a page, if all these three are identical, then it 
*doesn't* build the page (it is unchanged), and it can just copy data 
from the pickle into it's own data structures (which other pages may 
access).

If the page hasn't been built before, or has changed, then it saves the 
data into a pickle. This is saved out at the end.

I don't know how quickly I'll be able to implement this. It's not really 
that difficult. Testing will be a bugger of course.

The Hash Format

Pickle each page along with its breadcrumbs
Also pickle a hash of the contents and a hash for the template in use

If any page in a directory changes then the index file must be 
regenerated [#]_.

For each directory we must pickle the list of pages and the list of
sub-directories.

If this has changed then that directory is marked as changed (attribute 
on the
Processor - reset when a new directory is entered).

So enter a new page.
First check if it has a pickle (this maybe the first run through).
Hash the contents - and compare. If changed - generate the page and save the
hash.

If not changed need to compare the hash with the template in use - to see if
that has changed.

Do the same with the breadcrumbs.

If they all check out then instead of generating the page - the relevant
stuff can just be copied from the pickle.

If one of these *doesn't* check out, then the page must be generated - 
and the
correct data saved into the pickle.

If a page changes - then the directory marker needs to be set.
Index pages need to check this, and force regeneration if anything has 
changed.
Otherwise they follow the same procedure.

When processing a directory - if there are deleted entries, these need to be
removed from the pickle and the changed marker set.

---

The hash is kept as self.hash_data.
You can test if it is in use with if ``self.hash:`` (will be None) if 
hashing
is not in use.

The structure is a dictionary with an entry per directory. (This means 
that it
is not a tree.)

The keys are the values used for ``self.dir_as_url``.

Each directory has the following values :

index - an entry representing the index page
pages - a dictionary (keyed by page name) representing each page
dir_list - a list of the files in the directory [#]_

FIXME: How do we handle directories with ``__prune__`` ? Just skip them and
leave their pickle intact I guess.

.. [#] A later addition will allow you to specify whether a whole directory
       must be rebuilt if a file in a directory changes.
.. [#] This means that any extra files in the directory will cause the
       index page to be rebuilt.

[Rest2web-develop] Nailed the File Keyword Bug

[Michael F. <fuz...@vo...>]

Hello all,

I think I've finally nailed the file keyword bug.

The fix is now in subversion.

I've tested this with normal pages, index pages, and an index page using 
the 'index-file' keyword. For each of these I tested with a normal file 
(in the same directory) and using an absolute file location.

I would appreciate a test on Linux. I would also appreciate testing of 
the new verbosity settings - I'm sure they work, so perhaps feedback is 
a better word than testing. (Are the levels right - or would something 
between ``-v`` and ``-a`` be useful ?)

Testing with rest2web is a *pain*. I didn't design it with testing in 
mind, so there is no real way to leave regression tests in place. 
rest2web is also hard to maintain as there is some complicated logic in. 
Perhaps if I had built it from the ground up with tests in place the 
structure might be easier to maintain !

All this is just my excuse for not introducing tests now - but I realise 
this is a failing. I'm not about to refactor the whole program though 
I'm afraid.

rest2web currently does everything I need, *but* one of the improvements 
down the line will make rest2web better for building project 
documentation. This is a feature I *will* use. :-)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Rest2web-develop] rest2web Progress

[<fuz...@vo...>]

{ran_emo} There has been *some* tangible progress on `rest2web <http://www.voidspace.org.uk/python/rest2web/>`_ at last.

There is an updated version in the `SVN Repository <http://svn.rest2web.python-hosting.com>`_.

**rest2web** now supports three levels of verbosity via command line options :

* ``-v`` Verbose (the default)
* ``-a`` Actions & Warnings Only
* ``-w`` Warnings Only

There is currently a bug in the handling of the ``file`` keyword when use in conjunction with the ``index-file`` keyword.

After this there are several new features to be added before the next release.

In other rest2web news, the Leipzig Python User Group <http://www.python-academy.com/user-group/index.html>`_ are having a meeting on April 11th. Mike Muller is giving a talk on  *rest2web*. Shame it's a bit far for me to go, but it should be interesting. {sm;:wink:}

Re: [Rest2web-develop] RE: File Bug

[Fuzzyman <fuz...@vo...>]

Andrew Ittner wrote:
> Here is some debug output for the root's index:
> self.dir=; filename=index.txt; entry=rest2web140x62.gif;
> src=index.txt/rest2web140x62.gif
>
>   
So this works fine.

> And for projects/rest2web's index:
> self.dir=projects/rest2web; filename=projects/rest2web/index.txt;
> entry=halloffamelogo.png;
> src=projects/rest2web/projects/rest2web/index.txt/halloffamelogo.png
>
> This line:
> src = join(self.dir, dirname(filename), entry)
>
> will concatenate the current directory, AND the current directory of a file
> in the current directory, and THEN the filename.  So you end up with
> DIR/SUBDIR/DIR/SUBDIR/filename.ext.
>
>   
Yup - I understand.

I think the method ``get_real_restindex`` is doing this to the filename 
- so I will work out a generic method of solving this. I wonder why this 
worked for me though ?

> Since you want to merge these 2 loops, perhaps I should wait to add a fix.
> Here's what I recommend: compare self.dir and dirname(filename)'s results:
> if they match, use only one.  If they don't, use one.  I think we need a
> complete test suite on this one, because you talked about setting the
> index-file option, which I do not use in combination with the file keyword.
>   
I am not currently using the file keyword at all, I do however use the 
index-file keyword quite a lot, so it is important to me that this works 
correctly.

I will sort it out.

Thanks for your help in diagnosing this.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Andrew
>
> -----Original Message-----
> From: res...@li...
> [mailto:res...@li...]On Behalf Of Michael
> Foord
> Sent: Thursday, April 06, 2006 12:15 AM
> To: res...@li...
> Subject: Re: [Rest2web-develop] RE: File Bug
>
>
>
>
>
> On 06/04/06, Andrew Ittner <and...@us...> wrote:
> I love one-line fixes.
>
> Changed
>     src = join(self.dir, filename, entry)
>
>
> But in SVN I had already changed that line to :
>
>     src = join(self.dir, dirname(filename), entry)
>
> ??? Because I'd like to move the two chunks of code into a single function,
> that dirname will be needed for the case of index files.
>
> I'll make that change and you can test again. :-)
>
> Fuzzyman
> http://ww.voidspace.org.uk/python/index.shtml
>
>
>
> to
>     src = join(self.dir, entry)
>
> It appears that the filename value set the path to
> "index.txt /thefilename.ext" instead of just "thefilename.ext".  Once I
> removed it, both the index and non-index pages copied files fine (so I did
> not need to change the other loop's src line).
>
> Andrew
>
>   
>> -----Original Message-----
>> From: Fuzzyman [mailto:fuz...@vo...]
>> Sent: Tuesday, April 04, 2006 2:34 AM
>> To: res...@li...; Andrew Ittner
>> Subject: Re: File Bug
>>
>>
>> Hello Andrew,
>>
>> I would certainly appreciate your eyes in tracking down this bug. I
>> don't *think* it can be platform specific though, as there is no
>> platform specific code (I use platform independent functions from
>> ``os.path`` and ``shutil`` to do all the work).
>>
>> I can see where the bug is happening - for you it seems to be where the
>> location of the source file is determined. I've pasted the full code in
>> below - and explained the logic of the first part which determines the
>> source location. You could add in a print statement to see where it
>> thinks the source file *should be*, which will give us a clue as to what
>> is happening.
>>
>> The code exists twice (which means it ought to be turned into a method
>> of Processor). Once in the ``process`` method and once in the
>> ``processindex`` method.
>>
>> When rest2web starts, it changes the current directory to be the 'start'
>> directory (top level of the site directory tree).
>>
>> As it goes through the files, the current directory being worked on is
>> stored in ``self.dir`` (self being the processor instance). At the top
>> level directory the following is set in ``__init__``, ``self.dir =
>> os.curdir``. (Is it possible that this is the cause of the problem ?)
>>
>> Following code taken from -
>> http://rest2web.python-hosting.com/file/trunk/rest2web/restprocessor.py
>> - from the process method.
>>
>> # also copy files (from restindex) into same dir as target
>> for entry in restindex['file']:
>>     src = join(self.dir, dirname(filename), entry)
>>     ## XXX Add a ``print src`` here :-)
>>     if not os.path.isfile(src):
>>         print ('File "%s" referenced by file keyword missing.' %
>>             entry)
>>         continue
>>     #
>>     target_file_name = url2pathname(target)
>>     targetfile = os.path.normpath(join(
>>         self.target, self.dir, target_file_name))
>>     dest = join(dirname(targetfile), split(entry)[1])
>>     comparefile = os.path.normpath(join(
>>         self.compare, self.dir, target_file_name))
>>     cmp = join(dirname(comparefile), split(entry)[1])
>>     if comparefiles(src, cmp):
>>         print 'File "%s" identical, skipping.' % entry
>>         continue
>>     if not isdir(dirname(dest)):
>>         os.makedirs(dirname(dest))
>>     print 'Copying "%s".' % entry
>>     copy2(src, dest)   # alias for shutil.copy2
>>
>> I'll step through the code to explain the process.
>>
>>     for entry in restindex['file']:
>>
>> restindex['file'] should be a list of all your file keywords from the
>> page being processed.
>>
>>
>>     src = join(self.dir, dirname(filename), entry)
>>
>> ``join`` is an alias for ``os.path.join``.
>>
>> ``filename`` is the name of the file being processed. When we are
>> processing an index file, it could actually be a path relative to
>> ``self.dir`` if the ``index-file`` option is set.
>>
>> You were getting lots of  ``File "..." referenced by file keyword
>> missing.`` errors - so this logic seems to be faulty ?
>>
>> If you add the print statement I suggest, then you should be able to
>> tell what is happening. I'm sure the solution is simple enough - but I
>> can't see it.
>>
>> Thanks
>>
>> Fuzzyman
>> http://www.voidspace.org.uk/python/index.shtml
>>     
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>   

Re: [Rest2web-develop] Coding Standard

[Fuzzyman <fuz...@vo...>]

Fuzzyman wrote:
> Hello Stewart,
>

Oops... this should have gone to firedrop2-develop - it's relevant here 
too I guess, but a bit less so.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
> I've just posted to my blog an entry on my coding style and standard.
>
>    
> http://www.voidspace.org.uk/python/weblog/arch_d7_2006_04_01.shtml#e296
>
> It would help consistency if you could stick to as much of this as is 
> reasonable. This is how I code in all my projects, but these 'rules' 
> have developed over the three years I have been coding. That means 
> you'll see a lot of my code that *doesn't* follow these rules.
>
> Anyway, I think it will be helpful if we can be consistent - but only 
> where it is helpful. :-)
>
> Any comments or suggestions would be welcomed by the way.
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml
>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>

[Rest2web-develop] Coding Standard

[Fuzzyman <fuz...@vo...>]

Hello Stewart,

I've just posted to my blog an entry on my coding style and standard.

    http://www.voidspace.org.uk/python/weblog/arch_d7_2006_04_01.shtml#e296

It would help consistency if you could stick to as much of this as is 
reasonable. This is how I code in all my projects, but these 'rules' 
have developed over the three years I have been coding. That means 
you'll see a lot of my code that *doesn't* follow these rules.

Anyway, I think it will be helpful if we can be consistent - but only 
where it is helpful. :-)

Any comments or suggestions would be welcomed by the way.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

RE: [Rest2web-develop] RE: File Bug

[Andrew I. <and...@us...>]

Here is some debug output for the root's index:
self.dir=; filename=index.txt; entry=rest2web140x62.gif;
src=index.txt/rest2web140x62.gif

And for projects/rest2web's index:
self.dir=projects/rest2web; filename=projects/rest2web/index.txt;
entry=halloffamelogo.png;
src=projects/rest2web/projects/rest2web/index.txt/halloffamelogo.png

This line:
src = join(self.dir, dirname(filename), entry)

will concatenate the current directory, AND the current directory of a file
in the current directory, and THEN the filename.  So you end up with
DIR/SUBDIR/DIR/SUBDIR/filename.ext.

Since you want to merge these 2 loops, perhaps I should wait to add a fix.
Here's what I recommend: compare self.dir and dirname(filename)'s results:
if they match, use only one.  If they don't, use one.  I think we need a
complete test suite on this one, because you talked about setting the
index-file option, which I do not use in combination with the file keyword.

Andrew

-----Original Message-----
From: res...@li...
[mailto:res...@li...]On Behalf Of Michael
Foord
Sent: Thursday, April 06, 2006 12:15 AM
To: res...@li...
Subject: Re: [Rest2web-develop] RE: File Bug





On 06/04/06, Andrew Ittner <and...@us...> wrote:
I love one-line fixes.

Changed
    src = join(self.dir, filename, entry)


But in SVN I had already changed that line to :

    src = join(self.dir, dirname(filename), entry)

??? Because I'd like to move the two chunks of code into a single function,
that dirname will be needed for the case of index files.

I'll make that change and you can test again. :-)

Fuzzyman
http://ww.voidspace.org.uk/python/index.shtml



to
    src = join(self.dir, entry)

It appears that the filename value set the path to
"index.txt /thefilename.ext" instead of just "thefilename.ext".  Once I
removed it, both the index and non-index pages copied files fine (so I did
not need to change the other loop's src line).

Andrew

> -----Original Message-----
> From: Fuzzyman [mailto:fuz...@vo...]
> Sent: Tuesday, April 04, 2006 2:34 AM
> To: res...@li...; Andrew Ittner
> Subject: Re: File Bug
>
>
> Hello Andrew,
>
> I would certainly appreciate your eyes in tracking down this bug. I
> don't *think* it can be platform specific though, as there is no
> platform specific code (I use platform independent functions from
> ``os.path`` and ``shutil`` to do all the work).
>
> I can see where the bug is happening - for you it seems to be where the
> location of the source file is determined. I've pasted the full code in
> below - and explained the logic of the first part which determines the
> source location. You could add in a print statement to see where it
> thinks the source file *should be*, which will give us a clue as to what
> is happening.
>
> The code exists twice (which means it ought to be turned into a method
> of Processor). Once in the ``process`` method and once in the
> ``processindex`` method.
>
> When rest2web starts, it changes the current directory to be the 'start'
> directory (top level of the site directory tree).
>
> As it goes through the files, the current directory being worked on is
> stored in ``self.dir`` (self being the processor instance). At the top
> level directory the following is set in ``__init__``, ``self.dir =
> os.curdir``. (Is it possible that this is the cause of the problem ?)
>
> Following code taken from -
> http://rest2web.python-hosting.com/file/trunk/rest2web/restprocessor.py
> - from the process method.
>
> # also copy files (from restindex) into same dir as target
> for entry in restindex['file']:
>     src = join(self.dir, dirname(filename), entry)
>     ## XXX Add a ``print src`` here :-)
>     if not os.path.isfile(src):
>         print ('File "%s" referenced by file keyword missing.' %
>             entry)
>         continue
>     #
>     target_file_name = url2pathname(target)
>     targetfile = os.path.normpath(join(
>         self.target, self.dir, target_file_name))
>     dest = join(dirname(targetfile), split(entry)[1])
>     comparefile = os.path.normpath(join(
>         self.compare, self.dir, target_file_name))
>     cmp = join(dirname(comparefile), split(entry)[1])
>     if comparefiles(src, cmp):
>         print 'File "%s" identical, skipping.' % entry
>         continue
>     if not isdir(dirname(dest)):
>         os.makedirs(dirname(dest))
>     print 'Copying "%s".' % entry
>     copy2(src, dest)   # alias for shutil.copy2
>
> I'll step through the code to explain the process.
>
>     for entry in restindex['file']:
>
> restindex['file'] should be a list of all your file keywords from the
> page being processed.
>
>
>     src = join(self.dir, dirname(filename), entry)
>
> ``join`` is an alias for ``os.path.join``.
>
> ``filename`` is the name of the file being processed. When we are
> processing an index file, it could actually be a path relative to
> ``self.dir`` if the ``index-file`` option is set.
>
> You were getting lots of  ``File "..." referenced by file keyword
> missing.`` errors - so this logic seems to be faulty ?
>
> If you add the print statement I suggest, then you should be able to
> tell what is happening. I'm sure the solution is simple enough - but I
> can't see it.
>
> Thanks
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml