[PyCS-devel] big cvs commit

[Phillip P. <pp...@my...>]

I just pushed a whole bunch of stuff into CVS.  Some tidying / refactoring
(pycs.py and xmlStorageSystem.py are now a little shorter), some minor
changes (comments.py returns more helpful errors, referrers.py has a
different message) and one biggie: search.

I never managed to get enough time alone to finish the ht://Dig or Lucene
integration that I was working on, but I just hacked up a Python equivalent
to what I was doing with Lucene:

- added a new xmlStorageSystem.mirrorPosts(email,password,posts) method that
takes a list of posts (structs: {postid,date,url,guid,title,text}) and dumps
them in a new database table.  It deletes them (searches by postid) if they
already exist.

- added modules/system/search.py that searches through this table.

Examples:

    http://www.pycs.net/system/search.py?u=0000049&q=georg+%2Bxml-rpc

    http://www.pycs.net/system/search.py?u=0000049&q=georg+-bauer

    http://www.pycs.net/system/search.py?u=0000049&q=phil+-pearson

I've added support for this in to bzero ... it makes structs that look like
this:
    {postid: '200310151',
    'date': xmlrpclib.Date(...),
    'url': 'http://www.pycs.net/archive/2003/10/15/#200310151',
    'guid': 'http://www.pycs.net/archive/2003/10/15/#200310151',
    'title': 'post about something',
    'text': 'foo bar baz'}

It converts all incoming text into UTF-8 for storage in the database, and
displays everything as UTF-8.

TODO:

- make it respect access rules
- check that it works for non-english
- audit xml-rpc methods to make sure nothing else tries to put weird objects
in the database

------

Here are the full details of the CVS commit:

- modules/system/comments.py: added some helpful links when someone requests
comments.py?u=foo without 'p=...'

- pycs_settings.py: changed code to use standard python style; added new
mirrored_posts database table

- xmlStorageSystem.py: added xmlStorageSystem.mirrorPosts() call to allow
users to push posts into the search mirror (not really an index...)

- modules/system/rankings.py: fixed display bug - now everything appears at
the top of the screen and is spaced horizontally properly

- modules/system/referrers.py: fixed spelling

- modules/system/search.py: moved old search.py to htdig.py and made
search.py do a simple linear search through the mirrored_posts table.  NB:
it doesn't respect access rules yet.

- modules/system/users.py: added indication as to whether the user is using
the search mirror (and if so, how many posts are in there)

- www/pycs.css: added some css rules for the search results

Cheers,
Phil :-)