Menu
▾
▴
pyblosxom-devel
|
From: Blake W. <bl...@ph...> - 2003-02-04 16:47:46
|
Responding to some comments from Wari, Our Glorious Leader:
> I need an API for storage.
Okay, here's a first pass at the storage api.
----------------->8 storageApi.py 8<--------------------
"""
This module defines the api that any class implementing storage must
conform to.
"""
class Storage:
# Storage types.
COMMENT = 1
TRACKBACK = 2
PINGBACK = 3
SENT_TRACKBACK = 4
SENT_PINGBACK = 5
def allFilter( entry ):
return true
def get( self, type, entryId, filter=allFilter ):
# Returns a list of all the data for that entry,
# filtered by the filter.
pass
def store( self, type, entryId, data ):
# Stores a bit of data of the specified type for the
# passed-in entry.
pass
def search( self, searchTerm, type=None, entryId=None ):
# Searches in the specified type and entry (if supplied)
# for data containing the passed-in search term.
pass
----------------->8 storageApi.py 8<--------------------
|
|
From: Blake W. <bl...@ph...> - 2003-02-04 16:54:05
|
> Responding to some comments from Wari, Our Glorious Leader: > > I need an API for storage. > > Okay, here's a first pass at the storage api. > ----------------->8 storageApi.py 8<-------------------- > """ > This module defines the api that any class implementing storage must > conform to. > """ > class Storage: > > # Storage types. > COMMENT = 1 > TRACKBACK = 2 > PINGBACK = 3 > SENT_TRACKBACK = 4 > SENT_PINGBACK = 5 > > def allFilter( entry ): > return true > > def get( self, type, entryId, filter=allFilter ): > # Returns a list of all the data for that entry, > # filtered by the filter. > pass > > def store( self, type, entryId, data ): > # Stores a bit of data of the specified type for the > # passed-in entry. > pass > > def search( self, searchTerm, type=None, entryId=None ): > # Searches in the specified type and entry (if supplied) > # for data containing the passed-in search term. > pass > ----------------->8 storageApi.py 8<-------------------- And some day I'll remember to not hold down the Control key when I hit enter. Anyways, what I wanted to say was: 1) Does this fulfill all of our needs in terms of storing and retrieving comments/trackbacks/pingbacks/etc? 2) Is it easy enough to use, or can it be made easier? 3) Is it reasonable to implement, or nigh impossible? I think most of the trickyness would come in writing filters. Oh, and there's currently no simple way to sort the data, but if we returned the bit of data as a tuple containing the type, and whatever else we could parse out, then a sorter would be fairly simple to write. Any and all comments appreciated. Later, Blake. |
|
From: Wari W. <wa...@ce...> - 2003-02-05 02:52:31
|
* Blake Winton <bl...@ph...> [030205 00:51]: > > Responding to some comments from Wari, Our Glorious Leader: Glorious? :) > And some day I'll remember to not hold down the Control key when I hit > enter. Anyways, what I wanted to say was: > 1) Does this fulfill all of our needs in terms of storing and > retrieving comments/trackbacks/pingbacks/etc? Yes, it does fulfill all of the above :) It doesn't have a delete() or modify() though. Think of it this way, if you were to design a webui (or any UI) for that matter, managing comments, what would you need? > 2) Is it easy enough to use, or can it be made easier? A commentID would be nice I guess (makes delete and modify easier). > 3) Is it reasonable to implement, or nigh impossible? Not sure until it's done :) > I think most of the trickyness would come in writing filters. Oh, and > there's currently no simple way to sort the data, but if we returned > the bit of data as a tuple containing the type, and whatever else we > could parse out, then a sorter would be fairly simple to write. With a commentID (assuming it's actually a time int() at least) you can sort them 2 ways, ascending and descending. Unless we're slashdot or something like that, timestamps would give us a nice unique number between comments, btw, if you are slashdot, you can use a real data base, then the commentID can be an integer generated by the database, you know those autonumber types. Actually, if you are slashdot, you'd be using threaded discussions :) Comments for blogs does not need to be threaded (though it can break fluidity in comments). How about a dump() and load() so that you can migrate between storage implementations. Though not used by pyblosxom, but could be used by a utility. -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: Blake W. <bl...@ph...> - 2003-02-05 13:22:58
|
> > 1) Does this fulfill all of our needs in terms of storing and > > retrieving comments/trackbacks/pingbacks/etc? > Yes, it does fulfill all of the above :) It doesn't have a > delete() or modify() though. Sure it does. Put None, or put the changed data. > > 2) Is it easy enough to use, or can it be made easier? > A commentID would be nice I guess (makes delete and modify easier). Okay, it will be returned from the put. > How about a dump() and load() so that you can migrate between storage > implementations. Though not used by pyblosxom, but could be used by a > utility. Sounds like a good idea. I'll add these in, and fire off another email later today. Thanks, Blake. |
|
From: Blake W. <bl...@ph...> - 2003-02-05 23:12:40
|
> > > I need an API for storage. > > Okay, here's a first pass at the storage api. And the second pass at the storage api. http://www.latte.ca/cgi-bin/source.py?name=storage I felt it necessary to define the format that dump and load would use, so that we will have a hope of importing and exporting data from other sources. I'm open to discussing that format, since I pretty much made it up off the top of my head. Later, Blake. |
|
From: Wari W. <wa...@ce...> - 2003-02-06 06:10:11
|
* Blake Winton <bl...@ph...> [030206 07:09]: > And the second pass at the storage api. > http://www.latte.ca/cgi-bin/source.py?name=storage Looks better now :) > I felt it necessary to define the format that dump and load would use, > so that we will have a hope of importing and exporting data from other > sources. I'm open to discussing that format, since I pretty much made > it up off the top of my head. I'm just wondering, should the data dump really be in XML? I was thinking of a pickle at least, then all load() needs to do is eat up the pickle and import via the add() and store(). In fact if everything is standard, no new storageAPI plugins needs to override load() at all, if this is the case (either way, it's the same with XML) Maybe it's because I'm poor at XML, but anyone else agree on XML, or a pickle? -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: will <wi...@bl...> - 2003-02-06 15:37:04
|
On Thu, 6 Feb 2003, Wari Wahab wrote: > > I'm just wondering, should the data dump really be in XML? I was > thinking of a pickle at least, then all load() needs to do is eat up the > pickle and import via the add() and store(). In fact if everything is > standard, no new storageAPI plugins needs to override load() at all, if > this is the case (either way, it's the same with XML) > > Maybe it's because I'm poor at XML, but anyone else agree on XML, or a > pickle? The disadvantages to using XML would be that the performance of loading/saving is probably an order of magnitude slower than pickling. In addition to that, we also take on the burden of architecting the XML format we want to save in and how it works. We definitely get more bang for our buck with pickling. The advantage to using XML is that other non-Python processes can manipulate the data. I can't really think of other advantages--though it should be noted that this is somewhat of a big one. How does blosxom and the other *osxom systems store data? Would we want to store our data in a compatible format further enabling folks to switch between *osxoms? /will -- whatever it is, you can find it at http://www.bluesock.org/~willg/ except Will--you can only see him in real life. |
|
From: Wari W. <wa...@ho...> - 2003-02-06 15:58:03
|
will wrote: >The disadvantages to using XML would be that the performance of >loading/saving is probably an order of magnitude slower than pickling. >In addition to that, we also take on the burden of architecting the XML >format we want to save in and how it works. We definitely get more bang >for our buck with pickling. > > Sorry, the XML we were discussing about only relates to importing and exporting data between storageAPI compliant objects. If I don't like, say a file based storageAPI system, I can export to the pickle, and get the sqlite storageAPI object to get the data from the pickle export and store data in it's database, this is so that you don't lose data while changing 'drivers'. >The advantage to using XML is that other non-Python processes can >manipulate the data. I can't really think of other advantages--though it >should be noted that this is somewhat of a big one. > > Since this is a dump() and load() methods, maybe later we can define an export() method to export to a documented format like XML or maybe MovableType style comments data. A dump() and load() is for pyblosxom and pyblosxom only I guess (though other program could use its data) >How does blosxom and the other *osxom systems store data? Would we want >to store our data in a compatible format further enabling folks to switch >between *osxoms? > There's no common way, Rael's blosxom uses a modified standalone trackback tool for comments (which uses perl style dumps), I'm not sure what Sam Ruby uses, phosxom uses sqlite to store feedback. For pyblosxom I want it as flexible as possible yet can migrate data as well. Data storage is one thing, exporting it is another. |
|
From: Blake W. <bl...@ph...> - 2003-02-06 16:23:06
|
> Sorry, the XML we were discussing about only relates > to importing and exporting data between storageAPI > compliant objects. Really? I figured we would use the same API to handle both storageApi compliant objects and external systems, since I think people will switch between storageApi systems about as often as switching between non-pyBlosxom and pyBlosxom systems. Furthermore, if we had export() and import(), then dump() and load() wouldn't give us any extra functionality, so if we only get to pick one, I would choose export/import. (And I think that picking one or the other is better than having both, from a small-api point of view at the very least, as well as from the no-duplication-of-functionality point of view.) > > How does blosxom and the other *osxom systems store > > data? Would we want to store our data in a compatible > > format further enabling folks to switch between *osxoms? > There's no common way. Bummer. :( I guess if there was a single way to do this, we would already have done it. > For pyblosxom I want it as flexible as possible yet can > migrate data as well. Data storage is one thing, exporting > it is another. But migrating it from one driver to another seems to me to be very similar to migrating it from one system to another. Later, Blake. |
|
From: Wari W. <wa...@ho...> - 2003-02-06 16:48:06
|
Blake Winton wrote: >Really? I figured we would use the same API to handle >both storageApi compliant objects and external systems, >since I think people will switch between storageApi >systems about as often as switching between non-pyBlosxom >and pyBlosxom systems. > The thing is, how do you get people to use the same data format with differing comments system out there. If you want something that works with all *osxoms, then you need something standalone and outside of pyblosxom, like what Rael did with the Standalone Trackback Tool, or something like quicktopics on steroids :) >Furthermore, if we had export() and import(), then dump() >and load() wouldn't give us any extra functionality, so >if we only get to pick one, I would choose export/import. > > Ok, based on my previous mail, XML format is the way to go to exchange data when we need to migrate. >>For pyblosxom I want it as flexible as possible yet can >>migrate data as well. Data storage is one thing, exporting >>it is another. >> >> >But migrating it from one driver to another seems to me to >be very similar to migrating it from one system to another. > > We just need a well documented XML format then, agreed? |
|
From: Theodore W. L. <tw...@sa...> - 2003-02-08 09:22:11
|
> -----Original Message----- > From: pyb...@li...=20 > [mailto:pyb...@li...] On=20 > Behalf Of Wari Wahab > Sent: Thursday, February 06, 2003 9:17 AM > To: will > Cc: Wari Wahab; Blake Winton; pyb...@li... > Subject: Re: [Pyblosxom-devel] Supporting comments, etc. >=20 >=20 > will wrote: >=20 > >The disadvantages to using XML would be that the performance of=20 > >loading/saving is probably an order of magnitude slower than=20 > pickling. > >In addition to that, we also take on the burden of=20 > architecting the XML > >format we want to save in and how it works. We definitely=20 > get more bang=20 > >for our buck with pickling. > > =20 > > > Sorry, the XML we were discussing about only relates to importing and=20 > exporting data between storageAPI compliant objects. If I don't like,=20 > say a file based storageAPI system, I can export to the=20 > pickle, and get=20 > the sqlite storageAPI object to get the data from the pickle=20 > export and=20 > store data in it's database, this is so that you don't lose=20 > data while=20 > changing 'drivers'. >=20 > >The advantage to using XML is that other non-Python processes can > >manipulate the data. I can't really think of other=20 > advantages--though it=20 > >should be noted that this is somewhat of a big one. > > =20 > > > Since this is a dump() and load() methods, maybe later we can=20 > define an=20 > export() method to export to a documented format like XML or maybe=20 > MovableType style comments data. A dump() and load() is for pyblosxom=20 > and pyblosxom only I guess (though other program could use its data) >=20 > >How does blosxom and the other *osxom systems store data? Would we=20 > >want > >to store our data in a compatible format further enabling=20 > folks to switch=20 > >between *osxoms? > > > There's no common way, Rael's blosxom uses a modified standalone=20 > trackback tool for comments (which uses perl style dumps),=20 > I'm not sure=20 > what Sam Ruby uses, phosxom uses sqlite to store feedback.=20 > For pyblosxom=20 > I want it as flexible as possible yet can migrate data as well. Data=20 > storage is one thing, exporting it is another. >=20 Sam is using text files written into directories. They are not XML. There's a lot of value to using XML as an interchange format. In fact, = when I saw Sam 2 weeks ago (almost) we talked about using RSS as a = representation for articles as well as comments. Of course, we were also talking about wrapping the RSS item in a SOAP envelope for the next generation of the weblog / Metaweblog API -- that's an aside. I haven't played much with Python's XML support, but I know that at = least one of the python XML libraries is based on expat which is a very fast C parser. I'd like to try the XML format and see if performance is really a = problem. This might mean using more libraries then what's in the Python base, = though. Ted=20 |
|
From: Wari W. <wa...@ho...> - 2003-02-08 16:23:20
|
Theodore W. Leung wrote: >Sam is using text files written into directories. They are not XML. >There's a lot of value to using XML as an interchange format. > At least Sam did publish that changes to blosxom source code, unfortunately it's in perl :) I can't grok it. >I haven't played much with Python's XML support, but I know that at least >one of the python XML libraries is based on expat which is a very fast C >parser. > > It depends on the python distribution though, I'm not sure mine is based on expat or not, but what I've heard is that is expat is not available, python will fallback to a much slower pure python xml parser module. >I'd like to try the XML format and see if performance is really a problem. >This might mean using more libraries then what's in the Python base, though. > > I'm game if we need extra modules for a particular plugin, but pyblosxom must work with a standard distribution, no matter what we use, if there's a plugin or driver that requires external libraries and there's compelling reason to use it, then the user can install it himself, but we should have something that works by default, not matter what. |
|
From: Blake W. <bl...@ph...> - 2003-02-06 16:13:20
|
> Maybe it's because I'm poor at XML, but anyone else agree > > on XML, or a pickle? How are you poor at XML? Is it anything I can help with? > In addition to that, we also take on the burden of > architecting the XML format we want to save in and > how it works. Unless we can use someone else's format, which is what I've tried to do as much as possible. > We definitely get more bang for our buck with pickling. Agreed, but I've got the developer-bucks to spend on this, if it's the route we choose to go. > The advantage to using XML is that other non-Python > processes can manipulate the data. I can't really > think of other advantages--though it should be noted > that this is somewhat of a big one. Such as a vim process. This was my main reason for suggesting XML, in fact. I wanted to go through my list of referrers, and remove the ones that I really didn't want there, but since the referrers are pickled, I couldn't easily edit the file (and I didn't keep a backup, so I lost my referrer information twice. No big loss, but slightly annoying). Being able to edit comments with a text-editor is a big plus in my book. > How does blosxom and the other *osxom systems store data? > Would we want to store our data in a compatible format > further enabling folks to switch between *osxoms? We could do this, or write a tool to convert between the standard format and our format. (I mention this because of Leslie Orchard's post at: http://www.decafbad.com/news_archives/000410.phtml where he specifically mentions the "MT-to-blosxom converter" as one of the reasons he switched. Being able to switch back and forth from our format to other people's format is a huge win for getting users to convert. I've sent him email asking for more details on his comments system, both so that we can make it easy for him to convert, and so that we can get another perspective on how to implement these things. Oh, yeah, while I'm here. I'm subscribed to pyblosxom-devel, pyblosxom-users, and even pyblosxom-checkins, so you don't need to copy me in on messages that are going to those addresses, since I'll get a copy of them if you just send them to the list. Thanks, Blake. |
|
From: Wari W. <wa...@ho...> - 2003-02-06 16:41:26
|
Blake Winton wrote: >>Maybe it's because I'm poor at XML, but anyone else agree >> >> >>>on XML, or a pickle? >>> >>> >How are you poor at XML? Is it anything I can help with? > > Writing proper XML, I'm poor at that, until now I can't figure out how to write an XML file using DOM or SAX, you know writing proper XML files like RSS 2.0 without using regexp style escape, the most I got to do is a blosxom style rss dump, and that is not writing proper XML. Parsing XML, another story, looking at python's style of xml parsing from the examples I saw, I can't get to understand how it works, though I can modify code like Mark Pilgrim's rssparser.py to get some data I need, but when it comes to identicle tags like link in channels and link in channels->author (or something like that) I gave up :) I do know what XML is and can parse it by sight, not programmatically (who know I'll change that soon) >>We definitely get more bang for our buck with pickling. >> >> >Agreed, but I've got the developer-bucks to spend on this, >if it's the route we choose to go. > I guess I can learn something is we go XML route :) >>The advantage to using XML is that other non-Python >>processes can manipulate the data. I can't really >>think of other advantages--though it should be noted >>that this is somewhat of a big one. >> >> >Such as a vim process. This was my main reason for suggesting >XML, in fact. > That is the most compelling reason to use XML IMHO, hehe, you can dump the data, then edit the file (provided you keep to XML constraints like escapes and what nots) then get the 'driver' to load the data back. Cool, I like XML already :) >I wanted to go through my list of referrers, and >remove the ones that I really didn't want there, but since the >referrers are pickled, I couldn't easily edit the file (and I >didn't keep a backup, so I lost my referrer information twice. >No big loss, but slightly annoying). Being able to edit >comments with a text-editor is a big plus in my book. > Yeah, but isn't using XML as a native format for, say referer data, slow? I've tried marshal.xml before to store data, and it's painfully slow to load. >We could do this, or write a tool to convert between the >standard format and our format. (I mention this because of >Leslie Orchard's post at: >http://www.decafbad.com/news_archives/000410.phtml >where he specifically mentions the "MT-to-blosxom converter" >as one of the reasons he switched. Being able to switch >back and forth from our format to other people's format is >a huge win for getting users to convert. I've sent him >email asking for more details on his comments system, both >so that we can make it easy for him to convert, and so that >we can get another perspective on how to implement these >things. > It's a standard movabletype dump, here's what it looks like for my birthday entry in http://roughingit.subtlehints.net/ : -------- AUTHOR: wari TITLE: Happy birthday to me :) STATUS: Publish ALLOW COMMENTS: 1 CONVERT BREAKS: 1 ALLOW PINGS: 1 PRIMARY CATEGORY: Python CATEGORY: Python DATE: 02/02/2003 10:16:15 PM ----- BODY: So I'm 30, what do I do on my birthday? Release <a href="http://roughingit.subtlehints.net/pyblosxom">pyblosxom 0.6</a> :) ----- EXTENDED BODY: ----- EXCERPT: ----- KEYWORDS: ----- COMMENT: AUTHOR: Troy EMAIL: tad...@sp... IP: ipaddr URL: http://forestpirate.subtlehints.net DATE: 02/03/2003 01:10:55 AM Happy Birthday Wari. Hope you have a good one. (Or had a good one) ----- COMMENT: AUTHOR: nf0 EMAIL: nf...@10... IP: ipaddr URL: http://www.10500bc.org DATE: 02/03/2003 01:43:17 AM Happy Birthday! ----- COMMENT: AUTHOR: [rux] EMAIL: IP: ipaddr URL: DATE: 02/03/2003 08:19:47 PM happy birthday, dude ... ----- COMMENT: AUTHOR: wari EMAIL: IP: ipaddr URL: http://wari.per.sg/ DATE: 02/03/2003 10:48:53 PM Thanks guys :) ----- COMMENT: AUTHOR: Sam EMAIL: IP: ipaddr URL: http://www.deadman.org/ DATE: 02/04/2003 09:40:31 PM I'd forgotten that we have the same birthday. Happy birthday you old geezer! (I'm only 29!) ----- COMMENT: AUTHOR: dOh EMAIL: IP: ipaddr URL: DATE: 02/04/2003 11:43:59 PM Happy birthday to u... heh heh... congrats, the big 3-O, and a father soon. ----- COMMENT: AUTHOR: wari EMAIL: IP: ipaddr URL: http://wari.per.sg DATE: 02/06/2003 12:03:03 PM Hey Sam, I forgotten about your birthday too :) I'm not that old, 3 decades ain't old yet :) ----- -------- mt2blosxom.py only works on the entries, but not the comments. >Oh, yeah, while I'm here. >I'm subscribed to pyblosxom-devel, pyblosxom-users, and >even pyblosxom-checkins, so you don't need to copy me in >on messages that are going to those addresses, since I'll >get a copy of them if you just send them to the list. > > Sorry 'bout that. |
|
From: Blake W. <bl...@ph...> - 2003-02-06 17:14:24
|
> >How are you poor at XML? Is it anything I can help with? > Writing proper XML, I'm poor at that, until now I can't > figure out how to write an XML file using DOM or SAX, you > know writing proper XML files like RSS 2.0 without using > regexp style escape, the most I got to do is a blosxom > style rss dump, and that is not writing proper XML. http://uche.ogbuji.net:8080/uche.ogbuji.net/tech/pubs/ might help. or http://www.xml.com/pub/a/2002/11/13/py-xml.html or http://xml.coverpages.org/xmlPython.html But yeah, it's a little tricky. I stick with the standard "print" statements, myself. They seem to get the job done. > Parsing XML, another story, looking at python's style of > xml parsing from the examples I saw, I can't get to > understand how it works, though I can modify code like > Mark Pilgrim's rssparser.py to get some data I need, but > when it comes to identicle tags like link in channels > and link in channels->author (or something like that) I > gave up :) If it was a SAX-based parser (which I imagine it is), then you'll need to keep the state yourself, and when you see the "channels->author" tag(s), set a flag. That's why I tend to use DOM-based parsers, such as: http://www-106.ibm.com/developerworks/xml/library/x-tipulldom.html I agree that XML parsing isn't the easiest thing to do, but it might be worth learning, and there are a lot of resources out there. > >>The advantage to using XML is that other non-Python > >>processes can manipulate the data. > That is the most compelling reason to use XML IMHO, hehe, > you can dump the data, then edit the file (provided you > keep to XML constraints like escapes and what nots) then > get the 'driver' to load the data back. > Cool, I like XML already :) Sold! ;) > Yeah, but isn't using XML as a native format for, say > referer data, slow? I've tried marshal.xml before to > store data, and it's painfully slow to load. There are ways to optimize it, I'm sure. > >more details on his comments system. > It's a standard movabletype dump, here's what it looks > like for my birthday entry in > http://roughingit.subtlehints.net/ : > -------- [snip...] > -------- Could you mail me a file containing the dump for an entry or two? It looks easy enough to parse from what I can see here, and if I could write something to convert it to XML, that might get us most of the way there. > mt2blosxom.py only works on the entries, but not the comments. For now... ;) > >Oh, yeah, while I'm here. > >I'm subscribed to pyblosxom-devel, pyblosxom-users, and > >even pyblosxom-checkins, so you don't need to copy me in > >on messages that are going to those addresses, since I'll > >get a copy of them if you just send them to the list. > Sorry 'bout that. No worries. It was more a request for the future. Oh, one final question, are we happy with the location and name of the storageApy.py file? Shall I check it in, given that we can always edit it further? And more responses: > >Really? I figured we would use the same API to handle > >both storageApi compliant objects and external systems, > The thing is, how do you get people to use the same data > format with differing comments system out there. You either write converters, or get other people to write converters, or get so popular that everyone else writes to your standard. I'm hoping for option 3, but would be happy with option 2, and would even go with option 1 without too much complaining. ;) > If you want something that works with all *osxoms, then > you need something standalone and outside of pyblosxom, > like what Rael did with the Standalone Trackback Tool, > or something like quicktopics on steroids :) The format shouldn't be pyblosom specific, but pyblosxom will handle it, and it will (apparently) be driven by the pyblosxom developers. We could write code for the standard blosxom to support it, and then all the other blosxom clones would be forced to add support for it to keep up. > We just need a well documented XML format then, agreed? Agreed. Are there any we can snarf on the weeb? (I've just done a search and found the following: http://www.diaries.com/digiboy/stories/storyReader$41 Handles threads, doesn't handle trackback/pingback. Later, Blake. |
|
From: Wari W. <wa...@ce...> - 2003-02-07 05:34:03
|
* Blake Winton <bl...@ph...> [030207 01:11]: > http://uche.ogbuji.net:8080/uche.ogbuji.net/tech/pubs/ might help. or > http://www.xml.com/pub/a/2002/11/13/py-xml.html or > http://xml.coverpages.org/xmlPython.html I'll try and give it a read, again :) > But yeah, it's a little tricky. I stick with the standard "print" > statements, myself. They seem to get the job done. The only code I've seen creating a proper XML document with DOM is http://markpasc.org/code/tbpy/ , even that looks a bit heavy for me to process :) > That's why I tend to use DOM-based parsers, such as: > http://www-106.ibm.com/developerworks/xml/library/x-tipulldom.html > > I agree that XML parsing isn't the easiest thing to do, but it might > be worth learning, and there are a lot of resources out there. Well I guess there's money to be made on this :) Everyone's talking about how cool XML is. There also people who talks about how bad XML is, so it's subjected to debate. You love it or hate it, but can't ignore it. > Oh, one final question, are we happy with the location and name of the > storageApy.py file? Shall I check it in, given that we can always > edit it further? I'd prefer storageapi.py to sit in it's own directory, and you can place the drivers there, like preformatters, name it base or something, and the drivers can subclass it from there. > You either write converters, or get other people to write converters, > or get so popular that everyone else writes to your standard. I'm > hoping for option 3, but would be happy with option 2, and would even > go with option 1 without too much complaining. ;) Option 3 is possible if we could make something standalone out of our comments stuff and you can then sell it as a different package with its own SKU :) Option 2 and 1 will only come about if the feedback system is a killer application and people are dying to work with it :) > The format shouldn't be pyblosom specific, but pyblosxom will handle > it, and it will (apparently) be driven by the pyblosxom developers. Can be done. > We could write code for the standard blosxom to support it, and then > all the other blosxom clones would be forced to add support for it to > keep up. Yeah, embrace and extend :) > > We just need a well documented XML format then, agreed? > Agreed. Are there any we can snarf on the weeb? (I've just done a > search and found the following: > http://www.diaries.com/digiboy/stories/storyReader$41 Handles threads, > doesn't handle trackback/pingback. Should be easy to integrate pingbacks and trackbacks, I donno what post-it is, so I don't care about that yet :) But do we need something threaded? Blog users (your visitors) can get confused with this, I know because I used to have a PostNuke blog, and regular blog readers keep creating a child thread, realised what they've done, complain about it in a new thread, and the next comment becomes a child thread again, complain..... rinse lather and repeat :) People who visited slashdot or kuro5hin would have no problems with threads, me too, I love geeky threads, but the world is not ready for it. Could be a feature that you can turn on and off though. But to enable threading, do you need to change storageAPI? -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: Theodore W. L. <tw...@sa...> - 2003-02-09 08:51:51
|
Hi guys, I'm back and caught up and ready to tackle a few things. #1. I'm trying to figure out what is the best way to integrate my = Lucene code. It requires a change in pyblosxom.py to process the form that contains = the lucene search term. It also changes the paradigm in pyblosxom for getting the list of files = to turn into the blog because the list of files now becomes the list of = files that come back from lucene. So I think that we need pair of callback chains: 1 to handle the CGI = form argument and 1 to handle the generation of the file list. For the = CGIForm chain, I think that if a handler in the chain processes the form, that handler processing is done. For the file list, we need to turn the = regular directory walk into the default behavior (last in the chain) and let = each plugin before that return a list. If any plugin in the chain returns a list, then handler processing is done. #2. For comments I have the following issues We need a way to display a single article in a page, (so that we can put = the comment form up). This probably just means invoking a different = template for the single article case. We (again) need a way to handle the comment CGI form post (probably via = the chain above). We need a way to display the comment / trackback / pingback count = (retreived from the storage api) in story.html (via a variable) -- so every = iteration through valid_list needs to be able to talk to the storage api. This = seems like another place for a callback chain. I just wanted to get your thoughts on this before I plunge ahead. Ted |
|
From: Wari W. <wa...@ho...> - 2003-02-09 13:43:43
|
Theodore W. Leung wrote: >I'm back and caught up and ready to tackle a few things. > >#1. I'm trying to figure out what is the best way to integrate my Lucene >code.For the file list, we need to turn the regular >directory walk into the default behavior (last in the chain) and let each >plugin before that return a list. If any plugin in the chain returns a >list, then handler processing is done. > This was discuss @ http://sourceforge.net/mailarchive/forum.php?thread_id=1613664&forum_id=24361 but still there is no real solution yet. If you've got some ideas, or working code, we could try test it out and see if we can use it for all situations. >#2. For comments I have the following issues >We need a way to display a single article in a page, (so that we can put the >comment form up). This probably just means invoking a different template >for the single article case. > Yup. >We (again) need a way to handle the comment CGI form post (probably via the >chain above). > > Again, the need for your point #1. :) To make life simpler, I'm suggesting a comment.cgi type of program to handle comments. We can make pyblosxom handle everything, but as feedback mechanism goes, you'll find trackback and pingback requiring xmlrpc mechanism to input feedback data. Then you'll need some management of feedback data, especially deleting feedback spam if necessary (of course to know if you're spammed, to need to mail every feedbacks to the users). I'm not sure what Blake have in mind, he would also like pyblosxom to handle POST and GET requests too. XMLRPC uses POST too, incompatible with GET requests and cgi module won't handle it. >We need a way to display the comment / trackback / pingback count (retreived >from the storage api) in story.html (via a variable) -- so every iteration >through valid_list needs to be able to talk to the storage api. This seems >like another place for a callback chain. > Counts can be done by the standard load() plugins, iterating thru values of entryList at each call by using a standard incremental style of iteration. The plugin would call storageAPI to get relevant data of comment counts and what nots. >I just wanted to get your thoughts on this before I plunge ahead. > > More things to consider, how do you tell pyblosxom that you want to disable comments on a particular entry? Would some meta data like: #comments 0 in an entry work in order to disable comments? Also, what about making comments inactive for a particular entry as well? In my other blog I have to delete one entry because it recieved 100s of stupid comments and I can't disable it in MT (MT will fix that in 2.6)? Sorry if this mail sounds negative, I'mm all for feedbacks, I want it badly, but it does not seem to be an easy task with all these issue lying around. |
|
From: Blake W. <bl...@ph...> - 2003-02-09 14:39:30
|
> >#2. For comments I have the following issues > >We need a way to display a single article in a page, (so > >that we can put the comment form up). This probably just > >means invoking a different template for the single article > >case. > Yup. Or a comments flavour, which is a different way of saying the same thing, I guess. > >handle post > Again, the need for your point #1. :) To make life simpler, > I'm suggesting a comment.cgi type of program to handle > comments. We can make pyblosxom handle everything, but as > feedback mechanism goes, you'll find trackback and pingback > requiring xmlrpc mechanism to input feedback data. But XML-RPC still works over HTTP, doesn't it? I might look into this after we've got the chains set up. > Then you'll need some management of feedback data, especially > deleting feedback spam if necessary (of course to know if > you're spammed, to need to mail every feedbacks to the users). This would be easy enough with human-readable comment files. Other storage implementors would need to be more complicated. > I'm not sure what Blake have in mind, he would also like > pyblosxom to handle POST and GET requests too. XMLRPC uses > POST too, incompatible with GET requests and cgi module won't > handle it. I just want PyBlosxom to support a couple of other ways of adding comments. Although, if we supported PUT, we could add entries as well. Version 2, perhaps. ;) > >We need a way to display the comment / trackback / pingback > >count (retreived from the storage api) in story.html (via a > >variable) -- so every iteration through valid_list needs to > >be able to talk to the storage api. This seems like another > >place for a callback chain. > Counts can be done by the standard load() plugins, iterating > thru values of entryList at each call by using a standard > incremental style of iteration. The plugin would call > storageAPI to get relevant data of comment counts and what > nots. And the storageApi would cache various values if it was found to be too slow. (Getting the number of comments for a single entry should be as simple as: comments = storage.get( Storage.COMMENTS, entryId ) py['commentCount'] = len(comments) > >I just wanted to get your thoughts on this before I plunge ahead. > More things to consider, how do you tell pyblosxom that you want to > disable comments on a particular entry? Would some meta data like: > '#comments 0' in an entry work in order to disable comments? I'm not sure I like meta-data in comments. I mean, it's a sexy idea, and it gets a lot of stuff out of config.py that probably shouldn't be in it. I guess what I'm objecting to in this particular instance is the syntax. I've already put a python program listing in my weblog, which contained lines starting with "# comment" (or at least, it should have...) and to have those lines filtered out would be a royal pain. How do we feel about '<!-- pyblosxom-meta comments="0" -->' > Also, what about making comments inactive for a particular entry > as well? In my other blog I have to delete one entry because it > recieved 100s of stupid comments and I can't disable it in MT > (MT will fix that in 2.6)? How does this differ from the above suggestion? In either case, my recommendation is "Version 2!" Hey, MT didn't get around to adding it until version 2.6, so the way I figure it, we've got 2 whole versions to go before we need to have it on the agenda. :) > Sorry if this mail sounds negative, I'm all for feedbacks, I want it > badly, but it does not seem to be an easy task with all these issue > lying around. My take on it is that you should plunge ahead (using the work that's already been done, of course. Email me, and we can talk about which pieces I can write for you). Let's get something, and then make it work better, instead of trying to solve all our problems now, and never coming up with anything. As the IETF motto says, "Rough concensus and running code." I think we've got the rough concensus, let's get the running code. Later, Blake. |
|
From: Kai H. <he...@cs...> - 2003-02-09 19:07:43
|
On the subject of comments and what have you, have any of you had a look at Vellum ? http://www.kryogenix.org/code/vellum/ I tried it out in a fever yesterday, and I found it quite good. The way it handles plugins was particularly impressive. Much neater than Pyblosxom. :) Anyway, one of the plugins is "Comments": http://www.kryogenix.org/code/vellum/plugins/comments.py It makes use of another plugin, called "Audience": http://www.kryogenix.org/code/vellum/plugins/audience.py I like how it looks : http://www.kryogenix.org/days/414.html I prefer to write my blogs with vim, i.e. not using a web ui. However I hope Pyblosxom can take on some of the cool stuff Vellum has to offer. For example, I have a friend who needs a Web UI to blog, I will install Vellum for him. Regards, -Kai Hendry p.s. I do prefer pickle over xml. It is so much easier. |
|
From: Blake W. <bl...@ph...> - 2003-02-09 23:38:34
|
> On the subject of comments and what have you, have any > of you had a look at Vellum ? Not yet, but I will now... Okay, my initial thoughts are: The Audience is pretty much what I had come up with before, although that's a great name for it. The custom fields are an interesting idea, but I think they're a hang-over from C, since in Python you can add any fields you want to any object, so there should be no reason to have them there. It's tied down to bsddb, which bothers me, because I can't hand-edit the files, and because people are suggesting that it get replaced with other things. http://mail.python.org/pipermail/patches/2002-July/008828.html I can see what it's trying to do with the vellum.functions and it's a neat idea. We might want to use that, Wari. (calling vellum.cgi?a=foo will run the function named vellum.functions.foo, apparently with the form as a parameter. I'm not sure I entirely like it, but it's very similar in concept to our py dictionary. Actually, I think the vellum.Entry.Entry is closer to our py dictionary, but you get the idea.) > p.s. I do prefer pickle over xml. It is so much easier. Easier to write to the data files with vi? As a programmer, none of you will have to write the code to read or write XML. I'll do all of that coding for you. In fact, I'm half done. http://www.latte.ca/cgi-bin/source.py?name=file If anyone wants to write the code that uses it, let me know, and I'll forward you my test scripts, which will provide a pretty clear explanation of how I think the storageApi will be used. I did some tests with adding and removing data, and it was on the order of 0.02 ms to call get, passing it a type of Storage.COMMENTS on a document with three comments. I don't know if that scales yet, but I've got a whack of sample data from Wari, so I'll run some more tests, and see what happens. I am, however, cautiously optimistic. Further changes: Implement store and load in the Storage class, since the format to export and import will be the same throughout all the storage implementations. Also, move the marshal and unmarshal code up into the base, since it will be what's used to implement store and load. Oh, and to make everyone feel a little better, the page at http://www.python.org/doc/current/lib/module-xml.dom.minidom.html (which is the xml-api I'm using) says "New in version 2.0." So it seems like everything we need is included. Later, Blake. |
|
From: Wari W. <wa...@ce...> - 2003-02-10 11:41:23
|
* Blake Winton <bl...@ph...> [030210 07:35]: > Okay, my initial thoughts are: The Audience is pretty much what I had > come up with before, although that's a great name for it. Audience, that's a great name! Let's use Vellum :) > The custom fields are an interesting idea, but I think they're a > hang-over from C, since in Python you can add any fields you want to > any object, so there should be no reason to have them there. You can add custom fields? Hmm, entries can add custom variable, want more fields, we can do foo(**kwargs), what happens after that? > I can see what it's trying to do with the vellum.functions and it's a > neat idea. We might want to use that, Wari. That should be easy to do, right Will :) Will told me that our Callback transformation is more powerful, also ours seems limited now cause we have very little thinks to callback with our api, but it will grow I assure you :) >> pickle, db, etc... > Easier to write to the data files with vi? for DBM: db_dump -p dbfile > db.dump vim db.dump db_load db.dump etc etc for pickle: import pickle :) Stupid, but hey :) > As a programmer, none of you will have to write the code to read or > write XML. I'll do all of that coding for you. In fact, I'm half > done. http://www.latte.ca/cgi-bin/source.py?name=file Great piece of work there :) > I did some tests with adding and removing data, and it was on the > order of 0.02 ms to call get, passing it a type of Storage.COMMENTS on > a document with three comments. I don't know if that scales yet, but > I've got a whack of sample data from Wari, so I'll run some more > tests, and see what happens. I am, however, cautiously optimistic. Go load a page with the harry potter porn comments :) You'll see how fast it can render 100 comments I've got other sample data with 516 entries and 1376 comments and growing :) -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: Wari W. <wa...@ce...> - 2003-02-10 07:46:41
|
* Kai Hendry <he...@cs...> [030210 03:04]: > On the subject of comments and what have you, have any of you had a > look at Vellum ? I really think vellum is to pyblosxom as MT is to blosxom. It's a different creature altogether, different focus here and there. So if I think MT is great, why do blosxom users use perl blosxom when MT is quite a complete package. Different audience here Kai. > I prefer to write my blogs with vim, i.e. not using a web ui. However > I hope Pyblosxom can take on some of the cool stuff Vellum has to > offer. For example, I have a friend who needs a Web UI to blog, I > will install Vellum for him. If you want pyblosxom with webui, it's there in contrib/weblog-add.py, you can mould it and shape it to whatever you like, heck I think you can use it to start a subproject, webui for blosxoms :) supports every known blosxom out there. -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: Wari W. <wa...@ce...> - 2003-02-10 11:28:12
|
* Blake Winton <bl...@ph...> [030209 22:36]: > which contained lines starting with "# comment" (or at least, it > should have...) and to have those lines filtered out would be a royal > pain. > How do we feel about '<!-- pyblosxom-meta comments="0" -->' I need to explain my entryparsers :) just wait. > In either case, my recommendation is "Version 2!" Hey, MT didn't > get around to adding it until version 2.6, so the way I figure > it, we've got 2 whole versions to go before we need to have it > on the agenda. :) Hehehe, you are one cool dude :) > My take on it is that you should plunge ahead (using the work that's > already been done, of course. Email me, and we can talk about which > pieces I can write for you). Let's get something, and then make it > work better, instead of trying to solve all our problems now, and > never coming up with anything. As the IETF motto says, "Rough > concensus and running code." I think we've got the rough concensus, > let's get the running code. I don't have anything to contribute here, you guys can go ahead, I'm dealing with 'infrastructure' stuff now which I'll explain in another email -- Regards: Wari Wahab Senior R&D Engineer Celestix Networks http://www.celestix.com/ |
|
From: Theodore W. L. <tw...@sa...> - 2003-02-10 00:10:09
|
Ok, I've committed changes that make lucene work in a plugin style way. I added a new kind of execute method to api.py, so that the executed handlers can return a list (in the case of lucene, a list of files). I = then added two new callback chains, one for CGI and another for the filelist. When I added the callback chain for the filelist, I created a default = plugin that contains the code for the standard blosxom file traversal. I'm not sure that I like this approach because it puts the plugin directory back = as required. Seems like it might be better to have pyblosxom.py contain = the code and somehow register it, but the plugin discovery mechanism (as it stands) precludes that. Please let me know your thoughts on this. I totally don't mind comments that say "this sucks". Part of the reason = I asked for feedback was that I didn't like it myself. =20 I'll be looking at comments next, but I need to do a little Java hacking = the rest of today. Ted > -----Original Message----- > From: pyb...@li...=20 > [mailto:pyb...@li...] On=20 > Behalf Of Wari Wahab > Sent: Sunday, February 09, 2003 7:02 AM > To: tw...@sa... > Cc: pyb...@li... > Subject: Re: [Pyblosxom-devel] Lucene and comments >=20 >=20 > Theodore W. Leung wrote: >=20 > >I'm back and caught up and ready to tackle a few things. > > > >#1. I'm trying to figure out what is the best way to integrate my=20 > >Lucene code.For the file list, we need to turn the regular directory=20 > >walk into the default behavior (last in the chain) and let=20 > each plugin=20 > >before that return a list. If any plugin in the chain=20 > returns a list,=20 > >then handler processing is done. > > > This was discuss @=20 > http://sourceforge.net/mailarchive/forum.php?thread_id=3D1613664 > &forum_id=3D24361=20 > but still there is no real solution yet. If you've got some ideas, or=20 > working code, we could try test it out and see if we can use=20 > it for all=20 > situations. >=20 > >#2. For comments I have the following issues > >We need a way to display a single article in a page, (so that we can=20 > >put the comment form up). This probably just means invoking a=20 > >different template for the single article case. > > > Yup. >=20 > >We (again) need a way to handle the comment CGI form post=20 > (probably via=20 > >the chain above). > > =20 > > > Again, the need for your point #1. :) To make life simpler, I'm=20 > suggesting a comment.cgi type of program to handle comments.=20 > We can make=20 > pyblosxom handle everything, but as feedback mechanism goes,=20 > you'll find=20 > trackback and pingback requiring xmlrpc mechanism to input=20 > feedback data. >=20 > Then you'll need some management of feedback data, especially=20 > deleting=20 > feedback spam if necessary (of course to know if you're=20 > spammed, to need=20 > to mail every feedbacks to the users). >=20 > I'm not sure what Blake have in mind, he would also like pyblosxom to=20 > handle POST and GET requests too. XMLRPC uses POST too, incompatible=20 > with GET requests and cgi module won't handle it. >=20 > >We need a way to display the comment / trackback / pingback count=20 > >(retreived from the storage api) in story.html (via a=20 > variable) -- so=20 > >every iteration through valid_list needs to be able to talk to the=20 > >storage api. This seems like another place for a callback chain. > > > Counts can be done by the standard load() plugins, iterating=20 > thru values=20 > of entryList at each call by using a standard incremental style of=20 > iteration. The plugin would call storageAPI to get relevant data of=20 > comment counts and what nots. >=20 > >I just wanted to get your thoughts on this before I plunge ahead. > > =20 > > > More things to consider, how do you tell pyblosxom that you want to=20 > disable comments on a particular entry? Would some meta data=20 > like: #comments 0 in an entry work in order to disable=20 > comments? Also, what about making=20 > comments inactive for a particular entry as well? In my other blog I=20 > have to delete one entry because it recieved 100s of stupid=20 > comments and=20 > I can't disable it in MT (MT will fix that in 2.6)? >=20 > Sorry if this mail sounds negative, I'mm all for feedbacks, I want it=20 > badly, but it does not seem to be an easy task with all these issue=20 > lying around. >=20 >=20 >=20 > ------------------------------------------------------- > This SF.NET email is sponsored by: > SourceForge Enterprise Edition + IBM + LinuxWorld =3D Something=20 > 2 See! http://www.vasoftware.com=20 > _______________________________________________ > Pyblosxom-devel mailing list Pyb...@li... > https://lists.sourceforge.net/lists/listinfo/pyblosxom-devel >=20 |
Thanks for helping keep SourceForge clean.
X