From: Jim W. <spi...@us...> - 2006-02-18 17:24:38
|
<note> This was submitted to the list a while back and for some reason (probably a local issue) it did not go through. Here it is again with a few minor edits. </note> Does there continue to be any interest in making phpWebSite more portable in regards to databases? Originally I was drawn toward phpWebSite for its features and a statement somewhere that it would work with PostgreSQL. On closer examination I found that it did not, but read somewhere else that it would with minor modifications. As it turned out the modifications were minor, but involved changes on many lines of code. At this point I'm trying to decide if I should redo those changes against the latest CVS or if I should just move on and try and find a project that is more inclined to really support Postgres (and others). The decision for me really hinges on whether or not the project is willing to accept these modifications. I've been using a modified version phpWebSite on several sites for a couple years now with pgsql backends, backporting some security related patches as they came up. My goal is to run something more in line with phpWebSite releases that uses PostgreSQL. >From memory, the following problem areas needed to be addressed and were included in the modificatoins I made: 1). PostgreSQL folds SQL identifiers (table names, column names, index names, function names, etc) to lower case. This causes a problem mapping to case sensitive variables in phpWebSite that were coded with camel case. There seems to be a few ways to address the problem. One way (the one I have been using) is to make all the column names and corresponding php class variables lower case. Basically I added some underscore characters where CamelCase was previously used (e.g. "AdminEmail" becomes admin_email). The main problem with that approach is the fact that the SQL standard is to fold identifiers to UPPER case. So while this technique makes both PostgreSQL (that folds to lower) and MySQL (that doesn't fold at all) work, it would present problems for folks using Oracle or other systems. An alternative approach would be to quote all identifiers. This can bring some of its own issues (e.g. MySQL not in ansi mode). There are likely other ways of handling this issues, but I'm willing to implement a fix that makes sense to people. 2). The function Database::sqlGetIndex(tablename) is a problem because it it uses the "show index" command from MySQL. There is no easy fix here. We either have to restrict modules to always using a primary key column called "id" or have a way for modules to register primary key column names, either as a property in the data class (e.g. PHPWS_Item) or in some global location like a database table. In our local copy I have worked around this with a PostgreSQL method, but I would lean toward just changing all the modules so they use "id" for primary key and eliminating the use of sqlGetIndex in the standard modules. 3). The "Archive" function in phatform uses mysql specific utilities. Currently we are not using archive on our sites. The data dumps are automated daily and the forms so far are fairly static. In general users are discouraged from messing with form data and as a result they pretty much limit usage to getting web form submissions via email. In order to support multiple databases, we could probably just code recode this function for individual database dump utilties and simply return a message when the configured database isn't supported for the archive option. My best recollection is these were the only areas of trouble for getting PostgreSQL to work. My apologies for the long email, but I wanted to be specific about my intentions. Basically I'm volunteering to submit some work, but before proceeding I need to find out if the folks at Appalachian State feel that what I'm describing is compatible with the current goals the phpWebSite project. Best regards, Jim Wilson |
From: Greg M. <drk...@co...> - 2006-02-18 18:18:34
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Wilson wrote: > <note> > This was submitted to the list a while back and for some reason (probably a local issue) it did not go through. Here it is again with a few minor edits. > </note> > > Does there continue to be any interest in making phpWebSite more portable in regards to databases? > > Originally I was drawn toward phpWebSite for its features and a statement somewhere that it would work with PostgreSQL. On > closer examination I found that it did not, but read somewhere else that it would with minor modifications. As it turned out the > modifications were minor, but involved changes on many lines of code. The problem appears from developers using tools like phpmyadmin to extract a .sql file for a module. The tool is a great aid in the development process but it has some drawbacks. phpmyadmin tacks on quotes that make the create statement fail on postgresql. Postgresql and Oracle then treat these column and table names as mixed case. If the double quotes are removed, then they are created as uppercase in the database. You can send an undoublequoted table name like MyTableName and both Oracle and Postgresql will use MYTABLENAME and find the correct table. Moreover, these phpmyadmin scripts then create index clauses between the last column definition and the closing ); MySQl must create these in a non-standard way. If the index clauses are moved to table alter statements after the create table statement, then that makes the .sql file more portable. The other problem is that Postgresql and Oracle allow only 32 chars for a table name whereas mysql allows for 100 or so. So what is the SQL standard? Once you add a table prefix, and one for the module,mod_ to a long table name along with its supporting PEAR autonumber table, then the 32 char table name limit can be exceeded very quickly. If Oracle support is desired, then some of the phpWebSite tables create more than one text column where a smaller varchar column would work. Oracle converts these mysql text columns to their long column format. Oracle limits one long column per table because the LONG column supports 2gig+ in characters. Here's another post with examples http://sourceforge.net/mailarchive/forum.php?thread_id=7340026&forum_id=34704 . A 0.10.3 release is in the works. I'd think that your patches would be a good idea depending on how far the other developers want to take the 0.10.3 release. I think I have read in these emails that Matt is developing the next major release 1.0.0 or fallout on Postgresql. That in itself will provide your desired portability. However, third part module developers have to get the religion too. Again I think your patches would be great addition because the community would gain some experience with postgresql installs on 0.10.3+ before fallout is released. In addition, MySql may have added GIS support in 5.x. I don't recall. I think some slick business oriented phpWebSite modules could be developed with PostGIS http://postgis.refractions.net/ . GIS is not just for business. Some slick user modules could be developed too. At some point the PostGIS developers plan to submit PostGIS for OGIS www.opengis.org/ certification. Again, I'd say it would be a good idea but I do more documenting that programming on this project. What do the other developers feel? Greg -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFD92Txxyxe5L6mr7IRAhPWAJ9joxFO4enCoZQAlF3IEkLTWcUkRQCgmLRS g7HdrmXadfA5Y+RDmiq4CcY= =SFkK -----END PGP SIGNATURE----- |
From: Matthew M. <ma...@tu...> - 2006-02-20 15:20:30
|
> Does there continue to be any interest in making phpWebSite more portable in regards to databases? Yes but in 1.x only. I once believed that the pear db class would take care of the incompatibilities. How young and naive I was. It wasn't Pear's fault as that is not what the class was meant to do. Fallout (1.x) has been programmed and tested with both MySQL and PostgresSQL. What I have tried to do is make the installation sql as vanilla as possible. There are developer guidelines in place to prevent incompatibilities (table and column name character limits, no walking case, etc.). The database class in Fallout will also identify the DB OS and attempt to "fix" the query before it is put into the database. Once (if) people start using 1.x, I hope developers will identify what other databases need to work. -- Matthew McNaney Electronic Student Services Appalachian State University http://phpwebsite.appstate.edu |
From: Shaun M. <sh...@ae...> - 2006-02-21 10:35:36
|
It would also be useful if we could find out from the DB class which DB we're using so that we can automatically use features of MySQL that aren't in PostgreSQL for instance, and even which version. For instance I might want to use UPDATE LOW_PRIORITY for stats gathering which MySQL has but PostgreSQL doesn't. At the moment there's a config option in phpwsBB to select this. Shaun aegis design - http://www.aegisdesign.co.uk aegis hosting - http://www.aegishosting.co.uk |
From: Jim W. <spi...@us...> - 2006-02-21 14:30:23
|
> From: Matthew McNaney > > > Does there continue to be any interest in making phpWebSite more portable in regards to databases? > > Yes but in 1.x only. I once believed that the pear db class would take > care of the incompatibilities. How young and naive I was. It wasn't > Pear's fault as that is not what the class was meant to do. > Makes sense. Keep us up to date on progress then. Is 1.x another major rewrite compared to the current release(s)? How much module rework are we talking about? With my current time availabilty I'll need to wait until there's at least a chance of getting a maintainable site running with a few days effort before jumping in. Unfortunately, at the moment, this is only about producing results. Currently I am mostly working on a java/tapestry based project and cannot afford to put a lot of time into a side project that won't be usable for a year. Please let me know how things progress so that I can participate appropriately. Thanks, Jim Wilson |
From: Matthew M. <ma...@tu...> - 2006-02-21 18:41:59
|
On Tue, 2006-02-21 at 09:31 -0500, Jim Wilson wrote: > Is 1.x another major rewrite compared to the current release(s)? How much module rework are we talking about? Yes it is a major rewrite. It is over 3 years old. I am hoping the module rework won't be too bad. Modules I don't plan on rewriting for release are here: http://res.stddev.appstate.edu/cvs/conversions/ PhotoAlbum is the only one at this point. Here are the docs to convert an old module to work with the new core: http://res.stddev.appstate.edu/cvs/fallout/docs/Converting_Modules.txt?rev=1.3 The good news is that once you convert, your old module gets all the bells and whistles of the new core. I have rewritten all the core modules as well as announcements (now called Blog), Web Pages, Blocks, Fatcat (now just Categories), What's Related, and I am in the middle of Calendar. I have also written the conversion scripts to move the old data over: http://res.stddev.appstate.edu/cvs/fallout/convert/modules/ I am working on the User conversion but got buried in the foundation (i.e. users needs the alternate authorization to work properly, so I reinstalled and noticed a problem with permissions, which had to do with keys, which had to do with the Database class not performing joins well, but I digress). > Please > let me know how things progress so that I can participate appropriately. Thanks Jim. I know people have waited so long for 1.x that I can sense eyerolls every time I say it will be out "soon". That said, it is getting really close. There is still a lot of work to do but I feel a release will be coming soon. Developers out there can help me by testing, questioning, and writing documentation to fill in the blanks. Thanks, Matt -- Matthew McNaney Electronic Student Services Appalachian State University http://phpwebsite.appstate.edu |