Finally the time came when this talk has to be started. I have send the refreshing email to Joomla Development Team, to their core developers, on officially porting the Joomla CMS to CUBRID.
That said, I have a question to everybody familiar with Joomla. The latest stable release of Joomla is version 1.5.20 from 1.5.x family. As you know there is 1.6 coming soon with its 1.6 Beta 7 released yesterday on August 9th. The final stable release is expected sometime after September, 2010. As I am quite used to 1.5.x series, I was very surprised with huge improvements in 1.6. There are tons of modifications, huge improvement in user experience and overall system usage convenience. They say its at least 50% faster than Joomla 1.5.x, which I made certain of when I installed Joomla 1.6 beta.
Having said that, I have a question: which Joomla should we start with? The current stable and reliable 1.5, which has gazillion of plugins and components, or currently beta Joomla 1.6?
- It's in beta stage right now.
- But by the time we finish porting it will be definitely stable, so we do not need to be afraid of at all.
- By the time we finish, many major plugins and components will have provided 1.6 support. CommunityBuilder already provides 1.6 beta support, which will extend to 1.6 GA.
- If we provide 1.6, its official Joomla support will continue for couple more years for sure.
- Most users wait for 1.6 release, and they will move their current sites to the new system.
- 1.6 will require PHP 5.2+
- Several months later after 1.6 release Joomla Dev team plan to stop 1.5. support, so by the time when we finish it will become kind of out-fashioned with no official support.
- However, it has been proved by time that it is reliable and secure
- All plugins and components are compatible with 1.5.
So, what would be the best decision? It is important step. So, leave your comments below.
My vote goes for Joomla 1.6.
Let me get straight to the point…
1. First of all, after taking a quick look at the Joomla source code,
especially at the \libraries\joomla\database\… files,
maybe it should not really an issue which Joomla version we are coding
Cubrid support for…
Joomla uses it's own "database interface", which isolates the
interface/business logic from the db engine, so, again, I do not see (for
any issues with the Joomla version to port to.
2. Actually, I do see some possible issues with the Cubrid version…
Look at the code in there… We have things like:
return mysql_insert_id( $this->_resource );
$this->setQuery( 'SHOW CREATE table ' . $this->getEscaped( $tblval ) );
$this->setQuery( 'SHOW FIELDS FROM ' . $tblval );
…and so on…
And Cubrid has no support for any of these so far… But I know that there
are discussions to support them in the (immediate?) future….
3. However, there is an easy way to get the code working with Cubrid in one
Just use the Cubrid PHP compatibility library and that's it…:)
The library contains "workaround" for all the specific MySQL extensions
(LAST INSERT ID, SHOWs etc), so the above mentioned issues could be
addressed much more easier…
…Unless, of course, there are additional issues with the tables
structures/data types in Joomla (to investigate!)…
Otherwise, again, it would take at most one day to convert and that's it…!
Of course, on top of that, heavy testing is a definite thing to do, because,
I reiterate, there could be other incompabilities which might
not be as easy to "fix" as the MySQL PHP interface calls… (this would
include also executing "directly" MySQL-specific statements…)
I am very glad you have quite optimistic projections toward the Joomla porting. Glad to have you on board!
Here are couple of my comments. Personally, I do not think it is possible to get it done in one day, though, never afraid of trying. There reason is there are two things to alter: first the Database Tables, its data types, and you know there are over 30 tables in Joomla CMS. Second is the php source code. Therefore, I think it will take a bit more than you said. Anyways!
Another thing is, if we try to port Joomla with CUBRID 3.0, we should have even easier life. Take a look at http://wiki.cubrid.org/index.php/Port_MySQL_Compatible_PHP_Applications_to_CUBRID. This is that MySQL Compatibility Library customized for CUBRID 3.0 release. As you might already know CUBRID 3.0 has lots of new SQL extension that are just like in MySQL, so we can avoid changing many data types and SQL statements in Joomla code.
For the rest statements that are not implemented in CUBRID, I am sure we can find the workarounds.
Also I would like to share with everyone some of the useful links related to Joomla Database Structure.
To clear up with the Joomla Database Schema here is its illustration for the new 1.6 based on beta 2:
The original page for this Schema:
The Joomla Forum discussions page for 1.6 DB Schema:
Here is the DB Schema for 1.5 based on alpha 2:
And again the original page:
The Joomla Forum discussions page for 1.5 DB Schema:
Also, here is a link for Joomla Developers:
My opinion is that we should go for version 1.6.
There is no point to have a version with CUBRID that will not be officially supported by Joomla after few months.
Regarding the plugins, I think that will be changed in a very short time to support 1.6 as Joomla has a very active community.
This is an update message about porting Joomla. There is still a problem contacting with the Joomla Community. I haven't heard from them since my last message. Today, I will write to another mailing list. Hope to get their answer.
Meanwhile, I found very useful forum posts on how PostgreSQL and MSSQL tried to port Joomla, but seems like all of them failed. The reason is those porting was initiated by users, who finally gave up that hard task. But this is not our case. So, read these forums. There are useful information about MySQL specific SQL statements as well as porting hints.
PostgreSQL - Status? Can I Help?
SQL queries - too much MySQL. PostgreSQL is well possible
MSSQL for Joomla! 1.5
PDO database driver project
MSConnect Project Started
If anyone has some thoughts around, post them here. We have to keep discussing.
Based on the previous experience with similar projects, to get a better estimation of the effort from the start (and to avoid later effort-size estimation issues etc), I would suggest this approach:
- Enable full tracing of the MySQL statements
- Install Joomla
- Use Joomla; Go to to every functionality, operations etc. The scope would be to cover ~90% of the (MySQL) code.
- Analyze the SQL logs. Identify the MySQL specific statements, identify the issues and estimate.
The advantage here would be to get enough information from the beginning to understand what does it take (and how much) for the success of this project.
We want to identify the issues like the ones mentioned in the PostgreSQL/MSSQL threads below (CONCAT_WS, CREATE TABLE etc) as soon as possible.
I would like to ask you directly post your message on the forum. How do you think about it? Thus, the rest users could also keep track of all the conversation on one page.
Concerning your suggestion, I totally agree. Could you please proceed with the analysis and estimations? Roughly speaking we have to get it done until the end of this year by the time the Joomla 1.6 Stable is released. Meanwhile, I will continue my conversations with the Joomla team to get it officially posted together with their main project. In fact, I have already sent them a message.
Additionally, I have started creating a "JDatabaseCUBRID class" in "cubrid.php". This file represent the specific Joomla Database Connector for CUBRID, though does not mean it will solve the entire issue. Like Rienzi told, there are lots of places where MySQL specific SQL statements have been hard coded.
Now I am thinking of where we can have the code stored, which repository. There are several options.
One, more rational one, seems to be www.JoomlaCode.org.
Second, SF.net separate project.
Third, Github project.
Seems like JoomlaCode is more rational choice. If you agree, I will create an account there and upload the source. We will develop during the day, sync with the Repository and test locally.
Are there any more suggestions?
Concerning the CUBRID MySQL PHP Compatibility Library, I think we should not include it. We have to use the native CUBRID PHP API for this purpose. However, it is very useful just to refer to that Compatibility Library as it makes it easy to find the equivalents for particular MySQL PHP functions.
I think that if we are waiting for the community members to have a
version of Joomla that works with CUBRID this will not happen too soon.
So probably we should start working on it. We (Arnia) were thinking to
take care of porting Joomla to CUBRID. We will have an external (extra)
resource that will work on this project and it will not affect the
current schedule. This would be our contribution to the process of
increasing CUBRID's awareness.
Please let me know your opinion about this idea.
Good to go! Meanwhile I will be do my endeavor to get through Joomla Team.
Before that, please let me know if Rienzi is gong to join you? And where are you going to host the source? If locally, Rienzi cannot contribute.
Let me know the details of your plan and estimations by tomorrow.
Rienzi will probably help us with the analyze process and with the plan. Also he can help with some suggestions regarding different methods and workarounds necessary when porting PHP MYSQL applications to CUBRID.
Hosting of the source code will be on our SVN (that Rienzi can also access and you can also access it). It is a "public" SVN (visible from outside Arnia).
We will start the project on Monday and by the middle of next week we will also send you the plan and estimations. As Rienzi also said we need to have some time analyzing the application before start the actual development.
I have one question for you: is there any particular reason why you don't want to use MySQL Compatibility Library in porting Joomla? I my opinion this library will help us a lot to reduce the code size, the dev effort and in the future, when there will be updates/changes in CUBRID engine, we will have to modify the code only in one place.
As you understand Joomla is a strong hope to increase CUBRID awareness. There are so many users that almost no other project can provide CUBRID such awareness. So, I strongly hope you will start having the positive outcome already soon.
You can the development on your SVN. And I am very glad Rienzi is also on board.
Concerning the MySQL Compat. Lib. I'd rather not use it. Yes, I agree that at the beginning it will be very convenient to start with it. However, considering the further development I suggest you to use the native PHP functions. Here is the thing, you say when there are updates in CUBRID engine you will not need to modify the Joomla code but only MyCompatLib code. I do not think so. That CompatLib does not provide SQL conversion as well. However, throught the Joomla code you will need to change lots of SQL Statements to fit CUBRID rules. So, if CUBRID engine changes, you will need to modify the SQL statement in the Joomla code as well. If PHP API changes, like it did with 3.0 release, yes, we could change only MyCompatLib code. However, likewise, you can change Joomla JDatabaseCUBRID class in "cubrid.php", which is referred throughout the Joomla project. You have to modify majorly this class only. There rest place where DB is referred, the DB instance calles the functions of that JDatabaseCUBRID class. So, you have to look around and understand the project's structure. Another thing is, do you think, if we make it so that Joomla is officially accepts our changes, do you think they will be happy to see some external files like mysql_compat_library, blah, blah, blah. I do not think, they will allow this. They would not want to make structural changes in their project. I do not say I am completely right. So, tell me why I am not.
Since we have couple of days till you start the actual development, please, Catalin, Daniel, Rienzi participate in the active discussions now, expressing your opinion, so that we do not have to change our mind later.
Have a good day all of you!
Yesterday I took a closer look into Joomla and I have prepared some notes regarding the Cubrid porting.
Please take a look at the attached document and send your feedback/share your thoughts.
Summarizing, we should be able to fast start with the porting - I don't see any blocking issues so far.
As for the project effort estimation, maybe it's best first to complete the database structure porting…?
BLOB data type: At this point of time we can replace it temporarily with STRING. However, by the time we release the ported version of Joomla we will already have CUBRID 3.1 released (beginning of October), which will have new data types such as BLOB and CLOB. So, this issue will be gone as 3.1 rolls out.
There is one request: During the code analysis please pay attention if there is any SQL statement which required SORTING (ORDER BY). As you know we have some limitation in this matter, so it is best if we are sure no further issues can arise on this matter. Even if there are some places it requires ordered SELECTion, one solution is to SELECT and sort manually in PHP.
PRIMARY KEY: in CUBRID 3.0 we already support MySQL-like usage, so we won't have problems with it, I guess.
The rest notes by Rienzi are fine. So, below is possible scenario.
Port joo_* table schema with complete CUBRID support.
Create and port JDatabaseCUBRID and JTable classes.
Port SQL Statements
There is one more request. If it is possible let's have a table of all SQL statements in Joomla like:
| MySQL native SQL | CUBRID 3.0 Ported SQL | CUBRID 3.1 Ported SQL | CUBRID 3.2 Ported SQL |
| SELECT * FROM "joo_contens" | No Modification is required | …
This might help us in the future when CUBRID 3.2 rolls out with additional MySQL Compatible SQL extensions. We can easily look up which SQL needed modifications before, and see if with 3.2 they still need to be changed. The rest unchanged SQL can be thus skipped.
Any other suggestions are welcomed.
Esen/Catalin, let's do this:
1. I will take care next week of converting the databases structure/scripts; after that, in September, I'll finish all the the "installer" part.
2. The extra-resource mentioned by Catalin should start next developing the Cubrid specific classes.
By the middle of the next week, the structure will be ready and the development should be able to use the Joomla Cubrid db.
Further, in ~2 weeks from now, let's re-evaluate the project and finish up the estimations. By that time, we should have a pretty good idea of what it takes for this project to complete.
And let's start documenting, as suggested by Esen, the changes in SQL/DDL we will be doing for Cubrid compatibility.
I have also received an email from Catalin about the latest updates regarding Joomla porting. I should say I am glad the porting has been started.
Catalin, has mentioned that the development will be on their SVN http://220.127.116.11:8081/svnnhn/cubrid/cubrid-tools/joomla-Cubrid/branches/1.5.20/.
Based on the URL it seems like the porting has started for Joomla 1.5.x. Was that kind of mistyping and you are working on Joomla 1.6?
We have started porting using the latest stable release of Joomla, which is 1.5.20.
When 1.6 will be released (end of year?) then we will update it. But now there are only betas available for 1.6,
which still contain too many issues to deal with (if you take a look at the latest beta, no less then ~90 fixes were made in the last 2 weeks…)
We are taking care of fully documenting the entire process for adding Cubrid support for Joomla, so it won't be difficult to apply the changes to a future version.
Now, regarding the conversion itself, in addition to the db specific issues, here are some code issues we have encountered these days:
- zero DATE/DATETIME
- Insert ID
- CASE on non-logical expressions
- Implicit Casting/Convertions; this looks to be more difficult to deal with, due to a generic approach regarding data storage (using a generic procedure)
- Select DB
We will keep the thread updated with the progress and today we will send you also the effort estimation.
Thank you for your comments. Here are some more comments on CUBRID Engine related issues you are experiencing right now. Below is the list of additional issues from Catalin.
Catalin: "These items should be considered in the engine of CUBRID and would help us a lot the users to write CUBRID applications. Some of the items are ready, some of them in progress and other should be considere next:
a. (DROP TABLE) IF EXISTS
b. LAST INSERT ID
c. SHOW TABLES/INDEXES/COLUMNS/USERS/etc
d. FOREIGN KEYS support in Metadata
e. Data Types naming cleanup
g. STRING <->NUMBER automatic conversions/casting"
Regarding the data type casting CUBRID Dev team here in Korean together with the Romanian Dev team have already developed a Spec to address this issue in CUBRID 3.2 or 3.3 as a part of SQL Extension Phase II. So, in the future we will have most of the issues resolved. In the meantime, please find the work-around and apply them until we have a native solution.
The collaboration around the SQL Extension Specifications can be found here https://sourceforge.net/apps/trac/cubrid/.
Specifically the Type Conversion collaborations are here https://sourceforge.net/apps/trac/cubrid/wiki/SpecificationDrafts/TypeConversionInExpressionEvaluation?version=9.
You can find other interesting topics in the first link.
Porting of the Joomla to work with Cubrid database is under progress
* The front page is loading correctly together with article (clicking on article from fron page)
* You can view poll results, but voting is not functional yet
* HTTP sessions are correctly handled by the database backend.
* Users are able to login and logout
* ~30 queries were investigated and fixed appropriately
* Updated the document with modification on the Joomla code itself (aside from cubrid.php)
The above changes were commited into the repo.
I am glad to hear we are on the way to get the most famous CMS working with "the most famous DMBS"! Anyway, we'll make it come true, right!?
Please, don't forget to document every single change you make while porting. It will make our lives easier when updating/upgrading Joomla in the future.
Today I have ported around 100 SQL statements that were tested/adapted to work with Cubrid. These covers the administrator/ module and also I have started to work on libraries/ module.
Added the official Joomla functional tests into the trunk, but not ran them yet.
All commited changes are fully documented inside the document that covers the conversion. Also there are specified types of issues encountered while porting. I update the document before the commit to be sure that no change is missed.
Btw, one very easy way to test Joomla running on MySQL and compare it with the Cubrid version is to use an Uniform Server distribution already configured with Joomla & MySQL & Apache etc.
You can find it in here:
Hello Cristian and Rienzi!
I am glad the project is already moving toward something more feasible and soon testable version. I checked the Uniform Server with Joomla, and that looks very convenient. Thank you for that.
If there are any updates about Joomla (I heard this week we should have some working version), let me know!
Have a great day!
Unfortunately, the number of issues encountered porting Joomla to Cubrid seems to be increasing day by day…
As of last Friday, we counted:
- 200++ updates in Joomla code
- 150+ SQL quoting issues
- 25+ casting issues
and so on…
And this is only a part of the Joomla code and does not include the installer…
It become obvious that we need to explore other ways, because changing almost every SQL in Joomla is not an option.. :(
We had a meeting on Friday and Cristian is investigating now the possibility to change the query code "before" the execution,
based on some Cubrid mappings/rules.
He will post tomorrow the results of his investigation.
Overall, Cristian estimates an additional of 10-15 days required to fix all these issues with the incompatibilities between Cubrid and MySQL.
But there is no easy way to tell how many more SQL needs to be changed for Cubrid, we are taking them one by one…
We'll keep you posted.
Soon in couple of days we are planning to launch a "HELP WANTED" campaign on cubrid.org to attract the contributors to help us with "easy" tasks like:
1. Joomla porting: at this moment, you have encountered much difficulties due to the number of manual SQL changes required. I suppose there are many issues that are quite easy but just takes time to complete. So, we could ask users to help us with these tasks.
2. CUBRID page translation on Wikipedia: This is very easy, just requires some time. We can ask them translate the page from English to their languages so that more users have a chance to learn about CUBRID.
If you have in your mind other "easy" tasks let's think about them. I think we would have more chance to attract them if the task does not require too much time and expertize.
For this purpose please prepare a list of items/issues which can be delegated to the users so that they can quickly join and start fixing what is required. Along with the issues please prepare what is required to complete a particular task. You do not need to create a whole document. We can do that gradually. At this time we need to provide the quick tutorial to get involved, and a list of issues need to be solved. I will also join and see how much "easy" is to interact with the Joomla-porting team, and try to provide everything to make their life easier.
If you have additional ideas, let me know.
Sign up for the SourceForge newsletter:
You seem to have CSS turned off.
Please don't fill out this field.