[My question is in relation to WikiNDX4. Yes it's old and I'm working on migrating to the current version.] (I noticed that version, 6.3.4 does not have this issue.)
When editing my notes, I've just noticed that some of my math symbols aren't being saved correctly and TinyMCE is the culprit.
I am able to edit the _resource_text.resourcetextNote field directly using phpmyadmin and insert the "element of set" symbol '∈', and when I view the resource summary, the "element of set" symbol is displayed.
However, after editing my notes, the "element of set" symbol is replaced with a question mark when I save..
I've gone through the source and changed any references to entity_encoding and made sure they are "raw" or default to "raw" instead of "named". But it's still not working. The info I've found concerning the general problem through google hasn't helped me any further.
So I'm wondering if anyone here knows how this can be fixed?
Cheers, Nap.
Last edit: Nap 2020-07-25
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TinyMCE is fine but WKX4 is not UTF-8 safe (PHP functions used and MySQL shcema). There are no way to improve that with small changes. This take me several versions for a full support.
If you really want it you need to:
Set the size of all text field to the previous size x 4 (changes sometimes the type for a longer text field)
Change the type, remove or rebuild some index with a shortest size when the previous fields are two large.
Change the collation of the database and all text fields to utf8mb4_unicode_520_ci
In the SQL class, set the collation to utf8mb4_unicode_520_ci.
Change or replace all UTF-8 unsafe functions of PHP by a safe multibytes (mb string functions) equivalent or code you own functions is some cases.
Change all regexp to use trh preg family function with the u parameter.
I must have forgotten something but that's the biggest part.
Regards,
Stephane
Last edit: Stéphane Aulery 2020-07-25
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks Stephane, you did a good job of persuading me to migrate faster.
I've actually done the DB part already, though I'm using utf8mb4_general_ci.
I found the utf8mb4_general_ci character set useful when I was playing around with Google Scholar's results for all kinds of languages. Maybe I just didn't get to utf8mb4_unicode_520_ci because I found this one first.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
utf8mb4_general_ci is just for sorting. If you had not change the size of fields, don't do it now, because this will break the update script and prevent any easy migration.
utf8mb4_unicode_520_ci is more general than utf8mb4_general_ci despeate his name. utf8mb4_general_ci is an old collation. utf8mb4_unicode_520_ci is the last implementation from MySQL of the Unicode sort algo than work well with most language.
Do you know how to migrate? Why are you stuck with WKX4? Some pronblem?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you Stephane for the info.
WRT the DB, I only changed the encoding, not the size. I exported the SQL before I did it, so I'll revert to the original collation before proceeding with the migration.
Regarding the migration, I was putting it off until later (would be more convenient) but since I noticed this problem in the old version and the fact it doesn't exist in the new version I've decided now is the time.
I've just gone through the archive section of the files and found the 4.2.2 version I'm using along with its plugins. Since you asked about this, I do have a question.
In the plugins section, there are three there that may be relevant to migration importexportbib, backupMysql, and repairkit. If you could indicate which would be the appropriate to use?
(I am also able to export the SQL from the DB if that's a viable alternate procedure.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You can change safely th ecollation, not the size.
importexportbib will not help you backupMysql is the mysqldump for those you don't. I think for a yet proper tools. repairkit this plugin can repair a schema that was partially migrated by mistake.
Backup everything (code, data, db)
Push the code of 6.0.8 on the FTP
Use 6.0.8 to upgrade.
If all is fine, do it again with the very last version.
Since this is a big jump, after that use the repairkit to check if the db integrity is fine. Eventually repair it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I migrated from 4.2.2 to 5.1 to 6.0.8 to 6.3.4 and finally to 6.3.7.
For each version step, I exported the SQL, created a new virtual host, created a new DB, imported the SQL, copied the new version code, adjusted the config.php, and went through the prompted upgrade steps.
(This approach isn't really necessary but it it was a convenient way of keeping my existing version as a backup.)
I didn't need to install any plugins, and it all went rather smoothly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
[My question is in relation to WikiNDX4. Yes it's old and I'm working on migrating to the current version.]
(I noticed that version, 6.3.4 does not have this issue.)
When editing my notes, I've just noticed that some of my math symbols aren't being saved correctly and TinyMCE is the culprit.
I am able to edit the
_resource_text.resourcetextNotefield directly using phpmyadmin and insert the "element of set" symbol '∈', and when I view the resource summary, the "element of set" symbol is displayed.However, after editing my notes, the "element of set" symbol is replaced with a question mark when I save..
I've gone through the source and changed any references to
entity_encodingand made sure they are "raw" or default to "raw" instead of "named". But it's still not working. The info I've found concerning the general problem through google hasn't helped me any further.So I'm wondering if anyone here knows how this can be fixed?
Cheers, Nap.
Last edit: Nap 2020-07-25
Hi Nap,
TinyMCE is fine but WKX4 is not UTF-8 safe (PHP functions used and MySQL shcema). There are no way to improve that with small changes. This take me several versions for a full support.
If you really want it you need to:
I must have forgotten something but that's the biggest part.
Regards,
Stephane
Last edit: Stéphane Aulery 2020-07-25
Thanks Stephane, you did a good job of persuading me to migrate faster.
I've actually done the DB part already, though I'm using
utf8mb4_general_ci.I found the
utf8mb4_general_cicharacter set useful when I was playing around with Google Scholar's results for all kinds of languages. Maybe I just didn't get toutf8mb4_unicode_520_cibecause I found this one first.utf8mb4_general_ciis just for sorting. If you had not change the size of fields, don't do it now, because this will break the update script and prevent any easy migration.utf8mb4_unicode_520_ciis more general thanutf8mb4_general_cidespeate his name.utf8mb4_general_ciis an old collation.utf8mb4_unicode_520_ciis the last implementation from MySQL of the Unicode sort algo than work well with most language.Do you know how to migrate? Why are you stuck with WKX4? Some pronblem?
Thank you Stephane for the info.
WRT the DB, I only changed the encoding, not the size. I exported the SQL before I did it, so I'll revert to the original collation before proceeding with the migration.
Regarding the migration, I was putting it off until later (would be more convenient) but since I noticed this problem in the old version and the fact it doesn't exist in the new version I've decided now is the time.
I'm following Mark's advice in my earlier question: https://sourceforge.net/p/wikindx/discussion/326884/thread/98bc1cc7b4/ and have installed 6.0.8. I'm not working out how to get the data out of 4.2.2 and into this version.
I've just gone through the archive section of the files and found the 4.2.2 version I'm using along with its plugins. Since you asked about this, I do have a question.
In the plugins section, there are three there that may be relevant to migration
importexportbib,backupMysql, andrepairkit. If you could indicate which would be the appropriate to use?(I am also able to export the SQL from the DB if that's a viable alternate procedure.)
You can change safely th ecollation, not the size.
importexportbibwill not help youbackupMysqlis the mysqldump for those you don't. I think for a yet proper tools.repairkitthis plugin can repair a schema that was partially migrated by mistake.Backup everything (code, data, db)
Push the code of 6.0.8 on the FTP
Use 6.0.8 to upgrade.
If all is fine, do it again with the very last version.
Since this is a big jump, after that use the
repairkitto check if the db integrity is fine. Eventually repair it.Thanks Stephane. Here's how I did it.
I migrated from 4.2.2 to 5.1 to 6.0.8 to 6.3.4 and finally to 6.3.7.
For each version step, I exported the SQL, created a new virtual host, created a new DB, imported the SQL, copied the new version code, adjusted the config.php, and went through the prompted upgrade steps.
(This approach isn't really necessary but it it was a convenient way of keeping my existing version as a backup.)
I didn't need to install any plugins, and it all went rather smoothly.
Congratulations! You may be the first to do this through all three versions at the same time.
No alert, no problem, at any time?
Keep your backups for a few days / weeks. You never know if in use you discover a problem.