Modified History too long Error

Hi,

   We modified the History Layout by adding a few fields and the following error shows up when saving.
ERROR: field 'Other_Cancer'  is too long:
Maternal Aunt:|Maternal Grandmother:|…and other options from the assigned list.
Data Type is Textbox List.

I just took a look and noticed that size is set to 0.  Should it be something else?

Thanks

Ubuntu 11
OEMR 4.0

Well that did not work.  But here is what I found in the Errror Log.

   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/patient_file/report/patient_report.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/patient_file/report/custom_report.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php
   PHP Notice:  Use of undefined constant relatives_cancer - assumed 'relatives_cancer' in /var/www/openemr/library/report.inc on line 65, referer: http://10.0.0.72/openemr/interface/main/left_nav.php

Thanks

Hi,
Errors look benign and the size should be ok at 0, What does the structure of the mysql history_data table look like?
-brady
www.open-emr.org

No data gets stored.

Here is the Structure
`history_data` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `coffee` longtext,
  `tobacco` longtext,
  `alcohol` longtext,
  `sleep_patterns` longtext,
  `exercise_patterns` longtext,
  `seatbelt_use` longtext,
  `counseling` longtext,
  `hazardous_activities` longtext,
  `recreational_drugs` longtext,
  `last_breast_exam` varchar(255) DEFAULT NULL,
  `last_mammogram` varchar(255) DEFAULT NULL,
  `last_gynocological_exam` varchar(255) DEFAULT NULL,
  `last_rectal_exam` varchar(255) DEFAULT NULL,
  `last_prostate_exam` varchar(255) DEFAULT NULL,
  `last_physical_exam` varchar(255) DEFAULT NULL,
  `last_sigmoidoscopy_colonoscopy` varchar(255) DEFAULT NULL,
  `last_ecg` varchar(255) DEFAULT NULL,
  `last_cardiac_echo` varchar(255) DEFAULT NULL,
  `last_retinal` varchar(255) DEFAULT NULL,
  `last_fluvax` varchar(255) DEFAULT NULL,
  `last_pneuvax` varchar(255) DEFAULT NULL,
  `last_ldl` varchar(255) DEFAULT NULL,
  `last_hemoglobin` varchar(255) DEFAULT NULL,
  `last_psa` varchar(255) DEFAULT NULL,
  `last_exam_results` varchar(255) DEFAULT NULL,
  `relatives_cancer` longtext,
  `relatives_tuberculosis` longtext,
  `relatives_diabetes` longtext,
  `relatives_high_blood_pressure` longtext,
  `relatives_heart_problems` longtext,
  `relatives_stroke` longtext,
  `relatives_epilepsy` longtext,
  `relatives_mental_illness` longtext,
  `relatives_suicide` longtext,
  `cataract_surgery` datetime DEFAULT NULL,
  `tonsillectomy` datetime DEFAULT NULL,
  `cholecystestomy` datetime DEFAULT NULL,
  `heart_surgery` datetime DEFAULT NULL,
  `hysterectomy` datetime DEFAULT NULL,
  `hernia_repair` datetime DEFAULT NULL,
  `hip_replacement` datetime DEFAULT NULL,
  `knee_replacement` datetime DEFAULT NULL,
  `appendectomy` datetime DEFAULT NULL,
  `date` datetime DEFAULT NULL,
  `pid` bigint(20) NOT NULL DEFAULT '0',
  `name_1` varchar(255) DEFAULT NULL,
  `value_1` varchar(255) DEFAULT NULL,
  `name_2` varchar(255) DEFAULT NULL,
  `value_2` varchar(255) DEFAULT NULL,
  `additional_history` text,
  `exams` text NOT NULL,
  `usertext11` varchar(255) NOT NULL DEFAULT '',
  `usertext12` varchar(255) NOT NULL DEFAULT '',
  `usertext13` varchar(255) NOT NULL DEFAULT '',
  `usertext14` varchar(255) NOT NULL DEFAULT '',
  `usertext15` varchar(255) NOT NULL DEFAULT '',
  `usertext16` varchar(255) NOT NULL DEFAULT '',
  `usertext17` varchar(255) NOT NULL DEFAULT '',
  `usertext18` varchar(255) NOT NULL DEFAULT '',
  `usertext19` varchar(255) NOT NULL DEFAULT '',
  `usertext20` varchar(255) NOT NULL DEFAULT '',
  `usertext21` varchar(255) NOT NULL DEFAULT '',
  `usertext22` varchar(255) NOT NULL DEFAULT '',
  `usertext23` varchar(255) NOT NULL DEFAULT '',
  `usertext24` varchar(255) NOT NULL DEFAULT '',
  `usertext25` varchar(255) NOT NULL DEFAULT '',
  `usertext26` varchar(255) NOT NULL DEFAULT '',
  `usertext27` varchar(255) NOT NULL DEFAULT '',
  `usertext28` varchar(255) NOT NULL DEFAULT '',
  `usertext29` varchar(255) NOT NULL DEFAULT '',
  `usertext30` varchar(255) NOT NULL DEFAULT '',
  `userdate11` date DEFAULT NULL,
  `userdate12` date DEFAULT NULL,
  `userdate13` date DEFAULT NULL,
  `userdate14` date DEFAULT NULL,
  `userdate15` date DEFAULT NULL,
  `breast_cancer` varchar(255) DEFAULT NULL,
  `ovarian_cancer` varchar(255) DEFAULT NULL,
  `Menarche` varchar(255) DEFAULT NULL,
  `First_Pregnancy` varchar(255) DEFAULT NULL,
  `Gesta` varchar(255) DEFAULT NULL,
  `LMP` varchar(255) DEFAULT NULL,
  `Para` varchar(255) DEFAULT NULL,
  `Breast_Feeding` varchar(255) DEFAULT NULL,
  `C_Section` varchar(255) DEFAULT NULL,
  `Bra_Size` varchar(255) DEFAULT NULL,
  `Abortion` varchar(255) DEFAULT NULL,
  `Endometrial_Cancer` varchar(255) DEFAULT NULL,
  `Other_Cancer` varchar(255) DEFAULT NULL,
  `rfother` varchar(255) DEFAULT NULL,
  `abnmamo` varchar(255) DEFAULT NULL,
  `abnsono` varchar(255) DEFAULT NULL,
  `abnbrstmri` varchar(255) DEFAULT NULL,
  `breastpain` varchar(255) DEFAULT NULL,
  `breastmass` varchar(255) DEFAULT NULL,
  `nippledis` varchar(255) DEFAULT NULL,
  `rtinbrstexam` varchar(255) DEFAULT NULL,
  `famdxbc` varchar(255) DEFAULT NULL,
  `prsnldxbrstex` varchar(255) DEFAULT NULL,
  `coments` varchar(255) DEFAULT NULL,
  `piother` varchar(255) DEFAULT NULL,

Thanks

Hi,

This is coming from following code block in library/options.inc.php:

  // Better to die than to silently truncate data!
  if ($maxlength && ($data_type != 3 && $data_type != 34) && strlen($value) > $maxlength)
    die(htmlspecialchars( xl('ERROR: Field') . " '$field_id' " . xl('is too long'), ENT_NOQUOTES) .
    ":<br />&nbsp;<br />".htmlspecialchars( $value, ENT_NOQUOTES));

For the datatype (22), the $maxlength comes from the 'max_length' field in the layout_options sql row for this entry. Note if it is set to 0, then it should be 255. I may be mistaken, but I don't think the max_length is actually set in the layout editor, so it should be 255. Are you going over 255 characters?

-brady
www.open-emr.org

Hi,
If you are indeed breaking 255 characters, then we should consider going through the datatypes and creating a TEXT mysql column for the ones that have potential to break this limit.
-brady
www.open-emr.org

Hi,

I am getting that same type of error any solution yet? I added few items to the exam list and when trying to write a note in the note field I get this the error below, I run 4.1 (2) on windows 7.  Thanks. Mark
ERROR: Field 'exams' is too long:

brs:0:|cec:0:|ecg:2:,,,11-10 |gyn:0:|mam:0:|phy:1:11-10|pro:2:11-10 (E)|rec:1:11-10(-)|sic:0:|ret:0:|flu:0:DECLINED 2010-01-05|pne:0:|ldl:2:11-10 (114),,, 01-10 (113)|hem:1:11-10,,, 01-10|psa:1:4-11(3.9),1-11(4.1),10-10 (5.7),,, 01-10 (4.5)|fobt:0:|bmd:0:|tsco:0:|hgac:0:|bun:0:|gfr:0:|microalbum:0:|tg:0:|spiro:0:

Hi,

This bug has not been fixed. The problem is that those mysql fields only hold 255 characters usually (unless set to TEXT), which is why the above code checks it, otherwise it would truncate the field (ie. lose/corrupt data) silently. I think the following needs to be done to safely deal with this bug(codebase changes).
1) In the new items that have a chance of being bigger than 255 characters, have them create a TEXT mysql field (then can have thousands of characters)
2) In the validation check above, rather than check based on the datatype, figure out if there is a way to check the maximum size of the column through mysql (not sure if this can be done). This is important, because, solution in step 1 above will only fix it for users that create new columns; we still need to support/validate the older smaller columns.

Hope this makes some sense.
-brady
www.open-emr.org

Hi,
   Here is a quick easy fix which worked for me.
Go to Administration->lists->Exams and shorten  the names in the ID column.   Try and get the it to well below 255 so that it is useable.

Alfonso

This issue was fixed for LBF forms a while back, by changing the lbf_data.field_value type to "text".  So what I'd suggest is having the layout management logic use that type when creating new fields that aggregate data… I think that is also what Brady suggested.

Rod
www.sunsetsystems.com

Rod,

Can you please clarify, which field "the lbf_data.field_value type to "text". "  and how I can make "the management logic use that type when creating new fields that aggregate data".

Thanks

Mark

Rod,

But won't these datafields with the LBF still show the error? Although they have been changed to TEXT mysql fields, the validation will still break it, correct? I think this is what makes this issues a bit sticky. Even if we were to have all new elements created via the layout editor TEXT (like the LBF currently is), we are still stuck with the issue of ensuring the current and previously created elements via the layout editor do not break(ie. silently lose data if try to insert more data thatn the mysql column can hold). Thinking that instead of going by the datatype, maybe it's best to do something like here for each entry to ensure the user does not try to insert something over the size limit of the mysql column:
http://php.net/manual/en/function.mysql-field-type.php

-brady
www.open-emr.org

More to add from previous post,

Actually, if we just make a wrapper for the http://php.net/manual/en/function.mysql-field-len.php function in library/sql.inc which could takes as input the table and column name(and returns the maximum length of the field), should be able to work something in that checks this instead for validation on each data field for the insert. Then if the error gets thrown, it's easy to fix (user increases the size of the field in mysql). Something to look out for here is that it appears would need to do a correction if the field is in UTF8 (divide the length by 3).

-brady

Brady, what I had in mind (and neglected to state) for the rest of the solution was adding max_length to the layout editor. It is also desirable as the row count for multi-line text fields. Additional checking against the mysql maximum length does seem like a good idea.
Rod
www.sunsetsystems.com

bump,
Hoping to get this bug fixed before the next release:
http://open-emr.org/wiki/index.php/Active_Projects#Datafield_Max_Length_Bug
-brady
OpenEMR