#77 1.6 B2 CVS Line endings in the Database

[Im]

Hi,

You are impressivly fast! So I give you one more :)

In the database, in this case the mm_gedrec, the info is stored in different ways.
Some times with the hexa decimal x0D and sometimes without it. Please see attached picture.
On the pictur you could see, how different the line ending are.
This is after the GEDCOM was imported so no editing has been done yet.

I don't know what should it be. w/wo x0D line ending

This is just mm_gedrec, but this can be found else whare too.

For all the bugs... thanks
-Im

[Im]

mm_gedrec line endings

[Boudewijn Sjouke]

Logged In: YES
user_id=816380
Originator: NO

Well, thank you :-P

IMO the gedcom lines must be consistent, and all with x0D (I guess the \n) present.
I have no idea where to start here. Can you please check if the original gedcom has all x0D line endings present?

Boudewijn

[Im]

Logged In: YES
user_id=1262354
Originator: YES

Well, all the line endings in the original GEDCOM is \r\n. In the hex world that's: x0D, x0A

But I see some pathern. The line endings are missing from those which is on the LAST LINE in the INDI's record.
So, the @M9@ is missing it becouse that is the last line:
0 INDI
bla bla bla...
1 OBJE @M9@

In the DB this has line ending x0D
0 INDI
bla bla bla...
1 OBJE @M8@
bla bla bla...

because the object is NOT on the last line.

Hmmmm... I hope! Does this do the trick?
-Im

[Boudewijn Sjouke]

Logged In: YES
user_id=816380
Originator: NO

Yes, this brings us some steps further.
First of all, this doesn't seem to affect Genmod in its functions, I suspect the half-missing line ends are missing for a long time. But it's not very neat.
I know that during import all subrecords are checked for line endings, but after that the complete record is trimmed. That would explain why the \r is absent, but not why the \n is present.....

One more question to isolate the problem some more: in records where NOT a MM-link is the last subrecord, how does the line ending of the gedcom record in the database look like? Sorry I ask you to have a look, but my admin tool for MySQL doesn't show me line endings :-(

Boudewijn

Thanks,
Boudewijn

[Im]

Logged In: YES
user_id=1262354
Originator: YES

Hi,

Let me put it in this way. When using mysql command:
SELECT i_gedcom FROM gmcvs.gm_individuals WHERE i_gedcom LIKE '%OBJE%';
All line ending has x0D, x0A. No metter where the OBJE is, last line or not last line.

With this, I get 0 results, which is correct:
SELECT i_gedcom FROM gmcvs.gm_individuals WHERE i_gedcom LIKE '%OBJE%@';

Just to make things complicated... with this command:
SELECT i_gedcom FROM gmcvs.gm_individuals WHERE i_gedcom NOT LIKE '%OBJE%';
There are no line endings in the result

MySQL has it's own GUI package for both Windows and Linux, which I use. The program where I can see what's inside the DB (including the binary view as shown on the uploaded picture) is called MySQL Query Browser. In case you wonder...

I hope I undersod your question, and this is the answer you where looking for.

Regards,
-Im

[Boudewijn Sjouke]

Logged In: YES
user_id=816380
Originator: NO

Hi Im,

I removed some trimming from the code, but the x0A is still missing at some places.
There are many places where the gedcom tags are filtered, checked and modified so I am afraid if I strip more code, I will break something.
Without the x0A Genmod seems to function well, so I propose to leave it for this moment.

Btw how do you get the line ends showing in the Query Browser? I only see them in the hex window, so I assume I have some options set wrong :-(

Boudewijn

[Im]

Logged In: YES
user_id=1262354
Originator: YES

Thanks for your time. This one is not easy to fix, whitch is why I submited this as the last one :)

About the hex...
x0A = \n -> Line Feed
x0D = \r -> Carriege Return
So, it's not the x0A (\n) that is missing as you wrote.

This inconsistency applies to others too: sm_gedrec, s_name, f_gedcom, i_gedcom
These ones I have checked.

I can give you some idea to check your own DB. Here is my status where you can have some idea what I have:
SELECT * FROM gmcvs.gm_media_mapping;
268 rows
SELECT * FROM gmcvs.gm_media_mapping WHERE mm_gedrec LIKE '%\\\r';
192 rows, line endings with \r
SELECT * FROM gmcvs.gm_media_mapping WHERE mm_gedrec LIKE '%\\\n';
0 rows, line endings with \n
SELECT * FROM gmcvs.gm_media_mapping WHERE mm_gedrec LIKE '%@';
76 rows, line endings with @
SELECT * FROM gmcvs.gm_media_mapping WHERE mm_gedrec NOT LIKE '%\\\r';
76 rows, line ending with not \r

SELECT * FROM gmcvs.gm_individuals;
675 rows
SELECT * FROM gmcvs.gm_individuals WHERE i_gedcom LIKE '%OBJE%';
115 rows
SELECT * FROM gmcvs.gm_individuals WHERE i_gedcom LIKE '%OBJE%\\\r\\\n';
115 rows
SELECT * FROM gmcvs.gm_individuals WHERE i_gedcom NOT LIKE '%OBJE%' AND i_gedcom LIKE '%\\\n';
0 rows
SELECT * FROM gmcvs.gm_individuals WHERE i_gedcom NOT LIKE '%OBJE%' AND i_gedcom LIKE '%\\\r';
0 rows

It's not often you are placing something into the DB. Most of the time, you are just retriving the info.
If the DB would be ok, you would eliminate the code for checking, striping, altering the info you retrive, because it's already ready to use.
I just try to cheer you up! :)

I don't change anything in the Query Browser. But I have uploaded a zip file with an image how I can 'see' the binary code and a possible solution? And also a htm file where I found some \r" in the source.

Cheers and thanks!
-Im
File Added: source.zip

[Im]

Binary question and source