From: Kees B. <kee...@xs...> - 2007-02-23 21:10:21
|
Hi guys, Trying to read my GEDCOM in gramps23, just as a test, gave a few problems. First the Makefile.ams in svn are not up-to-date, so I had to tweak a few of them. Next, when reading my GEDCOM I get tracebacks. I never know which encoding to select, so until now I took the default. But with gramsp23 it gives this traceback. 23604: ERROR: DbLoader.py: line 504: Failed to import database. Traceback (most recent call last): File "/usr/local/share/gramps/DbLoader.py", line 500, in do_import importer(self.dbstate.db, filename, self.uistate.pulse_progressbar) File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 69, in importData import2(database, filename, callback, code_set, use_trans) File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 95, in import2 close = gedparse.parse_gedcom_file(use_trans) File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 1075, in parse_gedcom_file self.parse_header_head() File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 3953, in parse_header_head line = self.lexer.readline() File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 368, in readline self.readahead() File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 403, in readahead line = unicode(line) UnicodeDecodeError: 'utf8' codec can't decode bytes in position 26-28: invalid data In the old days (was it before gramps2?) there was always a sort of logging window with all the messages during GEDCOM import. Right now I have now idea which line causes that error. Can we get a linenumber in that message? -- Kees |
From: <rom...@ya...> - 2007-02-23 22:47:05
|
Hi, I have the same issue on some GEDCOMs (encodage = ANSI ?) ASCII for windows ... :( Seems Heredis-PAF-Parentele export ANSI GEDCOM :( Try GPL program (with wine ?) for windows (+ sources) : http://www.chez.com/ocado/convansel/convansel.zip Kees Bakker a écrit : > Hi guys, > > Trying to read my GEDCOM in gramps23, just as a test, gave a few problems. > > First the Makefile.ams in svn are not up-to-date, so I had to tweak a few > of them. > > Next, when reading my GEDCOM I get tracebacks. I never know which encoding > to select, so until now I took the default. But with gramsp23 it gives this > traceback. > > 23604: ERROR: DbLoader.py: line 504: Failed to import database. > Traceback (most recent call last): > File "/usr/local/share/gramps/DbLoader.py", line 500, in do_import > importer(self.dbstate.db, filename, self.uistate.pulse_progressbar) > File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 69, in importData > import2(database, filename, callback, code_set, use_trans) > File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 95, in import2 > close = gedparse.parse_gedcom_file(use_trans) > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 1075, in parse_gedcom_file > self.parse_header_head() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 3953, in parse_header_head > line = self.lexer.readline() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 368, in readline > self.readahead() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 403, in readahead > line = unicode(line) > UnicodeDecodeError: 'utf8' codec can't decode bytes in position 26-28: invalid data > > In the old days (was it before gramps2?) there was always a sort of logging window > with all the messages during GEDCOM import. Right now I have now idea which line > causes that error. Can we get a linenumber in that message? > -- > Kees > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > -- Jérôme Rapinat ___________________________________________________________________________ Yahoo! Mail rvente le mail ! Duvrez le nouveau Yahoo! Mail et son interface rlutionnaire. http://fr.mail.yahoo.com |
From: Don A. <don...@co...> - 2007-02-24 00:45:59
|
This is an crash message, not a syntax error. The problem here is that I'm working on improving the character set recognition, and I don't have every thing set correctly yet. The GEDCOM parser is a two pass parser, and it determines the character set on the first pass, and passes the information to the second pass so that the second pass can read the data correctly. Right now, the first pass is not passing the character set to the second pass yet. So any non-UTF8 characters that are found before the "CHAR" (character set) token in the GEDCOM file will cause a translation error. This will be corrected in the near future. Right now, I'm having issues of properly detecting UTF16, so the character sets are not fully working yet. However, if you don't have non-UTF8 characters in the first few lines of the file (which most files don't have), you should be okay. Typically this is a problem for GEDCOM files created by programs that have non-ASCII characters in their names or address (these lines appear before the CHAR line, so the character set is not set correctly). Personally, I find this to be a major flaw in GEDCOM. Don On Fri, 2007-02-23 at 22:10 +0100, Kees Bakker wrote: > Hi guys, >=20 > Trying to read my GEDCOM in gramps23, just as a test, gave a few problems= . >=20 > First the Makefile.ams in svn are not up-to-date, so I had to tweak a few > of them. >=20 > Next, when reading my GEDCOM I get tracebacks. I never know which encodin= g > to select, so until now I took the default. But with gramsp23 it gives th= is > traceback. >=20 > 23604: ERROR: DbLoader.py: line 504: Failed to import database. > Traceback (most recent call last): > File "/usr/local/share/gramps/DbLoader.py", line 500, in do_import > importer(self.dbstate.db, filename, self.uistate.pulse_progressbar) > File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 69, i= n importData > import2(database, filename, callback, code_set, use_trans) > File "/usr/local/share/gramps/GrampsDbUtils/_ReadGedcom.py", line 95, i= n import2 > close =3D gedparse.parse_gedcom_file(use_trans) > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 1075= , in parse_gedcom_file > self.parse_header_head() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomParse.py", line 3953= , in parse_header_head > line =3D self.lexer.readline() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 368, i= n readline > self.readahead() > File "/usr/local/share/gramps/GrampsDbUtils/_GedcomLex.py", line 403, i= n readahead > line =3D unicode(line) > UnicodeDecodeError: 'utf8' codec can't decode bytes in position 26-28: in= valid data >=20 > In the old days (was it before gramps2?) there was always a sort of loggi= ng window > with all the messages during GEDCOM import. Right now I have now idea whi= ch line > causes that error. Can we get a linenumber in that message? > -- > Kees >=20 > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share y= our > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Kees B. <kee...@xs...> - 2007-02-24 09:24:35
|
Op zaterdag 24 februari 2007 01:43, schreef Don Allingham: > This is an crash message, not a syntax error. The problem here is that > I'm working on improving the character set recognition, and I don't have > every thing set correctly yet. The GEDCOM parser is a two pass parser, > and it determines the character set on the first pass, and passes the > information to the second pass so that the second pass can read the data > correctly. >=20 > Right now, the first pass is not passing the character set to the second > pass yet. So any non-UTF8 characters that are found before the > "CHAR" (character set) token in the GEDCOM file will cause a translation > error. Ah, I see. But you (sort of) asked for testers of the GEDCOM import. That's why I tried. >=20 > This will be corrected in the near future. Right now, I'm having issues > of properly detecting UTF16, so the character sets are not fully working > yet. Linus Torvalds once said: "Sadly, when MS makes a whopper of a mistake (and they do it all too often), we're left having to work with the resulting breakage." UTF-16 is one of those mistakes. Anyway, support for reading GEDCOM in UTF-16 is needed to enable more people to switch to Gramps, right? But how much effort is making that worthwhile? >=20 > However, if you don't have non-UTF8 characters in the first few lines of > the file (which most files don't have), you should be okay. Typically > this is a problem for GEDCOM files created by programs that have > non-ASCII characters in their names or address (these lines appear > before the CHAR line, so the character set is not set correctly). That is probably it. My GEDCOM is created by ProGen, and the first lines ar= e: 0 HEAD 1 SOUR PRO-GEN 2 VERS 3.0b-p12 2 CORP PRO-GEN Genealogie =E1a la Carte 3 WWW www.pro-gen.nl 1 DEST PRO-GEN 1 DATE 23 FEB 2007 1 SUBM @S1@ 1 FILE BAKKER.GED 1 GEDC 2 VERS 5.5 2 FORM LINEAGE-LINKED 1 CHAR ANSEL 0 @S1@ SUBM 1 NAME A.C. Bakker The CORP line should in fact be "PRO-GEN Genealogie =E0 la Carte". I'll strip it for the time being, and see how far it goes. >=20 > Personally, I find this to be a major flaw in GEDCOM. >=20 > Don =2D- Kees |
From: Don A. <don...@co...> - 2007-02-24 16:42:37
|
On Sat, 2007-02-24 at 10:24 +0100, Kees Bakker wrote: >=20 > Ah, I see. But you (sort of) asked for testers of the GEDCOM import. That= 's > why I tried. And I appreciate it :-) > >=20 > > This will be corrected in the near future. Right now, I'm having issues > > of properly detecting UTF16, so the character sets are not fully workin= g > > yet. >=20 > Linus Torvalds once said: > "Sadly, when MS makes a whopper of a mistake (and they do it all too > often), we're left having to work with the resulting breakage." >=20 > UTF-16 is one of those mistakes. >=20 Yes, UTF-16 seems to be a big mistake to me. I can't see any advantage of UTF-16 over UTF-8. The problem with UTF-16 is that every character is mangled, so you can't parse a normal file. For example, in all other encodings, the first line should be: 0 HEAD However, under UTF-16, every character is padded with null characters to force a 2 byte character, so you can't look for "HEAD", but for something like "\000H\000E\000A\000D" This is something that we will be able to handle - it just takes a little time. If someone could generate some UTF-16 encoded files for me, this would help. Don |
From: Kees B. <kee...@xs...> - 2007-02-25 11:40:36
Attachments:
gramps23-Makefile.patch
|
Op vrijdag 23 februari 2007 22:10, schreef Kees Bakker: > Hi guys, > > Trying to read my GEDCOM in gramps23, just as a test, gave a few problems. > > First the Makefile.ams in svn are not up-to-date, so I had to tweak a few > of them. Some Makefiles were updated, but I still seem to need changes in the following: kees@spirit:~/src/gramps23> svn status M src/DataViews/Makefile.am M src/GrampsDb/Makefile.am M src/RelLib/Makefile.am M src/GrampsDbUtils/Makefile.am Any comment? -- Kees |
From: <rom...@ya...> - 2007-02-25 13:23:08
|
Kees, > Trying to read my GEDCOM in gramps23, just as a test, gave a few problems. to import GEDCOM, no ? :-D > Some Makefiles were updated, but I still seem to need changes in the following: > kees@spirit:~/src/gramps23> svn status > M src/DataViews/Makefile.am > M src/GrampsDb/Makefile.am > M src/RelLib/Makefile.am > M src/GrampsDbUtils/Makefile.am > > Any comment? On which SVN version ? I have a 'dirty' local gramps23 working copy !!! ./configure --prefix=/home/nono/gramps2.3 make make install I don't want to update gconf (I need to keep a safe GRAMPS-2.2.6 profil) but => GRAMPS launcher + some .py unreferenced modules not pasted yet. Maybe your patch will fix them for SVN 8231 (today, you need to add scr/GrampsDbUtils/GedcomChar.py) I just copy all the /src files into my /home/nono/gramps2.3/share It is like using $ python gramps.py but more user-friendly for us (localized local version with launcher) ;) Kees Bakker a écrit : > Op vrijdag 23 februari 2007 22:10, schreef Kees Bakker: >> Hi guys, >> >> Trying to read my GEDCOM in gramps23, just as a test, gave a few problems. >> >> First the Makefile.ams in svn are not up-to-date, so I had to tweak a few >> of them. > > Some Makefiles were updated, but I still seem to need changes in the following: > kees@spirit:~/src/gramps23> svn status > M src/DataViews/Makefile.am > M src/GrampsDb/Makefile.am > M src/RelLib/Makefile.am > M src/GrampsDbUtils/Makefile.am > > Any comment? > -- > Kees ___________________________________________________________________________ Yahoo! Mail rvente le mail ! Duvrez le nouveau Yahoo! Mail et son interface rlutionnaire. http://fr.mail.yahoo.com |
From: Kees B. <kee...@xs...> - 2007-02-25 14:01:09
|
Op zondag 25 februari 2007 14:27, schreef J=E9r=F4me: > Kees, >=20 > > Trying to read my GEDCOM in gramps23, just as a test, gave a few proble= ms. >=20 > to import GEDCOM, no ? :-D Qui. >=20 > > Some Makefiles were updated, but I still seem to need changes in the fo= llowing: > > kees@spirit:~/src/gramps23> svn status > > M src/DataViews/Makefile.am > > M src/GrampsDb/Makefile.am > > M src/RelLib/Makefile.am > > M src/GrampsDbUtils/Makefile.am > >=20 > > Any comment? >=20 > On which SVN version ? > I have a 'dirty' local gramps23 working copy !!! >=20 > ./configure --prefix=3D/home/nono/gramps2.3 > make > make install >=20 > I don't want to update gconf (I need to keep a safe GRAMPS-2.2.6 profil) Same here. But I use /usr/local for which I did a chown so that I can insta= ll as regular user. The gconf is a mystery to me, and I realize that it can ca= use problems running Gramps. >=20 > but =3D> GRAMPS launcher + some .py unreferenced modules not pasted yet. > Maybe your patch will fix them for SVN 8231 (today, you need to add=20 > scr/GrampsDbUtils/GedcomChar.py) >=20 > I just copy all the /src files into my /home/nono/gramps2.3/share > It is like using $ python gramps.py but more user-friendly for us=20 > (localized local version with launcher) ;) Well, yes, I know. But still the Makefile.am should be correct. That's why I mentioned it. =2D- Kees |
From: Don A. <don...@co...> - 2007-02-25 14:27:28
|
For the time being, things are changing quickly in SVN, so there are times when an install may not be fully functional. Until then, just run GRAMPS from the src directroy. $ cd src $ python gramps.py When things settle down, the install will be fixed. Don On Sun, 2007-02-25 at 12:40 +0100, Kees Bakker wrote: > Op vrijdag 23 februari 2007 22:10, schreef Kees Bakker: > > Hi guys, > >=20 > > Trying to read my GEDCOM in gramps23, just as a test, gave a few proble= ms. > >=20 > > First the Makefile.ams in svn are not up-to-date, so I had to tweak a f= ew > > of them. >=20 > Some Makefiles were updated, but I still seem to need changes in the foll= owing: > kees@spirit:~/src/gramps23> svn status > M src/DataViews/Makefile.am > M src/GrampsDb/Makefile.am > M src/RelLib/Makefile.am > M src/GrampsDbUtils/Makefile.am >=20 > Any comment? > -- > Kees > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share y= our > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV > _______________________________________________ Gramps-devel mailing list= Gra...@li... https://lists.sourceforge.net/lists/lis= tinfo/gramps-devel |
From: Kees B. <kee...@xs...> - 2007-02-25 14:50:21
|
Op zondag 25 februari 2007 15:24, schreef Don Allingham: > For the time being, things are changing quickly in SVN, so there are > times when an install may not be fully functional. > > Until then, just run GRAMPS from the src directroy. > > $ cd src > $ python gramps.py > > When things settle down, the install will be fixed. OK. no problem Here is a little fix for _GedcomUtils.py, otherwise it may crash reading certain GEDCOMs. kees@spirit:~/src/gramps23> svn diff src/GrampsDbUtils/_GedcomUtils.py Index: src/GrampsDbUtils/_GedcomUtils.py =================================================================== --- src/GrampsDbUtils/_GedcomUtils.py (revision 8231) +++ src/GrampsDbUtils/_GedcomUtils.py (working copy) @@ -143,7 +143,7 @@ def clean(self, gid): temp = gid.strip() - if temp[0] == '@' and temp[-1] == '@': + if len(temp) > 1 and temp[0] == '@' and temp[-1] == '@': temp = temp[1:-1] return temp -- Kees |
From: Don A. <don...@co...> - 2007-02-25 14:59:39
|
Thanks. On Sun, 2007-02-25 at 15:50 +0100, Kees Bakker wrote: > Op zondag 25 februari 2007 15:24, schreef Don Allingham: > > For the time being, things are changing quickly in SVN, so there are > > times when an install may not be fully functional. > >=20 > > Until then, just run GRAMPS from the src directroy. > >=20 > > $ cd src > > $ python gramps.py > >=20 > > When things settle down, the install will be fixed. >=20 > OK. no problem >=20 > Here is a little fix for _GedcomUtils.py, otherwise it may crash > reading certain GEDCOMs. >=20 > kees@spirit:~/src/gramps23> svn diff src/GrampsDbUtils/_GedcomUtils.py > Index: src/GrampsDbUtils/_GedcomUtils.py > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- src/GrampsDbUtils/_GedcomUtils.py (revision 8231) > +++ src/GrampsDbUtils/_GedcomUtils.py (working copy) > @@ -143,7 +143,7 @@ > =20 > def clean(self, gid): > temp =3D gid.strip() > - if temp[0] =3D=3D '@' and temp[-1] =3D=3D '@': > + if len(temp) > 1 and temp[0] =3D=3D '@' and temp[-1] =3D=3D '@': > temp =3D temp[1:-1] > return temp > =20 >=20 > -- > Kees >=20 > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share y= our > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |