From: Peter L. <pet...@te...> - 2010-03-26 19:11:15
|
Do you have Swedish locale installed? /Peter > Strange indeed ! > > No more "ascii" message, it seems to be related to my session ! > (Now error on testing nl_BE.utf8 !!!) > > > But, under Ubuntu 9.04 : > > LANG=sv_SV.utf8 python date_test.py > sv > > (process:3276): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > > RESULT: > slash-dates: 144 dates ok, 36 failed. > partial date: 372 dates ok, 60 failed. > Non-gregorian: 892 dates ok, 20 failed. > B. C. E.: 96 dates ok, 12 failed. > basic test: 5010 dates ok, 1056 failed. > > > Under OpenSuse 10.3 : > > LANG=sv_SV.utf8 python date_test.py > sv > > (date_test.py:4430): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > > slash-dates: 144 dates ok, 36 failed. > partial date: 372 dates ok, 60 failed. > Non-gregorian: 894 dates ok, 18 failed. > B. C. E.: 96 dates ok, 12 failed. > basic test: 5238 dates ok, 828 failed. > > Maybe failures are only on short months (in this case) because I do not > have swedish locale installed ? GrampsLocale.month_to_int > > Yes, test into Gramps has confirmed that date like "1 déc 2009" was parsed > as text. Some months with 3/4 letters (mai, juin) were the same (short = > long) and they were parsed. After change on _Date_fr.py, all short months > are parsed as DateObject. > > Date_pt and Date_sv do not set month_to_int > => GrampsLocale.month_to_int > > Date_fr uses some alternates names and translators could have set different > short month names than system. Sorry, but French people cannot use JUI for > the 6th and 7th months and JUN is not very friendly. Short months are not > only three letters and Ubuntu 9.04 and OpenSuse 10.3 do not display the > same short months. > > With my configuration the only one valuable tests are maybe with 'en' and > 'fr'. I cannot be the only one with failures with en_HK.utf8, en_IN, > en_PH.utf8, en_SG.utf8, en_NG (ubuntu) ? > > > Jérôme > > > --- En date de : Ven 26.3.10, Peter Landgren <pet...@te...> a > écrit : > > De: Peter Landgren <pet...@te...> > Objet: Re: [Gramps-devel] Date parser changed branch/trunk > À: "jerome" <rom...@ya...> > Cc: gra...@li..., "Benny Malengier" > <ben...@gm...> Date: Vendredi 26 mars 2010, 17h17 > > #yiv80361845 p, #yiv80361845 li {white-space:pre-wrap;}I get this when I > run "LANG=sv_SV.utf8 python src/date_test.py": > > RESULT: > > slash-dates: 180 dates ok, 0 failed. > > partial date: 432 dates ok, 0 failed. > > Non-gregorian: 912 dates ok, 0 failed. > > B. C. E.: 108 dates ok, 0 failed. > > basic test: 6066 dates ok, 0 failed. > > /Peter > > > > > I cannot test Date_sv !!! > > > > > > What happens? > > > > UnicoeEncodeError : 'ascii' codec can't encode character u'\xf6' in > > > > position 1: ordinal not in range(128) > > > > > > > > It seems to be on ISO date format and Julian calendar ? > > > > (not certain) > > > > > > > > Maybe you can reproduce it : > > > > > > > > 1. by downloading > > > > > > > > > > > > http://gramps.svn.sourceforge.net/viewvc/gramps/branches/maintenance/gram > >ps > > > >22/src/date_test.py > > > > > > > > 2. to rename relLib to gen.lib > > > > > > > > 3. to put the file under /src > > > > > > > > 4. run LANG=sv_SV.utf8 python src/date_test.py > > > > (or just python src/date_test.py) > > > > > > > > > > > > Note, I have a (better) patch for Date_fr.py : > > > > > > > > # Short months not the same as long months > > > > > > > > month_to_int[u"janv"] = 1 > > > > month_to_int[u"févr"] = 2 > > > > month_to_int[u"juil"] = 7 > > > > month_to_int[u"sept"] = 9 > > > > month_to_int[u"oct"] = 10 > > > > month_to_int[u"nov"] = 11 > > > > month_to_int[u"déc"] = 12 > > > > > > > > 7 short months were not the same as long months (number of characters) > > and > > > > were ignored for parsing date : "If the date cannot be converted, the > > text > > > > string is assigned". > > > > > > > > Change above will now convert these short months into Date object rather > > > > than text string ! :) > > > > > > > > I will commit the change on Date_fr.py but this also affect other > > localized > > > > DateHandlers. > > > > > > > > > > > > Regards, > > > > Jérôme > > > > --- En date de : Ven 26.3.10, Peter Landgren <pet...@te...> a écrit : > > > De: Peter Landgren <pet...@te...> > > > > > > Objet: Re: [Gramps-devel] Date parser changed branch/trunk > > > > > > À: gra...@li... > > > > > > Cc: "jerome" <rom...@ya...>, "Benny Malengier" > > > > > > <ben...@gm...> Date: Vendredi 26 mars 2010, 15h47 > > > > > > > Does something has been improved or modified on main > > > > > > way > > > > > > > (DateParser/DateDisplay) for > > > > > > managing/displaying/parsing "MON day, year" > > > > > > > date format ? > > > > > > > > > > > > > > > > It seems that Date_fr does not properly display few > > > > > > specific Dates into one > > > > > > > Date format : MON Day, year (short months and comma > > > > > > after the day). I do > > > > > > > not see any error with short months without comma or a > > > > > > dot. > > > > > > > Note, also minor errors on Date_nl, Date_es, Date_hr, > > > > > > Date_de > > > > > > > No errors on Date_pt > > > > > > > > I cannot test Date_sv !!! > > > > > > What happens? > > > > > > > > > > > > /Peter > > > > > > > Nevertheless, this should be related on regex into > > > > > > localized date handlers > > > > > > > and often very rare cases (not the common date format > > > > > > > :calendar, quality, > > > : > > > > etc ...). One space, dot or end of line characters > > > > > > might fix all failures ! > > > > > > > Regards, > > > > > > > > Jérôme > > > > > > > > > > > > > > > > --- En date de : Ven 26.3.10, jerome <rom...@ya...> > > > > > > a écrit : > > > > > De: jerome <rom...@ya...> > > > > > > > > > > Objet: Re: [Gramps-devel] Date parser changed > > > > > > branch/trunk > > > > > > > > À: "Benny Malengier" <ben...@gm...> > > > > > > > > > > Cc: gra...@li... > > > > > > > > > > Date: Vendredi 26 mars 2010, 12h20 > > > > > > > > > > >fr.FR.utf8 > > > > > > > > > > > > slash-dates: 186 > > > > > > dates > > > > > > > > ok, 54 failed. > > > > > > > > > > > partial date: 486 dates > > > > > > > > > > ok, 90 failed. > > > > > > > > > > >Non-gregorian: 1180 dates > > > > > > ok, 36 > > > > > > > > failed. > > > > > > > > > > > B. C. > > > > > > E.: 126 > > > > > > > > dates ok, 18 failed. > > > > > > > > > > > basic test: 5928 > > > > > > dates > > > > > > > > ok, 2160 failed. > > > > > > > > > > > > > > > > > > > > Note, I get better results under OpenSuse 10.3 > > > > > > ... ;) > > > > > > > > slash-dates: 240 dates ok, > > > > > > 0 > > > > > > > > failed. > > > > > > > > > > partial date: 576 dates > > > > > > ok, > > > > > > > > 0 failed. > > > > > > > > > > Non-gregorian: 1204 dates > > > > > > ok, 12 > > > > > > > > failed. > > > > > > > > > > B. C. > > > > > > E.: 144 > > > > > > > > dates ok, 0 failed. > > > > > > > > > > basic test: 6504 dates ok, > > > > > > > > > > 1584 failed. > > > > > > > > > > > > > > > > > > > > and no crash with en_NG.utf8 (known bug under > > > > > > Ubuntu) > > > > > > > > Parsing is also related to distributions > > > > > > (system/locales) > > > > > > > > "failed" does not mean a crash or lost data, > > > > > > maybe more > > > > > > > > related to GrampsLocale module than > > > > > > DateParser.py. > > > > > > > > We do not need to worry about "failed parsing", > > > > > > this often > > > > > > > > leads to minor display error. > > > > > > > > > > Maybe just crash might be annoying for Gramps. > > > > > > > > > > i.e under Ubuntu : LANG=en.NG.utf8 python > > > > > > gramps.py > > > > > > > > Jérôime > > > > > > > > > > > > > > > > > > > > --- En date de : Ven 26.3.10, jerome <rom...@ya...> > > > > > > > > > > a écrit : > > > > > > De: jerome <rom...@ya...> > > > > > > > > > > > > Objet: Re: [Gramps-devel] Date parser > > > > > > changed > > > > > > > > branch/trunk > > > > > > > > > > > À: "Benny Malengier" <ben...@gm...> > > > > > > > > > > > > Cc: gra...@li... > > > > > > > > > > > > Date: Vendredi 26 mars 2010, 11h41 > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I tried to re-use old testing scripts [1] > > > > > > and I get > > > > > > > > errors > > > > > > > > > > > between MOD_TEXTONLY and QUAL_CALCULATED or > > > > > > a > > > > > > > > non-gregorian > > > > > > > > > > > calendar issue : > > > > > > > > > > > > > > > > > > > > > > > > input was: > > > > > > > > > > > > calendar: CAL_JULIAN > > > > > > > > > > > > modifier: MOD_NONE > > > > > > > > > > > > quality: QUAL_NONE > > > > > > > > > > > > dateval: (4, 1, 1789, > > > > > > False) > > > > > > > > > text: > > > > > > 'Comment. Format: DAY MON YEAR' > > > > > > > > > DateDisplay gives: '4. janv 1789 (Julien)' > > > > > > > > > > > > parsed date was: > > > > > > > > > > > > calendar: CAL_GREGORIAN > > > > > > > > > > > > modifier: MOD_TEXTONLY > > > > > > > > > > > > quality: QUAL_NONE > > > > > > > > > > > > dateval: (0, 0, 0, > > > > > > False) > > > > > > > > > text: '4. janv > > > > > > 1789' > > > > > > > > > this gives:'4. janv 1789' > > > > > > > > > > > > > > > > > > > > > > > > input was: > > > > > > > > > > > > calendar: CAL_JULIAN > > > > > > > > > > > > modifier: MOD_NONE > > > > > > > > > > > > quality: QUAL_NONE > > > > > > > > > > > > dateval: (4, 2, 1789, > > > > > > False) > > > > > > > > > text: > > > > > > 'Comment. Format: DAY MON YEAR' > > > > > > > > > DateDisplay gives: '4. févr 1789 (Julien)' > > > > > > > > > > > > parsed date was: > > > > > > > > > > > > calendar: CAL_GREGORIAN > > > > > > > > > > > > modifier: MOD_TEXTONLY > > > > > > > > > > > > quality: QUAL_NONE > > > > > > > > > > > > dateval: (0, 0, 0, > > > > > > False) > > > > > > > > > text: '4. > > > > > > févr 1789' > > > > > > > > > this gives:'4. févr 1789' > > > > > > > > > > > > > > > > > > > > > > > > Some years ago (other distribution), I was > > > > > > able to fix > > > > > > > > most > > > > > > > > > > > of parsing errors on Date_fr.py. > > > > > > > > > > > > Today, I do not know if this is related to > > > > > > last > > > > > > > > changes (or > > > > > > > > > > > other distribution), but Date_fr.py does > > > > > > > > > > > > not properly parse all dates ... > > > > > > > > > > > > > > > > > > > > > > > > fr_LU.utf8 > > > > > > > > slash-dates: 180 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > partial date: 432 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > Non-gregorian: 904 dates > > > > > > ok, > > > > > > > > > 8 failed. > > > > > > > > > > > > B. C. > > > > > > E.: 108 > > > > > > > > > dates ok, 0 failed. > > > > > > > > > > > > basic test: 5010 dates > > > > > > ok, > > > > > > > > > 1056 failed. > > > > > > > > > > > > > > > > > > > > > > > > fr_BE.utf8, fr.CA.utf8, fr_CH.utf8 > > > > > > > > slash-dates: 240 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > partial date: 576 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > Non-gregorian: 1204 dates > > > > > > ok, 12 > > > > > > > > > failed. > > > > > > > > > > > > B. C. > > > > > > E.: 144 > > > > > > > > > dates ok, 0 failed. > > > > > > > > > > > > basic test: 6504 dates > > > > > > ok, > > > > > > > > > 1584 failed. > > > > > > > > > > > > > > > > > > > > > > > > fr.FR.utf8 > > > > > > > > slash-dates: 186 > > > > > > dates > > > > > > > > > ok, 54 failed. > > > > > > > > > > > > partial date: 486 > > > > > > dates > > > > > > > > > ok, 90 failed. > > > > > > > > > > > > Non-gregorian: 1180 dates > > > > > > ok, 36 > > > > > > > > > failed. > > > > > > > > > > > > B. C. > > > > > > E.: 126 > > > > > > > > > dates ok, 18 failed. > > > > > > > > > > > > basic test: 5928 dates > > > > > > ok, > > > > > > > > > 2160 failed. > > > > > > > > > > > > > > > > > > > > > > > > The difficulty is to get no errors on all > > > > > > locales > > > > > > > > using > > > > > > > > > > > french date handler. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > == others results (availables locales on my > > > > > > > > > > system)=== > > > > > > > > > > > LANG=.. python date_test.py > > > > > > > > > > > > > > > > > > > > > > > > en_AU.utf8, en_BW.utf8, en_CA.utf8, > > > > > > en_DK.utf8, > > > > > > > > > en_GB.utf8, > > > > > > > > > > > > en_IE.utf8, en_NZ.utf8, en_US.utf8, > > > > > > en_ZA.utf8, > > > > > > > > en_ZW.utf8 > > > > > > > > slash-dates: 180 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > partial date: 432 > > > > > > dates ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > Non-gregorian: 912 dates > > > > > > ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > B. C. > > > > > > E.: 108 > > > > > > > > > dates ok, 0 failed. > > > > > > > > > > > > basic test: 6066 dates > > > > > > ok, > > > > > > > > > 0 failed. > > > > > > > > > > > > > > > > > > > > > > > > en_HK.utf8, en_IN, en_PH.utf8, en_SG.utf8 > > > > > > > > slash-dates: 168 > > > > > > dates > > > > > > > > > ok, 12 failed. > > > > > > > > > > > > partial date: 378 > > > > > > dates > > > > > > > > > ok, 54 failed. > > > > > > > > > > > > Non-gregorian: 882 dates > > > > > > > > > > > > ok, 30 failed. > > > > > > > > > > > > B. C. E.: > > > > > > 90 dates > > > > > > > > > ok, 18 failed. > > > > > > > > > > > > basic test: 5058 dates > > > > > > ok, > > > > > > > > > 1008 failed. > > > > > > > > > > > > > > > > > > > > > > > > en_NG > > > > > > > > > > > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > File "date_test.py", line > > > > > > 49, in <module> > > > > > > > > > import DateHandler > > > > > > > > > > > > File > > > > > > "src/DateHandler/__init__.py", line 28, in > > > > > > > > > <module> > > > > > > > > > > > > from _DateHandler > > > > > > import (LANG, LANG_SHORT, > > > > > > > > > LANG_TO_PARSER, LANG_TO_DISPLAY, > > > > > > > > > > > > File > > > > > > "src/DateHandler/_DateHandler.py", line 56, > > > > > > > > in > > > > > > > > > > > <module> > > > > > > > > > > > > LANG = > > > > > > locale.getlocale(locale.LC_TIME)[0] > > > > > > > > > File > > > > > > "/usr/lib/python2.6/locale.py", line 497, in > > > > > > > > > getlocale > > > > > > > > > > > > return > > > > > > _parse_localename(localename) > > > > > > > > > File > > > > > > "/usr/lib/python2.6/locale.py", line 410, in > > > > > > > > > _parse_localename > > > > > > > > > > > > raise ValueError, > > > > > > 'unknown locale: %s' % > > > > > > > > > localename > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > http://gramps.svn.sourceforge.net/viewvc/gramps/branches/maintena > > > > > >nc > > > > > > > > > > > >e/gr amps22/test/dates.sh > > > > > > > > > > > > http://gramps.svn.sourceforge.net/viewvc/gramps/branches/maintena > > > > > >nc > > > > > > > > > > > >e/gr amps22/src/date_test.py (rename relLib to > > > > > > gen.lib) and put the file > > > > > > > > > under /src > > > > > > > > > > and > > > > > > > > > > > play with LANG variable. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > Jérôme > > > > > > _______________________________________________ > > > > > > > > > Gramps-devel mailing list > > > > > > > > > > > > > > > > > > > > > > > > Gra...@li... > > > > > > > > > > > > > > > > > > > > > > > > https://lists.sourceforge.net/lists/listinfo/gramps-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----------------------------------------------------------------------- > > >-- > > > > > > > >----- > > > > > > > > > > > Download Intel® Parallel Studio Eval > > > > > > > > > > > > Try the new software tools for yourself. > > > > > > Speed > > > > > > > > compiling, > > > > > > > > > > > find bugs > > > > > > > > > > > > proactively, and fine-tune applications for > > > > > > parallel > > > > > > > > > performance. > > > > > > > > > > > > See why Intel Parallel Studio got high marks > > > > > > during > > > > > > > > beta. |