From: Eero T. <ee...@us...> - 2006-06-12 19:29:20
|
Hi, I redirected this to gramps-devel and changed the subject appropriately. On Monday 12 June 2006 04:42, Alex Roitman wrote: > > Alex, is there a way to show error dialogs from the report generation > > class init method? The GraphViz class could catch UnicodeEncodeError > > exception and show user a dialog containing something like this: > > Your data contains characters cannot be converted to the Latin-1 > > charset you've selected from the "GraphViz Options" tab. Aborting the > > report generation so that you can correct this. > > I suppose we could add it. Is this only for GraphViz, or would other > plugins benefit from catching this exception as well? These source files seem to be doing encoding: ~/work/gramps-devel/src/plugins> grep encode *.py ./ansel_utf8.py: """Converts an UTF8 encoded string to ANSEL""" ./GenericFilter.py: f = open(self.file.encode('utf-8'),'w') ./docgen/PdfDoc.py: return new_s.encode('iso-8859-1') ./docgen/PSDrawDoc.py: new_text = orig.encode('iso-8859-1') ./docgen/OpenOfficeDoc.py: zipinfo = zipfile.ZipInfo(name.encode('latin-1')) ./docgen/ODFDoc.py: zipinfo = zipfile.ZipInfo(name.encode('latin-1')) ./DisplayModels.py: return unicode(data[2].encode('iso-8859-1')) ./WriteGedcom.py: return s.encode('iso-8859-1','replace') ./ReadGedcom.py: return s.encode('iso-8859-1','replace') ./latin_ansel.py: """Converts an ANSEL encoded string to ISO-8859-1""" ./soundex.py: strval = strval.encode('iso-8859-1') ./plugins/GraphViz.py: self.f.write(buffer.encode('iso-8859-1')) ./plugins/ExportVCalendar.py: self.g.write('%s\n' % (text.encode('iso-8859-1'))) ./plugins/WriteGeneWeb.py: return s.encode('iso-8859-1','xmlcharrefreplace') ./plugins/ExportVCard.py: self.g.write('%s\n' % (text.encode('iso-8859-1'))) ./GedcomInfo.py: f = open(filepath.encode('iso8859-1'),"r") How many of them don't handle properly characters that cannot be converted? > All that needs to be done is a try: except UnicodeEncodeError: clause > with the QuestionDialog.ErrorDialog() in the except part. There's a serious usability problem. I don't get the parent window in the report __init__ method so the ErrorDialog window tranciency would be borked (dialog wouldn't be trancient to any Gramps window/dialog). Couldn't there be some Gramps Exception which I could raise with the error string that the calling function would then should in an ErrorDialog that is properly trancient? - Eero |
From: Alex R. <sh...@gr...> - 2006-06-12 19:38:21
|
On Mon, 2006-06-12 at 22:49 +0300, Eero Tamminen wrote: > > I suppose we could add it. Is this only for GraphViz, or would other > > plugins benefit from catching this exception as well? >=20 > These source files seem to be doing encoding: > ~/work/gramps-devel/src/plugins> grep encode *.py > ./ansel_utf8.py: """Converts an UTF8 encoded string to ANSEL""" > ./GenericFilter.py: f =3D open(self.file.encode('utf-8'),'w') > ./docgen/PdfDoc.py: return new_s.encode('iso-8859-1') > ./docgen/PSDrawDoc.py: new_text =3D orig.encode('iso-8859-1') > ./docgen/OpenOfficeDoc.py: zipinfo =3D=20 > zipfile.ZipInfo(name.encode('latin-1')) > ./docgen/ODFDoc.py: zipinfo =3D zipfile.ZipInfo(name.encode('latin= -1')) > ./DisplayModels.py: return unicode(data[2].encode('iso-8859-1'= )) > ./WriteGedcom.py: return s.encode('iso-8859-1','replace') > ./ReadGedcom.py: return s.encode('iso-8859-1','replace') > ./latin_ansel.py: """Converts an ANSEL encoded string to ISO-8859-1""" > ./soundex.py: strval =3D strval.encode('iso-8859-1') > ./plugins/GraphViz.py: self.f.write(buffer.encode('iso-8859-1'= )) > ./plugins/ExportVCalendar.py: self.g.write('%s\n' %=20 > (text.encode('iso-8859-1'))) > ./plugins/WriteGeneWeb.py: return=20 > s.encode('iso-8859-1','xmlcharrefreplace') > ./plugins/ExportVCard.py: self.g.write('%s\n' %=20 > (text.encode('iso-8859-1'))) > ./GedcomInfo.py: f =3D open(filepath.encode('iso8859-1'),"r") >=20 > How many of them don't handle properly characters that cannot be converte= d? Can't tell right away, we'll have to check. > > All that needs to be done is a try: except UnicodeEncodeError: clause > > with the QuestionDialog.ErrorDialog() in the except part. >=20 > There's a serious usability problem. I don't get the parent window in > the report __init__ method so the ErrorDialog window tranciency would > be borked (dialog wouldn't be trancient to any Gramps window/dialog). Yes, I see. > Couldn't there be some Gramps Exception which I could raise with the erro= r > string that the calling function would then should in an ErrorDialog that= is > properly trancient? Errors.ReportError() can be raised as an exception. But then we need to catch it in the caller, just as you said, and display a proper error dialog. At this point, I don't think we should do these changes in 2.0.x branch. The whole thing will become a dead-end in a couple of month, when 2.2 releases. The dialog would be nice, but it won't be a panacea: just replacing the traceback with the nice "sorry, can't do" message. Not something that is really critical. Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Eero T. <ee...@us...> - 2006-06-12 20:33:25
Attachments:
GraphViz.py.diff
|
Hi, On Monday 12 June 2006 22:38, Alex Roitman wrote: ... > > > All that needs to be done is a try: except UnicodeEncodeError: clause > > > with the QuestionDialog.ErrorDialog() in the except part. > > > > There's a serious usability problem. I don't get the parent window in > > the report __init__ method so the ErrorDialog window tranciency would > > be borked (dialog wouldn't be trancient to any Gramps window/dialog). > > Yes, I see. > > > Couldn't there be some Gramps Exception which I could raise with the > > error string that the calling function would then should in an > > ErrorDialog that is properly trancient? > > Errors.ReportError() can be raised as an exception. > But then we need to catch it in the caller, just as you said, > and display a proper error dialog. Hm. I took a look at the report() in Report.py and it indeed seems to be catching ReportError with (msg1, msg2) tuple, and showing that with ErrorDialog. So that functionality actually is there. However, the report() function seems to have borked the tranciency too. It doesn't set any parent to ErrorDialogs it uses. > At this point, I don't think we should do these changes > in 2.0.x branch. The whole thing will become a dead-end > in a couple of month, when 2.2 releases. Fixing this for 2.2 is OK for me. The main thing is that it's fixed. :-) > The dialog > would be nice, but it won't be a panacea: just replacing > the traceback with the nice "sorry, can't do" message. > Not something that is really critical. Well, proper error dialog is much less scary than a traceback and actually tells user what is the problem and how to fix it. See the attached patch for the GraphViz.py part (untested and also changes self.f to local variable as it's not used anywhere else). - Eero |
From: Alex R. <sh...@gr...> - 2006-06-22 19:22:30
|
Eero, On Mon, 2006-06-12 at 23:53 +0300, Eero Tamminen wrote: > > Errors.ReportError() can be raised as an exception. > > But then we need to catch it in the caller, just as you said, > > and display a proper error dialog. >=20 > Hm. I took a look at the report() in Report.py and it indeed seems > to be catching ReportError with (msg1, msg2) tuple, and showing > that with ErrorDialog. So that functionality actually is there. >=20 > However, the report() function seems to have borked the tranciency too. > It doesn't set any parent to ErrorDialogs it uses. True. This all boils down to the fact that in gramps 2.0.x we don't handle the main context of the application right, neither db nor ui. In particular, the report dialogs are not aware of the main gramps window. In gramps 2.1 series we always have two variables, dbstate and uistate, which almost all classes take as constructor arguments. So every class has access to the database and ui state of the whole thing. And in 2.1 the report dialogs already are correctly transient_for the main gramps window. At this point, we probably don't want to fix it in 2.0.x. We just don't have enough resources to seriously test any extensive changes, as all developers are now hacking on gramps 2.1. So whether or not there will be a 2.0.12 release (not likely but possible), probably only small changes should be committed to 2.0.x tree. With some luck we'll be ready with 2.2 soon. Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Eero T. <ee...@us...> - 2006-06-22 19:41:14
|
Hi, On Thursday 22 June 2006 22:22, Alex Roitman wrote: > In gramps 2.1 series we always have two variables, dbstate > and uistate, which almost all classes take as constructor > arguments. So every class has access to the database and > ui state of the whole thing. And in 2.1 the report dialogs > already are correctly transient_for the main gramps window. Great, so the only thing needed in Relationship report is: --------------------- try: dotfile.write(buffer.encode('iso-8859-1')) except UnicodeEncodeError: raise Errors.ReportError(), (_("Your data contains characters that cannot be converted to latin-1. Unselect latin-1 option and try again."),) --------------------- Instead of just: dotfile.write(buffer.encode('iso-8859-1')) > At this point, we probably don't want to fix it in 2.0.x. OK with me. :-) - Eero |
From: Brian M. <br...@gr...> - 2006-06-24 19:01:16
|
>On Thursday 22 June 2006 22:22, Alex Roitman wrote: >> In gramps 2.1 series we always have two variables, dbstate >> and uistate, which almost all classes take as constructor >> arguments. So every class has access to the database and >> ui state of the whole thing. And in 2.1 the report dialogs >> already are correctly transient_for the main gramps window. > >Great, so the only thing needed in Relationship report is: >--------------------- > try: > dotfile.write(buffer.encode('iso-8859-1')) > except UnicodeEncodeError: > raise Errors.ReportError(), (_("Your data contains >characters that cannot be converted to latin-1. Unselect latin-1 option and >try again."),) >--------------------- >Instead of just: > dotfile.write(buffer.encode('iso-8859-1')) > The way I had fixed this previously in the 2.1 branch was like this: if self.latin: self.f.write(the_buffer.encode('iso-8859-1', 'replace')) else: self.f.write(the_buffer) The "replace" option tells the encode function to replace characters that can not be represented in the given encoding. I don't actually know what it replaces them with. But at least it doesn't throw an exception. This problem doesn't affect me since I don't use it, so I have no preference between my way and yours. If you change it, I certainly won't mind. ~Brian |
From: Eero T. <ee...@us...> - 2006-06-29 19:25:29
|
Hi, On Saturday 24 June 2006 22:01, Brian Matherly wrote: > >--------------------- > > try: > > dotfile.write(buffer.encode('iso-8859-1')) > > except UnicodeEncodeError: > > raise Errors.ReportError(), (_("Your data contains > >characters that cannot be converted to latin-1. Unselect latin-1 option > > and try again."),) > >--------------------- > >Instead of just: > > dotfile.write(buffer.encode('iso-8859-1')) > > The way I had fixed this previously in the 2.1 branch was like this: > > if self.latin: > self.f.write(the_buffer.encode('iso-8859-1', 'replace')) > else: > self.f.write(the_buffer) > > The "replace" option tells the encode function to replace characters that > can not be represented in the given encoding. I don't actually know what > it replaces them with. But at least it doesn't throw an exception. Interesting, I didn't know about that. > This problem doesn't affect me since I don't use it, so I have no > preference between my way and yours. If you change it, I certainly won't > mind. Producing something sounds nice, but it would also be nice to inform user about the problem with the selected option in case it was selected on purpose and/or the output wasn't what was expected. Arturas, could you test the 2.1.x version with Brian's fix using the "latin-1" option and tell whether the results of "replacing characters" are acceptable? If they are acceptable, I think Brian's fix is OK, otherwise it would be better to tell user what's the problem and how to fix it. - Eero |
From: Arturas S. <asl...@gm...> - 2006-06-30 16:24:30
|
SGVsbG8sCgpJJ20gbm90IHN1cmUgaWYgdGhpcyB3YXMgcmVxdWlyZWQgZnJvbSBtZSwgYnV0IEkg ZGlkIGZvbGxvd2luZzoKMS4gVXBkYXRlZCAgZ3JhbXBzIGZyb20gc3ZuIHRvIDY5NzUKMi4gQ29t cGlsZWQgcHJvZ3JtYSwgZXhwb3J0ZWQgZnJvbSBvbGQgdmVyc2lvbiBkYXRhLCBpbXBvcnRlZCB0 aGVtIHRvIG5ldwp2ZXJzaW9uLgozLiBSYW4gIHJlbGF0aW9uc2hpcCBkaWFncmFtICBjaGFuZ2lu ZyBvbmx5IG9uZSBwYXJhbWV0ZXIgLi4gKHJlcXVpcmUgbGF0aW4xCm91dHB1dCkuCldpdGggb25l IG9wdGlvbiAgKHVuY2hlY2tlZCkgbWluZSBuYW1lIGlzIEFydPtyYXMg0GxlaW5pdXMgKGNvcnJl Y3Qgb25lKQpXaGlsZSBjaGVja2VkIGl0IHNob3dzIEFydD9yYXMgP2xlaW5pdXMgICAoYWxsIG5v biBsYXRpbi0xIGNoYXJhY3RlcnMgYXJlCmNoYW5nZWQgYnkgY2hhcmFjdGVyICI/IikKCkl0cyB1 cCB0byB5b3UgdG8gIGRlY2lkZSBpcyBpdCBvayBvciBub3QuCgpJZiBtaW5lIHRlc3RzIGFyZSB3 cm9uZyBnaXZlIGF0ICBsZWFzdCBzb21lIGRpcmVjdGlvbnMgIGhvdyBJIHNob3VsZCB0ZXN0LgoK UmVnYXJkcywKQXJ0dXJhcwoKT24gNi8yOS8wNiwgRWVybyBUYW1taW5lbiA8ZWVyb3RAdXNlcnMu c291cmNlZm9yZ2UubmV0PiB3cm90ZToKPgo+IEhpLAo+Cj4gT24gU2F0dXJkYXkgMjQgSnVuZSAy MDA2IDIyOjAxLCBCcmlhbiBNYXRoZXJseSB3cm90ZToKPiA+ID4tLS0tLS0tLS0tLS0tLS0tLS0t LS0KPiA+ID4gICAgICAgICAgICB0cnk6Cj4gPiA+ICAgICAgICAgICAgICAgIGRvdGZpbGUud3Jp dGUoYnVmZmVyLmVuY29kZSgnaXNvLTg4NTktMScpKQo+ID4gPiAgICAgICAgICAgIGV4Y2VwdCBV bmljb2RlRW5jb2RlRXJyb3I6Cj4gPiA+ICAgICAgICAgICAgICAgIHJhaXNlIEVycm9ycy5SZXBv cnRFcnJvcigpLCAoXygiWW91ciBkYXRhIGNvbnRhaW5zCj4gPiA+Y2hhcmFjdGVycyB0aGF0IGNh bm5vdCBiZSBjb252ZXJ0ZWQgdG8gbGF0aW4tMS4gVW5zZWxlY3QgbGF0aW4tMSBvcHRpb24KPiA+ ID4gYW5kIHRyeSBhZ2Fpbi4iKSwpCj4gPiA+LS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gPiA+SW5z dGVhZCBvZiBqdXN0Ogo+ID4gPiAgICAgICAgICAgICAgICBkb3RmaWxlLndyaXRlKGJ1ZmZlci5l bmNvZGUoJ2lzby04ODU5LTEnKSkKPiA+Cj4gPiBUaGUgd2F5IEkgaGFkIGZpeGVkIHRoaXMgcHJl dmlvdXNseSBpbiB0aGUgMi4xIGJyYW5jaCB3YXMgbGlrZSB0aGlzOgo+ID4KPiA+ICAgICAgICAg aWYgc2VsZi5sYXRpbjoKPiA+ICAgICAgICAgICAgIHNlbGYuZi53cml0ZSh0aGVfYnVmZmVyLmVu Y29kZSgnaXNvLTg4NTktMScsICdyZXBsYWNlJykpCj4gPiAgICAgICAgIGVsc2U6Cj4gPiAgICAg ICAgICAgICBzZWxmLmYud3JpdGUodGhlX2J1ZmZlcikKPiA+Cj4gPiBUaGUgInJlcGxhY2UiIG9w dGlvbiB0ZWxscyB0aGUgZW5jb2RlIGZ1bmN0aW9uIHRvIHJlcGxhY2UgY2hhcmFjdGVycwo+IHRo YXQKPiA+IGNhbiBub3QgYmUgcmVwcmVzZW50ZWQgaW4gdGhlIGdpdmVuIGVuY29kaW5nLiBJIGRv bid0IGFjdHVhbGx5IGtub3cgd2hhdAo+ID4gaXQgcmVwbGFjZXMgdGhlbSB3aXRoLiBCdXQgYXQg bGVhc3QgaXQgZG9lc24ndCB0aHJvdyBhbiBleGNlcHRpb24uCj4KPiBJbnRlcmVzdGluZywgSSBk aWRuJ3Qga25vdyBhYm91dCB0aGF0Lgo+Cj4KPiA+IFRoaXMgcHJvYmxlbSBkb2Vzbid0IGFmZmVj dCBtZSBzaW5jZSBJIGRvbid0IHVzZSBpdCwgc28gSSBoYXZlIG5vCj4gPiBwcmVmZXJlbmNlIGJl dHdlZW4gbXkgd2F5IGFuZCB5b3Vycy4gSWYgeW91IGNoYW5nZSBpdCwgSSBjZXJ0YWlubHkgd29u J3QKPiA+IG1pbmQuCj4KPiBQcm9kdWNpbmcgc29tZXRoaW5nIHNvdW5kcyBuaWNlLCBidXQgaXQg d291bGQgYWxzbyBiZSBuaWNlIHRvIGluZm9ybSB1c2VyCj4gYWJvdXQgdGhlIHByb2JsZW0gd2l0 aCB0aGUgc2VsZWN0ZWQgb3B0aW9uIGluIGNhc2UgaXQgd2FzIHNlbGVjdGVkIG9uCj4gcHVycG9z ZSBhbmQvb3IgdGhlIG91dHB1dCB3YXNuJ3Qgd2hhdCB3YXMgZXhwZWN0ZWQuCj4KPiBBcnR1cmFz LCBjb3VsZCB5b3UgdGVzdCB0aGUgMi4xLnggdmVyc2lvbiB3aXRoIEJyaWFuJ3MgZml4IHVzaW5n Cj4gdGhlICJsYXRpbi0xIiBvcHRpb24gYW5kIHRlbGwgd2hldGhlciB0aGUgcmVzdWx0cyBvZiAi cmVwbGFjaW5nCj4gY2hhcmFjdGVycyIgYXJlIGFjY2VwdGFibGU/Cj4KPiBJZiB0aGV5IGFyZSBh Y2NlcHRhYmxlLCBJIHRoaW5rIEJyaWFuJ3MgZml4IGlzIE9LLCBvdGhlcndpc2UgaXQgd291bGQg YmUKPiBiZXR0ZXIgdG8gdGVsbCB1c2VyIHdoYXQncyB0aGUgcHJvYmxlbSBhbmQgaG93IHRvIGZp eCBpdC4KPgo+Cj4gICAgICAgICAtIEVlcm8KPgo= |
From: Brian M. <br...@gr...> - 2006-06-30 16:32:17
|
Your test was correct. Now we know that the 'replace' option replaces chara= cters with a question mark. Eero: Feel free to proceed as you see fit (or ask Alex - he usually has a h= elpful perspective on issues like this). ~Brian ----- Original Message ---- From: Arturas Sleinius <asl...@gm...> To: Eero Tamminen <ee...@us...> Cc: Brian Matherly <br...@gr...>; gra...@li...urcefor= ge.net; Alex Roitman <sh...@gr...> Sent: Friday, June 30, 2006 11:24:28 AM Subject: Re: [Gramps-devel] Catching UnicodeEncodeError exception from Grap= hViz report Hello, I'm not sure if this was required from me, but I did following: 1. Updated gramps from svn to 6975 2. Compiled progrma, exported from old version data, imported them to new v= ersion. 3. Ran relationship diagram changing only one parameter .. (require latin= 1 output). =20 With one option (unchecked) mine name is Art=C5=ABras =C5=A0leinius (corre= ct one) While checked it shows Art?ras ?leinius (all non latin-1 characters are c= hanged by character "?")=20 Its up to you to decide is it ok or not. =20 If mine tests are wrong give at least some directions how I should test. Regards, Arturas On 6/29/06, Eero Tamminen < ee...@us...> wrote:Hi, On Saturday 24 June 2006 22:01, Brian Matherly wrote:=20 > >--------------------- > > try: > > dotfile.write(buffer.encode('iso-8859-1')) > > except UnicodeEncodeError: > > raise Errors.ReportError (), (_("Your data contains > >characters that cannot be converted to latin-1. Unselect latin-1 option > > and try again."),) > >--------------------- > >Instead of just: > > dotfile.write(buffer.encode('iso-8859-1')) > > The way I had fixed this previously in the 2.1 branch was like this: > > if self.latin: > self.f.write(the_buffer.encode('iso-8859-1', 'replace'))=20 > else: > self.f.write(the_buffer) > > The "replace" option tells the encode function to replace characters that > can not be represented in the given encoding. I don't actually know what= =20 > it replaces them with. But at least it doesn't throw an exception. Interesting, I didn't know about that. > This problem doesn't affect me since I don't use it, so I have no > preference between my way and yours. If you change it, I certainly won't= =20 > mind. Producing something sounds nice, but it would also be nice to inform user about the problem with the selected option in case it was selected on purpose and/or the output wasn't what was expected.=20 Arturas, could you test the 2.1.x version with Brian's fix using the "latin-1" option and tell whether the results of "replacing characters" are acceptable? If they are acceptable, I think Brian's fix is OK, otherwise it would be=20 better to tell user what's the problem and how to fix it. - Eero Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easi= er Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D1= 21642 _______________________________________________ Gramps-devel mailing list Gra...@li... https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Eero T. <ee...@us...> - 2006-06-30 21:23:32
|
Hi, On Friday 30 June 2006 19:31, Brian Matherly wrote: > Your test was correct. Now we know that the 'replace' option replaces > characters with a question mark. Arturas, thanks! > Eero: Feel free to proceed as you see fit (or ask Alex - he usually has a > helpful perspective on issues like this). IMHO it would be better to give user the error dialog. When user sees the "?" characters, he doesn't know how to fix it. The whole latin-1 user option is unsatisfactory, but nobody's had the time to test all the output, font and encoding options to come up with rules where latin-1 is needed and when not. Therefore it's currently user option... Even if the latin-1 option would be enabled automatically based on the selected output format and font, user might still need to install some font, then select it from Relationship report and try again. For e.g. SVG format the font (and therefore latin-1) option can be ignored as SVG is UTF-8 and the fonts are SVG-viewer's headache. Eero |
From: Alex R. <sh...@gr...> - 2006-07-19 21:36:51
|
I finally got around to this, sorry for the delay. On Sat, 2006-07-01 at 00:44 +0300, Eero Tamminen wrote: > On Friday 30 June 2006 19:31, Brian Matherly wrote: > > Your test was correct. Now we know that the 'replace' option replaces > > characters with a question mark. [snip] > > Eero: Feel free to proceed as you see fit (or ask Alex - he usually has= a > > helpful perspective on issues like this). >=20 > IMHO it would be better to give user the error dialog. > When user sees the "?" characters, he doesn't know > how to fix it. >=20 > The whole latin-1 user option is unsatisfactory, but nobody's had the tim= e > to test all the output, font and encoding options to come up with rules > where latin-1 is needed and when not. Therefore it's currently user > option... How about we combine both: do replace and present an error window? Something like this: self.encoding_problem =3D False try: self.f.write(the_buffer) except UnicodeEncodeError: if self.latin: self.f.write(the_buffer.encode('iso-8859-1', 'replace')) self.encoding_problem =3D True else: raise Errors.ReporError( _("Your data contains characters that cannot " "be displayed. Most likely this is caused " "by the incorrect encoding.)) self.f.close() if self.encoding_problem: ErrorDialog( _("Your data contains characters that cannot be converted " "to latin-1. These characters were replaced with " "question marks in the output. To get these characters " "properly displayed, unselect latin-1 option and try=20 again.") ) So we first try to write as is. On exception, if latin1 is selected, we try the replace option as a fallback and then pop an error dialog to inform the user. If we get UnicodeEncode exception and it is not due to latin1 conversion, then we throw an exception and inform the user. Not much else we can do in this case really. Does that sound good to everybody? Should I commit this so that people can try it? Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Brian M. <br...@gr...> - 2006-07-19 22:02:31
|
Go for it. ~Brian ----- Original Message ---- From: Alex Roitman <sh...@gr...> To: Eero Tamminen <ee...@us...> Cc: Arturas Sleinius <asl...@gm...>; Brian Matherly <br...@gr...>; gra...@li... Sent: Wednesday, July 19, 2006 4:37:38 PM Subject: Re: [Gramps-devel] Catching UnicodeEncodeError exception from GraphViz report I finally got around to this, sorry for the delay. On Sat, 2006-07-01 at 00:44 +0300, Eero Tamminen wrote: > On Friday 30 June 2006 19:31, Brian Matherly wrote: > > Your test was correct. Now we know that the 'replace' option replaces > > characters with a question mark. [snip] > > Eero: Feel free to proceed as you see fit (or ask Alex - he usually has a > > helpful perspective on issues like this). > > IMHO it would be better to give user the error dialog. > When user sees the "?" characters, he doesn't know > how to fix it. > > The whole latin-1 user option is unsatisfactory, but nobody's had the time > to test all the output, font and encoding options to come up with rules > where latin-1 is needed and when not. Therefore it's currently user > option... How about we combine both: do replace and present an error window? Something like this: self.encoding_problem = False try: self.f.write(the_buffer) except UnicodeEncodeError: if self.latin: self.f.write(the_buffer.encode('iso-8859-1', 'replace')) self.encoding_problem = True else: raise Errors.ReporError( _("Your data contains characters that cannot " "be displayed. Most likely this is caused " "by the incorrect encoding.)) self.f.close() if self.encoding_problem: ErrorDialog( _("Your data contains characters that cannot be converted " "to latin-1. These characters were replaced with " "question marks in the output. To get these characters " "properly displayed, unselect latin-1 option and try again.") ) So we first try to write as is. On exception, if latin1 is selected, we try the replace option as a fallback and then pop an error dialog to inform the user. If we get UnicodeEncode exception and it is not due to latin1 conversion, then we throw an exception and inform the user. Not much else we can do in this case really. Does that sound good to everybody? Should I commit this so that people can try it? Alex -- Alexander Roitman http://www.gramps-project.org ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Gramps-devel mailing list Gra...@li... https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Eero T. <ee...@us...> - 2006-07-20 19:33:12
|
Hi, On Thursday 20 July 2006 00:37, Alex Roitman wrote: > How about we combine both: do replace and present an error window? > =C2=A0 =C2=A0 self.encoding_problem =3D False > =C2=A0 =C2=A0 try: > =C2=A0 =C2=A0 =C2=A0 =C2=A0 self.f.write(the_buffer) > =C2=A0 =C2=A0 except UnicodeEncodeError: > =C2=A0 =C2=A0 =C2=A0 =C2=A0 if self.latin: > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 self.f.write(the_buffer.encod= e('iso-8859-1', 'replace')) > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 self.encoding_problem =3D Tru= e > =C2=A0 =C2=A0 =C2=A0 =C2=A0 else: > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 raise Errors.ReporError( > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 _("Your data co= ntains characters that cannot " > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "be disp= layed. Most likely this is caused " > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "by the = incorrect encoding.)) > =C2=A0 =C2=A0 self.f.close() ... > So we first try to write as is. On exception, if latin1 > is selected, we try the replace option as a fallback and > then pop an error dialog to inform the user. > If we get UnicodeEncode exception and it is not due to latin1 > conversion, then we throw an exception and inform the user. Exception comes when you try to convert into latin-1, not when you try to write as-is. So, with the above code user would get everything in UTF-8 even although his output format font would support just latin-1. - Eero |
From: Alex R. <sh...@gr...> - 2006-07-20 19:49:15
|
On Thu, 2006-07-20 at 00:50 +0300, Eero Tamminen wrote: > > So we first try to write as is. On exception, if latin1 > > is selected, we try the replace option as a fallback and > > then pop an error dialog to inform the user. > > If we get UnicodeEncode exception and it is not due to latin1 > > conversion, then we throw an exception and inform the user. >=20 > Exception comes when you try to convert into latin-1, not when you try to > write as-is. Well, that can happen too, although this was not the original problem that you guys had :-) > So, with the above code user would get everything in UTF-8 > even although his output format font would support just latin-1. This is true, my mistake. How about this then: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D self.f =3D open(options_class.get_output(),'w') if self.latin: try: self.f.write(the_buffer.encode('iso-8859-1', 'strict')) except UnicodeEncodeError: self.f =3D open(options_class.get_output(),'w') self.f.write(the_buffer.encode('iso-8859-1', 'replace')) ErrorDialog( _("Your data contains characters that cannot be " "converted to latin-1. These characters were " "replaced with the question marks in the output. " "To get these characters properly displayed, " "unselect latin-1 option and try again.")) else: self.f.write(the_buffer) self.f.close() =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Eero T. <ee...@us...> - 2006-07-20 20:24:40
|
Hi, On Thursday 20 July 2006 22:50, Alex Roitman wrote: > > Exception comes when you try to convert into latin-1, not when you try > > to write as-is. > > Well, that can happen too, although this was not the original > problem that you guys had :-) Really? The text in Gramps is in UTF-8 and I don't see how just writing it to a file could generate Unicode exception... > > So, with the above code user would get everything in UTF-8 > > even although his output format font would support just latin-1. > > This is true, my mistake. How about this then: > ============ > self.f = open(options_class.get_output(),'w') > if self.latin: > try: > self.f.write(the_buffer.encode('iso-8859-1', 'strict')) > except UnicodeEncodeError: > self.f = open(options_class.get_output(),'w') > self.f.write(the_buffer.encode('iso-8859-1', 'replace')) > ErrorDialog( > _("Your data contains characters that cannot be " > "converted to latin-1. These characters were " > "replaced with the question marks in the output. " > "To get these characters properly displayed, " > "unselect latin-1 option and try again.")) > else: > self.f.write(the_buffer) > self.f.close() > ============ Looks good to me! :-) - Eero |