Menu
▾
▴
dbfpy-devel
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(6) |
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2007 |
Jan
(3) |
Feb
(7) |
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
|
Feb
(6) |
Mar
(2) |
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
(3) |
Oct
|
Nov
|
Dec
|
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(4) |
Jun
(1) |
Jul
|
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(8) |
Oct
|
Nov
|
Dec
(2) |
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(4) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(5) |
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2014 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(2) |
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
|
From: Yaroslav S. <ya...@gm...> - 2009-09-17 15:37:35
|
Hi, I personally don't think it's a good idea to pass DBF instance everywhere. Let's add this knowledge only if it's really needed (you say that FPT MEMO looks like the case). I would expect some factory (which always accepts DBF instances as an argument) which knows if DBF instance should be passed to the Field or not. -- Regards, Yaroslav > Hi, > > I have implemented FPT memo fields, but have a question how you would > like it plugged into the core. (I haven't implemented DBT memo > fields, but > if FPT is done correctly, it should be easy to add more memo field > types. > > The problem is that currently Fields are entirely ignorant of the > dbf, and > this is fine, except with memos they will need this information. > > I think the preferred way would be to pass the dbf instance to all > fields, > and there be some magic to guess the location of the FPT file, which > is > overridable on dbf instantiation. > > Please advise, > > Ali |
|
From: Ali A. <aa...@gm...> - 2009-09-10 09:00:23
|
Hi, I have implemented FPT memo fields, but have a question how you would like it plugged into the core. (I haven't implemented DBT memo fields, but if FPT is done correctly, it should be easy to add more memo field types. The problem is that currently Fields are entirely ignorant of the dbf, and this is fine, except with memos they will need this information. I think the preferred way would be to pass the dbf instance to all fields, and there be some magic to guess the location of the FPT file, which is overridable on dbf instantiation. Please advise, Ali |
|
From: SourceForge.net <no...@so...> - 2009-06-25 07:49:58
|
Bugs item #2811995, was opened at 2009-06-25 07:49 Message generated for change (Tracker Item Submitted) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=2811995&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: overflow numbers Initial Comment: I found a bug while working with dbf files that contain cells with overflow class class DbfNumericFieldDef, method decodeValue throw ValueError: invalid literal for int() with base 10: '****' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=2811995&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2009-05-27 14:57:03
|
Bugs item #2797420, was opened at 2009-05-27 16:57 Message generated for change (Tracker Item Submitted) made by mfrasca You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=2797420&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mario Frasca (mfrasca) Assigned to: Nobody/Anonymous (nobody) Summary: unclear exception text when accessing non existing column Initial Comment: ValueError: list.index(x): x not in list is the standard python exception plus text when looking for an item in a list that does not contain the item. this exception is raised when you attempt accessing a non existing column. that 'x' does not really help finding what column you were trying to access, what about this: def indexOfFieldName(self, name): """Index of field named ``name``.""" # FIXME: move this to header class try: return self.header.fields.index(name) except ValueError: raise ValueError("can't find index of field '%s'" % name) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=2797420&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2009-05-26 05:36:43
|
Feature Requests item #2796419, was opened at 2009-05-25 16:12 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Priority: 5 Private: No Submitted By: Mario Frasca (mfrasca) Assigned to: Nobody/Anonymous (nobody) Summary: exception decoding ^@ padded numeric field Initial Comment: we've got a dbf where numeric fields are padded with <NUL> characters. on these the library crashes: File "/usr/local/lib/python2.6/dist-packages/dbfpy/fields.py", line 240, in decodeValue # a float (has decimal separator) ValueError: null byte in argument for float() I would add this line just after stripping the value, before converting it to float/integer: value = value.replace('\0', '') ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2009-05-26 08:36 Message: done in version 2.2.4 ---------------------------------------------------------------------- Comment By: Mario Frasca (mfrasca) Date: 2009-05-25 18:11 Message: or even better, put this feature request under 'bugs', because it's there where it belongs... and maybe value=value.strip(" \0") ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2009-05-25 15:11:48
|
Feature Requests item #2796419, was opened at 2009-05-25 15:12 Message generated for change (Comment added) made by mfrasca You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Priority: 5 Private: No Submitted By: Mario Frasca (mfrasca) Assigned to: Nobody/Anonymous (nobody) Summary: exception decoding ^@ padded numeric field Initial Comment: we've got a dbf where numeric fields are padded with <NUL> characters. on these the library crashes: File "/usr/local/lib/python2.6/dist-packages/dbfpy/fields.py", line 240, in decodeValue # a float (has decimal separator) ValueError: null byte in argument for float() I would add this line just after stripping the value, before converting it to float/integer: value = value.replace('\0', '') ---------------------------------------------------------------------- >Comment By: Mario Frasca (mfrasca) Date: 2009-05-25 17:11 Message: or even better, put this feature request under 'bugs', because it's there where it belongs... and maybe value=value.strip(" \0") ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2009-05-25 13:12:20
|
Feature Requests item #2796419, was opened at 2009-05-25 15:12 Message generated for change (Tracker Item Submitted) made by mfrasca You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Priority: 5 Private: No Submitted By: Mario Frasca (mfrasca) Assigned to: Nobody/Anonymous (nobody) Summary: exception decoding ^@ padded numeric field Initial Comment: we've got a dbf where numeric fields are padded with <NUL> characters. on these the library crashes: File "/usr/local/lib/python2.6/dist-packages/dbfpy/fields.py", line 240, in decodeValue # a float (has decimal separator) ValueError: null byte in argument for float() I would add this line just after stripping the value, before converting it to float/integer: value = value.replace('\0', '') ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=2796419&group_id=140566 |
|
From: alexander s. <al...@ty...> - 2008-09-16 00:00:33
|
Dennis Follensbee wrote, at 15.09.2008 17:12: > > I’ve recently encountered a difficulty with reading a field that is > defined as a double, but has zero decimal places of precision. DBFPy > should treat it as a float or force it to be an integer. I cannot change > the original data because the data is produced elsewhere. > > The error message I receive is: > > *ValueError: invalid literal for int() with base 10: '3320.'* this should be fixed with dbfpy v2.2.2. best wishes, alex. |
|
From: SourceForge.net <no...@so...> - 2008-09-15 22:47:32
|
Bugs item #1731650, was opened at 2007-06-05 22:53 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1731650&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Works For Me Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Real use problems. Initial Comment: I found a few problems while converting dbf files to SQL. 1. class DbfNumericFieldDef, method decodeValue returns int or float, float is a problem cause Numeric field is accurate, but float is floating point. Some old financial systems use Numeric fields to keep monetary data. 2. class DbfCurrencyFieldDef, method decodeValue returns float. It's questionable, just as above. I worked around above problems by changing decodeValue method, so that Numeric and Currency fields return string value. 3. class DbfDateTimeFieldDef, method decodeValue fails on Python 2.5, on Windows. there is problem (exception) in line: _rv += datetime.timedelta(0, _msecs / 1000.0) It started to work when I changed it to _rv += datetime.timedelta(0, _msecs // 1000.0) ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2008-09-16 08:47 Message: finally tested the third issue with 2.5.2. no exception has been raised. as for numeric decoding, i am still not sure if changing float to decimal would be a good thing. perhaps not. anyway, you always can write your own field type handler (subclass of DbfFieldDef) and install it with registerField function. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1731650&group_id=140566 |
|
From: Dennis F. <Den...@te...> - 2008-09-15 14:12:21
|
Hello, I've been using DBFPy for a project I'm finishing at work. I've recently encountered a difficulty with reading a field that is defined as a double, but has zero decimal places of precision. DBFPy should treat it as a float or force it to be an integer. I cannot change the original data because the data is produced elsewhere. The error message I receive is: ValueError: invalid literal for int() with base 10: '3320.' As a work around, I altered line 243 in fields.py from return int(value) to return int(float(value)) I'm not sure what the best way to handle this case actually is, but this way works for me. Is there any chance that the code could be updated to include an update for this case? Thanks! Dennis ======================= Dennis Follensbee Software Engineer ======================= 1.800.331.7881 x1721 twiki.teleatlas.com/DennisFollensbee <http://twiki.teleatlas.com/twiki/bin/view/Main/DennisFollensbee> "Simple things should be easy. Hard things should be possible." -Larry Wall ======================= |
|
From: Albert K. <alb...@ly...> - 2008-06-10 17:04:40
|
<html><head><style type="text/css">body{font:12px Arial;margin:3px;overflow-y:auto;overflow-x:auto}p{margin:0px;}blockquote, ol, ul{margin-top:0px;margin-bottom:0px;}</style></head>
<body><div style="display: block; font-family: Arial; font-size: 12px;"><p class="MsoHeader" style="">Hello,</p><p class="MsoHeader" style=""><br></p>
<p class="MsoHeader" style="">I want to remove a colum in the
dbf. How does it work?</p>
<p class="MsoHeader" style=""><!--[if !supportEmptyParas]--> <!--[endif]--><o:p></o:p></p>
<p class="MsoHeader" style="">Thank you for your Help</p><p class="MsoHeader" style=""><br></p><p class="MsoHeader" style="">Best regards Albert<br></p>
<p class="MsoHeader" style=""><span style="font-family: "Arial Unicode MS";"><!--[if !supportEmptyParas]--> <!--[endif]--><o:p></o:p></span></p>
<span style="font-size: 12pt; font-family: Arial;"> </span></br><p style="margin-top:11px;padding-top:3px;background-image: url(http://mail.lycos.co.uk/Images/Mail/_content/dot.gif);background-repeat: repeat-x;background-position: 0px 0px;"><br>
_________________________________
<br>
LOVE@LYCOS ist jetzt ganz neu - melde dich kostenlos an, lerne neue Leute kennen und finde deinen Flirt!<br>
<A href="http://bc.lycos-europe.net/lnl/bc.php?loveatlycos_promo_mailsig|http://love.lycos.de/start.action?utm_campaign=mp&utm_source=mailsig">Jetzt flirten!</a>
</div></body></html> |
|
From: alexander s. <al...@ty...> - 2008-06-10 10:20:37
|
Albert Krueger wrote, at 10.06.2008 10:53: > > I want to remove a colum in the dbf. How does it work? create new dbf with columns you need and copy contents. cheers, alex. |
|
From: Albert K. <alb...@ly...> - 2008-06-10 10:14:33
|
<html><head><style type="text/css">body{font:12px Arial;margin:3px;overflow-y:auto;overflow-x:auto}p{margin:0px;}blockquote, ol, ul{margin-top:0px;margin-bottom:0px;}</style></head>
<body><div style="display: block; font-family: Arial; font-size: 12px;"><p class="MsoHeader" style="">Hello,</p><p class="MsoHeader" style=""><br></p>
<p class="MsoHeader" style="">I want to remove a colum in the
dbf. How does it work?</p>
<p class="MsoHeader" style=""><!--[if !supportEmptyParas]--> <!--[endif]--><o:p></o:p></p>
<p class="MsoHeader" style="">Thank you for your Help</p><br><p class="MsoHeader" style=""><br></p><p class="MsoHeader" style="">Bets regards Albert<br></p>
<p class="MsoHeader" style=""><span style="font-family: "Arial Unicode MS";"><!--[if !supportEmptyParas]--> <!--[endif]--><o:p></o:p></span></p>
<span style="font-size: 12pt; font-family: Arial;"><br> </span></br><p style="margin-top:11px;padding-top:3px;background-image: url(http://mail.lycos.co.uk/Images/Mail/_content/dot.gif);background-repeat: repeat-x;background-position: 0px 0px;"><br>
_________________________________
<br>
LOVE@LYCOS ist jetzt ganz neu - melde dich kostenlos an, lerne neue Leute kennen und finde deinen Flirt!<br>
<A href="http://bc.lycos-europe.net/lnl/bc.php?loveatlycos_promo_mailsig|http://love.lycos.de/start.action?utm_campaign=mp&utm_source=mailsig">Jetzt flirten!</a>
</div></body></html> |
|
From: alexander s. <al...@ty...> - 2008-03-13 06:13:06
|
Nacho Uve wrote, at 12.03.2008 21:51: > Hello, > I have created a dbf file using dbfpy without any error message, but any > program was able to open it. Finally, I realize that dbfpy doesn't test > if the fieldnames has 10 or less characters. > Maybe it will be a nice improvement to do! thank you. the change is checked in cvs and will appear in next release. best wishes, alex. |
|
From: Nacho U. <nac...@gm...> - 2008-03-12 19:51:17
|
Hello,
I have created a dbf file using dbfpy without any error message, but any
program was able to open it. Finally, I realize that dbfpy doesn't test if
the fieldnames has 10 or less characters.
Maybe it will be a nice improvement to do!
At the moment, I've made a little change at header.py to solve this problem.
Best regards,
Nacho Varela
### header.py
####################################
def _addField(self, *defs):
"""Internal variant of the `addField` method.
This method doesn't set `self.changed` field to True.
Return value is a length of the appended records.
Note: this method doesn't modify ``recordLength`` and
``headerLength`` fields. Use `addField` instead of this
method if you don't exactly know what you're doing.
"""
# insure we have dbf.DbfFieldDef instances first (instantiation
# from the tuple could raise an error, in such a case I don't
# wanna add any of the definitions -- all will be ignored)
_defs = []
_recordLength = 0
for _def in defs:
if isinstance(_def, fields.DbfFieldDef):
_obj = _def
else:
(_name, _type, _len, _dec) = (tuple(_def) + (None,) * 4)[:4]
_cls = fields.lookupFor(_type)
### [Nacho Uve] Just take the first 10 characters of the
fieldname
_obj = _cls(_name[0:10], _len, _dec,
ignoreErrors=self._ignore_errors)
_recordLength += _obj.length
_defs.append(_obj)
# and now extend field definitions and
# update record length
self.fields += _defs
return _recordLength
|
|
From: SourceForge.net <no...@so...> - 2008-02-27 04:39:46
|
Feature Requests item #1901734, was opened at 2008-02-26 00:32 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Delete Fields Initial Comment: Not sure if anyone is working on this still, but it would be useful to be able to delete fields from a dbf file. I looked through the source files and only saw addFields and delete records methods...no delete fields. ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2008-02-27 06:39 Message: Logged In: YES user_id=8719 Originator: NO there is no way to delete a piece of data in the middle of a file other than copy all remaining data to a new file. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2008-02-26 23:17 Message: Logged In: NO I was hoping to write a python script that i could run on hundreds of files, removing the undesired fields from each file. Instead of copying all the data I want into a new file...for one file, yea, that's fine, but not looping over hundreds. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2008-02-26 09:24 Message: Logged In: YES user_id=8719 Originator: NO .addField() works only while the dbf is empty (no records added) and serves to create the structure from scratch. what would be the purpose of field deletion in that context? what you need is just create new file without unwanted field and copy data - that's pretty trivial. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2008-02-26 21:17:48
|
Feature Requests item #1901734, was opened at 2008-02-25 14:32 Message generated for change (Comment added) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Delete Fields Initial Comment: Not sure if anyone is working on this still, but it would be useful to be able to delete fields from a dbf file. I looked through the source files and only saw addFields and delete records methods...no delete fields. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2008-02-26 13:17 Message: Logged In: NO I was hoping to write a python script that i could run on hundreds of files, removing the undesired fields from each file. Instead of copying all the data I want into a new file...for one file, yea, that's fine, but not looping over hundreds. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2008-02-25 23:24 Message: Logged In: YES user_id=8719 Originator: NO .addField() works only while the dbf is empty (no records added) and serves to create the structure from scratch. what would be the purpose of field deletion in that context? what you need is just create new file without unwanted field and copy data - that's pretty trivial. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2008-02-26 07:24:03
|
Feature Requests item #1901734, was opened at 2008-02-26 00:32 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Delete Fields Initial Comment: Not sure if anyone is working on this still, but it would be useful to be able to delete fields from a dbf file. I looked through the source files and only saw addFields and delete records methods...no delete fields. ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2008-02-26 09:24 Message: Logged In: YES user_id=8719 Originator: NO .addField() works only while the dbf is empty (no records added) and serves to create the structure from scratch. what would be the purpose of field deletion in that context? what you need is just create new file without unwanted field and copy data - that's pretty trivial. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2008-02-25 22:32:24
|
Feature Requests item #1901734, was opened at 2008-02-25 14:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Delete Fields Initial Comment: Not sure if anyone is working on this still, but it would be useful to be able to delete fields from a dbf file. I looked through the source files and only saw addFields and delete records methods...no delete fields. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746846&aid=1901734&group_id=140566 |
|
From: <Jef...@Da...> - 2008-02-19 15:40:36
|
i see that only 10 bytes are reserved for the name out of the 32 |
|
From: <Jef...@Da...> - 2008-02-19 15:11:20
|
Hi I'm a first time dbfpy user as of yesterday. I've basically copied the example provided, with one small change. In the code I provided, field names of length 11 characters or greater are not written correctly, and eventually field names are left blank. Printing the dbf object shows that the internal representation is fine. Unless I'm mistaken, in the dbf spec "field descriptors" are 32bytes each, so these longer names should be fine. Any thoughts? Is there a better place to send this newbie question/potential bug? Jeff Dartmouth College |
|
From: SourceForge.net <no...@so...> - 2007-06-05 19:53:38
|
Bugs item #1731650, was opened at 2007-06-05 12:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1731650&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Real use problems. Initial Comment: I found a few problems while converting dbf files to SQL. 1. class DbfNumericFieldDef, method decodeValue returns int or float, float is a problem cause Numeric field is accurate, but float is floating point. Some old financial systems use Numeric fields to keep monetary data. 2. class DbfCurrencyFieldDef, method decodeValue returns float. It's questionable, just as above. I worked around above problems by changing decodeValue method, so that Numeric and Currency fields return string value. 3. class DbfDateTimeFieldDef, method decodeValue fails on Python 2.5, on Windows. there is problem (exception) in line: _rv += datetime.timedelta(0, _msecs / 1000.0) It started to work when I changed it to _rv += datetime.timedelta(0, _msecs // 1000.0) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1731650&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2007-02-11 13:58:51
|
Bugs item #1656884, was opened at 2007-02-10 20:31 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Drew Ferguson (blacktav) Assigned to: Nobody/Anonymous (nobody) Summary: Bad dbf date field strings Initial Comment: Hi Thanks for a life-saver project An application I have (SwissPerfect) stores bad date strings formatted with spaces including incomplete dates (eg year only). Attached is a diff on utils.py 2.1.0 with a function to parse such bad date strings and fix them ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2007-02-11 15:58 Message: Logged In: YES user_id=8719 Originator: NO oh yes, it does tolerate, but you'll be getting INVALID_VALUE instead of bad dates (and instead of bad numbers too, but INVALID_VALUE can be converted to numeric zero by simple int() or float() call). if you need, say, the year number from such dates, you'll have to apply either raw access trick or field type substitute. ---------------------------------------------------------------------- Comment By: Drew Ferguson (blacktav) Date: 2007-02-11 15:51 Message: Logged In: YES user_id=701865 Originator: YES Hey Alex You are some dude! dbfpy 2.2 tolerates all bad data out of the box Great work, very elegant solution too Drew ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-11 12:01 Message: Logged In: YES user_id=8719 Originator: NO Drew, i have just released dbfpy-2.2.0. with ignoreErrors=True you should be able to handle things like zero bytes in numeric fields. as for date fields containing non-date values, i guess the best hack is to do:: from dbfpy import record class DateString(record.DbfCharacterFieldDef): typeCode = "D" length = 8 record.registerField(DateStrings) before opening your dbf file. then you will get strings instead of datetime objects, and process them in any way you wish. ---------------------------------------------------------------------- Comment By: Drew Ferguson (blacktav) Date: 2007-02-11 01:10 Message: Logged In: YES user_id=701865 Originator: YES Hi Alex Wow, quick response! Thanks Trap for embedded spaces is fine but partial dates (eg year only) continues to barf. Yes, enforcing a valid (python) date is not good. Even with access to "broken" raw data this is a scary can-of-worms; expecting a valid python date from applications which evidently permit partial dates (by accident or design) isn't good either. I hit another issue too where a numeric field barfs on int(value) with "null byte in argument for int()" I presume this is bad data too. I can't see easy long-term fixes here so I guess I'll live with my hacks and wish you the best. Again, thanks for all your hard-work in dbfpy, and allowing my hacks to be trivial to implement. If you want the broken source data files let me know. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 21:58 Message: Logged In: YES user_id=8719 Originator: NO yet another quick fix: i've added API to access raw DBF data. with current version you can catch conversion errors and then obtain offending values as:: dbf.header[field_name_or_number].decodeFromRecord( rec.rawFromStream(dbf, record_index)) where rec may be DbfRecord class or any valid record object. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 21:03 Message: Logged In: YES user_id=8719 Originator: NO thank you for reporting it. it's a +1 for the plan to make the Dbf class to optionally ignore all conversion errors, as i said in https://sourceforge.net/mailarchive/message.php?msg_id=37890707 however, i am not given to "fix" invalid values by replacing missing date parts by an arbitrary number. perhaps returning None instead of invalid dates would be more consistent... i have checked in a quick fix allowing date strings to have leading spaces instead of zeroes. if that solves your problem, i will release new dbfpy version as soon as possible. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2007-02-11 13:51:45
|
Bugs item #1656884, was opened at 2007-02-10 18:31 Message generated for change (Comment added) made by blacktav You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Drew Ferguson (blacktav) Assigned to: Nobody/Anonymous (nobody) Summary: Bad dbf date field strings Initial Comment: Hi Thanks for a life-saver project An application I have (SwissPerfect) stores bad date strings formatted with spaces including incomplete dates (eg year only). Attached is a diff on utils.py 2.1.0 with a function to parse such bad date strings and fix them ---------------------------------------------------------------------- >Comment By: Drew Ferguson (blacktav) Date: 2007-02-11 13:51 Message: Logged In: YES user_id=701865 Originator: YES Hey Alex You are some dude! dbfpy 2.2 tolerates all bad data out of the box Great work, very elegant solution too Drew ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-11 10:01 Message: Logged In: YES user_id=8719 Originator: NO Drew, i have just released dbfpy-2.2.0. with ignoreErrors=True you should be able to handle things like zero bytes in numeric fields. as for date fields containing non-date values, i guess the best hack is to do:: from dbfpy import record class DateString(record.DbfCharacterFieldDef): typeCode = "D" length = 8 record.registerField(DateStrings) before opening your dbf file. then you will get strings instead of datetime objects, and process them in any way you wish. ---------------------------------------------------------------------- Comment By: Drew Ferguson (blacktav) Date: 2007-02-10 23:10 Message: Logged In: YES user_id=701865 Originator: YES Hi Alex Wow, quick response! Thanks Trap for embedded spaces is fine but partial dates (eg year only) continues to barf. Yes, enforcing a valid (python) date is not good. Even with access to "broken" raw data this is a scary can-of-worms; expecting a valid python date from applications which evidently permit partial dates (by accident or design) isn't good either. I hit another issue too where a numeric field barfs on int(value) with "null byte in argument for int()" I presume this is bad data too. I can't see easy long-term fixes here so I guess I'll live with my hacks and wish you the best. Again, thanks for all your hard-work in dbfpy, and allowing my hacks to be trivial to implement. If you want the broken source data files let me know. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 19:58 Message: Logged In: YES user_id=8719 Originator: NO yet another quick fix: i've added API to access raw DBF data. with current version you can catch conversion errors and then obtain offending values as:: dbf.header[field_name_or_number].decodeFromRecord( rec.rawFromStream(dbf, record_index)) where rec may be DbfRecord class or any valid record object. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 19:03 Message: Logged In: YES user_id=8719 Originator: NO thank you for reporting it. it's a +1 for the plan to make the Dbf class to optionally ignore all conversion errors, as i said in https://sourceforge.net/mailarchive/message.php?msg_id=37890707 however, i am not given to "fix" invalid values by replacing missing date parts by an arbitrary number. perhaps returning None instead of invalid dates would be more consistent... i have checked in a quick fix allowing date strings to have leading spaces instead of zeroes. if that solves your problem, i will release new dbfpy version as soon as possible. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 |
|
From: SourceForge.net <no...@so...> - 2007-02-11 10:01:34
|
Bugs item #1656884, was opened at 2007-02-10 20:31 Message generated for change (Comment added) made by a1s You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Drew Ferguson (blacktav) Assigned to: Nobody/Anonymous (nobody) Summary: Bad dbf date field strings Initial Comment: Hi Thanks for a life-saver project An application I have (SwissPerfect) stores bad date strings formatted with spaces including incomplete dates (eg year only). Attached is a diff on utils.py 2.1.0 with a function to parse such bad date strings and fix them ---------------------------------------------------------------------- >Comment By: alexander smishlajev (a1s) Date: 2007-02-11 12:01 Message: Logged In: YES user_id=8719 Originator: NO Drew, i have just released dbfpy-2.2.0. with ignoreErrors=True you should be able to handle things like zero bytes in numeric fields. as for date fields containing non-date values, i guess the best hack is to do:: from dbfpy import record class DateString(record.DbfCharacterFieldDef): typeCode = "D" length = 8 record.registerField(DateStrings) before opening your dbf file. then you will get strings instead of datetime objects, and process them in any way you wish. ---------------------------------------------------------------------- Comment By: Drew Ferguson (blacktav) Date: 2007-02-11 01:10 Message: Logged In: YES user_id=701865 Originator: YES Hi Alex Wow, quick response! Thanks Trap for embedded spaces is fine but partial dates (eg year only) continues to barf. Yes, enforcing a valid (python) date is not good. Even with access to "broken" raw data this is a scary can-of-worms; expecting a valid python date from applications which evidently permit partial dates (by accident or design) isn't good either. I hit another issue too where a numeric field barfs on int(value) with "null byte in argument for int()" I presume this is bad data too. I can't see easy long-term fixes here so I guess I'll live with my hacks and wish you the best. Again, thanks for all your hard-work in dbfpy, and allowing my hacks to be trivial to implement. If you want the broken source data files let me know. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 21:58 Message: Logged In: YES user_id=8719 Originator: NO yet another quick fix: i've added API to access raw DBF data. with current version you can catch conversion errors and then obtain offending values as:: dbf.header[field_name_or_number].decodeFromRecord( rec.rawFromStream(dbf, record_index)) where rec may be DbfRecord class or any valid record object. ---------------------------------------------------------------------- Comment By: alexander smishlajev (a1s) Date: 2007-02-10 21:03 Message: Logged In: YES user_id=8719 Originator: NO thank you for reporting it. it's a +1 for the plan to make the Dbf class to optionally ignore all conversion errors, as i said in https://sourceforge.net/mailarchive/message.php?msg_id=37890707 however, i am not given to "fix" invalid values by replacing missing date parts by an arbitrary number. perhaps returning None instead of invalid dates would be more consistent... i have checked in a quick fix allowing date strings to have leading spaces instead of zeroes. if that solves your problem, i will release new dbfpy version as soon as possible. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=746843&aid=1656884&group_id=140566 |
;background-repeat:){kind=link}