Thread: [Passwordsafe-devel] Enhancement Topics (Format and Usage Bugs)
Popular easy-to-use and secure password manager
Brought to you by:
ronys
Menu
▾
▴
passwordsafe-devel
|
From: Wolfgang K. <91...@gm...> - 2007-05-07 16:08:57
|
Hello Early this year I released JPasswords 0.4.0 (http://sourceforge.net/projects/jpws) which kept track of the newer development of PWS file format. I am now working on the next release and would like to carry some of my observations and ideas to here in order to enhance fitness of the V3 file format for multi-application usage. It looks like I have 4 major topics which I will bring in as separate post for discussion. *** *Format Bugs and Shortcomings* *** I'll start with this one. The points here all concern the file header data including the header fields (referring to PWS 3.07). A. Unfortunate usage a) Field 02, the file-UUID, is recreated with every save operation of PWS. This in my view is against the genuine idea and purpose of an UUID, namely to identify a file over space and time. Looks like a bug to me and would be glad to see it improved. (The policy I have adopted in JPWS is that the original file and all of its backups keep the identical UUID value, while explicit copies (e.g. "save as") performed throught the program carry a new UUID.) b) The setting of security loops ("ITER" in format description) is likewise replaced by the PWS standard value of 2048 with every save. This also looks unfortunate to me as other applications might choose to set up a special value which then is lost. So my recommendation for these points is that the save process should impose new/standard values only if it doesn't find meaningful values already existing. B. Errors a) Header field 04 is obviousely falsely programmed. PWS generated a content value which is 9 bytes long, which is already non-conforming the format definition. According to definition this should be a bit-formatted integer value (of 4 or 8 bytes length). The factual value looks like a null-terminated string to me. So this should be corrected. b) With the definition of header field 05 (user) it looks to me that once again the "naturalistic fallacy" of seeing text strings as 1-to-1 symbol and encoding length might have struck. No encoding rule is provided (ASCII encoding seems too weak for international use!) and the length information has an unclear target (concerning encoded length or text length?). My suggestion is to replace the entire definition with just a normal text string; I don't see any urgent need for splitting it up. c) Interesting enough, a definition of the "Text" data type in FormatV3.txt has either never existed or disappeared (file revision 1216 / 17.01.2007). Can we fix it into something reliable like UTF-8 plain, without leading or trailing string or format or length tokens? Until points are clearified I will stay with format V3.00 with JPWS. Cheers! - Wolfgang Keller |
|
From: Frank P. <fp...@fp...> - 2007-05-07 23:59:38
|
On Mon, 07 May 2007 12:02:03 -0400, Wolfgang Keller <91...@gm...> wrote:
>
> looks like I have 4 major topics which I will bring in as separate post
> for discussion.
>
Hi Wolfgang,
just wanted to point out that these are implementation issues rather
than format issues. The formatV3.txt document should be definitive,
and implementations ("Password Safe Clones") should conform to that
document.
If Password Safe or any other implementation does not comply with the
format document, the implementation should be fixed rather than the
document. Once some clones begin copying other implementations'
issues and bugs, we lose interoperability.
The formatV3.txt shall be concise enough that no incompatible
interpretations can exist.
>
> b) The setting of security loops ("ITER" in format description) is
> likewise replaced by the PWS standard value of 2048 with every save.
> This also looks unfortunate to me as other applications might choose to
> set up a special value which then is lost.
>
Just FYI, my implementation retains the number of iterations across save
operations, and prints a warning when a value less than 2048 is used.
Again, this is an implementation detail.
>
> c) Interesting enough, a definition of the "Text" data type in
> FormatV3.txt has either never existed or disappeared (file revision 1216
> / 17.01.2007). Can we fix it into something reliable like UTF-8 plain,
> without leading or trailing string or format or length tokens?
>
Indeed. I seem to remember that we finalized on using UTF-8, with no null
byte at the end. Yet I can find no record of that decision in the current
formatV3.txt document. Rony, can you please add a note to the formatV3.txt
file to that effect, e.g.,
Text fields are stored using the UTF-8 character encoding. No null
character is stored in the record data or counted as part of the field
length. I.e., the ASCII string "Hello World" is stored as a single
block, with the field length set to 11.
Frank
--
Frank Pilhofer, fp...@fp...
|
|
From: Wolfgang K. <91...@gm...> - 2007-05-08 09:40:10
|
> > Frank Pilhofer wrote: > > Indeed. I seem to remember that we finalized on using UTF-8, with no null > byte at the end. Yet I can find no record of that decision in the current > formatV3.txt document. Rony, can you please add a note to the formatV3.txt > file to that effect, e.g., > > Text fields are stored using the UTF-8 character encoding. No null > character is stored in the record data or counted as part of the field > length. I.e., the ASCII string "Hello World" is stored as a single > block, with the field length set to 11. The definition is still to weak or misleading. Without being an expert in this, I know that UTF-8 may come with something they call "BOM" as a prefix token. Moreover, UTF-8 encoding is a bit-format, not a text format and I would not expect that null bytes are to be excluded from being a valid part of it. So I suggest the following text as definition for "Text": Text fields are stored using the plain UTF-8 character encoding without use of encoding signatures (BOM) or leading or trailing data length markers. I.e., the ASCII string "Hello World" is stored as a single block, with the field length set to 11. - Wolfgang |
|
From: Frank P. <fp...@fp...> - 2007-05-08 21:52:11
|
On Tue, 08 May 2007 02:53:27 -0400, Wolfgang Keller <91...@gm...> wrote:
>
> The definition is still to weak or misleading. Without being an expert
> in this, I know that UTF-8 may come with something they call "BOM" as a
> prefix token. Moreover, UTF-8 encoding is a bit-format, not a text
> format and I would not expect that null bytes are to be excluded from
> being a valid part of it. So I suggest the following text as definition
> for "Text":
>
Hi Wolfgang and all,
BOMs ("Byte Order Marks") may be used with UTF-16 or UTF-32 to indicate
that a code unit sequence is serialized in either big or little endian
order. UTF-8 is byte oriented, and has no need for BOMs.
Let's just delegate the responsibility to the appropriate party, with the
appropriate references.
Text fields are stored using the UTF-8 encoding scheme (see definition
D-39 of the Unicode Standard 4.0 at
http://www.unicode.org/versions/Unicode4.0.0/ch03.pdf). Note that a
Unicode string (D-29a) does not contain any terminating NUL character
that might exist in a C language implementation; consequently, no
NUL character is stored or counted as part of the field length. I.e.,
the ASCII string "Hello World" is stored as a single block, with the
field length set to 11.
We could strike the second sentence, because it is implicit from the first,
but given that I have been recently bitten by an incompatibility involving
the null character recently, I'd like to see it included.
Frank
--
Frank Pilhofer, fp...@fp...
|
|
From: ronys <ro...@gm...> - 2007-05-09 12:42:50
|
Hi folks, Good discussion. I agree that the format description is one of the more important parts of the project (probably what will outlive the application itself, not to mention myself). I've updated the document a bit, adding an explicit 'representation' section 3.1: http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pws afe/docs/formatV3.txt?view=markup Constructive comments, as usual, are welcome. Other comments > /dev/null. Cheers, Rony |
|
From: Wolfgang K. <91...@gm...> - 2007-05-09 15:11:24
|
Looks good to me. I have one point with the *time_t* format. - FYI, the 32-bit integer value has a final date of 2106-02-07 07:28:15 CEST. People, this is less than 100 years from now! :) - I'ld strongly suggest to declare the 64-bit value as a valid alternative for time_t. Note that this does not conflict existing 32-bit values in little endian orientation. Readers just may detect the length of a value; also readers who only read the first 4 bytes still gain the correct values (until 2106). - Wolfgang ronys wrote: > Hi folks, > > Good discussion. I agree that the format description is one of the more > important parts of the project (probably what will outlive the application > itself, not to mention myself). > > I've updated the document a bit, adding an explicit 'representation' section > 3.1: > http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pws > afe/docs/formatV3.txt?view=markup > > Constructive comments, as usual, are welcome. Other comments > /dev/null. > > Cheers, > > Rony > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Passwordsafe-devel mailing list > Pas...@li... > https://lists.sourceforge.net/lists/listinfo/passwordsafe-devel > > |
|
From: dk <dk...@gm...> - 2007-05-09 17:36:06
|
Actually... On most systems, time_32t is a signed 32-bit integer with a max. of 03:14:07 UTC on Tuesday, January 19, 2038, and only if made "unsigned" does it go on to 2106. time_64t on most systems is a signed 64-bit integer. If treated as "unsigned", it would have a max. on December 4, 292,277,026,596. I doubt very much that I (or PasswordSafe) will be here for the second date (290+ billion years time) and I think it is unlikely that I will really care about it on the first! David -----Original Message----- From: pas...@li... [mailto:pas...@li...] On Behalf Of Wolfgang Keller Sent: 09 May 2007 16:11 To: pas...@li... Subject: Re: [Passwordsafe-devel] Enhancement Topics (Format and Usage Bugs) Looks good to me. I have one point with the *time_t* format. - FYI, the 32-bit integer value has a final date of 2106-02-07 07:28:15 CEST. People, this is less than 100 years from now! :) - I'ld strongly suggest to declare the 64-bit value as a valid alternative for time_t. Note that this does not conflict existing 32-bit values in little endian orientation. Readers just may detect the length of a value; also readers who only read the first 4 bytes still gain the correct values (until 2106). - Wolfgang ronys wrote: > Hi folks, > > Good discussion. I agree that the format description is one of the > more important parts of the project (probably what will outlive the > application itself, not to mention myself). > > I've updated the document a bit, adding an explicit 'representation' > section > 3.1: > http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsa > fe/pws > afe/docs/formatV3.txt?view=markup > > Constructive comments, as usual, are welcome. Other comments > /dev/null. > > Cheers, > > Rony > > > ---------------------------------------------------------------------- > --- This SF.net email is sponsored by DB2 Express Download DB2 Express > C - the FREE version of DB2 express and take control of your XML. No > limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Passwordsafe-devel mailing list > Pas...@li... > https://lists.sourceforge.net/lists/listinfo/passwordsafe-devel > > ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Passwordsafe-devel mailing list Pas...@li... https://lists.sourceforge.net/lists/listinfo/passwordsafe-devel |
|
From: Wolfgang K. <91...@gm...> - 2007-05-10 06:27:19
|
dk wrote: > Actually... > > On most systems, time_32t is a signed 32-bit integer with a max. of 03:14:07 > UTC on Tuesday, January 19, 2038, and only if made "unsigned" does it go on > to 2106. > > time_64t on most systems is a signed 64-bit integer. If treated as > "unsigned", it would have a max. on December 4, 292,277,026,596. > > I doubt very much that I (or PasswordSafe) will be here for the second date > (290+ billion years time) and I think it is unlikely that I will really care > about it on the first! > > David > > > Are you saying that you are developing Password Safe for your own purposes? - Indeed in this case these considerations are irrelevant, but also your contradiction is! Allowing a 64-bit value as *alternative* does not cause problems but brings benefits, so why not admit it? - It does not cost more storage and all readers who are not interested in 8 bytes have to do is just read the first 4 bytes, which they probably are doing already anyway. - Wolfgang |
|
From: David G. <dk...@gm...> - 2007-05-09 22:10:01
|
Actually..... On most systems, time_32t is a signed 32-bit integer with a max. of 03:14:07 UTC on Tuesday, January 19, 2038, and only if made "unsigned" does it go on to 2106. time_64t on most systems is a signed 64-bit integer. If treated as "unsigned", it would have a max. on December 4, 292,277,026,596. I doubt very much that I (or PasswordSafe) will be here for the second date (290+ billion years time) and I think it is unlikely that I will really care about it on the first! David -----Original Message----- From: pas...@li... [mailto:pas...@li...] On Behalf Of Wolfgang Keller Sent: 09 May 2007 16:11 To: pas...@li... Subject: Re: [Passwordsafe-devel] Enhancement Topics (Format and Usage Bugs) Looks good to me. I have one point with the *time_t* format. - FYI, the 32-bit integer value has a final date of 2106-02-07 07:28:15 CEST. People, this is less than 100 years from now! :) - I'ld strongly suggest to declare the 64-bit value as a valid alternative for time_t. Note that this does not conflict existing 32-bit values in little endian orientation. Readers just may detect the length of a value; also readers who only read the first 4 bytes still gain the correct values (until 2106). - Wolfgang ronys wrote: > Hi folks, > > Good discussion. I agree that the format description is one of the > more important parts of the project (probably what will outlive the > application itself, not to mention myself). > > I've updated the document a bit, adding an explicit 'representation' > section > 3.1: > http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsa > fe/pws > afe/docs/formatV3.txt?view=markup > > Constructive comments, as usual, are welcome. Other comments > /dev/null. > > Cheers, > > Rony > > > ---------------------------------------------------------------------- > --- This SF.net email is sponsored by DB2 Express Download DB2 Express > C - the FREE version of DB2 express and take control of your XML. No > limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Passwordsafe-devel mailing list > Pas...@li... > https://lists.sourceforge.net/lists/listinfo/passwordsafe-devel > > ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Passwordsafe-devel mailing list Pas...@li... https://lists.sourceforge.net/lists/listinfo/passwordsafe-devel |
|
From: Wolfgang K. <91...@gm...> - 2007-05-10 06:49:39
|
> > > _A. Wolfgang's *New Header Data Fields*_ > > Field 07: (type "Text" maximum length 128) Database name. A logical > name for a database which can be used by applications in place of the > possibly lengthy filepath notion. > > Field 08: (type "Text" maximum length 256) Database Description. A > purely informative description concerning the purpose or other > practical use of the database. > > I don't mind but why the length limitation? What would we do with > them? How would they be entered and where would they be displayed > (apart from in the Database Property dialog "File Menu->Properties")? > > _B. Wolfgang's *Tolerating Unknown Fields*_ > > I was going to agree that we should copy "unknown" header fields found > in the header on opening a database back into the header when saving > it rather than just ignoring them. I was not so sure that we should > do this for unknown record fields. > > The problem is how/where to save them between reading them in and > writing them out! As we do not know their structure, they may have a > dependence on other fields that we do not know. Changing "our" PWS > fields may cause the other application to fail or corrupt data. What > about overlaps (someone somewhere unknown to PWS developers using > field X currently not used by PWS) but then PWS starts to use it? > > So I changed my mind - best to ignore unknown fields. If they are > there, the user should open the database in read-only mode to protect > them. In fact, maybe PWS should insist a database is read-only if it > detects unknown fields? > > _C. Wolfgang's *Format Bugs and Shortcomings*_ > > I agree that header field 02, the file-UUID, should not be recreated > with every save operation of PWS and I will try to get this in V3.08. > > I agree that the value of ITER should be at least 2048 but if an > existing database uses a greater value, then it should be retained. > Again, I will try to get this in V3.08. > > I agree that header field 04 is 'falsely programmed'. It is 8 > hexadecimal characters long (format %08x) representation of a time_t > 32-bit long integer rather than the raw "time_t" value. Two possible > solutions: a) change the file "formatV3.txt" to say its current format > b) change to write it out in raw 'time_t' format and program around > the data being in one or other form (I suppose based on the field > length of 4 or 8). > > Regards, > > David > * Fields 07 and 08: The length limitation is to prevent abuse and to give a reasonable frame for usage. The name clearly needs a limitation. The description might be argued to be longer but I prefer it 256 max. (Compare it with project descriptions in SF.) This way the user is forced to stay with the essential, which is its genuine purpose, second you can display it as a label without cutting considerations. - If and how you use these fields is up to you. * Format Bugs * Please remember that point B.b) with header field 05 (user) is still open to clearify. - Wolfgang _ _ |
|
From: dk <dk...@gm...> - 2007-05-10 19:36:24
|
>From the code, the current implementation for the user field 0x05 is a text string that identifies the user who made the last save and the host the user was logged on to at the time. The exact format is: User field length: 4 bytes (hexadecimal representation of the length of the username) - used to determine the start of the host name text field. User name: text Host name: text Total length of this field is: 4 + length of username + length of host name David _____ From: pas...@li... [mailto:pas...@li...] On Behalf Of Wolfgang Keller Sent: 10 May 2007 07:50 To: pas...@li... Subject: Re: [Passwordsafe-devel] Enhancement Topics (Format and Usage Bugs) A. Wolfgang's *New Header Data Fields* Field 07: (type "Text" maximum length 128) Database name. A logical name for a database which can be used by applications in place of the possibly lengthy filepath notion. Field 08: (type "Text" maximum length 256) Database Description. A purely informative description concerning the purpose or other practical use of the database. I don't mind but why the length limitation? What would we do with them? How would they be entered and where would they be displayed (apart from in the Database Property dialog "File Menu->Properties")? B. Wolfgang's *Tolerating Unknown Fields* I was going to agree that we should copy "unknown" header fields found in the header on opening a database back into the header when saving it rather than just ignoring them. I was not so sure that we should do this for unknown record fields. The problem is how/where to save them between reading them in and writing them out! As we do not know their structure, they may have a dependence on other fields that we do not know. Changing "our" PWS fields may cause the other application to fail or corrupt data. What about overlaps (someone somewhere unknown to PWS developers using field X currently not used by PWS) but then PWS starts to use it? So I changed my mind - best to ignore unknown fields. If they are there, the user should open the database in read-only mode to protect them. In fact, maybe PWS should insist a database is read-only if it detects unknown fields? C. Wolfgang's *Format Bugs and Shortcomings* I agree that header field 02, the file-UUID, should not be recreated with every save operation of PWS and I will try to get this in V3.08. I agree that the value of ITER should be at least 2048 but if an existing database uses a greater value, then it should be retained. Again, I will try to get this in V3.08. I agree that header field 04 is 'falsely programmed'. It is 8 hexadecimal characters long (format %08x) representation of a time_t 32-bit long integer rather than the raw "time_t" value. Two possible solutions: a) change the file "formatV3.txt" to say its current format b) change to write it out in raw 'time_t' format and program around the data being in one or other form (I suppose based on the field length of 4 or 8). Regards, David * Fields 07 and 08: The length limitation is to prevent abuse and to give a reasonable frame for usage. The name clearly needs a limitation. The description might be argued to be longer but I prefer it 256 max. (Compare it with project descriptions in SF.) This way the user is forced to stay with the essential, which is its genuine purpose, second you can display it as a label without cutting considerations. - If and how you use these fields is up to you. * Format Bugs * Please remember that point B.b) with header field 05 (user) is still open to clearify. - Wolfgang |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X