From: Jim S. <ja...@ne...> - 2004-09-29 15:48:19
|
Aleksey Karyakin wrote: >>This question has been around longer than Interbase. The great unwritten >>component is traditionally called the mega-database manager. >> >> > >Why unwritten ? I thought there are lots of Mega Database Managers on the >market. Most "big" DBMS have such capability not counting separate products. > > > It just never made it to the top of the list. A client willing to pay for it would have moved it up, but the client has never stepped forward. Even Sean has declined to pay for it. >What's the difference between performing a join (for example) using two >tables in the single database or from the separate database files on the >same Firebird server ? Two tables are just objects loaded in server memory >at some point. > > It's not nearly that simple. The table itself is materialized record by record, never as a single entity. What version of a record is appropriate for a particular transaction, the exact format of the record, whether garbage collection needs to be performed are all driven by relatively complex, interlocked data structures that drive a database engine. The data structures, in turn, are isolated from each other by the V-valve. Extending the engine to allow cross talk across two or more physical databases would required a total resign of the system. And while it well might benefit from a ground up redesign, it probably wouldn't be Firebird compatible when it was finished. For example, if you were going to start over, why keep BLR? But if you don't keep BLR, you've cut yourself off from the entire installed base. A better solution is to layer a solution on existing engines. The MDM would be a provider using the same API as the engine, the remote interface, and the gateway. It would prepare DSQL statements by decomposing it into SQL statements for execution on that target bases, calling back into the Y-valve for access, and coordinating the results. The implementation would be smaller and simpler than the changes required to do the same thing in the engine, and would automatically work for remote databases, engines of different version, and the gateway to Interbase and legacy Firebird. -- Jim Starkey Netfrastructure, Inc. 978 526-1376 |
From: Jim S. <ja...@ne...> - 2004-09-24 15:30:13
|
Rather than respond individual to so many interest posts, let me batch some comments. First, I'd like to ammend my proposal with the addition of upgrade user <username> ... The "upgrade user" command is syntactically parallel to "create user", and is semantically equivalent to "alter user" if the named user exists and semantically equivalent to "create user" if not. The upgrade form allows a single script to both create and update accounts. Dmitry proposed handling a user like other database objects and to separate "user name" used in the DDL command and an "identified by" string for logon purposes. Other database objects (tables, fields, triggers) are defined in core system tables. It seems obvious that users/accounts should be handled this way. There seems broad consensus that authentication should delegated to a security plugin, and different security plugins will have different ideas of what should be stored in a system table. We could define a core system table FB$USERS (folks, we're not rdb anymore) with account name as primary key. All other fields would necessarily be defined by whatever security plugin was configured. Defining a system table with one known field and an unknown number of other fields just doesn't feel satisfying. If someone could come up with a scheme that was both useful and open-ended, I could be convinced otherwise, but for now, an architecturally specified table to hold user data doesn't seem worth the effort, particularly when the legacy form of authentication doesn't handle database specific account information at all. I think I understand the motivation for separating user name and user indentification, but I don't think it's a good idea. It could be marginally useful, but it also be a source of endless confusion over which attributes, name or identification, was used in which context. For simplicity, which has virtual, let's stick with a single identifier for a database user. That said, there is merit to record on user information like full name, fax number, and mother's maiden name. Not all security models will want to track this stuff, but the fb_update_account_info mechanism should be able to accomodate it providing we can successfully manage the APB attribute space... For the purpose of discussion, I envision the set of security plugs would include: 1. Central security database for backwards compatibility (and somebody even might like it) 2. User/passwords stored in target database 3. PAM based for *ix systems 4. NT Domain based for Windows systems 5. LDAP based for large organizations There will probably be more, but there won't be any fewer. Someone raised the question as to why the network connection database parameter provided for a list of addresses. I did this because Interbase/Firebird/Vulcan TCP servers can redirect a connection across the network. I want to security plugin to have this information. In general, only the last (first in order) address is reliable (subject to Nickolay's final approval, of course), but if other addresses can be identified as trustworthy, the chain has some value. In any case, however, the other identification dpb parameters all provide lists of transmission agents by version, type, and product. For what it's worth, Interbase supposed TCP, DECnet, Apollo ring, and transciently, Jim's-really-crappy-asynchronous-line-protocol. Interbase servers were used as inter-network gateways all the time. The database handle in fd_update_account_info is the target the database. The security database is the business of a future plugin and, in general, inaccessible. The fact that SHA (and MD5) may have collisions is irrelevent for password hashing. The question is not whether collisions are possible, but whether you can compute a "collision" given a hash. Alex has proposed an spb parameter to specify security database for gsec. This is a very bad idea. Security configuration data should reside solely in a write protected file. Moving this stuff to a utility compounds mistakes made in the past. Alex has also suggested that fb_update_account_info need not be thread safe. This is incorrect. The entire API must be thread safe. All security plugins themselves must also be thread safe. No component, whether Y-valve, remote interface, engine, or gateway, presumes to manage threading for any other component. Every component is responsible for it's own thread safety. -- Jim Starkey Netfrastructure, Inc. 978 526-1376 |
From: Geoff W. <ge...@te...> - 2004-09-25 00:39:58
|
Hi Jim, In one email you said: > I think I understand the motivation for separating user name > and user indentification, but I don't think it's a good idea. So you are not a believer in surrogate keys? I know that not all database developers use them, but I do. The arguments for and against surrogate keys are the arguments for and against separating the user name from the user id - and I dont expect we will resolve those arguments conclusively here. If we do not use a surrogate identifier then we will need to discover a way of keeping user names synchronised, or ensure the API prevents changes to logon user names. Anything else provides the potential for holes in security. For example: Senior administrator John Citizen with the user name JCITIZEN leaves the company, and eventually his user name is deleted from the security database. With no sync of user details between databases all his privilegs remain defined. Sometime later his distant cousin Joe Citizen (from the dark side of the Citizen family) fakes his references and gets job with the same company. He is added to the security database as JCITIZEN with supposedly minor privileges, only to discover he has inherited all the privileges of his trustworthy cousin. I leave you with the image of Joe rubbing his hands together with glee! In a later email you said: > Sean, what does the security plugin that emulates the current > semantics do? Do it replicate FB$USERS from the security > database? Does it try to intercept references to FB$USERS > and redirect to the security database? Replication, of > course, is impossible for a readonly database. The > redirection, on the other hand, would be massively > complicated to implement, requiring a very "rich" security > plugin API. If we truly intend to create good and flexible security options for Firebird then I suspect that a "rich" API will indeed be necessary. We separate user authentication from the application databases completely. Databases may carry the users-as-objects table, if we decide username to userid translation is necessary. And of course they carry the existing roles/privileges stuff. It is up to the security API to have its own database of user definitions (alongs with passwords mgmt etc). Most of the systems we are talking about already have their own databases of one sort or another, so why would we try to carry a duplication in the application database? My preference would be that an application database that does not use the default security.fdb would need to be "registered" on the server to say what security system should be used to authenticate access (and so avoid the possibility that users from two separate security systems may duplicate a user name or user id and so allow unintended access). Such registration may even specify an equivalent security.fdb style database, just a separate one available on the same server (and so allow different applications to have different sets of users if this is desired). None of this solves how to give access to read-only databases. It seems to me that the security API may even need to be rich enough to support its own registration of databases in which user-to-role definitions can be added. This implies that such APIs (if they already have their own user database for "normal" user authentication requirements) may need to implement a new/separate database to support this requirement. (Or perhaps the server can be built to provide this service on behalf of the security plugins. Such plugins would register with the server and use the server to save/read these details to the server database provided for this purpose.) The result would be a recommendation that application databases use roles as the primary way of specifying permissions, as this allows access to be defined externally even if the database is made read-only. User name/id references inside an application database would be primarily audittrail type references, not permission references. Another result is that we would not want username to userid translation inside the application database. We would (if we use user-ids rather than user names) always store security system user ids (and so such ids provided by the API would have to fit with whatever field definition we decide on). The alternative to this would be that the server security service I spoke of above, would also provide the database user object name to plugin user id translation - and perhaps this indirection would be a good idea. I dont think the final system would be quite as convoluted as the above appears. There are many reasons to want the user-authentication kept separate, just as there are many reasons to want the distinction between server properties and database properties isolated more appropriately than they are now. Many server-specific things have crept into the database over the years, its time they were moved back out to the server where they belong. (All IMHO of course :-) -- Geoff Worboys Telesis Computing |
From: Mark O'D. <mar...@fi...> - 2004-09-25 00:55:51
|
Jim Starkey wrote: > > First, I'd like to ammend my proposal with the addition of > > upgrade user <username> ... > Like seemingly everyone else here, I also like the idea of an SQL "CREATE USER" command. But given we already have "recreate table", "recreate procedure" what about "recreate user" rather than "upgrade user"? Whatever is chosen it would be good to have a consistant keyword usage. (we could always do "create or replace user". :-). > > Dmitry proposed handling a user like other database objects and to > separate "user name" used in the DDL command and an "identified by" > string for logon purposes. > I also think Dmitry's idea of having the ability to store the user in the database has merit (and as I understand he implemented in a private tree last year). > We could define a core system table FB$USERS (folks, we're not rdb > anymore) with account name as primary key. All other fields would > necessarily be defined by whatever security plugin was configured. The table should only need to support the "internal" passwd hash mechanism as a default fallback mechanism if not using a plugin. Plugins are responsible for their own data, and it's unlikely they will need extra fields in a FB$USER table. For instance using LDAP, SASL, and NTLM will store the passwd hash in their own databases. If needed at all they should probably add a new table. > For the purpose of discussion, I envision the set of security plugs > would include: > > 1. Central security database for backwards compatibility (and > somebody even might like it) 2. User/passwords stored in target > database 3. PAM based for *ix systems 4. NT Domain based for Windows > systems 5. LDAP based for large organizations For unix there is PAM which then already has modules for everything else (including NTLM). so you only need PAM. Surely, since this is such a smart idea, on windows there is something similar? > Someone raised the question as to why the network connection database > parameter provided for a list of addresses. I did this because > Interbase/Firebird/Vulcan TCP servers can redirect a connection > across the network. Will this be similar to a prior issue where an IP address was passed back from the server to the client, for the client to make an extra connection for event processing? As this can causes havoc when tunnelling through NATed firewalls. Curiously Oracle has a similar problem, (I nearly mentioned it last time this came up). Oracle server has three modes, dedicated server, preallocated dedicated and multithreaded server. In all, except for the first, after the server/listener accepts a connection it passes back to the client a new port/ip address and the client reconnects using the new port/ip address. Again this causes problems mapping these IPs/ports through firewalls. NATing firewalls certainly did throw a lemon in the works, invalidating some of the assumptions with which these protocols were designed :-). Cheers Mark |
From: Mark O'D. <mar...@fi...> - 2004-09-24 06:10:07
Attachments:
md5sum.tar.gz
|
Hi Alex Alex Peshkov wrote: > Hi! > > During last two weeks I implemented crypt plugins facility for firebird > engine. Well done! And I like most of what you describe. Having now read the complete thread, there is a lot!, and since I'd previously given this issue quite some thought. Here is my outline of Alex's idea, plus a few extra bits. Hopefully it might help with the formalisation and acceptance of the design for fb2. The idea is to supplement the current password hash with something more modern, while retaining backward compatibility. The current system, roughly, and using my bad memory, has two cycles of DES to store/use a password. #define PASSWORD_SALT "9z" client: clienthash = DES(PASSWORD_SALT + password) ... sendToServer(username, clienthash) server: boolean isUserOK(user, password) { serverhash = DES(PASSWORD_SALT + clienthash) databaseValue = getHashForUserFromSecurityDB(username) if (serverHash == databaseValue) { return true; } return false; } The server side of this could be fairly easily updated to be: boolean isUserOK(username, password) { siteSalt = getSiteSalt() databaseValue = getHashForUserFromSecurityDB(username) // try md5 (or SHA-1 if you must :-) serverhash = MD5(sitesalt + username + clienthash) if (serverHash == databaseValue) { return true; } // try old DES serverhash = DES(PASSWORD_SALT + clienthash) if (serverHash == databaseValue) { return true; } return false; } // Get an array of random data that will be specific per site. // perhaps generated at install time. // The random prefix could either be stored in another table in // security.fdb. or in the firebird.conf file. // Either will do, there are plusses and minuses for both // It should also be longer than 1 byte, anyting over 4 should be // adequate, say 10 bytes? byte[] getSiteSalt() { data = getRandomFromSecurityDb() if (data == null) { buildAndStoreRandomInSecurityDB() data = getRandomFromSecurityDb() } return data; } Of course storing a password would also need to change to use md5. intermediatehash = DES(PASSWORD_SALT + password) newpassword = md5(getSiteSalt() + username + intermediatehash) For simplicity the new server hash method should be hardcoded and fixed as either MD5 or SHA-1. Choices confuse people and this is an intermediate solution. Eventually someone will implement security using PAM modules then any authentication, via SASL, unix-logon, NTLM, smartcard, kerberos, LDAP, even using MySQL or Postgresql, etc can all be plugged in. A slightly different approach to hashing user passwords that could also be used, but just as secure as the above is SSHA (most commonly used in LDAP). http://developer.netscape.com/docs/technote/ldap/pass_sha.html Thats it. It needs a bit of roadtesting, and a few more eyes, and Im sure a bit more polishing, but Alex's design means that: 1) old client can connect to new servers, 2) new clients still talk to old servers. 3) and even if really needed old security.fdb files could work with new servers. And I agree that most of Ivan's ideas are worth applying as well. http://www.volny.cz/iprenosil/interbase/ip_ib_isc4.htm I started thinking about this about two years ago, after all the PAM stuff looked like it would take a while to get done, but I have never had time to follow it up - but the idea looks good, and I think after a bit of tinkering it is worth including in fb2. Anyway, I've attached a standalone md5 implementation, if it helps, and Im guessing Jim has a SHA1. Cheers Mark PS: Later fb3? it might be useful to consider linking with openssl, since in additon to MD5/SHA-1 it also has RSA, SSL and easy use for random number generators, which are likely to be useful later. If needed I can provide openssl sample usages for both MD5/SHA1 code - it's a little different. But for inclusion in fb2.0, I'd say do it with a standalone MD5/SHA1 implentation. |
From: Dmitry Y. <di...@us...> - 2004-09-24 10:45:17
|
"Mark O'Donohue" <mar...@fi...> wrote: > > It needs a bit of roadtesting, and a few more eyes, and Im sure a bit > more polishing, but Alex's design means that: > > 1) old client can connect to new servers, > 2) new clients still talk to old servers. > 3) and even if really needed old security.fdb files could work > with new servers. If these points are true, then I support your proposal. Do I understand correctly that you suggest to have hardcoded two algorithms (MD5/SHA1 + DES) instead of firebird.conf-specified plugins list proprosed by Alex? Do you want to add the crypt plugins feature later and remove the hardcoded stuff or you think it's too complex solution for most hosts? Dmitry |
From: Alex P. <pe...@in...> - 2004-09-24 15:00:06
|
Dmitry Yemanov wrote: >"Mark O'Donohue" <mar...@fi...> wrote: > > >>It needs a bit of roadtesting, and a few more eyes, and Im sure a bit >>more polishing, but Alex's design means that: >> >> 1) old client can connect to new servers, >> >> Yes - except gsec. >> 2) new clients still talk to old servers. >> >> Yes - except gsec. It uses new spb parameter to specify security database to work with. >> 3) and even if really needed old security.fdb files could work >> with new servers. >> >> After fix of updatable view in cmp.cpp - yes. I revert back to update of view USERS instead of underlying table in security.epp, and this will work. But it's better to apply script I plan to write to update security database. > >If these points are true, then I support your proposal. Do I understand >correctly that you suggest to have hardcoded two algorithms (MD5/SHA1 + DES) >instead of firebird.conf-specified plugins list proprosed by Alex? Do you >want to add the crypt plugins feature later and remove the hardcoded stuff >or you think it's too complex solution for most hosts? > > > Looking at the number of mentioned in this thread hashing methods I still think that plugins will be useful. Moreover, to be secure we must be able to turn off DES hashes when needed - for example, in default install. Appears configuration parameter will be required for it anyway. Therefore, why not use it at that same time for selecting plugin? One drawback - API for this plugins should be specified. Currently I use extern "C" { FB_DLL_EXPORT char* makehash(const char* salt, const char* password); }; It need not be thread-safe (each plugin call is protected by mutex). Alex. |
From: Mark O'D. <mar...@fi...> - 2004-09-24 23:22:04
|
Hi Alex Alex Peshkov wrote: > Dmitry Yemanov wrote: > >> "Mark O'Donohue" <mar...@fi...> wrote: >> >>> >>> 1) old client can connect to new servers, 2) new clients still >>> talk to old servers. >>> > > Yes - except gsec. It uses new spb parameter to specify security > database to work with. > I assume gsec not being transportable, is not really a problem, what about other 3rd party tools? >>> 3) and even if really needed old security.fdb files could work >>> with new servers. >>> > > After fix of updatable view in cmp.cpp - yes. I revert back to update > of view USERS instead of underlying table in security.epp, and this > will work. But it's better to apply script I plan to write to update > security database. > I think it's acceptable that only the "new" security.fdb works post this change, as long as there is an upgrade path, and a clear error when someone trys it. I like your idea of an upgrade script to apply to old security.fdb files. > Looking at the number of mentioned in this thread hashing methods I > still think that plugins will be useful. I suppose Im arguing to upgrade the password hash algorithm which I see as highly desirable for fb2. A plugin architecture seems a larger change, desirable but needing a bit of time to let the design settle and time to implement it. But I think I can best comment if I can have a look at your plugin code. Do you mind sending me a patch or posting it somewhere. But, plugin asside, that shouldn't be an impediment to us upgrading the hash algorithm for fb2 even if plugin PAM modules make it into fb2 as well. > Moreover, to be secure we must be able to turn off DES hashes when > needed - for example, in default install. Appears configuration > parameter will be required for it anyway. Therefore, why not use it > at that same time for selecting plugin? If internally (and in gsec) SHA1/MD5 are used in preference and tried first, DES doesn't really need to be turned off. I really think we want to avoid giving a user a choice about something they mostly won't understand. Lets just design the new hash as safe, and make it a replacement. However, a warning script to show users still using the DES MAC for passwords would be helpful. But I'll look forward to seeing the patch. Cheers Mark |
From: Alex P. <pe...@in...> - 2004-09-29 10:25:27
|
Mark O'Donohue wrote: > > Hi Alex > > Alex Peshkov wrote: > >> Dmitry Yemanov wrote: >> >>> "Mark O'Donohue" <mar...@fi...> wrote: >>> >>>> >>>> 1) old client can connect to new servers, 2) new clients still talk >>>> to old servers. >>>> >> >> Yes - except gsec. It uses new spb parameter to specify security >> database to work with. >> > > I assume gsec not being transportable, is not really a problem, what > about other 3rd party tools? Tools, based upon service manager, work OK. isc_*_user() functions need some more testing, but I think that even if some problems remain, they may be easily fixed. >> Looking at the number of mentioned in this thread hashing methods I >> still think that plugins will be useful. > > > I suppose Im arguing to upgrade the password hash algorithm which I > see as highly desirable for fb2. > > A plugin architecture seems a larger change, desirable but needing a > bit of time to let the design settle and time to implement it. > Let's distinguish between AUTH plugins, which implementation is really none-trivial, and CRYPT plugins, which are as simple as you can't even imagine. The main problem was how to remove hash calculation from client, but I think we all agree it's anyway required step. > But I think I can best comment if I can have a look at your plugin > code. Do you mind sending me a patch or posting it somewhere. > > But, plugin asside, that shouldn't be an impediment to us upgrading > the hash algorithm for fb2 even if plugin PAM modules make it into fb2 > as well. I'm afraid AUTH plugins are not real for 2.0. At least we must support passing of non-ENC_crypt()ed passwords from client to server - I'm afraid PAM will not recognize our crypt hash as password :-). > > >> Moreover, to be secure we must be able to turn off DES hashes when >> needed - for example, in default install. Appears configuration >> parameter will be required for it anyway. Therefore, why not use it >> at that same time for selecting plugin? > > > If internally (and in gsec) SHA1/MD5 are used in preference and tried > first, DES doesn't really need to be turned off. > > I really think we want to avoid giving a user a choice about something > they mostly won't understand. Lets just design the new hash as safe, > and make it a replacement. > If we give users default install ONLY with new hashes, they will start to make replacements only when need to upgrade old installation. Upgrading, they will be not able to login, and (hopefully), will have to read documentation. If they don't revert bach to safe hashes only - what can we do? There many other ways to help hackers attack one's system - my friend was very surprised when got a $700/3days bill for internet after setting completely opened proxy on his w2k gateway. And when I asked him, has not he forgotten to setup terminal services on that box - answer was YES :-) > However, a warning script to show users still using the DES MAC for > passwords would be helpful. > Luckily, they differ in length. Therefore this script is possible. > > But I'll look forward to seeing the patch. > Please download from http://www.insi.yaroslavl.ru/fb/sec.rar (257K). Alex. |
From: Mark O'D. <mar...@fi...> - 2004-09-24 23:45:25
|
Dmitry Yemanov wrote: > "Mark O'Donohue" <mar...@fi...> wrote: > >>It needs a bit of roadtesting, and a few more eyes, and Im sure a bit >>more polishing, but Alex's design means that: >> >> 1) old client can connect to new servers, >> 2) new clients still talk to old servers. >> 3) and even if really needed old security.fdb files could work >> with new servers. > > > If these points are true, then I support your proposal. I think they should be true, and Alex seems to agree, (he's in a good position to be able to confirm it :-). I also think requiring an update script to be run over security.fdb is acceptable (and probably preferable) for 3). > Do I understand > correctly that you suggest to have hardcoded two algorithms (MD5/SHA1 + DES) > instead of firebird.conf-specified plugins list proprosed by Alex? Do you > want to add the crypt plugins feature later and remove the hardcoded stuff > or you think it's too complex solution for most hosts? > Yes, I think hardcoding SHA1/MD5 as an internal replacement for the current DES MAC is a good first step. It's a change with fairly low risk, and with good benefits for incuding in fb2. It then becomes the new fb default. Im glad Alex has done plugins, but Im not sure a plugin choice of DES or MD5 is enough, if done properly a plugin architecture should be able to support NTLM and PAM, pretty much straight out of the box. Plugins are good idea, but a larger architectual change, and may take longer to filter through all involved to settle into a design that is genrally acceptable. I'd like to see the plugin code, and would love to see it, or modified form also be included in fb2. Cheers Mark |
From: Geoff W. <ge...@te...> - 2004-09-24 14:04:48
|
Hi Dmitry, > My own vision is presented below. > User should become a database object, like tables, roles, [etc] I am pleased to see that I was not too far off base with my earlier email. From my limited understanding of the issues involved, something along the lines of what you suggest seems very appropriate. One particular issue that I can see arising from such an arrangement is the problem of multi-database applications. Its one of those "catch-22" situations. The current single security database per server enforces a single set of users for all databases on that server - even when some of the databases belong to different applications, and then neglects to provide any way to keep the databases in sync. However having user details inside each database makes it more difficult to implement applications that do make use of more than one database at a time. For example an application that has "archived" information made available from separate (possibly read-only) databases. I do implement a system along similar lines to this, not because FB has trouble with the database size, but because there are many administrative reasons to keep the changing/production database as small as practical. (Note: The above highlights a more specific issue, how to give access to a read-only database if the user definitions are inside the database?) I am not sure how this situation can be resolved, unless we can somehow differentiate between databases that carry such user details and databases that carry only role information for which the permissions are presumed to be managed via some separate database. (Or something to this effect.) I do have other ideas about all this stuff, but... well it gets very involved and when I posted an email related to these ideas (a few months ago) there was not much interest. So lets see what can develop with a less radical approach. -- Geoff Worboys Telesis Computing |
From: Kevin L. <kev...@gm...> - 2004-09-24 16:51:38
|
> One particular issue that I can see arising from such an > arrangement is the problem of multi-database applications. Would it be possible to maintain users in a security file, like it is now, but create the ability to bind a database to a specific one? Something like what MS Access does, actually. They are called workgroup files, which define roles and users. A database can be linked to any workgoup file and cannot be opened with any workgoup file other than the one defined by the database owner. Kevin Lingofelter |