Thread: Re: [Firebird-devel] RFC: Major changes in ODS 12 (Page 3)

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

> Additions are welcome

   Forgot to add: change record size from USHORT to ULONG to remove 64k 
limit for record length.

   SY, SD.

Dimitry Sibiryakov wrote:
> 
> Forgot to add: change record size from USHORT to ULONG to remove 64k 
> limit for record length.

I'm not sure it would be as easy as it sounds. I have no problem however 
to make this ODS change now to be utilized by some further FB version.

Dmitry

Dmitry Yemanov> 
>     * deprecate page buffers and sweep interval
>       (in favor of the per-database values defined in .conf)

One conf-file for all databases (i.e. current firebird.conf)
or per-database.conf file (i.e. my_db.conf) ? May be
this was already discussed early (i don't remeber) ?

WBR, GR.

Rustam Gadjimuradov wrote:
> 
> One conf-file for all databases (i.e. current firebird.conf)
> or per-database.conf file (i.e. my_db.conf) ?

You may have a common firebird.conf with defaults as well as a special 
section inside firebird.conf that define custom settings for my_db.fdb, 
or even a separate my_db.conf that's included by firebird.conf.

Dmitry

Dmitry Yemanov
> You may have a common firebird.conf with defaults as well as a special 
> section inside firebird.conf that define custom settings for my_db.fdb, 
> or even a separate my_db.conf that's included by firebird.conf.

OK, thanks.

But I partially agree with Mark. What about saving both solutions -
db-header options and options in conf-file - and two-way changing
(like "synchronizing") them ? Too excessive ?

WBR, GR

> But I partially agree with Mark. What about saving both solutions -
> db-header options and options in conf-file - and two-way changing
> (like "synchronizing") them ? Too excessive ?

I'd say: choose one method of storing per database settings and stick with it. Creating two different methods of storing settings is only confusing and prone to errors (direction of sync, which setting overrides what etc).

Mark
-- 
Neu: GMX DSL bis 50.000 kBit/s und 200,- Euro Startguthaben!
http://portal.gmx.net/de/go/dsl02

Mark Rotteveel> I'd say: choose one method of storing per database settings and stick with it.

In view of that per-database conf file will be used in any case
(for other goals), i'd prefer configuring curent db-header options
there (in files). But why already existed options must be cleaned
from db-header is not so obvious for me.

New options must be stored and configured via files, of course.

Mark Rotteveel> Creating two different methods of storing settings is only confusing
Mark Rotteveel> and prone to errors (direction of sync, which setting overrides what etc).

I think, they must be fully equal.

WBR, GR

Rustam Gadjimuradov wrote:
> 
> But why already existed options must be cleaned
> from db-header is not so obvious for me.

I won't be insisting. I just think their presence makes configuration 
even more confusing than now.

Dmitry

> Mark Rotteveel> I'd say: choose one method of storing per database
> settings and stick with it.
> 
> In view of that per-database conf file will be used in any case
> (for other goals), i'd prefer configuring curent db-header options
> there (in files). But why already existed options must be cleaned
> from db-header is not so obvious for me.
> 
> New options must be stored and configured via files, of course.

This is a middle option which is a disaster...

"So, if I move my database to another server/folder, I will need to move the separate conf file as well!"

Nuts to that!!!

Store all of the *specified* database settings (remembering not all settings are *database* settings) in the database header.

If a specific setting/parameter doesn't exist in the header, then the default value in the server/folder config file will apply.

In this way the database owner can define his configuration, while the server/host manager can define his settings.

Sean

Leyne, Sean napsal(a):
> 
>> Mark Rotteveel> I'd say: choose one method of storing per database 
>> settings and stick with it.
>> 
>> In view of that per-database conf file will be used in any case 
>> (for other goals), i'd prefer configuring curent db-header options 
>> there (in files). But why already existed options must be cleaned 
>> from db-header is not so obvious for me.
>> 
>> New options must be stored and configured via files, of course.
> 
> This is a middle option which is a disaster...
> 
> "So, if I move my database to another server/folder, I will need to
> move the separate conf file as well!"
> 
> Nuts to that!!!
> 
> Store all of the *specified* database settings (remembering not all
> settings are *database* settings) in the database header.
> 
> If a specific setting/parameter doesn't exist in the header, then the
> default value in the server/folder config file will apply.
> 
> In this way the database owner can define his configuration, while
> the server/host manager can define his settings.

Yeah, let's the flamewar begin :-) Seriously, both approaches how to
store the database configuration options have their pros & cons. Let's
summarize them a bit...

1) Configuration right in database

Pros:
- It goes where the database goes when it's moved: backup (both gbak and
nbackup) or new location or even shadow. However it's not propagated
when other replication techniques are used. Actually, this is a mixed
blessing as it could be also a drawback, because one has to re-check the
configuration after restore from backup or move (I saw
blast-from-the-past misconfigurations too many times, especially 0 sweep
interval stored in backup could be a disaster when one doesn't expect it
to reappear after restore).
- You don't need direct access to files on the server.

Cons:
- You need a special tool to see and change the configuration.
- Risk of Blast-from-the-past misconfiguration mentioned above.

2) Configuration in database-specific (text) configuration file

Pros:
- You don't need any special tool to see and change the configuration.
- The configuration file could be self-documented.

Cons:
- You need direct access to configuration file on the server.
- The configuration is not part of the backup (although I'd consider it
a plus, not drawback)
- You have to copy the configuration file along with database if you
*want* to retain it's configuration in new location.
- A danger of misconfiguration due to use of wrong file if there is not
clear (and simple) definition where this configuration file is located.

Personally, I would vote for 2) any sober day. However, I'd see two
things in it's initial implementation:

a) clear and simple rules for config file placement, for example file
must have the same name as the database (with .conf extension) and
located either in database directory or directory defined as "global
config directory" in firebird.conf.

b) ability to read/write config file via Services API.

best regards
Pavel Cisar
IBPhoenix

Pavel,

> Yeah, let's the flamewar begin :-)

I've got my flame-proof suit ready! ;-]

> 1) Configuration right in database

> Cons:
...
> - Risk of Blast-from-the-past misconfiguration mentioned above.

I really don't think those are real problems, more human misunderstandings...

> 2) Configuration in database-specific (text) configuration file
> 
> Pros:
> - You don't need any special tool to see and change the configuration.
> - The configuration file could be self-documented.
> 
> Cons:
> - You need direct access to configuration file on the server.
> - The configuration is not part of the backup (although I'd consider it
> a plus, not drawback)
> - You have to copy the configuration file along with database if you
> *want* to retain it's configuration in new location.
> - A danger of misconfiguration due to use of wrong file if there is not
> clear (and simple) definition where this configuration file is located.
> 
> Personally, I would vote for 2) any sober day.

It's 2:00pm in Toronto, so I'm quite sober!

Sean

Pavel Cisar wrote:
> Personally, I would vote for 2) any sober day. However, I'd see two
> things in it's initial implementation:
>   

I'd like option 1, I think everything inside the database less error 
prone... Perhaps because I think that the database specific 
configuration (sweep, page buffers, etc.) is tight related to the 
physical file and the expected usage pattern.

but if there is a .conf file for a database, I would love to see it side 
by side with the database.

> a) clear and simple rules for config file placement, for example file
> must have the same name as the database (with .conf extension) and
> located either in database directory

Yes !

>  or directory defined as "global
> config directory" in firebird.conf.
>   

I prefer the former...

And what about store in the database header the .conf file name ? when 
restoring/accessing a database with a missing .conf file, an entry could 
be added to firebird.log. So it would be easy to note that some config 
file is missing.

see you !

-- 
Alexandre Benson Smith
Development
THOR Software e Comercial Ltda
Santo Andre - Sao Paulo - Brazil
www.thorsoftware.com.br

Alexandre Benson Smith napsal(a):
> Pavel Cisar wrote:
>> Personally, I would vote for 2) any sober day. However, I'd see two
>> things in it's initial implementation:
>>   
> 
> I'd like option 1, I think everything inside the database less error 
> prone... Perhaps because I think that the database specific 
> configuration (sweep, page buffers, etc.) is tight related to the 
> physical file and the expected usage pattern.

You just prove my point that there is 50:50 split in preferences where
such options should be stored, so we can't reach the consensus by
counting votes :)

> And what about store in the database header the .conf file name ? when 
> restoring/accessing a database with a missing .conf file, an entry could 
> be added to firebird.log. So it would be easy to note that some config 
> file is missing.

That's very bad idea as it complicates database relocations, and it's
definitely not error prone. From my field experience, people *never*
look into firebird.log until something bad happens. As database would
work without the config file (with default values), nobody would notice
until it crashes and burn. Also if there would be a requirement that
config file must be of the same name as database, there is no point to
note it's name in database itself.

best regards
Pavel Cisar
IBPhoenix

Pavel Cisar wrote:
>
> Also if there would be a requirement that
> config file must be of the same name as database, there is no point to
> note it's name in database itself.

I don't think this requirement serves the flexibility goal.

Dmitry

Dmitry Yemanov napsal(a):
> Pavel Cisar wrote:
>> Also if there would be a requirement that
>> config file must be of the same name as database, there is no point to
>> note it's name in database itself.
> 
> I don't think this requirement serves the flexibility goal.

Yeah, that's true. You know, I'm so scared of Vulcan-type configuration
that's profound example of over-engineering with it's "layering" and
"wildcarding" that I tend to run toward opposite extreme. :-)

Certainly the best option would be to have these configuration options
as part of aliases.conf (or any config file we would use as database
catalog) with possibility to include these values from another text file
(to separate and externalize database or group specific options). We
just need to solve the riddle of multiply defined (aka redefined) values
in sensible (i.e. not Vulcan) way.

best regards
Pavel Cisar
IBPhoenix

On Tuesday 27 October 2009 10:31:19 Pavel Cisar wrote:
> Alexandre Benson Smith napsal(a):
> > Pavel Cisar wrote:
> >> Personally, I would vote for 2) any sober day. However, I'd see two
> >> things in it's initial implementation:
> >
> > I'd like option 1, I think everything inside the database less error
> > prone... Perhaps because I think that the database specific
> > configuration (sweep, page buffers, etc.) is tight related to the
> > physical file and the expected usage pattern.
>
> You just prove my point that there is 50:50 split in preferences where
> such options should be stored, so we can't reach the consensus by
> counting votes :)
>
> > And what about store in the database header the .conf file name ? when
> > restoring/accessing a database with a missing .conf file, an entry could
> > be added to firebird.log. So it would be easy to note that some config
> > file is missing.
>
> That's very bad idea as it complicates database relocations, and it's
> definitely not error prone. From my field experience, people *never*
> look into firebird.log until something bad happens. 

Really, who will look in that crazy log file? But if we leave database 
in 'sysdba-only single-attach' mode (like we do when some restore errors 
happen) and on an attempt to attach to it produce additional error message 
(missing conf file abc.conf), this can work.

Hi !

Pavel Cisar wrote:
> Alexandre Benson Smith napsal(a):
>   
>> Pavel Cisar wrote:
>>     
>>> Personally, I would vote for 2) any sober day. However, I'd see two
>>> things in it's initial implementation:
>>>   
>>>       
>> I'd like option 1, I think everything inside the database less error 
>> prone... Perhaps because I think that the database specific 
>> configuration (sweep, page buffers, etc.) is tight related to the 
>> physical file and the expected usage pattern.
>>     
>
> You just prove my point that there is 50:50 split in preferences where
> such options should be stored, so we can't reach the consensus by
> counting votes :)
>   

yes ! :)

>   
>> And what about store in the database header the .conf file name ? when 
>> restoring/accessing a database with a missing .conf file, an entry could 
>> be added to firebird.log. So it would be easy to note that some config 
>> file is missing.
>>     
>
> That's very bad idea as it complicates database relocations,

That would not be the full file path, but the relative one (perhaps even 
a rule to store on the very same directory), so the relocation would be 
just to copy the .conf file too.. I don't think it would complicate

>  and it's
> definitely not error prone.

And should not be, since a database could be used without the .conf file.

>  From my field experience, people *never*
> look into firebird.log until something bad happens. 

But from the support point of view, one can just tell the user to look 
into firebird.log for some message about missing database config file

> As database would
> work without the config file (with default values), nobody would notice
> until it crashes and burn.

Agreed, but at least we would know that it burned because of a missing 
config than to try to guess what changed (work load, transaction 
management, etc.)

>  Also if there would be a requirement that
> config file must be of the same name as database, there is no point to
> note it's name in database itself.
>   

Agreed, that was just an idea, I sugested the file name because there is 
no agreement that the config file should have the same name as the 
database yet. Perhaps you don't need the file name stored in the header, 
but a flag that indicates that this database have a customized config file.

> best regards
> Pavel Cisar
> IBPhoenix
>   

see you !

-- 
Alexandre Benson Smith
Development
THOR Software e Comercial Ltda
Santo Andre - Sao Paulo - Brazil
www.thorsoftware.com.br

Pavel Cisar wrote:
> Yeah, let's the flamewar begin :-) Seriously, both approaches how to
> store the database configuration options have their pros & cons. Let's
> summarize them a bit...

;)

> 1) Configuration right in database

...

> Cons:
> - You need a special tool to see and change the configuration.

You can also see that as a pro in my mind: having a specific API (+ 
implementation of a tool) makes it a lot easier for DBA to configure 
settings (and for Firebird to validate the settings).

> 2) Configuration in database-specific (text) configuration file
> 
> Pros:
> - You don't need any special tool to see and change the configuration.
> - The configuration file could be self-documented.
> 
> Cons:
> - You need direct access to configuration file on the server.
> - The configuration is not part of the backup (although I'd consider it
> a plus, not drawback)

This last drawback can be addressed by (conf-file solution: backup the 
database-specific conf file as well; in-database solution: make 
restoring the configuration optional)

> - You have to copy the configuration file along with database if you
> *want* to retain it's configuration in new location.
> - A danger of misconfiguration due to use of wrong file if there is not
> clear (and simple) definition where this configuration file is located.
> 
> Personally, I would vote for 2) any sober day.

I myself would vote for 1; although I can live with 2 but then I would 
still like an API for configuration and a possibility for remote 
retrieval or backup of that configuration file.

-- 
Mark Rotteveel

Dmitry Yemanov
> Everything you can find in firebird.conf, excluding (or better say 
> ignoring) those making sense for the remote subsystem.

Ahh... OK, thanks.

Then my vote for cleaning this options (and all others, possibly)
from db-header and manage them via files. If we can manage
options remotely (via ServiceAPI or another) - it would be great
but it's not necesarry and not priority, as for me.

WBR, GR

Leyne, Sean wrote:
> 
> "So, if I move my database to another server/folder, I will need to move the separate conf file as well!"

Is it the real problem? Place the appropriate .conf file next to the 
database file and you're unlikely to forget it when moving...

> In this way the database owner can define his configuration, while the server/host manager can define his settings.

I can easily imagine a database configured by the owner to be 
unmanageable on many hosts, regardless of what the server manager thinks 
about it...

Dmitry

Dmitry Yemanov wrote:
> Leyne, Sean wrote:
>> "So, if I move my database to another server/folder, I will need to move the separate conf file as well!"
> 
> Is it the real problem? Place the appropriate .conf file next to the 
> database file and you're unlikely to forget it when moving...

That doesn't really work that well if you only backup and restore remotely.

> 
>> In this way the database owner can define his configuration, while the server/host manager can define his settings.
> 
> I can easily imagine a database configured by the owner to be 
> unmanageable on many hosts, regardless of what the server manager thinks 
> about it...

Could be, although maybe allowing system administrators to enforce a 
certain setting globally is an option (ie the system setting takes 
precedence over database specific settings).

-- 
Mark Rotteveel

> > "So, if I move my database to another server/folder, I will need to move
> the separate conf file as well!"
> 
> Is it the real problem? Place the appropriate .conf file next to the
> database file and you're unlikely to forget it when moving...
> 
> > In this way the database owner can define his configuration, while the
> server/host manager can define his settings.
> 
> I can easily imagine a database configured by the owner to be
> unmanageable on many hosts, regardless of what the server manager thinks
> about it...

OK, that's a fair point/comment but that doesn't speak to issue at hand, where should the database settings be stored.

If we want to talk about limiting the scope of the database setting, then we should talk about adding new parameters to define the min/max values which are appropriate for a server.

Such, that if a server has min = 10 and max = 100, if a database had a setting of 150, then it would be limited to 100.

But if that database was moved to another server, which had a max = 200, then the database setting would be "fine".

Sean

> That's very bad idea as it complicates database relocations, and it's
> definitely not error prone. From my field experience, people *never*
> look into firebird.log until something bad happens.

Yes, but that is also partly because the logging done by Firebird isn't really useful. I think that is something that could be improved.

Mark
-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

Mark,

> Yes, but that is also partly because the logging done by Firebird isn't
> really useful. I think that is something that could be improved.

You're welcome to start a new thread with your suggestions ;-)

Dmitry

Stefan Heymann napsal(a):
> 
> We already have settings like Sweep Interval, SQL Dialect, and Buffer
> Pages in the database. Why introduce a new file for others? 

The point was to move these values (except SQL Dialect which actually
isn't a configuration option) out from database to text configuration
file. The config file could be firebird.conf with [databases] section,
separate databases.conf file or database specific file, whatever, as
this wasn't settled yet.

> Where do
> you draw the line? Will GBAK also put that file into the .fbk backup
> or will I have to copy that? If it does, will a GBAK restore also
> restore that file?

Configuration isn't actually what should be part of the backup file
itself. It should contain only your data, not configuration as there is
no direct link between your data and server configuration (although
database specific). You can use the backup just fine on another system
or on the same one it was created but under different conditions (for
example you switched from SuperServer to Classic meanwhile), where the
configuration values would be not valid any more. If you want to backup
your configuration, then make a backup copy of your configuration file(s).

best regards
Pavel Cisar
IBPhoenix

Thread: Re: [Firebird-devel] RFC: Major changes in ODS 12 (Page 3)

A powerful, cross platform, SQL database system

firebird-devel