openinteract-dev

[Openinteract-dev] example of new server.ini @INCLUDE syntax

[Chris W. <ch...@cw...>]

Since I didn't get much of a response from my previous inquiry about a
possible change to the server configuration files I've created an example:

  http://www.openinteract.org/new_config_example/

The example uses the live configuration from my website (with passwords
and such changed of course) and shows the full configuration vs. what it
would look like broken down into files. It's all the same data. The main
configuration contains the data users are most likely to change (database,
session, email server + addresses, deployment context) and that's it.

See what you think.

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.

Re: [Openinteract-dev] example of new server.ini @INCLUDE syntax

[A. <ant...@he...>]

Hi!

I think the @INCLUDE syntax might not be a very good idea. Some quick points:

-- FIND
* Editors find makes include quite useless.
* All editors have find and all users can use it.
* But includes break editors find.

-- OVERLAPPING ENTRIES
* Overlapping entries are hard to spot with include.
* The way include works with overlapping entries is not trivial.
* Errors resulting from this might be hard to find.

- Antti

Re: [Openinteract-dev] example of new server.ini @INCLUDE syntax

[Chris W. <ch...@cw...>]

> I think the @INCLUDE syntax might not be a very good idea. Some quick
> points:
>
> -- FIND
> * Editors find makes include quite useless.
> * All editors have find and all users can use it.
> * But includes break editors find.
>
> -- OVERLAPPING ENTRIES
> * Overlapping entries are hard to spot with include.
> * The way include works with overlapping entries is not trivial.
> * Errors resulting from this might be hard to find.

These are all excellent points. I'm convinced: the multiple server.ini
files won't be the default. (I'll keep the feature in the INI parser
though.)

Thanks!

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.

Re: [Openinteract-dev] example of new server.ini @INCLUDE syntax

[Chris W. <ch...@cw...>]

> Given the possibility that the amount of configuration entries
> (sections, key/value pairs) become large enough, you might want to
> consider complexity of "depth" (lots of info at few places - a "deep"
> configuration tree structure) vs. complexity of breadth (lots of places
> with just a little info everywhere - a "shallow" or "flat" config tree.)

The INI format ensures we don't get too deep. I find configurations that
get too deep very confusing, which makes for confusing software.

> Proximity of configuration entries (wether two unrelated entries can be
> found in the same file, directory or even filesystem) should be part of
> your consideration. Obviously, keeping all configuration close together
> has some benefits (searchability, easy access) but too much in one place
> can increase the learning curve and/or make comprehension more difficult.

Yes, that's what I'm responding to.

It hit me when looking over the Quick Start guide that to get OI started
you only need to edit a few items. But they're spread all over the
configuration file! So why not have the stuff you absolutely need to edit
up front and the rest of it easily available but not right in front of
your eyes?

> This is roughly the same problem as "having too much documentation" vs.
> "having too little documentation easily available"
>
> I think the sweet spot most likely lays somewhere in-between the two,
> and that the clue to make this information accessible (easy to
> find/scan, read, understand) lays with good (sensible, intuitive,
> natural, conventions/standards-abiding) abstractions and categorization
> of the information, and to keep the "depth" at a minimum while still
> staying within a "width" that is comprehensible for the casual observer.

Right, that's what I'm shooting for and why I used topic-oriented include
files.

> Given this, I'd make the following suggestions:
>
>   * Reduce the amount of configuration files, if possible.

A good goal, but we also don't want to make them encompass too much.

>   * Group related config sections more tightly together (e.g. put the
> "main" datasource together with the other datasources.)

You're right that this is a bit of a disconnect, but I did it so that a
new user would only have to edit 'server.ini' to get started quickly.

>   * KISS. Is it necessary to differentiate between Controllers and
> Content Generators? (Perhaps a different issue, but maybe worthwhile to
> look into if your're trying to clean up configuration anyway. Reducing
> the amount of necessary concepts to learn helps both with file size and
> understandability. :)

Very, very true.

>   * Package-related config should be placed somewhere very close, but
> seperatly from the server config (e.g. the "fulltext" package config has
> IMHO nothing to do in the conf/server.ini file, and the base_user
> action.ini files are too far away in pkg/base_user/conf.) I'd like to
> suggest that all package config files are accessible from a suitable
> "package.d" directory (e.g. "conf/packages.d/fulltext.ini"), and that
> the main config file @INCLUDEs these en-masse. The conf/packages.d/*.ini
> files need not to be more than symlinks to the packages in order to get
> a positive effect.

That's a really interesting idea. With the global package configuration
override file I've tried to keep everything in one place, but the global
override stuff is probably too confusing for most people to use. (Then
again, most people probably don't need to modify their package/SPOPS
configurations either.)

The other part we have to balance is package upgrades -- how to allow the
user to keep any changes she's made with the potentially new package
configuration? Maybe with a package upgrade we read in the existing
configurations and compare them to the new ones, writing out any
differences to a separate file?

I don't think we can do symlinks (not cross-platform), but I like the idea
of keeping all the SPOPS + Action data in one location....

>   * When upgrading some software (OI2 or .zip packages,) I'd like to
> keep my old config, but still get the new features. Splitting up files
> (as you described) may help with this, so do place commonly changed
> config entries in one or more seperate files.

OI2 should never, never overwrite your server.ini file with an upgrade.
Once we write it at server creation we don't touch it. (In fact we don't
touch anything in conf/ except for repository.ini.)

>   * BTW, I'd like to keep my timezone when I upgrade the server. :)

This shouldn't get overwritten. Does it?

>   * s/server_action_info.ini/server_actions.ini/g (Naming consistency is
> good. :)

Actually 'server_actions.ini' was my first name but I thought it implied
that you'd actually find actions there, which isn't true.

>   * The ini file parser should warn about inconsistencies and duplicate
> entries if they're not allowed.

Dupes we can do. Inconsistencies maybe not. But I'd imagine duplicated
configuration would be the main problem.

>   * Configuration entries that have related entries that for some
> greater reason can't be placed "close by", should at least reference the
> related entries.

Absolutely.

> Just my USD 0.02. Feel free to criticise! :)

Thanks!

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.

Re: [Openinteract-dev] example of new server.ini @INCLUDE syntax

[Salve J N. <sal...@me...>]

Chris Winters wrote:
> Salve J. Nilsen wrote:
>> 
>> * Package-related config should be placed somewhere very close, but
>>   seperatly from the server config (e.g. the "fulltext" package
>>   config has IMHO nothing to do in the conf/server.ini file, and the
>>   base_user action.ini files are too far away in pkg/base_user/conf.)
>>   I'd like to suggest that all package config files are accessible
>>   from a suitable "package.d" directory (e.g.
>>   "conf/packages.d/fulltext.ini"), and that the main config file
>>   @INCLUDEs these en-masse. The conf/packages.d/*.ini files need not
>>   to be more than symlinks to the packages in order to get a positive
>>   effect.
> 
> That's a really interesting idea. With the global package
> configuration override file I've tried to keep everything in one
> place, but the global override stuff is probably too confusing for
> most people to use. (Then again, most people probably don't need to
> modify their package/SPOPS configurations either.)
> 
> The other part we have to balance is package upgrades -- how to allow
> the user to keep any changes she's made with the potentially new
> package configuration? Maybe with a package upgrade we read in the
> existing configurations and compare them to the new ones, writing out
> any differences to a separate file?
> 
> I don't think we can do symlinks (not cross-platform), but I like the
> idea of keeping all the SPOPS + Action data in one location....

How about using a file that just contain the following lines?

=======8<----< $OPENINTERACT2/conf/packages.d/my_package.ini >-------

# This is a OI2 config file, belonging to the OI2 package referenced to
# below. See the package config files in that directory for more
# information.

@INCLUDE = $OPENINTERACT2/pkg/my_package-0.01/conf.d

-------------->8=====================================================


>> * When upgrading some software (OI2 or .zip packages,) I'd like to 
>> keep my old config, but still get the new features. Splitting up
>> files (as you described) may help with this, so do place commonly
>> changed config entries in one or more seperate files.
> 
> OI2 should never, never overwrite your server.ini file with an
> upgrade. Once we write it at server creation we don't touch it. (In
> fact we don't touch anything in conf/ except for repository.ini.)

Hm. In that case, there's not much reason for splitting up files, right? 
We should now only be left with the problem of "how to keep all 
configuration close together".

>> * BTW, I'd like to keep my timezone when I upgrade the server. :)
> 
> This shouldn't get overwritten. Does it?

It doesn't. I assumed (in error) it was the server.ini file you wanted 
to be able to change.

But anyway (somewhat unrelated to this) I _do_ have a problem with the 
timezone enry in server.ini, but only because it is so closely placed to 
the CVS revision string in the file (making my patch to that config file 
fail every time server.ini gets updated in CVS. :)


>> * s/server_action_info.ini/server_actions.ini/g (Naming consistency
>>   is good. :)
> 
> Actually 'server_actions.ini' was my first name but I thought it
> implied that you'd actually find actions there, which isn't true.

Oh. Hm. I got confused by the [box] definition in 
server_action_info.ini, where it says

=========================8<--------------------

[box]
action             = box_display
default_template   = base_box::main_box_shell

# ....

--------------------->8=========================

Should it say "use_action", "bind_action" or something other, less 
confusing then "action"?

(See! I'm showing off my ignorance! :)


>> * The ini file parser should warn about inconsistencies and
>> duplicate entries if they're not allowed.
> 
> 
> Dupes we can do. Inconsistencies maybe not. But I'd imagine
> duplicated configuration would be the main problem.

How about checking for legal keywords? IIRC, some config file parsers on 
CPAN force you to define which keywords and/or value types are legal in 
a given file. Would that be sensible for OI2?


- Salve

-- 
Salve J. Nilsen <salvejn at met dot no> / Systems Developer
Norwegian Meteorological Institute           http://met.no/
Information Technology Department / Section for Development

Re: [Openinteract-dev] example of new server.ini @INCLUDE syntax

[Salve J N. <sal...@me...>]

Some thoughts...

Chris Winters wrote:
> Since I didn't get much of a response from my previous inquiry about a
> possible change to the server configuration files I've created an example:
> 
>   http://www.openinteract.org/new_config_example/
> 
> The example uses the live configuration from my website (with passwords
> and such changed of course) and shows the full configuration vs. what it
> would look like broken down into files. It's all the same data. The main
> configuration contains the data users are most likely to change (database,
> session, email server + addresses, deployment context) and that's it.
> 
> See what you think.

Given the possibility that the amount of configuration entries
(sections, key/value pairs) become large enough, you might want to
consider complexity of "depth" (lots of info at few places - a "deep"
configuration tree structure) vs. complexity of breadth (lots of places
with just a little info everywhere - a "shallow" or "flat" config tree.)

Proximity of configuration entries (wether two unrelated entries can be
found in the same file, directory or even filesystem) should be part of
your consideration. Obviously, keeping all configuration close together
has some benefits (searchability, easy access) but too much in one place
can increase the learning curve and/or make comprehension more difficult.

This is roughly the same problem as "having too much documentation" vs.
"having too little documentation easily available"

I think the sweet spot most likely lays somewhere in-between the two,
and that the clue to make this information accessible (easy to
find/scan, read, understand) lays with good (sensible, intuitive,
natural, conventions/standards-abiding) abstractions and categorization
of the information, and to keep the "depth" at a minimum while still
staying within a "width" that is comprehensible for the casual observer.

Given this, I'd make the following suggestions:

  * Reduce the amount of configuration files, if possible.
  * Group related config sections more tightly together (e.g. put the
"main" datasource together with the other datasources.)
  * KISS. Is it necessary to differentiate between Controllers and
Content Generators? (Perhaps a different issue, but maybe worthwhile to
look into if your're trying to clean up configuration anyway. Reducing
the amount of necessary concepts to learn helps both with file size and
understandability. :)
  * Package-related config should be placed somewhere very close, but
seperatly from the server config (e.g. the "fulltext" package config has
IMHO nothing to do in the conf/server.ini file, and the base_user
action.ini files are too far away in pkg/base_user/conf.) I'd like to
suggest that all package config files are accessible from a suitable
"package.d" directory (e.g. "conf/packages.d/fulltext.ini"), and that
the main config file @INCLUDEs these en-masse. The conf/packages.d/*.ini
files need not to be more than symlinks to the packages in order to get
a positive effect.
  * When upgrading some software (OI2 or .zip packages,) I'd like to
keep my old config, but still get the new features. Splitting up files
(as you described) may help with this, so do place commonly changed
config entries in one or more seperate files.
  * BTW, I'd like to keep my timezone when I upgrade the server. :)
  * s/server_action_info.ini/server_actions.ini/g (Naming consistency is
good. :)
  * The ini file parser should warn about inconsistencies and duplicate
entries if they're not allowed.
  * Configuration entries that have related entries that for some
greater reason can't be placed "close by", should at least reference the
related entries.

Just my USD 0.02. Feel free to criticise! :)


- Salve

-- 
Salve J. Nilsen <salvejn at met dot no> / Systems Developer
Norwegian Meteorological Institute           http://met.no/
Information Technology Department / Section for Development