naviserver-devel — Developer discussion

Re: [naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

On 23.10.2007, at 19:25, Stephen Deasey wrote:

>
> test cfg-4.1 {global type check bool} -body {
>     ns_setconfig section cfg-4.1 true
>     expr {[ns_getconfig -bool section cfg-4.1] ? 1 : 0}
> } -result 1

Try again...

Re: [naviserver-devel] nsconfigrw

[Stephen D. <sd...@gm...>]

On 10/23/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> On 23.10.2007, at 19:25, Stephen Deasey wrote:
>
> > Two interesting failing test cases:
> >
> > test cfg-5.4 {defaults propagate to global store} -body {
> >     ns_getconfig section cfg-5.4 foo
> >     ns_getconfig section
> > } -match glob -result {*cfg-5.4*}
> >
> > ---- Result was:
> > cfg-4.1 cfg-3.2 cfg-4.2 cfg-3.1
> > ---- Result should have been (glob matching):
> > *cfg-5.4*
> > ==== cfg-5.4 FAILED
> >
>
> This can happen if "cfg-5.4" was never declared.
> The  "ns_getconfig section cfg-5.4 foo" will NOT
> trigger error if the section/key is missing. It
> will just return empty string (as ns_config does).
> Correspondingly, the "ns_getconfig section" will
> return just those keys that have been ns_setconfig'ed
> so far.


But one of the things we want to support is introspection, right?


  foreach section [ns_getconfig] {
      foreach {k v} [ns_getconfig $section] {
          ...
      }
  }


The only way to make it work atm is to pre-declare all config values:


  # mylib.tcl

  ns_setconfig section enabled true
  ...

  proc foo {} {
      if {[ns_getconfig section enabled]} {
          ...
      }
  }
  ...


Pre-declaring is a new restriction. Is this the plan?

I could also imagine having a hybrid system, where you *could*
pre-declare, but you could also have ad-hoc calls. Both have their
benefits:

- Config which is only referenced once, and rarely if ever needs to be
changed, is ideally handled with inline calls. Everything is in one
place.

- Config which cannot be guessed, e.g. a host name, or which is
accessed from multiple pieces of code, would benefit from a
declaration. This would allow the default to be written once and make
options for configuring code more obvious.


A system of pre-declaring might have benefits for C-code especially.


Or..?

Re: [naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

On 23.10.2007, at 19:25, Stephen Deasey wrote:

> Two interesting failing test cases:
>
>
>
> test cfg-5.4 {defaults propagate to global store} -body {
>     ns_getconfig section cfg-5.4 foo
>     ns_getconfig section
> } -match glob -result {*cfg-5.4*}
>
> ---- Result was:
> cfg-4.1 cfg-3.2 cfg-4.2 cfg-3.1
> ---- Result should have been (glob matching):
> *cfg-5.4*
> ==== cfg-5.4 FAILED
>
>

This can happen if "cfg-5.4" was never declared.
The  "ns_getconfig section cfg-5.4 foo" will NOT
trigger error if the section/key is missing. It
will just return empty string (as ns_config does).
Correspondingly, the "ns_getconfig section" will
return just those keys that have been ns_setconfig'ed
so far.

>
>
> test cfg-4.1 {global type check bool} -body {
>     ns_setconfig section cfg-4.1 true
>     expr {[ns_getconfig -bool section cfg-4.1] ? 1 : 0}
> } -result 1
>
> ---- Test generated error; Return code was: 1
> ---- Return code should have been one of: 0 2
> ---- errorInfo: configuration parameter is not a boolean

This is of course more tricky, as we do not store
strings in Param, just 1/0. I think I must investigate
string in Param on yes/y/no/n/true/t/false etc etc when
converting string to boolean.

Eagle-Eye-Stephen...

Re: [naviserver-devel] nsconfigrw

[Stephen D. <sd...@gm...>]

Two interesting failing test cases:



test cfg-5.4 {defaults propagate to global store} -body {
    ns_getconfig section cfg-5.4 foo
    ns_getconfig section
} -match glob -result {*cfg-5.4*}

---- Result was:
cfg-4.1 cfg-3.2 cfg-4.2 cfg-3.1
---- Result should have been (glob matching):
*cfg-5.4*
==== cfg-5.4 FAILED




test cfg-4.1 {global type check bool} -body {
    ns_setconfig section cfg-4.1 true
    expr {[ns_getconfig -bool section cfg-4.1] ? 1 : 0}
} -result 1

---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: configuration parameter is not a boolean

Re: [naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

On 23.10.2007, at 18:53, Stephen Deasey wrote:

> Groovy. Fixed.

What would be intersting are some speed figures.
The one I gave were from my MPB with 2GHZ Intel.
I guess that weshould be not slower than ns_config.

As looking in the nsd/config.c whole lotta things
got into there... some arcane backslash parsing
code etc pp... brrr...
Ultimately the nsconfigrw should be plug-replacement
for most of the things in config.c, or?

What we haven't figured out is: how the changed
params are saved (made persistent) when a change
occurs? This is by no means trivial to do.
Perhaps the easiest would be to have some kind of
dbm-like storage (qdbm is nice and LGPL) plus
some utility to display the values inthere...
Just a thought...

Re: [naviserver-devel] nsconfigrw

[Stephen D. <sd...@gm...>]

On 10/23/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> On 23.10.2007, at 17:50, Vasiljevic Zoran wrote:
>
> >
> > Anyways... this must be some trivia...
>
> Indeed.
>
> Tests ended at Tue Oct 23 18:03:40 CEST 2007
> all.tcl:     Total  29   Passed  25   Skipped  4   Failed  0


Groovy. Fixed.

Re: [naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

On 23.10.2007, at 17:50, Vasiljevic Zoran wrote:

>
> Anyways... this must be some trivia...

Indeed.

Tests began at Tue Oct 23 18:03:40 CEST 2007
ns_config.test
[23/Oct/2007:18:03:40][3655.1800600][-command-] Warning: configrw: ns/ 
testconfig:intval value=42, rounded up to 43
[23/Oct/2007:18:03:40][3655.1800600][-command-] Warning: configrw: ns/ 
testconfig:intval value=42, rounded down to 41
[23/Oct/2007:18:03:40][3655.1800600][-command-] Warning: configrw: ns/ 
testconfig:intval value=42, rounded up to 43
[23/Oct/2007:18:03:40][3655.1800600][-command-] Warning: configrw: ns/ 
testconfig:missing value=42, rounded down to 41
ns_getconfig.test
[23/Oct/2007:18:03:40][3655.1803800][-ns_job_0-] Notice: Starting  
thread: -ns_job_1-

Tests ended at Tue Oct 23 18:03:40 CEST 2007
all.tcl:        Total   29      Passed  25      Skipped 4        
Failed  0
Sourced 2 Test Files.
Number of tests skipped for each constraint:
         4       knownBug

Please checkout, recompile and try again.

Re: [naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

On 23.10.2007, at 17:08, Stephen Deasey wrote:

> Fails to run.
>

Amazing. For me it passes all tests...
This is perhaps I'm on Mac and you're on Linux? ;-)

Anyways... this must be some trivia...

Re: [naviserver-devel] nsconfigrw

[Stephen D. <sd...@gm...>]

Fails to run.


$ make gdbtest TCLTESTARGS="-debug 3"

Tests began at Tue Oct 23 04:01:45 PM BST 2007
ns_config.test

::tcltest::Eval called
Running ns_config-2.1 {
    expr {[ns_getconfig -bool ns/testconfig trueval] ? 1 : 0}
}

::tcltest::Eval called
Running ns_config-2.2 {
    ns_getconfig -bool ns/testconfig missing
}
make: *** [test] Segmentation fault


[Switching to Thread 1077488528 (LWP 4409)]
0x400063d1 in CopyParam (dest=0x9fbb030, src=0x0) at configrw.c:958
958	    dest->type = src->type;

(gdb) backtrace

#0  0x400063d1 in CopyParam (dest=0x9fbb030, src=0x0) at configrw.c:958
#1  0x4000625e in SetParam (secPtr=0x9fba960, name=0x9fbabb0
"missing", param=0x0) at configrw.c:858
#2  0x40005fbd in CfgGet (section=0x9fb0cf0 "ns/testconfig",
key=0x9fbabb0 "missing", outPtr=0x4038e6c0) at configrw.c:708
#3  0x40004f9f in Cfg_GetBool (section=0x9fb0cf0 "ns/testconfig",
key=0x9fbabb0 "missing", def=0x0, value=0x4038e7e4) at configrw.c:224
#4  0x400058d6 in CfgGetObjCmd (data=0x0, interp=0x9eeeaf0, objc=4,
objv=0x4038e98c) at configrw.c:540

[naviserver-devel] nsconfigrw

[Vasiljevic Z. <zv...@ar...>]

Hi!

Stephen added the module "nsconfigw" into the modules
section, based on my example from RFE.
He did quite a lot of work there lately to clean it up,
debug and write some test cases.

Good Work, Stephen!

I changed the way how parameters are handled to avoid
excessive allocations and conversions where possible
(this can still be improved). I compared the ns_config
(NS built-in read-only config) speed with the new module:

lexxsrv:nscp 2> time {ns_config -bool ns/parameters logdebug} 1000
3.368 microseconds per iteration

lexxsrv:nscp 4> time {ns_getconfig -bool ns/parameters logdebug} 1000
2.331 microseconds per iteration

Well, 1 microsec is not much but in this case this is 30% (faster).

Strings is similar, yet not so large difference (as expected):

lexxsrv:nscp 16> time {ns_config ns/parameters tcllibrary} 1000
3.163 microseconds per iteration

lexxsrv:nscp 18> time {ns_getconfig ns/parameters tcllibrary} 1000
2.966 microseconds per iteration

The config store now has typed-params so we avoid conversions
similarily to Tcl objects. Per-definition, the contention on
readers is minimal per-design: just one reader and one writer
may contend for the same mutex. Multiple readers will never
contend, except on the very first request for a param that
is not privately cached. This all comes with a price of more
memory usage. Not really much, but still. OTOH, you get true
read/write config which is reasonably (very!) fast so you can
even put calls to it in tight loops.

I think this is very good now. It needs adding WideInt type
and some way of making the conig persistent. Perhaps some
other param qualifiers (readonly, readwrite...)

Cheers
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver/nsd config.c, 1.14, 1.15 fastpath.c, 1.39, 1.40 str.c, 1.8, 1.9

[Stephen D. <sd...@gm...>]

> + Tcl_WideInt
> + Ns_StrToWideInt(CONST char *string, Tcl_WideInt *intPtr)
> + {
> +     Tcl_WideInt  lval;
> +     char *ep;
> +
> +     errno = 0;
> +     lval = strtoll(string, &ep, string[0] == '0' && string[1] == 'x' ? 16 : 10);
> +     if (string[0] == '\0' || *ep != '\0') {
> +         return NS_ERROR;
> +     }
> +     if ((errno == ERANGE && (lval == LLONG_MAX || lval == LLONG_MIN))) {
> +         return NS_ERROR;
> +     }
> +     *intPtr = (int) lval;
> +
> +     return NS_OK;
> + }


   *intPtr = (int) lval;


Is this right?

[naviserver-devel] ns_setpriveleges

[Stephen D. <sd...@gm...>]

> +
> + /*
> +  *----------------------------------------------------------------------
> +  * Ns_SetPrivileges --
> +  *
> +  *      Set the effective user ID/group ID of the current process, uid as -1 or
> +  *      gid as -1 will be ignored
> +  *
> +  * Results:
> +  *      -1 on error


Should be NS_OK / NS_ERROR.


> +  * Side effects:
> +  *      Process privileges will be different after success


This is not a side effect, it's the intended effect.


> + int
> + Ns_SetPrivileges(int uid, int gid)
> + {


This is really two calls in one. It should be something more like:

  int Ns_SetUser(CONST char *user);
  int Ns_SetGroup(CONST char *group);


Then Ns_GetPrivileges() can be deleted and rolled into the above.

For example, this:


  nsd/nsmain.c:

  if (Ns_SetPrivileges(-1, gid) == -1) {
      Ns_Fatal("nsmain: failed to switch to group %d", gid);
  }
  NsForkBinder();

  if (Ns_SetPrivileges(uid, -1) == -1) {
      Ns_Fatal("nsmain: failed to switch to user %d", uid);
  }


Can be turned into this:


  if (Ns_SetGroup(group) != NS_OK) {
      Ns_Fatal("nsmain: failed to switch to group %s", group);
  }
  NsForkBinder();

  if (Ns_SetUser(user) != NS_OK) {
      Ns_Fatal("nsmain: failed to switch to user %s", user);
  }


And this:


> RCS file: /cvsroot/naviserver/naviserver/nsproxy/nsproxylib.c,v
>
> --- 459,465 ----
>      }
>
> !     if (Ns_GetPrivileges(user, group, &uid, &gid) == -1 ||
> !         Ns_SetPrivileges(uid, gid) == -1) {
> !         Ns_Fatal("nsproxy: unable to switch to user '%s', group '%s'", user, group);
>      }


Can be turned into this:


  if (Ns_SetUser(user) != NS_OK
          || Ns_SetGroup(group) != NS_OK) {
      Ns_Fatal("nsproxy: unable to switch to user '%s', group '%s'",
user, group);
  }

(Always put the || or && at the beginning of the continuation line.)


> + /*
> +  *----------------------------------------------------------------------
> +  *
> +  * NsTclSetPrivilegesObjCmd --
> +  *
> +  *      Implements ns_setprivileges as ObjCommand.


Should be ns_setuser and ns_setgroup.


> +  *
> +  * Results:
> +  *      Tcl result, 1 if sucessful, -1 on error


Should throw a Tcl error, not return -1.


> +  * Side effects:
> +  *      Error message will be output in the log file, not returned as Tcl result


Use Tcl_PosixError().


>       NsTclUrlDecodeObjCmd,
>       NsTclUrlEncodeObjCmd,
>       NsTclVarObjCmd,
>       NsTclWriteContentObjCmd,
>       NsTclWriteFpObjCmd,
>       NsTclWriteObjCmd,
>       NsTclWriterObjCmd,
> -     NsTclFileStatObjCmd;
> +     NsTclFileStatObjCmd,
> +     NsTclSetPrivilegesObjCmd;


'S' comes after 'R' and before 'T'...


> Log Message:
>        * doc/src/ns_setpriveleges.man: Added documentation file


Checked, reasonably complete man pages go in doc/src/mann/*.


> + --- NEW FILE: ns_setpriveleges.man ---
> + [manpage_begin ns_setpriveleges n 4.99]


(Misspelled privilege)

Don't hard-code the version number. Do this:


  [include version_include]
  [manpage_begin ns_setprivileges n [vset version]]


> + [see_also nsd]
> + [keywords ns_time ns_conn]


...?!


> Index: tclmisc.c
> ===================================================================
> RCS file: /cvsroot/naviserver/naviserver/nsd/tclmisc.c,v
> retrieving revision 1.23
> retrieving revision 1.24
> diff -C2 -d -r1.23 -r1.24
> *** tclmisc.c   14 Aug 2007 08:49:10 -0000      1.23
> --- tclmisc.c   20 Oct 2007 17:20:32 -0000      1.24
> ***************
> *** 1071,1074 ****
> --- 1071,1084 ----
>   *
>   * $Log$
> +  * Revision 1.24  2007/10/20 17:20:32  seryakov
> +  *     * include/ns.h:
> +  *     * nsd/unix.c: Added 2 new public API functions Ns_SetPriveleges and Ns_GetPriveleges
> +  *       which are resolve and assign user uid/gid
> +  *
> +  *     * nsproxy/nsproxylib.c:
> +  *     * nsd/nsmain.c: Switched to use new function that assign user real uid/gid
> +  *
> +  *     * nsd/tclmisc.c: New Tcl command ns_setpriveleges that sets real uid and gid
> +  *


We don't embed change logs in source files.


Also, *please* don't change white space in the same commit as other substantial
changes as it makes the update notifications very difficult to read.

Re: [naviserver-devel] Set uid/gid in Tcl

[Vasiljevic Z. <zv...@ar...>]

On 18.10.2007, at 23:41, Vasiljevic Zoran wrote:

>
> The nsmain.c contains some uid/gid fiddling code
> that is indef WIN32. If you du something like
> ns_pid/gid please do the same (ifndef WIN32).
>

Similar code is used in nsproxy/nsproxylib.c
This all should really be ifndef WIN32 ...

Re: [naviserver-devel] Set uid/gid in Tcl

[Vasiljevic Z. <zv...@ar...>]

On 18.10.2007, at 23:20, Vlad Seryakov wrote:

> I do it in some modules as well, but would it make sense to provide  
> this
> as core Tcl command so no need to copy code from module to module

The nsmain.c contains some uid/gid fiddling code
that is indef WIN32. If you du something like
ns_pid/gid please do the same (ifndef WIN32). 

Re: [naviserver-devel] Set uid/gid in Tcl

[Vlad S. <vl...@cr...>]

I do it in some modules as well, but would it make sense to provide this 
as core Tcl command so no need to copy code from module to module

Vasiljevic Zoran wrote:
> On 18.10.2007, at 22:53, Vlad Seryakov wrote:
> 
>> Will it be useful to have internal Tcl command for changing real  
>> uid/gid?
>>
>> for example i want to start as root, prepare data, load modules and  
>> then
>> in Tcl switch to non-privileged user.
> 
> Yes. We already do that in a module.
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Set uid/gid in Tcl

[Vasiljevic Z. <zv...@ar...>]

On 18.10.2007, at 22:53, Vlad Seryakov wrote:

> Will it be useful to have internal Tcl command for changing real  
> uid/gid?
>
> for example i want to start as root, prepare data, load modules and  
> then
> in Tcl switch to non-privileged user.

Yes. We already do that in a module.

[naviserver-devel] Set uid/gid in Tcl

[Vlad S. <vl...@cr...>]

Question,

Will it be useful to have internal Tcl command for changing real uid/gid?

for example i want to start as root, prepare data, load modules and then 
in Tcl switch to non-privileged user.

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 15.10.2007, at 21:53, Jeff Rogers wrote:

> However this gets resolved, I think there should also be a way to  
> easily
> write out a config file based on the current dynamic configuration, so
> that if the server does get restarted the configuration can stay  
> the same.

... which is called persistence and I thought about that of course.
I haven't figure out how this would be achieved but one can think
of anything between generating the ns_section/ns_param lines in a
Tcl file on one extreme, to putting all in somekind of database
(gdbm or similar).

Cheers
Zoran

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 15.10.2007, at 23:27, Andrew Piskorski wrote:

>
> Making nsv usable from C code is very useful, and not all that hard.
> I've done it in the past (for AOLserver 3.x and 4.0.x).  So, just how
> are the use cases for dynamic ns_config different from those for nsv?

Good question. Tt happens that we already use the ns_section/ns_param
all over the code (c and tcl). furthermore we need to be able to tune
server params during runtime as well. All this is not what nsv's are.

>
> Since the existig ns_config is totally static, it's safe to take the
> pointer returned by Ns_ConfigGetValue() and just assume it never
> changes and is always valid process-wide.  But of course that can't
> work when calling Nsv from C code.  What does your dynamic ns_config
> code do for that case?

It gets the value from it's per-thread cache. But, the source-code
is pretty simple and documented so you should be able to get the
idea by skipping thru the code (takes about 10-15 minutes).
It tries to avoid lock contention by greedily caching all conf params
in all threads that access them. OTOH, writers have a "tough" job
of updating all those private thread caches everytime some config
parameter is changed.

Cheers
Zoran

Re: [naviserver-devel] ns_config read/write

[Stephen D. <sd...@gm...>]

I cleaned up the module a bit and imported it to CVS.

  http://naviserver.cvs.sourceforge.net/naviserver/nsconfigrw
  http://naviserver.sourceforge.net/n/nsconfigrw/files/ns_getconfig.html

(btw. if you create a man page for your module and put it in the right
place, doc/src/mann or doc/src/man3, it will appear on the website...)


Most of the tests are failing. Looks like the code can't distinguish
missing values for bool and int because a default is always required.
Is this OK..?

Step two would be to fix the excessive copying.

Re: [naviserver-devel] ns_config read/write

[Andrew P. <at...@pi...>]

On Mon, Oct 15, 2007 at 12:53:57PM -0700, Jeff Rogers wrote:
> However this gets resolved, I think there should also be a way to easily 
> write out a config file based on the current dynamic configuration, so 
> that if the server does get restarted the configuration can stay the same.

Excellent idea.  It's also awfully nice to be able to just diff two
files to see what you changed.

-- 
Andrew Piskorski <at...@pi...>
http://www.piskorski.com/

Re: [naviserver-devel] ns_config read/write

[Andrew P. <at...@pi...>]

On Mon, Oct 15, 2007, Zoran Vasiljevic wrote:
> On 15.10.2007, at 20:37, Stephen Deasey wrote:

>> 2: This scheme would nicely handle non performance critical key/value
>> config called from Tcl code. But if you can already do that with
>> nsv...?
>
> Well, not really. This needs to be also accessible from C and from
> various plugin-modules.

Making nsv usable from C code is very useful, and not all that hard.
I've done it in the past (for AOLserver 3.x and 4.0.x).  So, just how
are the use cases for dynamic ns_config different from those for nsv?

Since the existig ns_config is totally static, it's safe to take the
pointer returned by Ns_ConfigGetValue() and just assume it never
changes and is always valid process-wide.  But of course that can't
work when calling Nsv from C code.  What does your dynamic ns_config
code do for that case?

-- 
Andrew Piskorski <at...@pi...>
http://www.piskorski.com/

Re: [naviserver-devel] ns_config read/write

[Rick C. <rc...@Kn...>]

In the version we did for AOLServer 3.4.2, which relies on our
proprietary event-notification subsystem, we:

(1) Put a place in the startup .tcl script where over-rides from a local
file would be used (using "source"). That way, the file output from live
settings could be the equivalent of:
   ns_section "x"
   ns_param p v
	...

without changing the file we ship for customers. This made it somewhat
easier to edit with Tcl (since it only uses these two commands and is
always generated, not hand-edited).

(I say "the equivalent" above because we do some fairly strange things
to map between sections/parameters and our configuration "topicspace"
that I don't want to go into here.)

(2) Our configuration variables are marked with whether they are "no
restart" or not.  The backend that implements "change this variable"
returns a flag indicating whether restart is required, and our GUI puts
an indicator up if that's happened.  Actually, we also set another
global state variable to "configuration changes make restart required"
so any instance of any console can show an appropriate flag.

This mostly works OK, though we do occasionally have to train
administrators with the question "When you changed that, did the console
tell you you needed to restart? What happened when you restarted?"

As far as calculated or enabled configuration settings go, we model all
of those as "restart required" settings.

-- ReC



-----Original Message-----
From: nav...@li...
[mailto:nav...@li...] On Behalf Of
Jeff Rogers
Sent: Monday, October 15, 2007 12:54 PM
To: nav...@li...
Subject: Re: [naviserver-devel] ns_config read/write

Vasiljevic Zoran wrote:

> Do you have some other idea how (if) we should build the chnageable
> config? It seems pretty silly to me that we need to restart the server
> to change some marginal parameter... In the 21. century...

However this gets resolved, I think there should also be a way to easily

write out a config file based on the current dynamic configuration, so=20
that if the server does get restarted the configuration can stay the
same.

-J

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_config read/write

[Jeff R. <dv...@di...>]

Vasiljevic Zoran wrote:

> Do you have some other idea how (if) we should build the chnageable
> config? It seems pretty silly to me that we need to restart the server
> to change some marginal parameter... In the 21. century...

However this gets resolved, I think there should also be a way to easily 
write out a config file based on the current dynamic configuration, so 
that if the server does get restarted the configuration can stay the same.

-J

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 15.10.2007, at 20:37, Stephen Deasey wrote:

>
> One problem is that there seems to be a lot of memory allocation and
> copying going on, even on the fast path. For example, NsConf_GetBool()
> does three mallocs, a couple of copies, conversion back and forth
> between string and int rep, two hash lookups and a lock/unlock.  It's
> not exactly (opt & FOO_OPT)  :-(

This all is true in the current implementation just
because I was lazy... The important part where you
need to look at is the:

     ConfGetParam(section, key, &defpar, &outpar);

This one we must/should take apart. Even here we have
some headroom.


>
> Another problem is that the model is extremely basic. It seems to be
> identical to nsv (with different locking) so the only way to have
> dynamic configuration is to do this:
>
>     AtRuntime()
>     {
>         if (ConfigGet("enabled")) {
>             ...
>         }
>     }
>
>
> - This will be a problem for some code because it's too slow.

It very different from nsv. Locking is completely different.
The only similarity is that it stores key/value pairs.

>
> - Some configuration parameters depend on other ones or on some
> computation like converting units. ATM this is done once at startup.
> But with this scheme it would have to be inline with the code that
> used the param (or perhaps wrappers..).

True. But, again, this would be done only once, as I cache
the values in the per-thread private hash table.

>
> - Some changes in config values should trigger other code to run -- it
> is an event, it is not polled. For example, some threads might need
> killed off or created within a pool, or a proc should be scheduled.

By all means. Bu this is not in the scope of the config module,
rather the "user" code. If the user-code is written so that it
reacts to changes of the config, then there is no problem.
At the moment no code does that because the config is read-only.

>
> - One of the problems mentioned was inventorying config params. For
> example, some config params are guarded behind a check for 'enabled'
> -- if it's not enabled the other params won't be checked, and they
> remain hidden. I guess this module doesn't help with this, maybe it
> shouldn't..?

This is the functionality that I did not handle at all. The emphasis
was to create read/write config more/less with the same functionality
as the current one.

>
>
> One way of looking at it might be that this will not handle all config
> situations. I think the problem with that is:
>
> 1: It is confusing. We are saying: config is read write. But when you
> write some config params, nothing changes. It is doubly confusing
> because some of those params *will* be changeable, but only by calling
> ns_fooctl.

This is true. If the rest of the code is not programmed with the
notion of changeable config, there will be "confusing" situations.

>
> 2: This scheme would nicely handle non performance critical key/value
> config called from Tcl code. But if you can already do that with
> nsv...?

Well, not really. This needs to be also accessible from C and from
various plugin-modules.

>
>
> I still agree that a more runtime configurable server is desirable.

One thing to remember: if we set the funtional-bar too high we will
not achieve much on the short/midterm.

What I see is that in order to make the config writable, all kind of
parts in the server needs to be rewritten in order to gain benefit
from it (less important) and behave "logically" (more important).
Given number of such places that would be potentially changed are
so high that I will not attempt to do so. Simply because of the lack
of time.

Do you have some other idea how (if) we should build the chnageable
config? It seems pretty silly to me that we need to restart the server
to change some marginal parameter... In the 21. century...



>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a  
> browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel