naviserver-devel — Developer discussion

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

yes, i know, i am complaining about myself relying on particular formatting.

Stephen Deasey wrote:
> On 10/15/07, Vlad Seryakov <vl...@cr...> wrote:
>> The only thing which broke my old code regarding switching to vsnprintf
>> is that old Ns_DStringPrintf handled empty string differently, vsnprintf
>> now puts (null), before that empty string did not put anything.
>>
>> I still need to review si the damage is critical enough or never upgrade
>> until all code updated. That sucks
> 
> 
> The printf familly calls make no guarantee about NULLs. If we're
> relying on that anywhere in the code it's a bug and we need to fix it.
> 
> e.g.:
> 
>     Ns_Log(Warning, "huh?: %s", nullable ? nullable : "uh-oh");
> 
> 
> Otherwise, we can expect crashes.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] ns_config read/write

[Stephen D. <sd...@gm...>]

On 10/15/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> On 14.10.2007, at 23:34, Stephen Deasey wrote:
>
> > Here's the thread from last time (lots of good ideas):
>
> Yep. Too bad nobody peeked at the suggested
> (bare-bones) implementation. I guess I should
> just go and get it in, tagging the tree before
> the change so we can easily revert back if this
> founds to be problematic?


:-/


> Out of the numerous ideas, the really important
> one would be an option (compile or runtime) wether
> to allow config to be writable or not....


I don't think this is the most important problem.

One problem is that there seems to be a lot of memory allocation and
copying going on, even on the fast path. For example, NsConf_GetBool()
does three mallocs, a couple of copies, conversion back and forth
between string and int rep, two hash lookups and a lock/unlock.  It's
not exactly (opt & FOO_OPT)  :-(

Another problem is that the model is extremely basic. It seems to be
identical to nsv (with different locking) so the only way to have
dynamic configuration is to do this:

    AtRuntime()
    {
        if (ConfigGet("enabled")) {
            ...
        }
    }


- This will be a problem for some code because it's too slow.

- Some configuration parameters depend on other ones or on some
computation like converting units. ATM this is done once at startup.
But with this scheme it would have to be inline with the code that
used the param (or perhaps wrappers..).

- Some changes in config values should trigger other code to run -- it
is an event, it is not polled. For example, some threads might need
killed off or created within a pool, or a proc should be scheduled.

- One of the problems mentioned was inventorying config params. For
example, some config params are guarded behind a check for 'enabled'
-- if it's not enabled the other params won't be checked, and they
remain hidden. I guess this module doesn't help with this, maybe it
shouldn't..?


One way of looking at it might be that this will not handle all config
situations. I think the problem with that is:

1: It is confusing. We are saying: config is read write. But when you
write some config params, nothing changes. It is doubly confusing
because some of those params *will* be changeable, but only by calling
ns_fooctl.

2: This scheme would nicely handle non performance critical key/value
config called from Tcl code. But if you can already do that with
nsv...?


I still agree that a more runtime configurable server is desirable.

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Stephen D. <sd...@gm...>]

On 10/15/07, Vlad Seryakov <vl...@cr...> wrote:
> The only thing which broke my old code regarding switching to vsnprintf
> is that old Ns_DStringPrintf handled empty string differently, vsnprintf
> now puts (null), before that empty string did not put anything.
>
> I still need to review si the damage is critical enough or never upgrade
> until all code updated. That sucks


The printf familly calls make no guarantee about NULLs. If we're
relying on that anywhere in the code it's a bug and we need to fix it.

e.g.:

    Ns_Log(Warning, "huh?: %s", nullable ? nullable : "uh-oh");


Otherwise, we can expect crashes.

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

The only thing which broke my old code regarding switching to vsnprintf 
is that old Ns_DStringPrintf handled empty string differently, vsnprintf 
now puts (null), before that empty string did not put anything.

I still need to review si the damage is critical enough or never upgrade 
until all code updated. That sucks

Vlad Seryakov wrote:
> Yes, works fine now, Thanks
> 
> Stephen Deasey wrote:
>> On 10/13/07, Vlad Seryakov <vl...@cr...> wrote:
>>> I was able to find the location which causes the crash but i am not sure
>>> how to fix it
>>>
>>> In function Ns_VALog i call vsnprintf directly, it works, when it is
>>> called via Ns_DStringVPintf it crashes. using va_copy does not help
>>>
>>
>> diff -r 22a912b584eb nsd/dstring.c
>> --- a/nsd/dstring.c	Fri Oct 12 20:35:13 2007 +0100
>> +++ b/nsd/dstring.c	Sat Oct 13 19:46:22 2007 +0100
>> @@ -176,10 +176,11 @@ Ns_DStringPrintf(Ns_DString *dsPtr, CONS
>>   */
>>
>>  char *
>> -Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list ap)
>> -{
>> -    char *buf;
>> -    int   origLength, newLength, bufLength, result;
>> +Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list apSrc)
>> +{
>> +    char    *buf;
>> +    int      origLength, newLength, bufLength, result;
>> +    va_list  ap;
>>
>>      origLength = dsPtr->length;
>>
>> @@ -205,11 +206,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
>>      buf = dsPtr->string + origLength;
>>      bufLength = newLength - origLength;
>>
>> +    va_copy(ap, apSrc);
>>  #ifdef __WIN32
>>      result = _vsnprintf_s(buf, bufLength, fmt, ap);
>>  #else
>>      result = vsnprintf(buf, bufLength, fmt, ap);
>>  #endif
>> +    va_end(ap);
>>
>>      /*
>>       * Check for overflow and retry. For win32 just double the buffer size
>> @@ -229,11 +232,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
>>          buf = dsPtr->string + origLength;
>>          bufLength = newLength - origLength;
>>
>> +    va_copy(ap, apSrc);
>>  #ifdef __WIN32
>>          result = _vsnprintf_s(buf, bufLength, fmt, ap);
>>  #else
>>          result = vsnprintf(buf, bufLength, fmt, ap);
>>  #endif
>> +        va_end(ap);
>>      }
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems?  Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
> 

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 14.10.2007, at 23:34, Stephen Deasey wrote:

> Here's the thread from last time (lots of good ideas):

Yep. Too bad nobody peeked at the suggested
(bare-bones) implementation. I guess I should
just go and get it in, tagging the tree before
the change so we can easily revert back if this
founds to be problematic?

Out of the numerous ideas, the really important
one would be an option (compile or runtime) wether
to allow config to be writable or not.... This I
may need to implement beforehand. 

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 14.10.2007, at 23:34, Stephen Deasey wrote:

> Did you eventually use this code? Did it work for you?

It is still "lab" quality. I have it running here for
some time but not in the production (i.e in the product).
I guess it is safe but you obviously never know. I also
have no speed metrics although the speed should not be
the problem, given the low mutex contention. Except
if you have very large number of simultaneous writers
which is practically never the case...

Of course, before I do any step I wanted to hear some
comments on the design to see if everybody really understands
what is going on there and judge if this is going to
open some new benefits, or (god forbid) troubles.
The latter must/should be really zero.

[naviserver-devel] errorCode and errorInfo

[Vasiljevic Z. <zv...@ar...>]

Hi!

It seems like errorInfo/errorCode cannot just simply be
set by Tcl_SetVar() as Tcl will overwrite those if the
interp flags were not correctly set. The latter is
unfortunaltely not "exported" to the outside, so the
only chance you have is to call Tcl_SetErrorCode and/or
Tcl_AddObjErrorInfo/Tcl_AddErrorInfo.

I found that because this seems not to work correctly:

lexxsrv:nscp 1> ns_job create test
test
lexxsrv:nscp 2> ns_job queue test "error A B C"
job0
lexxsrv:nscp 3> ns_job wait test job0
A
lexxsrv:nscp 4> set errorCode
NONE
lexxsrv:nscp 5> set errorInfo
A
     while executing
"ns_job wait test job0"

Cheers
Zoran

Re: [naviserver-devel] ns_config read/write

[Stephen D. <sd...@gm...>]

On 10/12/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> So:
>
>    The configuration is global and read-only. Parameters may be
>    retrieved at run-time using [cmd ns_config], although usually
>    configuration parameters are used by Tcl library files at start-up.


This is from here:

    http://naviserver.sourceforge.net/n/naviserver/files/ns_config.html

(which describes how it is, not necessarily how it should be)


> I submitted a patch/module to RFE section since some time
> already, that implements the config storage
> that is read/write. The emphasis was to minimize
> locking for readers (making readers fast) on the cost
> of making writers slow(er).


http://sourceforge.net/tracker/index.php?func=detail&aid=1549952&group_id=130646&atid=719006

(Also, there's now an nsconf module in CVS, but that's something different...)


> Has anybody checked that? Because I think that we might
> really use that module as a replacement for the read-only
> version that is currently in place. We may even make the
> (new module) read-only per switch (default) but allow
> read-write for those who might need it (we, for example).
>
> Any ideas, objections?


Here's the thread from last time (lots of good ideas):

http://sourceforge.net/mailarchive/message.php?msg_id=9AD66047-D15A-4C93-B705-AFF1F2E5F330%40archiware.com


Did you eventually use this code? Did it work for you?

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

Yes, works fine now, Thanks

Stephen Deasey wrote:
> On 10/13/07, Vlad Seryakov <vl...@cr...> wrote:
>> I was able to find the location which causes the crash but i am not sure
>> how to fix it
>>
>> In function Ns_VALog i call vsnprintf directly, it works, when it is
>> called via Ns_DStringVPintf it crashes. using va_copy does not help
>>
> 
> 
> diff -r 22a912b584eb nsd/dstring.c
> --- a/nsd/dstring.c	Fri Oct 12 20:35:13 2007 +0100
> +++ b/nsd/dstring.c	Sat Oct 13 19:46:22 2007 +0100
> @@ -176,10 +176,11 @@ Ns_DStringPrintf(Ns_DString *dsPtr, CONS
>   */
> 
>  char *
> -Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list ap)
> -{
> -    char *buf;
> -    int   origLength, newLength, bufLength, result;
> +Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list apSrc)
> +{
> +    char    *buf;
> +    int      origLength, newLength, bufLength, result;
> +    va_list  ap;
> 
>      origLength = dsPtr->length;
> 
> @@ -205,11 +206,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
>      buf = dsPtr->string + origLength;
>      bufLength = newLength - origLength;
> 
> +    va_copy(ap, apSrc);
>  #ifdef __WIN32
>      result = _vsnprintf_s(buf, bufLength, fmt, ap);
>  #else
>      result = vsnprintf(buf, bufLength, fmt, ap);
>  #endif
> +    va_end(ap);
> 
>      /*
>       * Check for overflow and retry. For win32 just double the buffer size
> @@ -229,11 +232,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
>          buf = dsPtr->string + origLength;
>          bufLength = newLength - origLength;
> 
> +    va_copy(ap, apSrc);
>  #ifdef __WIN32
>          result = _vsnprintf_s(buf, bufLength, fmt, ap);
>  #else
>          result = vsnprintf(buf, bufLength, fmt, ap);
>  #endif
> +        va_end(ap);
>      }
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Stephen D. <sd...@gm...>]

On 10/13/07, Vlad Seryakov <vl...@cr...> wrote:
> I was able to find the location which causes the crash but i am not sure
> how to fix it
>
> In function Ns_VALog i call vsnprintf directly, it works, when it is
> called via Ns_DStringVPintf it crashes. using va_copy does not help
>


diff -r 22a912b584eb nsd/dstring.c
--- a/nsd/dstring.c	Fri Oct 12 20:35:13 2007 +0100
+++ b/nsd/dstring.c	Sat Oct 13 19:46:22 2007 +0100
@@ -176,10 +176,11 @@ Ns_DStringPrintf(Ns_DString *dsPtr, CONS
  */

 char *
-Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list ap)
-{
-    char *buf;
-    int   origLength, newLength, bufLength, result;
+Ns_DStringVPrintf(Ns_DString *dsPtr, CONST char *fmt, va_list apSrc)
+{
+    char    *buf;
+    int      origLength, newLength, bufLength, result;
+    va_list  ap;

     origLength = dsPtr->length;

@@ -205,11 +206,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
     buf = dsPtr->string + origLength;
     bufLength = newLength - origLength;

+    va_copy(ap, apSrc);
 #ifdef __WIN32
     result = _vsnprintf_s(buf, bufLength, fmt, ap);
 #else
     result = vsnprintf(buf, bufLength, fmt, ap);
 #endif
+    va_end(ap);

     /*
      * Check for overflow and retry. For win32 just double the buffer size
@@ -229,11 +232,13 @@ Ns_DStringVPrintf(Ns_DString *dsPtr, CON
         buf = dsPtr->string + origLength;
         bufLength = newLength - origLength;

+    va_copy(ap, apSrc);
 #ifdef __WIN32
         result = _vsnprintf_s(buf, bufLength, fmt, ap);
 #else
         result = vsnprintf(buf, bufLength, fmt, ap);
 #endif
+        va_end(ap);
     }

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

with included old dsprintf.c it works fine

Vlad Seryakov wrote:
> I was able to find the location which causes the crash but i am not sure 
> how to fix it
> 
> In function Ns_VALog i call vsnprintf directly, it works, when it is 
> called via Ns_DStringVPintf it crashes. using va_copy does not help
> 
> 
>      {
>            char buf[4096];
>            vsnprintf(buf, sizeof(buf), fmt, *vaPtr);
>            Ns_DStringAppend(&cachePtr->buffer, buf);
>      }
>      //Ns_DStringVPrintf(&cachePtr->buffer, fmt, *vaPtr);
> 
> 
> Gustaf Neumann wrote:
>> there is nothing wrong on using *va_list in general.
>> the problem is rather in vsnprintf, which does most probably
>> a loop like in Ns_DStringVarAppend
>>
>> char *
>> Ns_DStringVarAppend(Ns_DString *dsPtr, ...)
>> {
>>     register char   *s;
>>     va_list         ap;
>>
>>     va_start(ap, dsPtr);
>>     while ((s = va_arg(ap, char *)) != NULL) {
>>         Ns_DStringAppend(dsPtr, s);
>>     }
>>     va_end(ap);
>>
>>     return dsPtr->string;
>> }
>>
>> which fails, when the va_list is terminated by 0 and not (char*)NULL.
>> however, this should not account for the problem with vsnprintf(),
>> which should not read more arguments as indicated by the fmt string.
>> You are the, the number of %-codes corresponds to the arguments?
>> Is the error produced via Ns_TclLogErrorInfo() ?
>>
>> What happens, if you add at the end of the vararglist in the call
>> causing the problem a (char*)NULL)?
>>
>> -gustaf neumann
>>
>>
>>  
>> Vlad Seryakov schrieb:
>>> I use ns_log or Ns_Log, so it should take care about it, it ends up in 
>>> Ns_DStringPrintf which calls vsnprintf.
>>>
>>>
>>> Is it correct in x64 to pass *va_list?
>>>
>>> Ns_VALog(Ns_LogSeverity severity, CONST char *fmt, va_list *vaPtr)
>>>
>>> Gustaf Neumann wrote:
>>>   
>>>> be sure to terminate va_* argument lists with NULL (or to be on the safe 
>>>> side
>>>> with  (char*)NULL) and not with 0. when you have sizeof(int) != 
>>>> sizeof(char *),
>>>> the compiler will put a 32bit  for 0, a comparison with a 64bit null 
>>>> pointer
>>>> will fail. I am pretty sure, the vsnprintf()  tries to write an error 
>>>> message....
>>>> and misses the end of the var arg list....
>>>>
>>>> -gustaf neumann
>>>>
>>>> PS: fixed a similar issue in xotcl more than a year ago.
>>>>
>>>> Vlad Seryakov schrieb:
>>>>     
>>>>> Hi,
>>>>>
>>>>> This is an example when it crashes inside vsnprintf, experimenting i 
>>>>> found  that issuing long unknown command crashes as well. In my case i 
>>>>> noticed that trying to log long lines does not work as well. In x32 
>>>>> everything works fine
>>>>>
>>>>> db:vlad[18:01:19]#uname -a
>>>>> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
>>>>> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>>>>>
>>>>>
>>>>> ossweb:nscp 1>
>>>>> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>>>>>
>>>>>       
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems?  Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
> 

-- 
Vlad Seryakov
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

I was able to find the location which causes the crash but i am not sure 
how to fix it

In function Ns_VALog i call vsnprintf directly, it works, when it is 
called via Ns_DStringVPintf it crashes. using va_copy does not help


     {
           char buf[4096];
           vsnprintf(buf, sizeof(buf), fmt, *vaPtr);
           Ns_DStringAppend(&cachePtr->buffer, buf);
     }
     //Ns_DStringVPrintf(&cachePtr->buffer, fmt, *vaPtr);


Gustaf Neumann wrote:
> there is nothing wrong on using *va_list in general.
> the problem is rather in vsnprintf, which does most probably
> a loop like in Ns_DStringVarAppend
> 
> char *
> Ns_DStringVarAppend(Ns_DString *dsPtr, ...)
> {
>     register char   *s;
>     va_list         ap;
> 
>     va_start(ap, dsPtr);
>     while ((s = va_arg(ap, char *)) != NULL) {
>         Ns_DStringAppend(dsPtr, s);
>     }
>     va_end(ap);
> 
>     return dsPtr->string;
> }
> 
> which fails, when the va_list is terminated by 0 and not (char*)NULL.
> however, this should not account for the problem with vsnprintf(),
> which should not read more arguments as indicated by the fmt string.
> You are the, the number of %-codes corresponds to the arguments?
> Is the error produced via Ns_TclLogErrorInfo() ?
> 
> What happens, if you add at the end of the vararglist in the call
> causing the problem a (char*)NULL)?
> 
> -gustaf neumann
> 
> 
>  
> Vlad Seryakov schrieb:
>> I use ns_log or Ns_Log, so it should take care about it, it ends up in 
>> Ns_DStringPrintf which calls vsnprintf.
>>
>>
>> Is it correct in x64 to pass *va_list?
>>
>> Ns_VALog(Ns_LogSeverity severity, CONST char *fmt, va_list *vaPtr)
>>
>> Gustaf Neumann wrote:
>>   
>>> be sure to terminate va_* argument lists with NULL (or to be on the safe 
>>> side
>>> with  (char*)NULL) and not with 0. when you have sizeof(int) != 
>>> sizeof(char *),
>>> the compiler will put a 32bit  for 0, a comparison with a 64bit null 
>>> pointer
>>> will fail. I am pretty sure, the vsnprintf()  tries to write an error 
>>> message....
>>> and misses the end of the var arg list....
>>>
>>> -gustaf neumann
>>>
>>> PS: fixed a similar issue in xotcl more than a year ago.
>>>
>>> Vlad Seryakov schrieb:
>>>     
>>>> Hi,
>>>>
>>>> This is an example when it crashes inside vsnprintf, experimenting i 
>>>> found  that issuing long unknown command crashes as well. In my case i 
>>>> noticed that trying to log long lines does not work as well. In x32 
>>>> everything works fine
>>>>
>>>> db:vlad[18:01:19]#uname -a
>>>> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
>>>> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>>>>
>>>>
>>>> ossweb:nscp 1>
>>>> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>>>>
>>>>       
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

When i replaced Ns_VALog with pure vfprintf it does not crash anymore, 
so it is definitely something with passing va_list.

I tried as discussed here but it still crashes:
http://osdir.com/ml/redhat.amd64/2006-12/msg00001.html

void
Ns_Log(Ns_LogSeverity severity, CONST char *fmt, ...)
{
     va_list ap;

     va_start(ap, fmt);
vfprintf(stderr, fmt, ap);
fprintf(stderr, "\n");
     //Ns_VALog(severity, fmt, ap);
     va_end(ap);
}


Gustaf Neumann wrote:
> there is nothing wrong on using *va_list in general.
> the problem is rather in vsnprintf, which does most probably
> a loop like in Ns_DStringVarAppend
> 
> char *
> Ns_DStringVarAppend(Ns_DString *dsPtr, ...)
> {
>     register char   *s;
>     va_list         ap;
> 
>     va_start(ap, dsPtr);
>     while ((s = va_arg(ap, char *)) != NULL) {
>         Ns_DStringAppend(dsPtr, s);
>     }
>     va_end(ap);
> 
>     return dsPtr->string;
> }
> 
> which fails, when the va_list is terminated by 0 and not (char*)NULL.
> however, this should not account for the problem with vsnprintf(),
> which should not read more arguments as indicated by the fmt string.
> You are the, the number of %-codes corresponds to the arguments?
> Is the error produced via Ns_TclLogErrorInfo() ?
> 
> What happens, if you add at the end of the vararglist in the call
> causing the problem a (char*)NULL)?
> 
> -gustaf neumann
> 
> 
>  
> Vlad Seryakov schrieb:
>> I use ns_log or Ns_Log, so it should take care about it, it ends up in 
>> Ns_DStringPrintf which calls vsnprintf.
>>
>>
>> Is it correct in x64 to pass *va_list?
>>
>> Ns_VALog(Ns_LogSeverity severity, CONST char *fmt, va_list *vaPtr)
>>
>> Gustaf Neumann wrote:
>>   
>>> be sure to terminate va_* argument lists with NULL (or to be on the safe 
>>> side
>>> with  (char*)NULL) and not with 0. when you have sizeof(int) != 
>>> sizeof(char *),
>>> the compiler will put a 32bit  for 0, a comparison with a 64bit null 
>>> pointer
>>> will fail. I am pretty sure, the vsnprintf()  tries to write an error 
>>> message....
>>> and misses the end of the var arg list....
>>>
>>> -gustaf neumann
>>>
>>> PS: fixed a similar issue in xotcl more than a year ago.
>>>
>>> Vlad Seryakov schrieb:
>>>     
>>>> Hi,
>>>>
>>>> This is an example when it crashes inside vsnprintf, experimenting i 
>>>> found  that issuing long unknown command crashes as well. In my case i 
>>>> noticed that trying to log long lines does not work as well. In x32 
>>>> everything works fine
>>>>
>>>> db:vlad[18:01:19]#uname -a
>>>> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
>>>> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>>>>
>>>>
>>>> ossweb:nscp 1>
>>>> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>>>>
>>>>       
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Gustaf N. <ne...@wu...>]

there is nothing wrong on using *va_list in general.
the problem is rather in vsnprintf, which does most probably
a loop like in Ns_DStringVarAppend

char *
Ns_DStringVarAppend(Ns_DString *dsPtr, ...)
{
    register char   *s;
    va_list         ap;

    va_start(ap, dsPtr);
    while ((s = va_arg(ap, char *)) != NULL) {
        Ns_DStringAppend(dsPtr, s);
    }
    va_end(ap);

    return dsPtr->string;
}

which fails, when the va_list is terminated by 0 and not (char*)NULL.
however, this should not account for the problem with vsnprintf(),
which should not read more arguments as indicated by the fmt string.
You are the, the number of %-codes corresponds to the arguments?
Is the error produced via Ns_TclLogErrorInfo() ?

What happens, if you add at the end of the vararglist in the call
causing the problem a (char*)NULL)?

-gustaf neumann


 
Vlad Seryakov schrieb:
> I use ns_log or Ns_Log, so it should take care about it, it ends up in 
> Ns_DStringPrintf which calls vsnprintf.
>
>
> Is it correct in x64 to pass *va_list?
>
> Ns_VALog(Ns_LogSeverity severity, CONST char *fmt, va_list *vaPtr)
>
> Gustaf Neumann wrote:
>   
>> be sure to terminate va_* argument lists with NULL (or to be on the safe 
>> side
>> with  (char*)NULL) and not with 0. when you have sizeof(int) != 
>> sizeof(char *),
>> the compiler will put a 32bit  for 0, a comparison with a 64bit null 
>> pointer
>> will fail. I am pretty sure, the vsnprintf()  tries to write an error 
>> message....
>> and misses the end of the var arg list....
>>
>> -gustaf neumann
>>
>> PS: fixed a similar issue in xotcl more than a year ago.
>>
>> Vlad Seryakov schrieb:
>>     
>>> Hi,
>>>
>>> This is an example when it crashes inside vsnprintf, experimenting i 
>>> found  that issuing long unknown command crashes as well. In my case i 
>>> noticed that trying to log long lines does not work as well. In x32 
>>> everything works fine
>>>
>>> db:vlad[18:01:19]#uname -a
>>> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
>>> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>>>
>>>
>>> ossweb:nscp 1>
>>> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>>>
>>>       

Re: [naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

On 13.10.2007, at 00:06, Vlad Seryakov wrote:

> In it will not hurt performance when constantly asking for config
> parameter with locking i am fine with it.

Locking is of course necessary. But it is limited to locking
one mutex and, more importantly, the locking contention is
only between one reading and one or more writing threads.
Locking a mutex is per-se not problematic, speedwise. It is
the contention on the mutex that can slow you down.
Assuming that writing (changing config) is far less to expect
tnat reading, I'm sure there will be no speed penalty.
I can also make the thing read-only per-config parameter so
to avoid locking altogether if any concerns arise in future.

It is good to peek in the code so you can see why I think
this approach will not hurt.

Cheers
Zoran

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

I use ns_log or Ns_Log, so it should take care about it, it ends up in 
Ns_DStringPrintf which calls vsnprintf.


Is it correct in x64 to pass *va_list?

Ns_VALog(Ns_LogSeverity severity, CONST char *fmt, va_list *vaPtr)

Gustaf Neumann wrote:
> be sure to terminate va_* argument lists with NULL (or to be on the safe 
> side
> with  (char*)NULL) and not with 0. when you have sizeof(int) != 
> sizeof(char *),
> the compiler will put a 32bit  for 0, a comparison with a 64bit null 
> pointer
> will fail. I am pretty sure, the vsnprintf()  tries to write an error 
> message....
> and misses the end of the var arg list....
> 
> -gustaf neumann
> 
> PS: fixed a similar issue in xotcl more than a year ago.
> 
> Vlad Seryakov schrieb:
>> Hi,
>>
>> This is an example when it crashes inside vsnprintf, experimenting i 
>> found  that issuing long unknown command crashes as well. In my case i 
>> noticed that trying to log long lines does not work as well. In x32 
>> everything works fine
>>
>> db:vlad[18:01:19]#uname -a
>> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
>> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>>
>>
>> ossweb:nscp 1>
>> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems?  Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>   
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Crash in x64 mode in Ns_Log

[Gustaf N. <ne...@wu...>]

be sure to terminate va_* argument lists with NULL (or to be on the safe 
side
with  (char*)NULL) and not with 0. when you have sizeof(int) != 
sizeof(char *),
the compiler will put a 32bit  for 0, a comparison with a 64bit null 
pointer
will fail. I am pretty sure, the vsnprintf()  tries to write an error 
message....
and misses the end of the var arg list....

-gustaf neumann

PS: fixed a similar issue in xotcl more than a year ago.

Vlad Seryakov schrieb:
> Hi,
>
> This is an example when it crashes inside vsnprintf, experimenting i 
> found  that issuing long unknown command crashes as well. In my case i 
> noticed that trying to log long lines does not work as well. In x32 
> everything works fine
>
> db:vlad[18:01:19]#uname -a
> Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
> Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux
>
>
> ossweb:nscp 1>
> ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>   

Re: [naviserver-devel] ns_config read/write

[Vlad S. <vl...@cr...>]

In it will not hurt performance when constantly asking for config 
parameter with locking i am fine with it. Can be also compile time 
option for configure

Vasiljevic Zoran wrote:
> So:
> 
> 
>    The configuration is global and read-only. Parameters may be  
> retrieved at
>    run-time using [cmd ns_config], although usually configuration  
> parameters are
>    used by Tcl library files at start-up.
> 
> I submitted a patch/module to RFE section since some time
> already, that implements the config storage
> that is read/write. The emphasis was to minimize
> locking for readers (making readers fast) on the cost
> of making writers slow(er).
> 
> Has anybody checked that? Because I think that we might
> really use that module as a replacement for the read-only
> version that is currently in place. We may even make the
> (new module) read-only per switch (default) but allow
> read-write for those who might need it (we, for example).
> 
> Any ideas, objections?
> 
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

[naviserver-devel] Crash in x64 mode in Ns_Log

[Vlad S. <vl...@cr...>]

Hi,

This is an example when it crashes inside vsnprintf, experimenting i 
found  that issuing long unknown command crashes as well. In my case i 
noticed that trying to log long lines does not work as well. In x32 
everything works fine

db:vlad[18:01:19]#uname -a
Linux db 2.6.22-ARCH #1 SMP PREEMPT Thu Oct 4 11:47:51 EDT 2007 x86_64 
Intel(R) Xeon(R) CPU           X5355  @ 2.66GHz GenuineIntel GNU/Linux


ossweb:nscp 1>
ossweb:nscp 1> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

[naviserver-devel] ns_config read/write

[Vasiljevic Z. <zv...@ar...>]

So:


   The configuration is global and read-only. Parameters may be  
retrieved at
   run-time using [cmd ns_config], although usually configuration  
parameters are
   used by Tcl library files at start-up.

I submitted a patch/module to RFE section since some time
already, that implements the config storage
that is read/write. The emphasis was to minimize
locking for readers (making readers fast) on the cost
of making writers slow(er).

Has anybody checked that? Because I think that we might
really use that module as a replacement for the read-only
version that is currently in place. We may even make the
(new module) read-only per switch (default) but allow
read-write for those who might need it (we, for example).

Any ideas, objections?

Re: [naviserver-devel] [naviserver-commits] naviserver/include nsthread.h, 1.25, 1.26

[Vasiljevic Z. <zv...@ar...>]

On 09.10.2007, at 15:49, Stephen Deasey wrote:

>
> This will fail silently and someone will be surprised...

Not any more.

Re: [naviserver-devel] [naviserver-commits] naviserver/include nsthread.h, 1.25, 1.26

[Vasiljevic Z. <zv...@ar...>]

On 09.10.2007, at 15:49, Stephen Deasey wrote:

> Solaris 2.8 is 32-bit only?
>
> This will fail silently and someone will be surprised...

So the suggestion is to make it 32/64 bits?

Re: [naviserver-devel] [naviserver-commits] naviserver/include nsthread.h, 1.25, 1.26

[Stephen D. <sd...@gm...>]

On 10/9/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> On 09.10.2007, at 15:07, Stephen Deasey wrote:
>
> >
> > 32-bit only -- is this a good assumption?
>
> No. But I found only Solaris 2.8 missing them.


Solaris 2.8 is 32-bit only?

This will fail silently and someone will be surprised...

Re: [naviserver-devel] [naviserver-commits] naviserver/include nsthread.h, 1.25, 1.26

[Vasiljevic Z. <zv...@ar...>]

On 09.10.2007, at 15:07, Stephen Deasey wrote:

>
> 32-bit only -- is this a good assumption?

No. But I found only Solaris 2.8 missing them.
Solaris 2.10 and all the other have those
macros defined. Solaris 2.8 is old and nobody
that I know really needs that any more.

Hence I did not want to make a "project"
arround that, just get our own code compiled
for 2.8 solaris.

>
> Lot's of examples of working around these missing macros:
>
>   http://www.google.com/codesearch?q=PRIdMAX
>

Well, yes. But, practically, this is all old hat.
I checked all boxes that we compile on and only
Solaris 2.8 is missing those from inttypes.h.
All other platforms are basically OK so I see no
need to invest much work there. Unless we want to
be "clean" on older HP/UX or AIX'es or other
more esoteric machines... I guess not. All "newer"
Linuxes, Mac's and Solaris'es are perfectly OK.

Cheers
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver/include nsthread.h, 1.25, 1.26

[Stephen D. <sd...@gm...>]

On 10/9/07, Zoran Vasiljevic <vas...@us...> wrote:
> Update of /cvsroot/naviserver/naviserver/include
> In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14795/include
>
> Modified Files:
>         nsthread.h
> Log Message:
> * include/nsthread.h: Added macro definitions for
> some formatted IO macros, if not defined, as is
> the case for older Solaris versions (2.8-).



>   /*
> +  * Older Solaris version (2.8-)  lack formatted IO
> +  * macros in inttypes.h. Declare some of those here,
> +  * assuming a 32-bit system. This is not a complete
> +  * list, it only reflects what's used in the code.
> +  */


32-bit only -- is this a good assumption?

Lot's of examples of working around these missing macros:

  http://www.google.com/codesearch?q=PRIdMAX