naviserver-devel — Developer discussion

Re: [naviserver-devel] nsstats.tcl

[Gustaf N. <ne...@wu...>]

this was a bug. many thnaks for noting! -g

On 14.10.20 13:26, Maksym Zinchenko wrote:
> Hello, I've updated Naviserver from the latest commit in bitbucket and 
> nsstats.tcl stopped authenticating me with plain authentication. I've 
> checked and noticed that "ns_conn authpassword" and "ns_conn 
> *authuser*" return empty values. Is it a bug ?

[naviserver-devel] nsstats.tcl

[Maksym Z. <siq...@gm...>]

Hello, I've updated Naviserver from the latest commit in bitbucket and
nsstats.tcl stopped authenticating me with plain authentication. I've
checked and noticed that "ns_conn authpassword" and "ns_conn *authuser*"
return empty values. Is it a bug ?

Re: [naviserver-devel] ns_conn status

[David O. <da...@qc...>]

Thank you very much Gustaf

On Tue, 8 Sep 2020 at 19:12, Gustaf Neumann <ne...@wu...> wrote:

> Dear David,
>
> You assumption is right, ... AFIKT there is no problem with the case,
> when  "ns_conn status" is called without additional argument in a trace.
> i have added code to  bitbucket handle the case with and without
> arguments differently.
>
> all the best
>
> -g

Re: [naviserver-devel] Fw: ns_http and 8-bit enodings

[Zoran V. <zv...@ar...>]

pls take the time off  to relax. work is not going to run away...
i am also off until 20th of sept.
have a great time!

Am 10. September 2020 12:51:24 MESZ schrieb Gustaf Neumann <ne...@wu...>:
>On 05.09.20 14:50, Gustaf Neumann wrote:
>> I did not have time yet to dig into TCLHTTP_USE_EXTERNALTOUTF the 
>> other test cases depending on that. My gut feeling says "this is 
>> necessary", but my brain says "why this was no problem so far" and 
>> "what will be the collateral damage"  if we do so.
>>
>> The flag TCLHTTP_USE_EXTERNALTOUTF is still in the code but needs 
>> further investigation. However, i am going mid of next week on
>holiday 
>> (boat), where i will not be reachable ... until then many things are 
>> piled up on my side, so i have no chance to look into this deeper 
>> right now.
>
>My plan would be as follows: Prepare a release for the near future
>(i'll 
>be at 10 days offline, maybe i can work on change summary etc.). For 
>this release, leave the TCLHTTP_USE_EXTERNALTOUTF for the time being in
>
>the code (deactivated by default), and address the encoding issue in 
>detail after that.
>
>Users depending on releases will get a long list of new features with 
>4.99.20 and they will not be afraid of changed encodings etc. Other 
>users with complex encoding/recoding requirements can use the new 
>"-binary" flag of ns_http.
>
>all the best
>
>-g
>
>
>
>_______________________________________________
>naviserver-devel mailing list
>nav...@li...
>https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Fw: ns_http and 8-bit enodings

[Gustaf N. <ne...@wu...>]

On 05.09.20 14:50, Gustaf Neumann wrote:
> I did not have time yet to dig into TCLHTTP_USE_EXTERNALTOUTF the 
> other test cases depending on that. My gut feeling says "this is 
> necessary", but my brain says "why this was no problem so far" and 
> "what will be the collateral damage"  if we do so.
>
> The flag TCLHTTP_USE_EXTERNALTOUTF is still in the code but needs 
> further investigation. However, i am going mid of next week on holiday 
> (boat), where i will not be reachable ... until then many things are 
> piled up on my side, so i have no chance to look into this deeper 
> right now.

My plan would be as follows: Prepare a release for the near future (i'll 
be at 10 days offline, maybe i can work on change summary etc.). For 
this release, leave the TCLHTTP_USE_EXTERNALTOUTF for the time being in 
the code (deactivated by default), and address the encoding issue in 
detail after that.

Users depending on releases will get a long list of new features with 
4.99.20 and they will not be afraid of changed encodings etc. Other 
users with complex encoding/recoding requirements can use the new 
"-binary" flag of ns_http.

all the best

-g

Re: [naviserver-devel] ns_conn status

[Gustaf N. <ne...@wu...>]

Dear David,

You assumption is right, ... AFIKT there is no problem with the case, 
when  "ns_conn status" is called without additional argument in a trace. 
i have added code to  bitbucket handle the case with and without 
arguments differently.

all the best

-g

On 07.09.20 17:53, David Osborne wrote:
> I notice that "ns_conn status" is flagged in conn.c as requiring an 
> active connection
> ...
> I suspect the flag is required because "ns_conn status" is also used 
> to SET the http status code?

[naviserver-devel] ns_conn status

[David O. <da...@qc...>]

Hi,

I notice that "ns_conn status" is flagged in conn.c as requiring an active
connection (NS_CONN_REQUIRE_CONNECTED rather than
NS_CONN_REQUIRE_CONFIGURED).
The http status code is useful to be able to query in a trace after the
connection is closed.
Obviously that use-case is now incompatible given this flag.

I suspect the flag is required because "ns_conn status" is also used to SET
the http status code?
Is there still a way of querying the status code retrospectively in a trace?

Regards,
-- 

*David *

Re: [naviserver-devel] CMD exec, CMD 1row,CMD select

[Gustaf N. <ne...@wu...>]

On 06.09.20 15:30, Maksym Zinchenko wrote:
> /usr/bin/ld: libnsd.so: undefined reference to `NsTclCryptoScryptObjCmd'
That happened when compiled without HAVE_OPENSSL_3. Fixed now on 
bitbucket. Sorry for that. -g

Re: [naviserver-devel] CMD exec, CMD 1row,CMD select

[Maksym Z. <siq...@gm...>]

Ok, but now its giving me compilation error in nsd:

/usr/bin/ld: libnsd.so: undefined reference to `NsTclCryptoScryptObjCmd'

What i'm doing wrong?

On Sun, Sep 6, 2020 at 11:26 AM Gustaf Neumann <ne...@wu...> wrote:

> On 06.09.20 12:00, Maksym Zinchenko wrote:
> > Hello, I have a lot of lines in my logs like:
> this was escaped debug output on the (unreleasaed) master branch. this
> is removed by now.
>
> Many thanks for noting this
>
> -g
>
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] CMD exec, CMD 1row,CMD select

[Gustaf N. <ne...@wu...>]

On 06.09.20 12:00, Maksym Zinchenko wrote:
> Hello, I have a lot of lines in my logs like:
this was escaped debug output on the (unreleasaed) master branch. this 
is removed by now.

Many thanks for noting this

-g

[naviserver-devel] CMD exec, CMD 1row,CMD select

[Maksym Z. <siq...@gm...>]

Hello, I have a lot of lines in my logs like:

CMD exec: <----query-----> expire> (none)

CMD select: <----query-----> (string)

CMD 1row: <----query-----> (none)

I've searched my code, but it looks like it's not mine :) Where do I switch
off this sql logging?

Thanks

Re: [naviserver-devel] Fw: ns_http and 8-bit enodings

[Gustaf N. <ne...@wu...>]

On 04.09.20 13:53, oleg wrote:
> After the patch, ns_http became similar to ns_return: understands charset and -binary )

exactly.

Zoran:

 > OK. I see no problem with that

I've added the "-binary" flag to "ns_http" as provided by Oleg to 
bitbucket. I've also added a short documentation and the 
ns_http_charsets.test, where i've added a "-binary" flag all occurrences 
where "encoding convertfrom" is explicitly used (either we provide an 
automatic conversion or none).

I did not have time yet to dig into TCLHTTP_USE_EXTERNALTOUTF the other 
test cases depending on that. My gut feeling says "this is necessary", 
but my brain says "why this was no problem so far" and "what will be the 
collateral damage"  if we do so.

The flag TCLHTTP_USE_EXTERNALTOUTF is still in the code but needs 
further investigation. However, i am going mid of next week on holiday 
(boat), where i will not be reachable ... until then many things are 
piled up on my side, so i have no chance to look into this deeper right now.

The change form oleg has high quality, including many test cases which 
are releated to TCLHTTP_USE_EXTERNALTOUTF but not committed yet.

all the best, and many thanks to Oleg!

-g

Re: [naviserver-devel] ns_http and 8-bit enodings

[Zoran V. <zv...@ar...>]

On Fri, 4 Sep 2020 15:07:32 +0300
oleg <oo...@ua...> wrote:

> Yes. In fact, I have been using -binary for two years.

OK. I see no problem with that. Actually I cannot see 
any other way of handling such cases, to be honest.

If this is still open when I come back from holidays
in about two weeks, I will put that in.

Cheer's
Zoran

Re: [naviserver-devel] ns_http and 8-bit enodings

[oleg <oo...@ua...>]

On Fri, 4 Sep 2020 13:55:05 +0200
Zoran Vasiljevic <zv...@ar...> wrote:

> Does the -binary option alone solves your problem?

Yes. In fact, I have been using -binary for two years.

Oleg.

[naviserver-devel] Fw: ns_http and 8-bit enodings

[oleg <oo...@ua...>]

On Fri, 4 Sep 2020 11:38:53 +0200
Gustaf Neumann <ne...@wu...> wrote:

> since HTTP has means to include encodings, which NaviServer uses
> acting as a server, it should behave the same way when acting as a
> client and not burdening the application to dig into the content-type
> charsets to call the right conversion stuff.  

After the patch, ns_http became similar to ns_return: understands charset and -binary )

> Having NaviServer versions leading to different
> results depending on compile flags is not a good idea.  

Totally agree. #ifdefs were inserted only for safety because this code changes the usual behavior of the ns_http command

> To get a more detailed understanding, i have to dig into your examples
> to understand whether this is indeed a problem on the Tcl side or
> in NaviServer, ... but for this, i need a certain block of time, which
> is hard to get for me right now.  

Please see my answer to Zoran. There I wrote my opinion (possibly wrong) about using the Tcl_NewStringObj function.


Oleg.

[naviserver-devel] Fw: ns_http and 8-bit enodings

[oleg <oo...@ua...>]

On Fri, 4 Sep 2020 10:31:53 +0200
Zoran Vasiljevic <zv...@ar...> wrote:

> Hi!    
Hi!

> What is the purpose of signalling the binary content way up from the
> command line? For this we have the Ns_IsBinaryMimeType(cType) test.
>  

Sometimes we have to deal with misconfigured legacy sites, which transmit data in 8-bit encoding and report "Content-Type: text/plain" without specifying charset. Such data will be broken, since it will be processed ns_http as utf-8.

Receiving binary content in this case will allow us to correctly recover data using 'encoding convertfrom' command 

>     
> > Sometimes converted strings become
> > corrupted. For example, there is a server with output encoding
> > iso-8859-2      
> 
> This is I guess out of the scope as we rely on Tcl to do 
> encoding conversions. So if this comes bad, then you must
> post a bug to the Tcl project.    

As far as I understand, this is not a Tcl problem.

ns_http uses Tcl_NewStringObj for nonbinary data received, even for 8-bit chars.
Input data for Tcl_NewStringObj must be correct utf-8 string. An 8-bit string is not.
For 8-bit strings, we must either use Tcl_NewByteArrayObj or prepare the utf-8 string using Tcl_ExternalToUtf before Tcl_NewStringObj .


Oleg.

Re: [naviserver-devel] ns_http and 8-bit enodings

[Gustaf N. <ne...@wu...>]

Hi Oleg,

since HTTP has means to include encodings, which NaviServer uses acting
as a server, it should behave the same way when acting as a client and
not burdening the application to dig into the content-type charsets
to call the right conversion stuff.

A "-binary" flag still makes sense in cases there is no content-type
given or to let the developer overrule other mechanisms.
The usage of the "-binary" flag + convertfrom/to should
always be applicable. Having NaviServer versions leading to different
results depending on compile flags is not a good idea.

To get a more detailed understanding, i have to dig into your examples
to understand whether this is indeed a problem on the Tcl side or
in NaviServer, ... but for this, i need a certain block of time, which
is hard to get for me right now.

-g

On 03.09.20 13:52, oleg wrote:
> Hello!
>
> We are having some difficulties when using the ns_http command with
> sites using 8-bit encoding.
>
> The ns_http command does not convert the received data, so we must use
> the 'encoding convertfrom' command. Sometimes converted strings become
> corrupted. For example, there is a server with output encoding
> iso-8859-2:
> if the server passes 'äöüŁ', then after conversion we get 'äöüŁ'
> (correct);
> if the server passes 'ÄÖÜŁ', then after conversion we get 'ÄÖ#'
> (corrupted).
> See attached ns_http.test1 for example (test 1.2 fails).
>
> Such strings can be found in any 8-bit encoding (to see run attached
> http_charsets.test with 'pairsTest' constraint enabled).
> The source for the ns_http command (tclhttp.c) shows that the problem is
> using the Tcl_NewStringObj on binary input data (8-bit chars).
>
> Two solutions come up:
> 1) Using Tcl_NewByteArrayObj instead of Tcl_NewStringObj;
> 2) Using Tcl_ExternalToUtf before using Tcl_NewStringObj, i.e. built-in
> 'encoding convertfrom'.
>
> Attached tclhttp.c.binary-externaltoutf patch modifies the ns_http
> command:
> 1) the -binary switch is added to the queue/wait/run sub-commands to use
> of Tcl_NewByteArrayObj on text pages;
> 2) without -binary the text page will be converted according to the
> Content-Type header.
>
> Note that the second change requires the TCLHTTP_USE_EXTERNALTOUTF to
> be defined at compile time.
>
> The fixed ns_http command can be tested with the attached ns_http.test2
> (see 1.2.1 and 1.2.2). More intensive testing of changes can be done
> with the http_charsets.test (note commented pairsTest
> constraint).
> Also I replaced the 'nstest :: http-0.9 -encoding xxx' with 'ns_http
> run' in existing encoding.test (see encoding_ns_http.test). All data
> transformations are successfully performed without explicit decoding.
>
> Automatic data decoding is convenient to use, but it changes the
> behavior of ns_http on 8-bit inputs. These changes could break existing
> code if someone uses ns_http to inter with 8-bit sites (with risk of
> data corruption). To use the patched version of ns_http, either remove
> the 'encoding convertfrom' or add the -binary switch.
>
> It should be noted that the -binary switch followed by 'encoding
> convertfrom' will also be useful for 8-bit sites with missing or
> incorrect Content-Type.
>
> Regards,
> Oleg Oleinick.
>
> PS. Attached files:
>
> ns_http.test1 - tests for the current version, shows corruption of
> 8-bit text;
>
> ns_http.test2 - tests for the patched version, shows the correct
> receipt of 8-bit text;
>
> tclhttp.c.binary-externaltoutf.patch - patch for changing the ns_http
> command, adds the -binary switch and text data auto-decoding;
>
> http_charsets.test - tests for ns_http, suitable for both the current
> and the patched version;
>
> encoding_ns_http.test - like existing encoding.test, with 'nstest ::
> http-0.9 -encoding xxx' replaces by new 'ns_http run';
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_http and 8-bit enodings

[Zoran V. <zv...@ar...>]

On Thu, 3 Sep 2020 14:52:44 +0300
oleg <oo...@ua...> wrote:

Hi!
Thanks for looking into this.

> The ns_http command does not convert the received data, so we must use
> the 'encoding convertfrom' command. 

I see you added the Ns_GetTypeEncoding(cType) test for non-binary content
and then use Tcl_ExternalToUtfDString() to convert. 
This seems OK for me.

What is the purpose of signalling the binary content way up from the command 
line? For this we have the Ns_IsBinaryMimeType(cType) test.

> Sometimes converted strings become
> corrupted. For example, there is a server with output encoding
> iso-8859-2

This is I guess out of the scope as we rely on Tcl to do 
encoding conversions. So if this comes bad, then you must
post a bug to the Tcl project.

Cheers
Zoran

[naviserver-devel] ns_http and 8-bit enodings

[oleg <oo...@ua...>]

Hello!

We are having some difficulties when using the ns_http command with
sites using 8-bit encoding.

The ns_http command does not convert the received data, so we must use
the 'encoding convertfrom' command. Sometimes converted strings become
corrupted. For example, there is a server with output encoding
iso-8859-2:
if the server passes 'äöüŁ', then after conversion we get 'äöüŁ'
(correct);
if the server passes 'ÄÖÜŁ', then after conversion we get 'ÄÖ#'
(corrupted).
See attached ns_http.test1 for example (test 1.2 fails).

Such strings can be found in any 8-bit encoding (to see run attached
http_charsets.test with 'pairsTest' constraint enabled).
The source for the ns_http command (tclhttp.c) shows that the problem is
using the Tcl_NewStringObj on binary input data (8-bit chars).

Two solutions come up:
1) Using Tcl_NewByteArrayObj instead of Tcl_NewStringObj;
2) Using Tcl_ExternalToUtf before using Tcl_NewStringObj, i.e. built-in
'encoding convertfrom'.

Attached tclhttp.c.binary-externaltoutf patch modifies the ns_http
command:
1) the -binary switch is added to the queue/wait/run sub-commands to use
of Tcl_NewByteArrayObj on text pages;
2) without -binary the text page will be converted according to the
Content-Type header.

Note that the second change requires the TCLHTTP_USE_EXTERNALTOUTF to
be defined at compile time.

The fixed ns_http command can be tested with the attached ns_http.test2
(see 1.2.1 and 1.2.2). More intensive testing of changes can be done
with the http_charsets.test (note commented pairsTest
constraint).
Also I replaced the 'nstest :: http-0.9 -encoding xxx' with 'ns_http
run' in existing encoding.test (see encoding_ns_http.test). All data
transformations are successfully performed without explicit decoding.

Automatic data decoding is convenient to use, but it changes the
behavior of ns_http on 8-bit inputs. These changes could break existing
code if someone uses ns_http to inter with 8-bit sites (with risk of
data corruption). To use the patched version of ns_http, either remove
the 'encoding convertfrom' or add the -binary switch.

It should be noted that the -binary switch followed by 'encoding
convertfrom' will also be useful for 8-bit sites with missing or
incorrect Content-Type.

Regards,
Oleg Oleinick.

PS. Attached files:

ns_http.test1 - tests for the current version, shows corruption of
8-bit text;

ns_http.test2 - tests for the patched version, shows the correct
receipt of 8-bit text;

tclhttp.c.binary-externaltoutf.patch - patch for changing the ns_http
command, adds the -binary switch and text data auto-decoding;

http_charsets.test - tests for ns_http, suitable for both the current
and the patched version;

encoding_ns_http.test - like existing encoding.test, with 'nstest ::
http-0.9 -encoding xxx' replaces by new 'ns_http run';

Re: [naviserver-devel] Cookie Same-Site "None"

[Gustaf N. <ne...@wu...>]

Dear Wolfgang,

i've added a followup change, such that that the omission of "-samesite" 
flag
on ns_cookie does not result into a "samesite=none". This is more
conservative.

-g

On 01.09.20 10:18, Wolfgang Winkler via naviserver-devel wrote:
>
> Dear Gustaf,
>
> bthanks for the speedy fix. We've cherry picked the commit for 4.99.19 
> and it works flawlessly.
>
> Cookie handling has become a catch up game lately, as browser vendors 
> are getting more and more creative without a proper standardization 
> process.
>
> Regards,
>
> wiwo
>
> Am 31.08.20 um 12:54 schrieb Gustaf Neumann:
>> Wolfgang,
>>
>> you are right, explicit setting of same-site=none is necessary now.
>>
>> In previous versions of browsers, no explicit setting
>> of the same-site flag was exactly the same as explicit setting
>> (an implicit default of same-site=none)
>>
>> Since some browsers switched to a default of "lax", explicit
>> setting became necessary.
>>
>> Fixed now on bitbucket.
>>
>> -gn
>>
>> PS: it is not developer-friendly that the behavior is changed
>> on the fly.... On the client site, the disruptive behavior
>> change was intended, so changing the default value on the
>> server is probably not good - and is left unchanged.
>>
>>
>>
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> -- 
>
> *Wolfgang Winkler*
> Geschäftsführung
> wol...@di...
> mobil +43.699.19971172
>
> dc:*büro*
> digital concepts Novak Winkler OG
> Software & Design
> Landstraße 68, 5. Stock, 4020 Linz
> www.digital-concepts.com <http://www.digital-concepts.com>
> tel +43.732.997117.72
> tel +43.699.1997117.2
>
> Firmenbuchnummer: 192003h
> Firmenbuchgericht: Landesgericht Linz
>
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Cookie Same-Site "None"

[Wolfgang W. <wol...@di...>]

Dear Gustaf,

thanks for the speedy fix. We've cherry picked the commit for 4.99.19 
and it works flawlessly.

Cookie handling has become a catch up game lately, as browser vendors 
are getting more and more creative without a proper standardization process.

Regards,

wiwo

Am 31.08.20 um 12:54 schrieb Gustaf Neumann:
> Wolfgang,
>
> you are right, explicit setting of same-site=none is necessary now.
>
> In previous versions of browsers, no explicit setting
> of the same-site flag was exactly the same as explicit setting
> (an implicit default of same-site=none)
>
> Since some browsers switched to a default of "lax", explicit
> setting became necessary.
>
> Fixed now on bitbucket.
>
> -gn
>
> PS: it is not developer-friendly that the behavior is changed
> on the fly.... On the client site, the disruptive behavior
> change was intended, so changing the default value on the
> server is probably not good - and is left unchanged.
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] using rwlocks for nsv variables

[Gustaf N. <ne...@wu...>]

Dear all,

on June 30, i wrote:

> However, there might be NaviServer applications with nsvs out there, 
> for which the change of using rwlocks for nsv variables might lead to 
> a reduced performance. So we have in general the following options for 
> the forthcoming release:
>
> a) hardwire nsvs to rwlocks
> b) make it a compile-time decision to choose between rwlocks and mutex 
> locks for nsvs
> c) provide a configuration variable in the config file to choose 
> between rwlocks and mutex locks for nsvs at startup
> d) provide a runtime API for creating nsv arrays with rwlock or mutex
>
Since there were some concerns that rwlocks might not be the best choice 
for all nsvs, i did some more extensive tests and metering on several 
servers. These tests convinced me that rwlocks are best default value 
for web server applications, since there is an overwhelming amount of 
read operations compared to write operations (see below, e.g. less the 
5% writer operations, these are real-world figures from our production 
code at the university):

     Name	                Locks	Busy	Read	Write	Write %
     nsv:56:live	                192.91M	14	192.87M	47.79K	0.02%
     nsv:138:live	        92.15M	1	92.13M	16.21K	0.02%
     nsv:169:live	        44M	0	43.99M	14.46K	0.03%
     nsv:164:live	        31.81M	21	31.52M	294.13K	0.92%
     nsv:157:live	        26.96M	0	26.96M	1.5K	0.01%
     nsv:76:live	                23.53M	0	23.53M	1.26K	0.01%
     nsv:146:live	        20.59M	0	20.55M	38.33K	0.19%
     nsv:27:live	                13.26M	0	13.26M	801	0.01%
     nsv:50:live	                8.98M	0	8.98M	1.15K	0.01%
     nsv:185:live	        8.57M	0	8.28M	288.21K	3.36%
     nsv:184:live	        8.02M	0	7.72M	290.49K	3.62%
     nsv:107:live	        7.05M	0	7.05M	924	0.01%

One more interesting fact is we see very little number of busy 
operations, which was in the case of mutex locks on the same server as 
least by a factor of 1000 higher. So, we can achieve a much higher 
degree of parallelism using rwlocks. These numbers can be obtained from 
the updated versions of the nstats module.

One more interesting comparison is potting different kind of operations 
into relation. The numbers below are in the sense of "Numbers everyone 
should know" in [1].

As we can see one very simple DB query (0.068ms) costs as much as 500 
Tcl variable lookups (130 ns), but is about 70 times faster then an 
"exec ls /". The operations in the table are either NaviServer 
primitives, NSF commands, or simple OpenACS commands, from the point of 
view of an application developer (the exec is over nsproxy).

The results show that the nsv read operation (based on rwlock) is faster 
than a proc invocation or an "info command", where in the case of the 
mutex, it is slower.

           86 ns   time {dict get {a 1 b 2 c 3} b} 100000
          130 ns   set x 1; time {info exists x} 100000
          131 ns   set x 1; time {set x} 100000
          140 ns   time {set x 1} 100000
          198 ns   time {ns_quotehtml "hello world"} 100000
          214 ns   set x 1; time {expr {$x + $x}} 100000
          216 ns   nsv_set foo x 1; time {nsv_get foo x} 100000
          248 ns   proc foo {x} {return $x}; time {foo 1} 100000
          273 ns   time {info commands ::db_string} 100000
          313 ns   time {ns_cache_eval ns:memoize 1 {set x 1}} 100000
          319 ns   time {nsv_set foo x 1} 100000
          322 ns   time {array set x {a 1 b 2 c 3}} 100000
          348 ns   time {ns_md5 foo} 100000
          362 ns   time {ns_sha1 foo} 100000
          373 ns   time {lang::util::localize "hello world"} 100000
          485 ns   nx::Class create Foo {:public method bar {} {return 0};:create ::foo}; time {::foo bar} 100000
          776 ns   time {ad_conn subsite_id} 100000
         1820 ns   time {nx::Object create ::o} 100000
         4104 ns   time {nx::Object new} 100000
         6945 ns   time {parameter::get -package_id [ad_conn subsite_id] -parameter DefaultMaster -default "x"} 100000
        25611 ns   time {md5::md5 foo} 100000
        27423 ns   time {sha1::sha1 foo} 100000
        68492 ns   set id 252; time {xo::dc get_value -prepare int qn {select title from acs_objects where object_id=:id}} 100000
        90712 ns   time {xo::dc get_value dbqd..qn {select title from acs_objects where object_id=252}} 100000
       103241 ns   time {db_string dbqd..qn {select title from acs_objects where object_id=252}} 100000
       156529 ns   time {set F [open /tmp/nix w]; puts $F x; close $F} 10000
      4760448 ns   time {exec ls /} 1000

times with mutex locks

          293 ns   nsv_set foo x 1; time {nsv_get foo x} 100000
          354 ns   time {nsv_set foo x 1} 100000

This is a test with very little contention, where the previous tests 
i've posted were with very high contention. But certainly, applications 
might be different.

Since we want to have on the longer range binary distributions of 
NaviServer, i implemented the option (c) from above, such that the 
decision of using rwlocks or mutex operation can be done (1) at startup 
time and (2) per server. Also, the documentation of NaviServer is updated.

all the best

-g

[1] 
https://stackoverflow.com/questions/4087280/approximate-cost-to-access-various-caches-and-main-memory

Re: [naviserver-devel] Cookie Same-Site "None"

[Gustaf N. <ne...@wu...>]

Wolfgang,

you are right, explicit setting of same-site=none is necessary now.

In previous versions of browsers, no explicit setting
of the same-site flag was exactly the same as explicit setting
(an implicit default of same-site=none)

Since some browsers switched to a default of "lax", explicit
setting became necessary.

Fixed now on bitbucket.

-gn

PS: it is not developer-friendly that the behavior is changed
on the fly.... On the client site, the disruptive behavior
change was intended, so changing the default value on the
server is probably not good - and is left unchanged.

[naviserver-devel] Cookie Same-Site "None"

[Wolfgang W. <wol...@di...>]

Hello!

We are using "ns_setcookie" with the flag "-samesite". If used with 
"-samesite none", the SameSite value of "None" is not passed to the 
browser. An empty "SameSite" value is different from "SameSite=None", at 
least in Chromium based browsers.

I checked the source code:

https://bitbucket.org/naviserver/naviserver/src/master/nsd/cookies.c

Obviously "none" is treated like an empty value.

Yours,

Wolfgang Winkler

-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] some comments on windows build

[oleg <oo...@ua...>]

On Fri, 28 Aug 2020 12:36:20 +0200
Gustaf Neumann <ne...@wu...> wrote:

> One question though: is the trick with '--data @- << $string' really
> the recommended
> way to pass utf-8 data in windows?

There are a large number of posts on the stackowerflow/etc about using
utf-8 on the windows command line. I could not find a working option,
except that the original test works in Windows 7 and does not work in
Windows 10. The failed test looks like this:

==== http-5.3-b-default check encoding ns_conn content POST,
    content-type application/x-www-form-urlencoded,
    test via curl FAILED
==== Contents of test case:

    set string "Testing <äöüß☀>"
    exec curl -g -s --data $string [ns_config test listenurl]/post 2>
    /dev/null

---- Result was:
utf-8 <application/x-www-form-urlencoded> AÄATesting <aou??>ZÜZ
---- Result should have been (exact matching):
utf-8 <application/x-www-form-urlencoded> AÄATesting <äöüß☀>ZÜZ
==== http-5.3-b-default FAILED

The trick with '--data @- << $string' was tested successfully on all
Windows boxes I found around.

Oleg.