naviserver-devel — Developer discussion

[naviserver-devel] keepalive performance issue

[Jeff R. <dv...@di...>]

Hi all,

I am running a test using ab against a local naviserver instance with a 
vanilla config, and I am seeing requests with http keepalive enabled 
having a delay of ~38ms per request compared to non-keepalive.   This 
rather surprised me;  is there a config setting to avoid that delay, or 
is it otherwise to be expected?

My config:

ns_section      "ns/servers"
ns_param         default         Naviserver

ns_section      "ns/server/default"
ns_param minthreads 20

ns_section      "ns/server/default/modules"
ns_param         nscp            nscp.so
ns_param         nssock          nssock.so
ns_param         nslog           nslog.so

ns_section      "ns/server/default/module/nssock"
ns_param port 8080


Tests:

$ ab -n 30 -c 1  http://localhost:8080/hello.html
...

Time per request:       0.170 [ms] (mean)

$ ab -k -n 30 -c 1  http://localhost:8080/hello.html
...

Time per request:       38.666 [ms] (mean)

Re: [naviserver-devel] Installing NS 4.99.17 on Debian 7 Wheezy

[Iuri S. <iu...@iu...>]

Indeed. I’m doing it in the near future. I’m just waiting for new resources. 
In the meanwhile, I’m going to apply your changes.

Thanks Gustaf!


> On Nov 20, 2018, at 12:04, Gustaf Neumann <ne...@wu...> wrote:
> 
> It seems, that the function EC_curve_nist2nid() was introduced in OpenSSL 1.0.2, and you have probably on that machine a version of OpenSSL between 1.0 and that version. I have just now committed a change to bitbucket [1] that should address this compilation issue. 
> Applying the change to NaviServer sources will help, but also upgrading debian will work, and should be performed regardless of this issue: Debian 7 (wheezy) has ended its LTS life in May 2018 [2], going to Debian 8 or 9 is therefore the way to go.
> all the best
> -gn
> [1] https://bitbucket.org/naviserver/naviserver/commits/49c314f2b1e9410de0f61935c0bb07457846bd66 <https://bitbucket.org/naviserver/naviserver/commits/49c314f2b1e9410de0f61935c0bb07457846bd66>
> [2] https://wiki.debian.org/LTS/ <https://wiki.debian.org/LTS/>
> 
> 
> On 20.11.18 04:16, Iuri Sampaio wrote:
>> Hi there,
>> 
>> Has anyone got a similar error related to tclcrypto library?
>> 
>> Right now, I’m search for the package, which could contain the source for the reference EC_curve_nist2nid, but Debian 7 is an old distro…
>> I’m already using openssl latest version for Debian 7
>>  
>> Next attempt will be upgrading to Debian 9. Shall I?
>> 
>> 
>> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H     -c -o tclcrypto.o tclcrypto.c
>> tclcrypto.c: In function 'GetCurve':
>> tclcrypto.c:463:9: warning: implicit declaration of function 'EC_curve_nist2nid' [-Wimplicit-function-declaration]
>> /bin/rm -Rf libnsd.so
>> gcc -shared -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H    -L../nsthread -L../nsd -L../nsdb -o libnsd.so adpcmds.o adpeval.o adpparse.o adprequest.o auth.o binder.o cache.o callbacks.o cls.o compress.o config.o conn.o connio.o cookies.o connchan.o crypt.o dns.o driver.o dstring.o encoding.o event.o exec.o fastpath.o fd.o filter.o form.o httptime.o index.o info.o init.o limits.o lisp.o listen.o log.o mimetypes.o modload.o nsconf.o nsmain.o nsthread.o op.o pathname.o pidfile.o proc.o progress.o queue.o quotehtml.o random.o range.o request.o return.o returnresp.o rollfile.o sched.o server.o set.o sls.o sock.o sockcallback.o sockfile.o str.o task.o tclcache.o tclcallbacks.o tclcmds.o tclconf.o tclenv.o tclfile.o tclhttp.o tclimg.o tclinit.o tcljob.o tclmisc.o tclobj.o tclobjv.o tclrequest.o tclresp.o tclsched.o tclset.o tclsock.o sockaddr.o tclthread.o tcltime.o tclvar.o tclxkeylist.o tls.o stamp.o url.o url2file.o urlencode.o urlopen.o urlspace.o uuencode.o unix.o watchdog.o nswin32.o tclcrypto.o   -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
>> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H     -c -o main.o main.c
>> /bin/rm -Rf nsd
>> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
>> libnsd.so: undefined reference to `EC_curve_nist2nid'
>> collect2: error: ld returned 1 exit status
>> make[1]: *** [nsd] Error 1
>> make[1]: Leaving directory `/usr/local/src/naviserver-4.99.17/nsd'
>> make: *** [all] Error 1
>> root@iurix:/usr/local/src/install-ns#
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Installing NS 4.99.17 on Debian 7 Wheezy

[Gustaf N. <ne...@wu...>]

It seems, that the function EC_curve_nist2nid() was introduced in 
OpenSSL 1.0.2, and you have probably on that machine a version of 
OpenSSL between 1.0 and that version. I have just now committed a change 
to bitbucket [1] that should address this compilation issue.

Applying the change to NaviServer sources will help, but also upgrading 
debian will work, and should be performed regardless of this issue: 
Debian 7 (wheezy) has ended its LTS life in May 2018 [2], going to 
Debian 8 or 9 is therefore the way to go.

all the best
-gn
[1] 
https://bitbucket.org/naviserver/naviserver/commits/49c314f2b1e9410de0f61935c0bb07457846bd66
[2] https://wiki.debian.org/LTS/


On 20.11.18 04:16, Iuri Sampaio wrote:
> Hi there,
>
> Has anyone got a similar error related to tclcrypto library?
>
> Right now, I’m search for the package, which could contain the source 
> for the reference EC_curve_nist2nid, but Debian 7 is an old distro…
> I’m already using openssl latest version for Debian 7
> Next attempt will be upgrading to Debian 9. Shall I?
>
>
> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe -I../include 
> -I"/usr/local/ns/include"  -DHAVE_CONFIG_H -c -o tclcrypto.o tclcrypto.c
> tclcrypto.c: In function 'GetCurve':
> tclcrypto.c:463:9: warning: implicit declaration of function 
> 'EC_curve_nist2nid' [-Wimplicit-function-declaration]
> /bin/rm -Rf libnsd.so
> gcc -shared -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC -pipe  
> -I../include -I"/usr/local/ns/include" -DHAVE_CONFIG_H    
> -L../nsthread -L../nsd -L../nsdb -o libnsd.so adpcmds.o adpeval.o 
> adpparse.o adprequest.o auth.o binder.o cache.o callbacks.o cls.o 
> compress.o config.o conn.o connio.o cookies.o connchan.o crypt.o dns.o 
> driver.o dstring.o encoding.o event.o exec.o fastpath.o fd.o filter.o 
> form.o httptime.o index.o info.o init.o limits.o lisp.o listen.o log.o 
> mimetypes.o modload.o nsconf.o nsmain.o nsthread.o op.o pathname.o 
> pidfile.o proc.o progress.o queue.o quotehtml.o random.o range.o 
> request.o return.o returnresp.o rollfile.o sched.o server.o set.o 
> sls.o sock.o sockcallback.o sockfile.o str.o task.o tclcache.o 
> tclcallbacks.o tclcmds.o tclconf.o tclenv.o tclfile.o tclhttp.o 
> tclimg.o tclinit.o tcljob.o tclmisc.o tclobj.o tclobjv.o tclrequest.o 
> tclresp.o tclsched.o tclset.o tclsock.o sockaddr.o tclthread.o 
> tcltime.o tclvar.o tclxkeylist.o tls.o stamp.o url.o url2file.o 
> urlencode.o urlopen.o urlspace.o uuencode.o unix.o watchdog.o 
> nswin32.o tclcrypto.o   -lz -lcrypt -lnsthread -L/usr/local/ns/lib 
> -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic 
> -L/usr/local/ns/lib -lssl -lcrypto -Wl,-rpath,:/usr/local/ns/lib
> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe -I../include 
> -I"/usr/local/ns/include"  -DHAVE_CONFIG_H -c -o main.o main.c
> /bin/rm -Rf nsd
> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz 
> -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s -lieee -lm  
> -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   
> -Wl,-rpath,:/usr/local/ns/lib
> libnsd.so: undefined reference to `EC_curve_nist2nid'
> collect2: error: ld returned 1 exit status
> make[1]: *** [nsd] Error 1
> make[1]: Leaving directory `/usr/local/src/naviserver-4.99.17/nsd'
> make: *** [all] Error 1
> root@iurix:/usr/local/src/install-ns#

[naviserver-devel] Installing NS 4.99.17 on Debian 7 Wheezy

[Iuri S. <iu...@iu...>]

Hi there,

Has anyone got a similar error related to tclcrypto library?

Right now, I’m search for the package, which could contain the source for the reference EC_curve_nist2nid, but Debian 7 is an old distro…
I’m already using openssl latest version for Debian 7
 
Next attempt will be upgrading to Debian 9. Shall I?


gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H     -c -o tclcrypto.o tclcrypto.c
tclcrypto.c: In function 'GetCurve':
tclcrypto.c:463:9: warning: implicit declaration of function 'EC_curve_nist2nid' [-Wimplicit-function-declaration]
/bin/rm -Rf libnsd.so
gcc -shared -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H    -L../nsthread -L../nsd -L../nsdb -o libnsd.so adpcmds.o adpeval.o adpparse.o adprequest.o auth.o binder.o cache.o callbacks.o cls.o compress.o config.o conn.o connio.o cookies.o connchan.o crypt.o dns.o driver.o dstring.o encoding.o event.o exec.o fastpath.o fd.o filter.o form.o httptime.o index.o info.o init.o limits.o lisp.o listen.o log.o mimetypes.o modload.o nsconf.o nsmain.o nsthread.o op.o pathname.o pidfile.o proc.o progress.o queue.o quotehtml.o random.o range.o request.o return.o returnresp.o rollfile.o sched.o server.o set.o sls.o sock.o sockcallback.o sockfile.o str.o task.o tclcache.o tclcallbacks.o tclcmds.o tclconf.o tclenv.o tclfile.o tclhttp.o tclimg.o tclinit.o tcljob.o tclmisc.o tclobj.o tclobjv.o tclrequest.o tclresp.o tclsched.o tclset.o tclsock.o sockaddr.o tclthread.o tcltime.o tclvar.o tclxkeylist.o tls.o stamp.o url.o url2file.o urlencode.o urlopen.o urlspace.o uuencode.o unix.o watchdog.o nswin32.o tclcrypto.o   -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC  -pipe  -I../include -I"/usr/local/ns/include"  -DHAVE_CONFIG_H     -c -o main.o main.c
/bin/rm -Rf nsd
gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
libnsd.so: undefined reference to `EC_curve_nist2nid'
collect2: error: ld returned 1 exit status
make[1]: *** [nsd] Error 1
make[1]: Leaving directory `/usr/local/src/naviserver-4.99.17/nsd'
make: *** [all] Error 1
root@iurix:/usr/local/src/install-ns#

Re: [naviserver-devel] NaviServer 4.99.17 available

[David O. <da...@qc...>]

Thanks for all the work on this Gustaf.
Looks like a great release!

Regards,
Dave

On Sun, 4 Nov 2018 at 17:54, Gustaf Neumann <ne...@wu...> wrote:

>
> Dear all,
>
> on sourceforge is now the release of NaviServer 4.99.17 [1].
> Below is a summary of the changes. This release was tested
> under Windows 8.1 (32bit) and 10 (64bit) many thanks to
> Oleg Oleinik), various Linux versions (e.g. Ubuntu 18.04,
> FC 27, including Raspberry PI Raspbian GNU/Linux 9.4
> with ARM processor), and OmniOS and OpenBSD, with
> Tcl 8.5, 8.6 and 8.7 (alpha). Many thanks to all contributors!
>
> all the best
>
> -gustaf
> [1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.17/
>
>
> =======================================
> NaviServer 4.99.17, released 2018-11-04
> =======================================
>
>  421 files changed, 17805 insertions(+), 10275 deletions(-)
>
> New Features:
> -------------
>
>   - Added brotli support for delivering static content
>     ---------------------------------------------------
>     The brotli compression format (RFC 7932) achieves better results for
>     compression for typical web content compared to e.g. gzip and is
>     supported by most current browsers. The definition of static
>     content delivery is exactly the same for "brotli" as for static
>     "gzip" content.
>
>     New configuration options for ns/fastpath:
>     * "brotli_static": should static brotli delivery be checked?
>     * "brotli_refresh": should outdated brotli compressed files be refreshed?
>     * "brotli_cmd": OS-level command with options used for brotli compression
>
>   - "ns_conn" additions:
>     * New subcommand "ns_conn acceptedcompression" to return accepted
>       compressions for the current connection (gzip or brotli)
>
>     * New subcommands "ns_conn currentaddr" and "ns_conn currentport"
>       to refer to the address and port of the currently open server side
>       of the socket.
>
>
>   - New feature for GDPR: add mask IP addresses optionally in access log
>     --------------------------------------------------------------------
>     This feature is similar to "anonip" IP anonymizer of the Swiss
>     privacy foundation): When this feature is activated, all IP
>     addresses are masked in the log file, such that the host-specific
>     (= person- specific) information is masked out and the IP-address
>     does not match a particular individual anymore. Still, with the
>     masking turned on, reverse DNS lookup and/or geolocation is
>     possible.
>
>     The option can be configured via the following parameters in the
>     ".../module/nslog" section of the config file of NaviServer:
>      * "masklogaddr": boolean value to turn feature on/off (default off)
>      * "maskipv4": mask for IPv4 addresses (default 255.255.255.0)
>      * "maskipv6": mask for IPv6 addresses (default ff:ff:ff:ff::)
>
>
>   - New atomic "nsv_set" commands/options
>     -------------------------------------
>
>     * Obtain (old) value from nsv ARRAY and set it to a new value set foo
>
>         set foo [nsv_set -reset ARRAY KEY NEWVALUE]
>
>       After the operation, ARRAY(KEY) has a new value.
>       The operation is similar to "nsv_set ARRAY KEY NEWVALUE",
>       but the variant with "-reset" returns the old value.
>       Similar to GETSET in REDIS.
>
>    * Obtain a value from nsv ARRAY and unset it (no new value is provided).
>
>         set foo [nsv_set -reset ARRAY KEY]
>
>      The operations returns the old value of ARRAY(KEY) or empty, if
>      it does not exist.  After the operation, the ARRAY(KEY) is unset.
>
>    * Set a default value for a nsv ARRAY
>
>         nsv_set -default ARRAY KEY DEFAULTVALUE
>
>       Do nothing when ARRAY(KEY) has already a value.
>       Return an error, when DEFAULTVALUE is not provided.
>       Similar to SETNX in REDIS.
>
>
>   - Extended functionality of "ns_http"
>     -----------------------------------
>
>     * Changed return value for "ns_http run" and "ns_http wait".
>       Now the commands return attribute value pairs (Tcl dict).
>       Previously, the commands returned different results,
>       depending on the options
>
>       - "ns_http run" returned the name of the queued command (which
>         was completely useless, this the queued command was already
>         terminated)
>       - "ns_http wait" returned sometimes the body of the reply (when
>         the reply was not spooled) or empty (spooled) or "1", when the
>         option "-R" was specified
>
>       Now the result is always a dict containing "status" (HTTP status
>       code), "time" (elapsed time), "headers" (ns_set of the reply
>       header fields, and either the "body" (reply body) or "file" (the
>       name of the spooled file). The values in the dict might mirror
>       the result of other (optional) output variables, which continue
>       to work.
>
>       Since the results were irregular and not documented in the man
>       pages, the backward compatibility should be high. Also, the
>       regression test of NaviServer was not effected by this change.
>
>
>     * Support for asynchronous “ns_http” tasks:
>
>       When the new flag "-donecallback” is provided, then a “ns_http”
>       task is started in the background and does not require an
>       “ns_http wait” anymore. Instead, when it finishes it executes
>       the provided donecallback which receives as additional arguments
>       the Tcl result (in form of an integer) and the result dict,
>       returned otherwise from e.g. “ns_http run”.
>
>       As a consequence, requests of the form
>          ns_http queue … -donecallback …
>       differ from client requests without the callback
>       in the following points:
>
>       a) the command does not return a handle for “ns_http wait”
>       b) The command is not automatically cleaned up (canceled)
>          at the end of a connection requests
>       c) Typically, donecallbacks are executed in a different
>          thread than it was started.
>
>       Some more options, which were previously only valid on
>       “ns_http wait” (such as e.g. spoolsize) are now valid
>       for “ns_http queue” as well.
>
>       The new flag allows a developer to start many requests in
>       parallel without having to start many threads or complex
>       synchronization tasks.
>
>     * New option "-body_file" for "ns_http wait" (similar to "ns_http
>       queue" which can be used to specify a filename for the
>       downloaded content)
>
>     * New option "-outputfile" to "ns_http run" and "ns_http wait":
>       by specifying the option "-outputfile" the received file will be
>       always written to the disk, no matter how large it is.
>
>     * New subcommand "ns_http stats": returns a dict (flat list of
>       attributes and values). The list contains "task", "url",
>       "requestlength", "sent", "replylength" and "received".
>
>
>   - Support resolving a hostname against multiple IP addresses
>     ----------------------------------------------------------
>
>     * In case, one domain name has multiple IP addresses registered,
>       previous versions of NaviServer tried only the first returned
>       address and returned an error, when this failed. This happened
>       when using client functions (e.g. ns_http).
>
>     * host names with multiple IP addresses are nowadays quite common,
>       e.g. "localhost" having an IPv4 and an IPv6 address registered. In
>       previous versions, when e.g. localhost has as first address the IPv6
>       address, but IPv6 was not active, it returned an error.
>
>     * resolving against IP addresses continues to work as always.
>
>
>   - Extended crypto functionality
>     -----------------------------
>
>      * Added HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
>           ::ns_crypto::md hkdf -digest sha256 ....
>        (requires OpenSSL 1.1.0 or newer)
>
>      * Added support for elliptic curve cryptography
>         . ns_crypto::eckey import ?-string string? ?-encoding encoding?
>         . ns_crypto::eckey priv ?-pem pem? ?-encoding encoding?
>         . ns_crypto::eckey pub ?-pem pem? ?-encoding encoding?
>         . ns_crypto::eckey sharedsecret ?-pem pem? ?-encoding encoding? pubkey
>         . ns_crypto::eckey generate ?-name name? ?-pem pem?
>
>      * Added support for getting an arbitrary number of random bytes
>        based on crypto support on OpenSSL:
>         . ns_crypto::randombytes ?-encoding encoding? bytes
>
>      * Added support for Authenticated Encryption with Associated Data
>        (AEAD) scheme, which provides confidentiality, integrity, and authenticity.
>
>         . ns_crypto::aead::encrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? input"
>         . ns_crypto::aead::decrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? ?-tag tag? input"
>
>      * Added "-encoding hex|base64url|base64" option for output encoding
>         for the crypto functions. Previously, all these functions returned the
>         result in "hex" format. If the option is not specified, the result
>         is the same as before (hex). The encoding option was added to the
>         following functions:
>         - ns_crypto::hmac string
>         - ns_crypto::hmac get
>         - ns_crypto::md string
>         - ns_crypto::md get
>         - ns_crypto::md vapidsign
>         - ns_crypto::enc
>
>
>   - "ns_connchan" improvements
>     --------------------------
>
>     * Callback suspension: a Tcl socket callback function might now
>       return the value "2" to flag that the callback will be
>       suspended, but it keeps the socket open. Previously, the Tcl
>       callback could return just "0" (callback is cancelled, socket is
>       closed) or "1" (continue callback handling, keep socket open).
>       This can be used to suspend callback handling on bottlenecks and
>       continue later, when situation improves.
>
>     * "ns_connchan write": The function returns now the number of
>       bytes sent, which might be less than the length of the data to
>       be sent.
>
>     * "ns_connchan list": the function returns now two more fields per
>       entry: a) cmd name of the callback (eases debugging, when
>       callbacks are changed dynamically) b) conditions used for
>       registering the callback.
>
>     * The "ns_connchan" callbacks are now listed via the nsstats
>       interface as well (with proper callback info).
>
>
>   - Added Possibility to add per-server "initcmds" to config file
>     -------------------------------------------------------------
>
>     This new feature provides an easy means to add command which
>     should be executed after server initialization into a configuration
>     files. By using the "initcmds" one can start multiple instances
>     of NaviServer differing just in the initialization command
>     by providing different config files. The "initcmds" approach
>     has the advantage over e.g. "ns_atserverstart" that it is
>     executed in a state when the server is fully initialized.
>
>
>
>   - Improved readability for config files
>     -------------------------------------
>
>     The "ns_section" command has new an optional last argument for the
>     parameters of its configuration section. Instead of writing e.g.
>
>        ns_section ns/parameters
>        ns_param serverlog    $logroot/error.log
>        ns_param pidfile      $logroot/nsd.pid
>        ns_param home         $homedir
>        #...
>
>     one can write now
>
>        ns_section ns/parameters {
>            ns_param serverlog   $logroot/error.log
>            ns_param pidfile     $logroot/nsd.pid
>            ns_param home        $homedir
>            #...
>        }
>
>     This block notation has two advantages: a) it shows clearly, where
>     the section ends and b) when using an editor with automatic
>     indentation, the log file looks nicer without manual indentation
>     work. This change is fully backward compatible, old style config
>     files continue to work.
>
>
>   - Further new commands and features
>     ---------------------------------
>
>     * "ns_hash": provide a interface for the HashStringKey() function
>       (very useful for e.g. cache partitioning)
>
>     * "ns_reflow_text".
>           ns_reflow_text ?-width width? ?-offset offset? ?-prefix prefix? ?--? text
>       Tcl-based solutions are surprisingly slow on largish input.
>
>     * "ns_base64urlendcode", "ns_base64urldedcode": URL-save variants of base64,
>       used in various new RFCs for passing base64 values in HTTP requests.
>
>    * "ns_uuid": runtime efficient version of a version 4 UUID
>       according to RFC 4122: A Universally Unique IDentifier (UUID)
>       URN Namespace
>
>    * Improved results of [ns_cache_stats -contents ...]. The command
>      returns now a tcl list instead of a string, including reuse count.
>
>   - Improved results of "ns_striphtml":
>      * Added resolving of all HTML4 entities (including &nbsp; etc.)
>      * Stripped as well HTML comments.
>
>
> Performance Improvements:
> -------------------------
>
>   - Check for epoch updates on blue-prints as well in cleanup phase
>     (deallocate trace) to reduce latency for client requests.
>
>
> Bug Fixes:
> ----------
>
>   - Don't try to serve redirected files in case the connection is
>     already closed (will lead only to confusing messages)
>
>   - Ns_IsBinaryMimeType(): When a chartype is specified, never assume
>     the mimetype is binary.
>
>   - ns_getformfile: Don't rely on global variables
>     (see https://sourceforge.net/p/naviserver/bugs/80/)
>     Before this change, Depending on configured values,
>     ns_getformfile could have missed uploaded files.
>
>   - Added missing fetch commands for obtaining handles (on obviously
>     seldom used subcommands "ns_db setexception|sp_setparam")
>
>   - ns_http:
>     * Timeout and connection behavior show works now consistent
>       for HTTP and HTTPS URLs.
>     * Fixed a potential bug (infinite loop) when requests
>       run into timeouts.
>
>   - "ns_set print": align with documented behavior
>
>   - "ns_parseurl": make results more robust, when a URL with a colon
>     in the path was given, but no port provided (detected a port
>     incorrectly)
>
>   - "ns_parseformfile": handle also www-form-urlencoded (in
>      addition to multipart/form-data as before)
>
>   - Windows:
>     * Make sure to create temporary files correctly under windows (many
>       thanks to Oleg Oleinik)
>     * Incorrect handle was used to perform operations on the
>       process in Ns_WaitForProcess (many thanks to Oleg Oleinik)
>
>
> Documentation improvements:
> ---------------------------
>
>   - Updated several man pages
>     * admin-db.man:
>     * admin-install.man
>     * admin-maintenance.man
>     * adp-overview.man
>     * commandlist.man
>     * main-features.man
>     * main-history.man
>     * ns_accesslog.man
>     * ns_adp.man
>     * ns_adp_argc.man
>     * ns_adp_argv.man
>     * ns_adp_bind_args.man
>     * ns_adp_eval.man
>     * ns_adp_flush.man
>     * ns_adp_info.man
>     * ns_adp_safeeval.man
>     * ns_atexit.man
>     * ns_atexit.man
>     * ns_atshutdown.man
>     * ns_atsignal.man
>     * ns_base64.man
>     * ns_base64encode.man
>     * ns_conn.man
>     * ns_connchan.man
>     * ns_cookie.man
>     * ns_db.man
>     * ns_driver.man
>     * ns_fmttime.man
>     * ns_getcontent.man
>     * ns_getform.man
>     * ns_hmac.man
>     * ns_hotp.man
>     * ns_http.man
>     * ns_httpget.man
>     * ns_httpopen.man
>     * ns_ictl.man
>     * ns_job.man
>     * ns_kill.man
>     * ns_md.man
>     * ns_memoize.man
>     * ns_mutex.man
>     * ns_parsequery.man
>     * ns_perm.man
>     * ns_proxy.man
>     * ns_queryexists.man
>     * ns_queryget.man
>     * ns_querygetall.man
>     * ns_quotehtml.man
>     * ns_rand.man
>     * ns_register.man
>     * ns_return.man
>     * ns_rlimit.man
>     * ns_schedule_proc.man
>     * ns_sendmail.man
>     * ns_server.man
>     * ns_set.man
>     * ns_set.man:
>     * ns_setexpires.man
>     * ns_sockopen.man
>     * ns_sockselect.man
>     * ns_striphtml.man
>     * ns_thread.man
>     * ns_time.man
>     * ns_tmpnam.man
>     * ns_totp.man
>     * ns_urlspace.man
>     * ns_uuencode.man
>     * ns_writer.man
>     * nscgi.man
>     * nsperm.man
>     * nssock.man
>     * nsv.man
>     * returnstatus-cmds.man
>     * tcl-lib-file.man
>     * tcl-lib-nstrace.man
>     * tcl-libraries.man
>     * tcl-overview.man
>
>
> Tcl API Changes:
> ----------------
>
>   - "ns_conn isconnected": returns now false, when the connection is
>      already closed. Otherwise we have to add a new subcommand
>      "ns_conn closed", which seems somewhat odd (isconnected true and
>      closed true?)
>
>   - "ns_cache_stats" returns now hitrate in form of a float instead of
>     an integer.
>
>
> C API Changes:
> --------------
>
>    - Ns_SockTimedConnect2() returns now Ns_ReturnCode to be able to
>      distinguish between timeouts and errors
>
>    - Ns_SockConnectError(): new call to provide consistent error messages
>      and to set Tcl error info consistently
>
>
> Configuration Changes:
> ----------------------
>
>   - Use same initialization for "enablelclpages" when configured over
>     "ns/server/$server" or over "ns/server/$server/adp" to avoid subtle
>     differences on global variables.
>
>   - Extended sample config files:
>
>     * nsd-config.tcl
>        . Added parameter "masklogaddr" for "nslog" section
>        . Added parameter for "gzip" and "brotli" static delivery
>        . Added parameter for "initcmds"
>        . Added setting for recommended environment variables
>          HOME, LANG and RANDFILE. RANDFILE is sometimes needed
>           to avoid surprises with permissions, when OpenSSL
>           crypto functions are called from the server
>
>     * openacs-config.tcl
>        . Switched to PostgreSQL's conninfo notation in the datasource
>          to hint that more such parameters are possible as well
>          (e.g. connections via SSL)
>        . Added comments and examples for configuring OpenACS caches
>        . Added comments how to configure EmailDeliveryMode via config file
>        . Added comments how to configure IncludeCallingInfo of OpenACS API browser
>        . Added comments how to configure WithDeprecatedCode
>        . Added example for activating more intense SQL logging
>        . Added subsite-based error pages
>        . Added section indicating how to configure for nsstats module
>        . Added parameter settings for "reuseport"
>        . Added parameter "masklogaddr" for "nslog" section
>        . Added parameter for "brotli" static delivery
>        . Preconfigure customized error pages based on subsites
>
>   - Config files changed to block notation (see above)
>
>   - Improved handling of configuration errors:
>     * Provided better log message, when configured value is out of range
>     * Updated config values when range check corrects the value to get the
>       finally configured value via introspection.
>
>
> Code Changes:
> -------------
>
>   - Extended regression test:
>         * tests/adp.test
>         * tests/compat.test
>         * tests/http.test
>         * tests/ns_base64.test
>         * tests/ns_base64encode.test
>         * tests/ns_cache.test
>         * tests/ns_conn.test
>         * tests/ns_crypto.test
>         * tests/ns_nsv.test
>         * tests/ns_parseurl.test
>         * tests/ns_reflow_text.test
>         * tests/ns_server.test
>         * tests/ns_striphtml.test
>
>
>  - Added version tag of NaviServer to "starting" message in log file
>    to ease analysis in case of problems.
>
>  - Implemented deprecated commands as Tcl proc and complain on
>    its usage
>       . ns_adp_eval
>       . ns_adp_safeeval
>
>   - Improved Platform support:
>     * Extended list of platforms for manual provisioning of __PRIPTR_PREFIX
>       in case it is not defined.
>     * Do not rely on implementation defined print/scan format "%p",
>       but use instead standardized integer types.
>     * Windows:
>       . Improved type-cleanness (many thanks to Oleg Oleinik)
>       . Support to compile with MSVC 2017 (many thanks to Oleg Oleinik)
>       . When compiling with VS 2012, use built-in support for
>         intptr_t and uintptr_t.
>       . Added support to run regression test under Windows
>         (many thanks to Oleg Oleinik)
>
>   - Changed preset encodings (when not specified in config file) for
>     files with extensions .htm, .html, and .adp from iso8859-1 to
>     utf-8.
>
>   - Improved debugging support:
>
>     * Added compiler-flag NS_MUTEX_NAME_DEBUG for ease locating
>       anonymous mutexes.
>     * Provided names for all remaining mutexes to make it easier
>       to pinpoint bottlenecks due to mutexes
>     * Improve debugging output in various messages, when debugging
>       options are activated.
>
>
>   - Minor update of mime types based on actual IANA media types.
>
>   - Provided compatibility with Tcl 8.7.2 (removed CONST and VOID macros)
>   - Further improved robustness by avoiding potential NULL dereferences
>     found be static code analyzers (mostly error cases)
>   - Improved configure scripts
>   - Make sure, we have enough space for the terminating null character
>   - Added "pure" specifier for improved code generation
>
>   - Various code smell removals
>     * Prefer boolean type over int
>     * Improved type cleanness
>     * Reduced implicit conversions
>     * Reduced number of return statements before end of function
>     * Don't pass implementation-defined NULL after the last typed argument
>       to a variadic function
>     * Removed dead assignments
>     * Added "const" declarations
>     * Fixed typos, improved comments
>
>
> Modules:
> --------
>
>  - nsdbbdb (many thanks to Brendan Graves):
>    * Changed to work with FreeTDS 1.00.86
>    * Updated outdated paths.
>
>  - nsdbpg:
>    * Added sample configuration to include hints how to use SSL
>      connections to the database
>    * Provided compatibility with Tcl 8.7.2
>    * Minor code cleanups
>
>  - nssmtpd, nsdns, nstftpd, nsradiusd, nsdbi, nsdbipg:
>    * Brought code in sync with the current code base (including IPv6, 8.7.2 compat)
>    * Improved spelling
>
>  - nsphp (many thanks to Anthony Bennett):
>    * Refactored for newer versions of NaviServer
>    * Improved README and spelling
>
>  - nsstats:
>    * Added results to "ns_connchan list" to per-server info
>    * Added optional query parameter "raw=0|1" to switch between
>      raw and prettified numbers
>    * Improved number prettification
>    * Added optional query parameter "max=..." for controlling the number of
>      per-cache entries shown (default 50)
>    * Improved cache statistics: added hits/req, saved/req, saved/hit; dropped saved/KB
>    * Improved mutex statistics: added locks/req, potential locks/req, potential locks per second
>    * Added summative statistics to mutex locks page
>    * Made background locks (locks, not included in per-request statistics)
>      configurable via NaviServer config file
>    * Various other small changes and beautifications (use more CSS)
>    * Bumped version to 1.8
>
>  - nsoracle:
>    * Fixed potential crash when oci_error_p is called with dbh == NULL
>      (many thanks to Oleg Oleinik)
>    * Some more code cleanup and improved spellings.
>
>  - websocket:
>    * Fixed potential bug: make sure that upgrade header is sent as a single block.
>    * More minor code cleanup.
>
>  - revproxy:
>    * Fixed potential blocking conditions when send buffer of OS overflows.
>      With "-sendtimeout 0" a call to "ns_connchan write" might write less
>      bytes than desired; In this case, callbacks are suspended and continue
>      when the situation improves.
>    * Added new switches "-sendtimeout" and "-receivetimeout" to revproxy::upstream
>    * Added "-backend_reply_callback" to ::revproxy::upstream to allow proxy the
>      modification of backend header fields per request.
>    * Bumped version number to 0.11
>
>  - letsencrypt:
>    * Modernized code: use built-in ns_base64urlencode (requires 4.99.17)
>    * Changed order of certificates in pem file (private key first)
>    * Don't hard-code OpenSSL dir on OS (many thanks to Stefan Sobernig)
>
>  - nswebpush:
>    * New module for implementing Web Push via NaviServer
>      (Many thanks to Georg Prohaska)
>    * For details, see: https://bitbucket.org/naviserver/nswebpush/src/default/
>
>  _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

[naviserver-devel] NaviServer 4.99.17 available

[Gustaf N. <ne...@wu...>]

Dear all,

on sourceforge is now the release of NaviServer 4.99.17 [1].
Below is a summary of the changes. This release was tested
under Windows 8.1 (32bit) and 10 (64bit) many thanks to
Oleg Oleinik), various Linux versions (e.g. Ubuntu 18.04,
FC 27, including Raspberry PI Raspbian GNU/Linux 9.4
with ARM processor), and OmniOS and OpenBSD, with
Tcl 8.5, 8.6 and 8.7 (alpha). Many thanks to all contributors!

all the best

-gustaf
[1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.17/


=======================================
NaviServer 4.99.17, released 2018-11-04
=======================================

  421 files changed, 17805 insertions(+), 10275 deletions(-)

New Features:
-------------

   - Added brotli support for delivering static content
     ---------------------------------------------------
     The brotli compression format (RFC 7932) achieves better results for
     compression for typical web content compared to e.g. gzip and is
     supported by most current browsers. The definition of static
     content delivery is exactly the same for "brotli" as for static
     "gzip" content.

     New configuration options for ns/fastpath:
     * "brotli_static": should static brotli delivery be checked?
     * "brotli_refresh": should outdated brotli compressed files be refreshed?
     * "brotli_cmd": OS-level command with options used for brotli compression

   - "ns_conn" additions:
     * New subcommand "ns_conn acceptedcompression" to return accepted
       compressions for the current connection (gzip or brotli)

     * New subcommands "ns_conn currentaddr" and "ns_conn currentport"
       to refer to the address and port of the currently open server side
       of the socket.


   - New feature for GDPR: add mask IP addresses optionally in access log
     --------------------------------------------------------------------
     This feature is similar to "anonip" IP anonymizer of the Swiss
     privacy foundation): When this feature is activated, all IP
     addresses are masked in the log file, such that the host-specific
     (= person- specific) information is masked out and the IP-address
     does not match a particular individual anymore. Still, with the
     masking turned on, reverse DNS lookup and/or geolocation is
     possible.

     The option can be configured via the following parameters in the
     ".../module/nslog" section of the config file of NaviServer:
      * "masklogaddr": boolean value to turn feature on/off (default off)
      * "maskipv4": mask for IPv4 addresses (default 255.255.255.0)
      * "maskipv6": mask for IPv6 addresses (default ff:ff:ff:ff::)


   - New atomic "nsv_set" commands/options
     -------------------------------------

     * Obtain (old) value from nsv ARRAY and set it to a new value set foo

         set foo [nsv_set -reset ARRAY KEY NEWVALUE]

       After the operation, ARRAY(KEY) has a new value.
       The operation is similar to "nsv_set ARRAY KEY NEWVALUE",
       but the variant with "-reset" returns the old value.
       Similar to GETSET in REDIS.

    * Obtain a value from nsv ARRAY and unset it (no new value is provided).

         set foo [nsv_set -reset ARRAY KEY]

      The operations returns the old value of ARRAY(KEY) or empty, if
      it does not exist.  After the operation, the ARRAY(KEY) is unset.

    * Set a default value for a nsv ARRAY

         nsv_set -default ARRAY KEY DEFAULTVALUE

       Do nothing when ARRAY(KEY) has already a value.
       Return an error, when DEFAULTVALUE is not provided.
       Similar to SETNX in REDIS.


   - Extended functionality of "ns_http"
     -----------------------------------

     * Changed return value for "ns_http run" and "ns_http wait".
       Now the commands return attribute value pairs (Tcl dict).
       Previously, the commands returned different results,
       depending on the options

       - "ns_http run" returned the name of the queued command (which
         was completely useless, this the queued command was already
         terminated)
       - "ns_http wait" returned sometimes the body of the reply (when
         the reply was not spooled) or empty (spooled) or "1", when the
         option "-R" was specified

       Now the result is always a dict containing "status" (HTTP status
       code), "time" (elapsed time), "headers" (ns_set of the reply
       header fields, and either the "body" (reply body) or "file" (the
       name of the spooled file). The values in the dict might mirror
       the result of other (optional) output variables, which continue
       to work.

       Since the results were irregular and not documented in the man
       pages, the backward compatibility should be high. Also, the
       regression test of NaviServer was not effected by this change.


     * Support for asynchronous “ns_http” tasks:

       When the new flag "-donecallback” is provided, then a “ns_http”
       task is started in the background and does not require an
       “ns_http wait” anymore. Instead, when it finishes it executes
       the provided donecallback which receives as additional arguments
       the Tcl result (in form of an integer) and the result dict,
       returned otherwise from e.g. “ns_http run”.

       As a consequence, requests of the form
          ns_http queue … -donecallback …
       differ from client requests without the callback
       in the following points:

       a) the command does not return a handle for “ns_http wait”
       b) The command is not automatically cleaned up (canceled)
          at the end of a connection requests
       c) Typically, donecallbacks are executed in a different
          thread than it was started.

       Some more options, which were previously only valid on
       “ns_http wait” (such as e.g. spoolsize) are now valid
       for “ns_http queue” as well.

       The new flag allows a developer to start many requests in
       parallel without having to start many threads or complex
       synchronization tasks.

     * New option "-body_file" for "ns_http wait" (similar to "ns_http
       queue" which can be used to specify a filename for the
       downloaded content)

     * New option "-outputfile" to "ns_http run" and "ns_http wait":
       by specifying the option "-outputfile" the received file will be
       always written to the disk, no matter how large it is.

     * New subcommand "ns_http stats": returns a dict (flat list of
       attributes and values). The list contains "task", "url",
       "requestlength", "sent", "replylength" and "received".


   - Support resolving a hostname against multiple IP addresses
     ----------------------------------------------------------

     * In case, one domain name has multiple IP addresses registered,
       previous versions of NaviServer tried only the first returned
       address and returned an error, when this failed. This happened
       when using client functions (e.g. ns_http).

     * host names with multiple IP addresses are nowadays quite common,
       e.g. "localhost" having an IPv4 and an IPv6 address registered. In
       previous versions, when e.g. localhost has as first address the IPv6
       address, but IPv6 was not active, it returned an error.

     * resolving against IP addresses continues to work as always.


   - Extended crypto functionality
     -----------------------------

      * Added HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
           ::ns_crypto::md hkdf -digest sha256 ....
        (requires OpenSSL 1.1.0 or newer)

      * Added support for elliptic curve cryptography
         . ns_crypto::eckey import ?-string string? ?-encoding encoding?
         . ns_crypto::eckey priv ?-pem pem? ?-encoding encoding?
         . ns_crypto::eckey pub ?-pem pem? ?-encoding encoding?
         . ns_crypto::eckey sharedsecret ?-pem pem? ?-encoding encoding? pubkey
         . ns_crypto::eckey generate ?-name name? ?-pem pem?

      * Added support for getting an arbitrary number of random bytes
        based on crypto support on OpenSSL:
         . ns_crypto::randombytes ?-encoding encoding? bytes

      * Added support for Authenticated Encryption with Associated Data
        (AEAD) scheme, which provides confidentiality, integrity, and authenticity.

         . ns_crypto::aead::encrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? input"
         . ns_crypto::aead::decrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? ?-tag tag? input"

      * Added "-encoding hex|base64url|base64" option for output encoding
         for the crypto functions. Previously, all these functions returned the
         result in "hex" format. If the option is not specified, the result
         is the same as before (hex). The encoding option was added to the
         following functions:
         - ns_crypto::hmac string
         - ns_crypto::hmac get
         - ns_crypto::md string
         - ns_crypto::md get
         - ns_crypto::md vapidsign
         - ns_crypto::enc


   - "ns_connchan" improvements
     --------------------------

     * Callback suspension: a Tcl socket callback function might now
       return the value "2" to flag that the callback will be
       suspended, but it keeps the socket open. Previously, the Tcl
       callback could return just "0" (callback is cancelled, socket is
       closed) or "1" (continue callback handling, keep socket open).
       This can be used to suspend callback handling on bottlenecks and
       continue later, when situation improves.

     * "ns_connchan write": The function returns now the number of
       bytes sent, which might be less than the length of the data to
       be sent.

     * "ns_connchan list": the function returns now two more fields per
       entry: a) cmd name of the callback (eases debugging, when
       callbacks are changed dynamically) b) conditions used for
       registering the callback.

     * The "ns_connchan" callbacks are now listed via the nsstats
       interface as well (with proper callback info).


   - Added Possibility to add per-server "initcmds" to config file
     -------------------------------------------------------------

     This new feature provides an easy means to add command which
     should be executed after server initialization into a configuration
     files. By using the "initcmds" one can start multiple instances
     of NaviServer differing just in the initialization command
     by providing different config files. The "initcmds" approach
     has the advantage over e.g. "ns_atserverstart" that it is
     executed in a state when the server is fully initialized.



   - Improved readability for config files
     -------------------------------------

     The "ns_section" command has new an optional last argument for the
     parameters of its configuration section. Instead of writing e.g.

        ns_section ns/parameters
        ns_param serverlog    $logroot/error.log
        ns_param pidfile      $logroot/nsd.pid
        ns_param home         $homedir
        #...

     one can write now

        ns_section ns/parameters {
            ns_param serverlog   $logroot/error.log
            ns_param pidfile     $logroot/nsd.pid
            ns_param home        $homedir
            #...
        }

     This block notation has two advantages: a) it shows clearly, where
     the section ends and b) when using an editor with automatic
     indentation, the log file looks nicer without manual indentation
     work. This change is fully backward compatible, old style config
     files continue to work.


   - Further new commands and features
     ---------------------------------

     * "ns_hash": provide a interface for the HashStringKey() function
       (very useful for e.g. cache partitioning)

     * "ns_reflow_text".
           ns_reflow_text ?-width width? ?-offset offset? ?-prefix prefix? ?--? text
       Tcl-based solutions are surprisingly slow on largish input.

     * "ns_base64urlendcode", "ns_base64urldedcode": URL-save variants of base64,
       used in various new RFCs for passing base64 values in HTTP requests.

    * "ns_uuid": runtime efficient version of a version 4 UUID
       according to RFC 4122: A Universally Unique IDentifier (UUID)
       URN Namespace

    * Improved results of [ns_cache_stats -contents ...]. The command
      returns now a tcl list instead of a string, including reuse count.

   - Improved results of "ns_striphtml":
      * Added resolving of all HTML4 entities (including   etc.)
      * Stripped as well HTML comments.


Performance Improvements:
-------------------------

   - Check for epoch updates on blue-prints as well in cleanup phase
     (deallocate trace) to reduce latency for client requests.


Bug Fixes:
----------

   - Don't try to serve redirected files in case the connection is
     already closed (will lead only to confusing messages)

   - Ns_IsBinaryMimeType(): When a chartype is specified, never assume
     the mimetype is binary.

   - ns_getformfile: Don't rely on global variables
     (see https://sourceforge.net/p/naviserver/bugs/80/)
     Before this change, Depending on configured values,
     ns_getformfile could have missed uploaded files.

   - Added missing fetch commands for obtaining handles (on obviously
     seldom used subcommands "ns_db setexception|sp_setparam")

   - ns_http:
     * Timeout and connection behavior show works now consistent
       for HTTP and HTTPS URLs.
     * Fixed a potential bug (infinite loop) when requests
       run into timeouts.

   - "ns_set print": align with documented behavior

   - "ns_parseurl": make results more robust, when a URL with a colon
     in the path was given, but no port provided (detected a port
     incorrectly)

   - "ns_parseformfile": handle also www-form-urlencoded (in
      addition to multipart/form-data as before)

   - Windows:
     * Make sure to create temporary files correctly under windows (many
       thanks to Oleg Oleinik)
     * Incorrect handle was used to perform operations on the
       process in Ns_WaitForProcess (many thanks to Oleg Oleinik)


Documentation improvements:
---------------------------

   - Updated several man pages
     * admin-db.man:
     * admin-install.man
     * admin-maintenance.man
     * adp-overview.man
     * commandlist.man
     * main-features.man
     * main-history.man
     * ns_accesslog.man
     * ns_adp.man
     * ns_adp_argc.man
     * ns_adp_argv.man
     * ns_adp_bind_args.man
     * ns_adp_eval.man
     * ns_adp_flush.man
     * ns_adp_info.man
     * ns_adp_safeeval.man
     * ns_atexit.man
     * ns_atexit.man
     * ns_atshutdown.man
     * ns_atsignal.man
     * ns_base64.man
     * ns_base64encode.man
     * ns_conn.man
     * ns_connchan.man
     * ns_cookie.man
     * ns_db.man
     * ns_driver.man
     * ns_fmttime.man
     * ns_getcontent.man
     * ns_getform.man
     * ns_hmac.man
     * ns_hotp.man
     * ns_http.man
     * ns_httpget.man
     * ns_httpopen.man
     * ns_ictl.man
     * ns_job.man
     * ns_kill.man
     * ns_md.man
     * ns_memoize.man
     * ns_mutex.man
     * ns_parsequery.man
     * ns_perm.man
     * ns_proxy.man
     * ns_queryexists.man
     * ns_queryget.man
     * ns_querygetall.man
     * ns_quotehtml.man
     * ns_rand.man
     * ns_register.man
     * ns_return.man
     * ns_rlimit.man
     * ns_schedule_proc.man
     * ns_sendmail.man
     * ns_server.man
     * ns_set.man
     * ns_set.man:
     * ns_setexpires.man
     * ns_sockopen.man
     * ns_sockselect.man
     * ns_striphtml.man
     * ns_thread.man
     * ns_time.man
     * ns_tmpnam.man
     * ns_totp.man
     * ns_urlspace.man
     * ns_uuencode.man
     * ns_writer.man
     * nscgi.man
     * nsperm.man
     * nssock.man
     * nsv.man
     * returnstatus-cmds.man
     * tcl-lib-file.man
     * tcl-lib-nstrace.man
     * tcl-libraries.man
     * tcl-overview.man


Tcl API Changes:
----------------

   - "ns_conn isconnected": returns now false, when the connection is
      already closed. Otherwise we have to add a new subcommand
      "ns_conn closed", which seems somewhat odd (isconnected true and
      closed true?)

   - "ns_cache_stats" returns now hitrate in form of a float instead of
     an integer.


C API Changes:
--------------

    - Ns_SockTimedConnect2() returns now Ns_ReturnCode to be able to
      distinguish between timeouts and errors

    - Ns_SockConnectError(): new call to provide consistent error messages
      and to set Tcl error info consistently


Configuration Changes:
----------------------

   - Use same initialization for "enablelclpages" when configured over
     "ns/server/$server" or over "ns/server/$server/adp" to avoid subtle
     differences on global variables.

   - Extended sample config files:

     * nsd-config.tcl
        . Added parameter "masklogaddr" for "nslog" section
        . Added parameter for "gzip" and "brotli" static delivery
        . Added parameter for "initcmds"
        . Added setting for recommended environment variables
          HOME, LANG and RANDFILE. RANDFILE is sometimes needed
           to avoid surprises with permissions, when OpenSSL
           crypto functions are called from the server

     * openacs-config.tcl
        . Switched to PostgreSQL's conninfo notation in the datasource
          to hint that more such parameters are possible as well
          (e.g. connections via SSL)
        . Added comments and examples for configuring OpenACS caches
        . Added comments how to configure EmailDeliveryMode via config file
        . Added comments how to configure IncludeCallingInfo of OpenACS API browser
        . Added comments how to configure WithDeprecatedCode
        . Added example for activating more intense SQL logging
        . Added subsite-based error pages
        . Added section indicating how to configure for nsstats module
        . Added parameter settings for "reuseport"
        . Added parameter "masklogaddr" for "nslog" section
        . Added parameter for "brotli" static delivery
        . Preconfigure customized error pages based on subsites

   - Config files changed to block notation (see above)

   - Improved handling of configuration errors:
     * Provided better log message, when configured value is out of range
     * Updated config values when range check corrects the value to get the
       finally configured value via introspection.


Code Changes:
-------------

   - Extended regression test:
         * tests/adp.test
         * tests/compat.test
         * tests/http.test
         * tests/ns_base64.test
         * tests/ns_base64encode.test
         * tests/ns_cache.test
         * tests/ns_conn.test
         * tests/ns_crypto.test
         * tests/ns_nsv.test
         * tests/ns_parseurl.test
         * tests/ns_reflow_text.test
         * tests/ns_server.test
         * tests/ns_striphtml.test


  - Added version tag of NaviServer to "starting" message in log file
    to ease analysis in case of problems.

  - Implemented deprecated commands as Tcl proc and complain on
    its usage
       . ns_adp_eval
       . ns_adp_safeeval

   - Improved Platform support:
     * Extended list of platforms for manual provisioning of __PRIPTR_PREFIX
       in case it is not defined.
     * Do not rely on implementation defined print/scan format "%p",
       but use instead standardized integer types.
     * Windows:
       . Improved type-cleanness (many thanks to Oleg Oleinik)
       . Support to compile with MSVC 2017 (many thanks to Oleg Oleinik)
       . When compiling with VS 2012, use built-in support for
         intptr_t and uintptr_t.
       . Added support to run regression test under Windows
         (many thanks to Oleg Oleinik)

   - Changed preset encodings (when not specified in config file) for
     files with extensions .htm, .html, and .adp from iso8859-1 to
     utf-8.

   - Improved debugging support:

     * Added compiler-flag NS_MUTEX_NAME_DEBUG for ease locating
       anonymous mutexes.
     * Provided names for all remaining mutexes to make it easier
       to pinpoint bottlenecks due to mutexes
     * Improve debugging output in various messages, when debugging
       options are activated.


   - Minor update of mime types based on actual IANA media types.

   - Provided compatibility with Tcl 8.7.2 (removed CONST and VOID macros)
   - Further improved robustness by avoiding potential NULL dereferences
     found be static code analyzers (mostly error cases)
   - Improved configure scripts
   - Make sure, we have enough space for the terminating null character
   - Added "pure" specifier for improved code generation

   - Various code smell removals
     * Prefer boolean type over int
     * Improved type cleanness
     * Reduced implicit conversions
     * Reduced number of return statements before end of function
     * Don't pass implementation-defined NULL after the last typed argument
       to a variadic function
     * Removed dead assignments
     * Added "const" declarations
     * Fixed typos, improved comments


Modules:
--------

  - nsdbbdb (many thanks to Brendan Graves):
    * Changed to work with FreeTDS 1.00.86
    * Updated outdated paths.
    
  - nsdbpg:
    * Added sample configuration to include hints how to use SSL
      connections to the database
    * Provided compatibility with Tcl 8.7.2
    * Minor code cleanups

  - nssmtpd, nsdns, nstftpd, nsradiusd, nsdbi, nsdbipg:
    * Brought code in sync with the current code base (including IPv6, 8.7.2 compat)
    * Improved spelling

  - nsphp (many thanks to Anthony Bennett):
    * Refactored for newer versions of NaviServer
    * Improved README and spelling

  - nsstats:
    * Added results to "ns_connchan list" to per-server info
    * Added optional query parameter "raw=0|1" to switch between
      raw and prettified numbers
    * Improved number prettification
    * Added optional query parameter "max=..." for controlling the number of
      per-cache entries shown (default 50)
    * Improved cache statistics: added hits/req, saved/req, saved/hit; dropped saved/KB
    * Improved mutex statistics: added locks/req, potential locks/req, potential locks per second
    * Added summative statistics to mutex locks page
    * Made background locks (locks, not included in per-request statistics)
      configurable via NaviServer config file
    * Various other small changes and beautifications (use more CSS)
    * Bumped version to 1.8

  - nsoracle:
    * Fixed potential crash when oci_error_p is called with dbh == NULL
      (many thanks to Oleg Oleinik)
    * Some more code cleanup and improved spellings.
    
  - websocket:
    * Fixed potential bug: make sure that upgrade header is sent as a single block.
    * More minor code cleanup.

  - revproxy:
    * Fixed potential blocking conditions when send buffer of OS overflows.
      With "-sendtimeout 0" a call to "ns_connchan write" might write less
      bytes than desired; In this case, callbacks are suspended and continue
      when the situation improves.
    * Added new switches "-sendtimeout" and "-receivetimeout" to revproxy::upstream
    * Added "-backend_reply_callback" to ::revproxy::upstream to allow proxy the
      modification of backend header fields per request.
    * Bumped version number to 0.11

  - letsencrypt:
    * Modernized code: use built-in ns_base64urlencode (requires 4.99.17)
    * Changed order of certificates in pem file (private key first)
    * Don't hard-code OpenSSL dir on OS (many thanks to Stefan Sobernig)

  - nswebpush:
    * New module for implementing Web Push via NaviServer
      (Many thanks to Georg Prohaska)
    * For details, see: https://bitbucket.org/naviserver/nswebpush/src/default/

Re: [naviserver-devel] Towards NaviServer 4.99.17

[Gustaf N. <ne...@wu...>]

I've uploaded version 4.99.17b2 to bitbucket.

did some more tests on uncommon platforms (such as OmniOS, OpenBSD,
which might be tricky with header files and OpenSSL versions etc...),
did some more cleanups (whitespace changes).

The release should be done on the weekend...

all the best

-gn

Re: [naviserver-devel] Towards NaviServer 4.99.17

[Gustaf N. <ne...@wu...>]

On 30.10.18 19:47, Oleg wrote:
> I tested the 32-bit Naviserver for Windows 8.1/32bit (using MSVC 2015) and Windows 10/64bit (using MSVC 2017).
> Compiling required some tricks but the testing was almost successful.
> See testing output in the attachment (buildall.ooa.test.log).
>
> Some notes.
> 1) Makefile.win32.patch - to compile nsdbtest.
> 2) nsthread.h.patch - stdint.h is needed here because of the int32_t used just below.
> 3) test.nscfg.patch - for loading .dll modules
>
> And some hacks for problems with sockets - sock.c.patch (windows returns WSAWOULDBLOCK after connect on async socket) and win32readable.patch (to avoid "premature end of data")
>
> Thank you for your work.

Great and many thanks for the tests and (very good) patches!

I have applied most of these without changes, with one exception:
In the patch "test.nscfg.patch" you changed

    .../nssock.so
(and similar) to
    .../nssock[info sharedlib]

which looks very plausible, but on some platforms (e.g. macOS) the extension
of the modules is ".so", although its [info sharedlibextension] is 
".dylib". I guess
this was to make config files more portable, and due to the fact that the
modules are not regular shared libraries in nature.

Since i am not sure, whether windows forces you to use .dll, i make a small
proc to keep the config-file fo the test server compatible with windows.
I would hate to change the extensions for modules under macOS (such
that [info sharedlibextension] wornks always, since this would break
existing installations/configurations.

A way out might be to handle this on the .c code where the libraries
are loaded (e.g. to allow module names without the extension), but
this is nothing for the current release.

Due to the fact that probably nobody was running the regression test
under windows since many year, it is great to see, that 1243 out of 1271
tests succeeded (97.8%). i have made some more changes for low
hanging gruits, such that the number should go up further.

all the best and many thanks!

-g

> Thank you for your work.
>
> Regards,
>
> Oleg.

Re: [naviserver-devel] Towards NaviServer 4.99.17

[Oleg <oo...@ua...>]

28.10.2018 16:53, Gustaf Neumann <ne...@wu...>
> 
> on sourceforge is a release candidate for NaviServer 4.99.17 [1].
> Please test if possible.
> 

Hi.

I tested the 32-bit Naviserver for Windows 8.1/32bit (using MSVC 2015) and Windows 10/64bit (using MSVC 2017).
Compiling required some tricks but the testing was almost successful.
See testing output in the attachment (buildall.ooa.test.log).

Some notes.
1) Makefile.win32.patch - to compile nsdbtest.
2) nsthread.h.patch - stdint.h is needed here because of the int32_t used just below.
3) test.nscfg.patch - for loading .dll modules

And some hacks for problems with sockets - sock.c.patch (windows returns WSAWOULDBLOCK after connect on async socket) and win32readable.patch (to avoid "premature end of data")

Thank you for your work.

Regards,

Oleg.

[naviserver-devel] Towards NaviServer 4.99.17

[Gustaf N. <ne...@wu...>]

Dear all,

on sourceforge is a release candidate for NaviServer 4.99.17 [1].
Below is a - preliminary - summary of changes; especially the
summary of changes in the modules is missing. In case you have more
changes for this release, please commit or get in touch with me.
If everything goes well, the release should follow next week.

Please test if possible.

best regards

-gustaf
[1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.17/


=======================================
NaviServer 4.99.17, released 2018-XX-XX
=======================================

  398 files changed, 14060 insertions(+), 6760 deletions(-)

New Features:
-------------

   - Added brotli support for delivering static content
     ---------------------------------------------------
     The brotli compression format (RFC 7932) achieves better results for
     compression for typical web content compared to e.g. gzip and is
     supported by most current browsers. The definition of static
     content delivery is exactly the same for "brotli" as for static
     "gzip" content.

     New configuration options for ns/fastpath:
     * "brotli_static": should static brotli delivery be checked?
     * "brotli_refresh": should outdated brotli compressed files be refreshed?
     * "brotli_cmd": OS-level command with options used for brotli compression

   - "ns_conn" additions:
     * new subcommand "ns_conn acceptedcompression" to return accepted
       compressions for the current connection (gzip or brotli)

     * new subcommands "ns_conn currentaddr" and "ns_conn currentport"
       to refer to the address and port of the currently open server side
       of the socket.


   - New feature for GDPR: add mask IP addresses optionally in access log
     --------------------------------------------------------------------
     This function is similar to "anonip" IP anonymizer of the Swiss
     privacy foundation).  When this parameter is set, all IP addresses
     are masked in the log file, such that the host-specific (= person-
     specific) information is masked out and the IP-address does not
     match a particular individual anymore. Still, with the masking
     turned on, reverse DNS lookup and/or geolocation is possible.

     The option can be configured via the following parameters in the
     ".../module/nslog" section of the config file of NaviServer:
      * "masklogaddr": boolean value to turn feature on/off (default off)
      * "maskipv4": mask for IPv4 addresses (default 255.255.255.0)
      * "maskipv6": mask for IPv6 addresses (default ff:ff:ff:ff::)


   - New atomic "nsv_set" commands/options
     -------------------------------------

     * Obtain (old) value from nsv ARRAY and set it to a new value set foo

	set foo [nsv_set -reset ARRAY KEY NEWVALUE]

       After the operation, ARRAY(KEY) has a new value.
       The operation is similar to "nsv_set ARRAY KEY NEWVALUE",
       but the variant with "-reset" returns the old value.
       Similar to GETSET in REDIS.

    * Obtain a value from nsv ARRAY and unset it (no new value is provided).

	set foo [nsv_set -reset ARRAY KEY]

      The operations returns the old value of ARRAY(KEY) or empty, if
      it does not exist.  After the operation, the ARRAY(KEY) is unset.

    * Set a default value for a nsv ARRAY

	nsv_set -default ARRAY KEY DEFAULTVALUE

       Do nothing when ARRAY(KEY) has already a value.
       Return an error, when DEFAULTVALUE is not provided.
       Similar to SETNX in REDIS.


   - Extended functionality of "ns_http"
     -----------------------------------

     * Changed return value for "ns_http run" and "ns_http wait".
       Now the commands return attribute value pairs (Tcl dict).
       Previously, the commands returned different results,
       depending on the options

       - "ns_http run" returned the name of the queued command (which
	was completely useless, this the queued command was already
	terminated)
       - "ns_http wait" returned sometimes the body of the reply (when
	the reply was not spooled) or empty (spooled) or "1", when the
	option "-R" was specified

       Now the result is always a dict containing "status" (HTTP status
       code), "time" (elapsed time), "headers" (ns_set of the reply
       header fields, and either the "body" (reply body) or "file" (the
       name of the spooled file). The values in the dict might mirror
       the result of other (optional) output variables, which continue
       to work.

       Since the results were irregular and not documented in the man
       pages, the backward compatibility should be high. Also the
       regression test of NaviServer was not effected by this change.


     * Support for asynchronous “ns_http” tasks:

       When the new flag "-donecallback” is provided, then a “ns_http”
       task is started in the background and does not require an
       “ns_http wait” anymore. Instead, when it finishes it executes
       the provided donecallback which receives as additional arguments
       the Tcl result (in form of an integer) and the result dict,
       returned otherwise from e.g. “ns_http run”.

       As a consequence, requests of the form
	 ns_http queue … -donecallback …
       differ from client requests without the callback
       in the following points:

       a) the command does not return a handle for “ns_http wait”
       b) The command is not automatically cleaned up (canceled)
	 at the end of a connection requests
       c) Typically, donecallbacks are executed in a different
	 thread than it was started.

       Some more options, which were previously only valid on
       “ns_http wait” (such as e.g. spoolsize) are now valid
       for “ns_http queue” as well.

       The new flag allows a developer to start many requests in
       parallel without having to start many threads or complex
       synchronization tasks.

     * New option "-body_file" for "ns_http wait" (similar to "ns_http
       queue" which can be used to specify a filename for the
       downloaded content)

     * New option "-outputfile" to "ns_http run" and "ns_http wait":
       by specifying the option "-outputfile" the received file will be
       always written to the disk, no matter how large it is.

     * New subcommand "ns_http stats": returns a dict (flat list of
       attributes and values). The list contains "task", "url",
       "requestlength", "sent", "replylength" and "received".


   - Support resolving a hostname against multiple IP addresses
     ----------------------------------------------------------

     * In case, one domain name has multiple IP addresses registered,
       previous versions of NaviServer tried only the first returned
       address and returned an error, when this failed. This happened
       when using client functions (e.g. ns_http).

     * host names with multiple IP addresses are nowadays quite common,
       e.g. "localhost" having an IPv4 and an IPv6 address registered. In
       previous versions, when e.g. localhost has as first address the IPv6
       address, but IPv6 was not active, it returned an error.

     * resolving against IP addresses continues to work as always.


   - Extended crypto functionality
     -----------------------------

      * Added HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
	  ::ns_crypto::md hkdf -digest sha256 ....
        (requires OpenSSL 1.1.0 or newer)

      * Added support for elliptic curve cryptography
	. ns_crypto::eckey import ?-string string? ?-encoding encoding?
	. ns_crypto::eckey priv ?-pem pem? ?-encoding encoding?
	. ns_crypto::eckey pub ?-pem pem? ?-encoding encoding?
	. ns_crypto::eckey sharedsecret ?-pem pem? ?-encoding encoding? pubkey
	. ns_crypto::eckey generate ?-name name? ?-pem pem?

      * Added support for getting an arbitrary number of random bytes
        based on crypto support on OpenSSL:
	. ns_crypto::randombytes ?-encoding encoding? bytes

      * Added support for Authenticated Encryption with Associated Data
        (AEAD) scheme, which provides confidentiality, integrity, and authenticity.

	. ns_crypto::aead::encrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? input"
	. ns_crypto::aead::decrypt string ?-aad aad? ?-cipher cipher? ?-encoding encoding? ?-iv iv? ?-key key? ?-tag tag? input"

      * Added "-encoding hex|base64url|base64" option for output encoding
	for the crypto function. Previously, all these functions returned the
	result in "hex" format. If the option is not specified, the result
	is the same as before (hex). The encoding option was added to the
	following functions:
	- ns_crypto::hmac string
	- ns_crypto::hmac get
	- ns_crypto::md string
	- ns_crypto::md get
	- ns_crypto::md vapidsign
	- ns_crypto::enc


   - "ns_connchan" improvements
     --------------------------

     * Callback suspension: a Tcl socket callback function might now
       return the value "2" to flag that the callback will be
       suspended, but it keeps the socket open. Previously, the Tcl
       callback could return just "0" (callback is cancelled, socket is
       closed) or "1" (continue callback handling, keep socket open).
       This can be used to suspend callback handling on bottlenecks and
       continue later, when situation improves.

     * "ns_connchan write": The function returns now the number of
       bytes sent, which might be less than the length of the data to
       be sent.

     * "ns_connchan list": the function returns now two more fields per
       entry: a) cmd name of the callback (eases debugging, when
       callbacks are changed dynamically) b) conditions used for
       registering the callback.

     * The "ns_connchan" callbacks are now listed via the nsstats
       interface as well (with proper callback info).


   - Added Possibility to add per-server "initcmds" to config file
     -------------------------------------------------------------

     This new features provides an easy means to add command which
     should be executed after server initialization into a configuration
     files. By using the "initcmds" one can start multiple instances
     of NaviServer differing just in the initialization command
     by providing different config files. The "initcmds" approach
     has the advantage over e.g. "ns_atserverstart" that it is
     executed in a state when the server is full initiated.



   - Improved readability for config files
     -------------------------------------

     The "ns_section" command has new an optional last argument for the
     parameters of its configuration section. Instead of writing e.g.

        ns_section ns/parameters
        ns_param serverlog    $logroot/error.log
        ns_param pidfile      $logroot/nsd.pid
        ns_param home         $homedir
        #...

     one can write now

        ns_section ns/parameters {
	   ns_param serverlog   $logroot/error.log
	   ns_param pidfile     $logroot/nsd.pid
	   ns_param home        $homedir
	   #...
        }

     This has two advantages: a) it shows clearly, where the section
     ends and b) when using an editor with automatic indentation, the
     log file looks nicer without manual indentation work. This change
     is fully backward compatible, old style config files continue to
     work.


   - Further new commands and features
     ---------------------------------

     * "ns_hash": provide a interface for the HashStringKey() function
       (very useful for e.g. cache partitioning)

     * "ns_reflow_text".
	  ns_reflow_text ?-width width? ?-offset offset? ?-prefix prefix? ?--? text
       Tcl-based solutions are surprisingly slow on largish input.

     * "ns_base64urlendcode", "ns_base64urldedcode": URL-save variants of base64,
       used in various new RFCs for passing base64 values in HTTP requests.

    * "ns_uuid": runtime efficient version of a version 4 UUID
       according to RFC 4122: A Universally Unique IDentifier (UUID)
       URN Namespace

    * Improved results of [ns_cache_stats -contents ...]. The command
      returns now a tcl list instead of a string, including reuse count.

   - Improved results of "ns_striphtml":
      * Added resolving of all HTML4 entities (including   etc.)
      * Stripped as well HTML comments.


Performance Improvements:
-------------------------

   - Check for epoch updates on blue-prints as well in cleanup phase
     (deallocate trace) to reduce latency for client requests.


Bug Fixes:
----------

   - Don't try to serve redirected files in case the connection is
     already closed (will lead only to confusing messages)

   - Ns_IsBinaryMimeType(): When a chartype is specified, never assume
     the mimetype is binary.

   - ns_getformfile: Don't rely on global variables
     (see https://sourceforge.net/p/naviserver/bugs/80/)
     Before this change, Depending on configured values,
     ns_getformfile could have missed uploaded files.

   - Added missing fetch commands for obtaining handles (on obviously
     seldom used subcommands "ns_db setexception|sp_setparam")

   - ns_http:
     * Timeout and connection behavior show works now consistent
       for HTTP and HTTPS URLs.
     * Fixed a potential bug (infinite loop) when requests
       run into timeouts.

   - "ns_set print": align with documented behavior

   - "ns_parseurl": make results more robust, when a URL with a colon
     in the path was given, but no port provided (detected a port
     incorrectly)

   - "ns_parseformfile": handle also www-form-urlencoded (in
      addition to multipart/form-data as before)

   - Windows:
     * Make sure to create temporary files correctly under windows (many
       thanks to Oleg Oleinik)
     * Incorrect handle was used to perform operations on the
       process in Ns_WaitForProcess (many thanks to Oleg Oleinik)


Documentation improvements:
---------------------------

   - Updated several man pages
     * admin-db.man:
     * admin-install.man
     * admin-maintenance.man
     * adp-overview.man
     * commandlist.man
     * main-features.man
     * main-history.man
     * ns_accesslog.man
     * ns_adp.man
     * ns_adp_argc.man
     * ns_adp_argv.man
     * ns_adp_bind_args.man
     * ns_adp_eval.man
     * ns_adp_flush.man
     * ns_adp_info.man
     * ns_adp_safeeval.man
     * ns_atexit.man
     * ns_atexit.man
     * ns_atshutdown.man
     * ns_atsignal.man
     * ns_base64.man
     * ns_base64encode.man
     * ns_conn.man
     * ns_connchan.man
     * ns_cookie.man
     * ns_db.man
     * ns_driver.man
     * ns_fmttime.man
     * ns_getcontent.man
     * ns_getform.man
     * ns_hmac.man
     * ns_hotp.man
     * ns_http.man
     * ns_httpget.man
     * ns_httpopen.man
     * ns_ictl.man
     * ns_job.man
     * ns_kill.man
     * ns_md.man
     * ns_memoize.man
     * ns_mutex.man
     * ns_parsequery.man
     * ns_perm.man
     * ns_proxy.man
     * ns_queryexists.man
     * ns_queryget.man
     * ns_querygetall.man
     * ns_quotehtml.man
     * ns_rand.man
     * ns_register.man
     * ns_return.man
     * ns_rlimit.man
     * ns_schedule_proc.man
     * ns_sendmail.man
     * ns_server.man
     * ns_set.man
     * ns_set.man:
     * ns_setexpires.man
     * ns_sockopen.man
     * ns_sockselect.man
     * ns_striphtml.man
     * ns_thread.man
     * ns_time.man
     * ns_tmpnam.man
     * ns_totp.man
     * ns_urlspace.man
     * ns_uuencode.man
     * ns_writer.man
     * nscgi.man
     * nsperm.man
     * nssock.man
     * nsv.man
     * returnstatus-cmds.man
     * tcl-lib-file.man
     * tcl-lib-nstrace.man
     * tcl-libraries.man
     * tcl-overview.man


Tcl API Changes:
----------------

   - "ns_conn isconnected": returns now false, when the connection is
      already closed. Otherwise we have to add a new subcommand
      "ns_conn closed", which seems somewhat odd (isconnected true and
      closed true?)

   - "ns_cache_stats" returns now hitrate in form of a float instead of
     an integer.


C API Changes:
--------------

    - Ns_SockTimedConnect2() returns now Ns_ReturnCode to be able to
      distinguish between timeouts and errors

    - Ns_SockConnectError(): new call to provide consistent error messages
      and to set Tcl error info consistently


Configuration Changes:
----------------------

   - use same initialization for enablelclpages when configured over
     "ns/server/$server" or over "ns/server/$server/adp" to avoid subtle
     differences on global variables

   - Extended sample config files:

     * nsd-config.tcl

     * openacs-config.tcl
        . Added OpenACS specific comments for configuring caches
        . Added subsite based error pages
        . Added section indicating how to configure for nsstats module
        . Added comments how to configure EmailDeliveryMode via config file
        . Use PostgreSQL's conninfo notation in the datasource to hint, that
	 more parameters are possible as well (e.g. connections via SSL)

     * sample-config.tcl:

     * nsd-config.tcl:
	Added commen environment variables HOME, LANG and RANDFILE.
	RANDFILE is sometimes needed to avoid surprises with permissions,
	when OpenSSL crypto functions are called from the server

   - Improved handling of configuration errors:
     * Provide better log message, when configured value is out of range
     * Update config values when range check corrects the value to get the
       finally configured value via introspection.


Code Changes:
-------------

   - Extended regression test:
	* tests/adp.test
	* tests/compat.test
	* tests/http.test
	* tests/ns_base64.test
	* tests/ns_base64encode.test
	* tests/ns_cache.test
	* tests/ns_conn.test
	* tests/ns_crypto.test
	* tests/ns_nsv.test
	* tests/ns_parseurl.test
	* tests/ns_reflow_text.test
	* tests/ns_server.test
	* tests/ns_striphtml.test


  - Added version tag of NaviServer to "starting" message in log file
    to ease analysis in case of problems.

  - Implemented deprecated commands as Tcl proc and complain on
    its usage
       . ns_adp_eval
       . ns_adp_safeeval

   - Improved Platform support:
     * Extend list of platforms for manual provisioning of __PRIPTR_PREFIX
       in case it is not defined
     * Windows:
       . improve type-cleanness (many thanks to Oleg Oleinik)
       . when compiling with VS 2012, use built-in support for
	intptr_t and uintptr_t.

   - Changed preset encodings (when not specified in config file) for
     files with extensions .htm, .html, and .adp from iso8859-1 to
     utf-8.

   - Improved Debugging support:

     * Added compiler-flag NS_MUTEX_NAME_DEBUG for ease locating
       anonymous mutexes.
     * Provided names for all remaining mutexes to make it easier
       to pinpoint bottlenecks due to mutexes
     * Improve debugging output in various messages, when debugging
       options are activated.



   - Minor update of mime types based on actual IANA media types.

   - Provide compatibility with Tcl 8.7.2 (removed CONST and VOID macros)
   - further improved robustness by avoiding potential NULL dereferences
     found be static code analyzers (mostly error cases)
   - improved configure scripts
   - make sure, we have enough space for the terminating null character
   - prefer boolean type over int
   - added "pure" specifier for improved code generation
   - improved type cleanness, reduce implicit conversions
   - reduced number of return statements before end of function
   - Don't pass implementation-defined NULL after the last typed argument
     to a variadic function
   - Remove dead assignments
   - Added const declarations
   - Fixed typos, improved comments


Modules:
--------




-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] Using qmail with naviserver

[Brian F. <Bri...@qu...>]

Hi Thorpe

there is some old documentation here that may be of some help.
https://openacs.org/doc/install-qmailhttps://openacs.org/doc/install-qmail

Brian
________________________________
From: THORPE MAYES via naviserver-devel [nav...@li...]
Sent: 23 October 2018 15:58
To: THORPE MAYES via naviserver-devel
Cc: THORPE MAYES
Subject: [naviserver-devel] Using qmail with naviserver

Hi,

Is it possible to use qmail with naviserver? If so, how can I go about doing it?

Here is what is in the config file regarding sendmail:

set mailhost               mail.somthing.com<http://mail.somthing.com>
set smtphost               mail.something.com<http://mail.something.com>
set smtpport               25
set smtptimeout            60
set smtpmsgidhostname      mail.something.com<http://mail.something.com>

ns_section "ns/parameters"
…
ns_param smtphost $smtphost
ns_param mailhost $mailhost
ns_param smtpport $smtpport
ns_param smtptimeout $smtptimeout
ns_param smtpmsgidhostname $smtpmsgidhostname

Thank you.

Thorpe

[naviserver-devel] Using qmail with naviserver

[THORPE M. <ta...@me...>]

Hi,

Is it possible to use qmail with naviserver? If so, how can I go about doing it?

Here is what is in the config file regarding sendmail:

set mailhost               mail.somthing.com
set smtphost               mail.something.com
set smtpport               25
set smtptimeout            60
set smtpmsgidhostname      mail.something.com

ns_section "ns/parameters"
…
ns_param smtphost $smtphost
ns_param mailhost $mailhost
ns_param smtpport $smtpport
ns_param smtptimeout $smtptimeout
ns_param smtpmsgidhostname $smtpmsgidhostname

Thank you.

Thorpe

[naviserver-devel] Naviserver SSL connections to Postgresql DB

[David O. <da...@qc...>]

Hi,

I wanted to ask how NaviServer users typically go about encrypting their
connection to a remote PostgreSQL backend under Linux?

In the past we've used the nsdbpg drivers (which don't support SSL
connections) and encrypted using stunnel SSL wrapper, but in later versions
of libpq5 this seems to have stopped working well.

Would the main other option be to use the nsdbipg driver with a conninfo
string including sslmode=require?

Any other options?

Regards,
-- 
David

Re: [naviserver-devel] Install Error via install-ns.sh

[Gustaf N. <ne...@wu...>]

Glad, that the problems are solved. There are many ways to address the 
issue.

Now, since OpenSSL 1.1.1 is finally released, i hope it is adoped soon 
by the common distros to give users little reason to use their own 
compiled version of OpenSSL.

all the best
-gn

On 23.09.18 17:23, THORPE MAYES via naviserver-devel wrote:
> Dear Gustav,
>
> Thank you very much for your help.
>
> Here is what I ended up doing:
>
> 1. I installed OpenSSL in its standard location (/usr/local)
>
> 2. I added -with-openssl=/usr/local/ to the ./config command in your 
> bash script per your direction
>
> 3. I added these symlinks to /usr/local/ns/lib:
>
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libssl.so.1.1 libssl.so.1.1
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libcrypto.so.1.1 libcrypto.so.1.1
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libssl.so libssl.so
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libcrypto.so libcrytpo.so
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libcrypto.a libcrytpo.a
> [root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s 
> /usr/local/lib64/libssl.a libssl.a
>
> I am not sure that I needed them all, but no harm.
>
> Those changes resulted in naviserver being installed via the script.
>
> Again, thank you.
>
> Thorpe
>
>
>
>> On Sep 23, 2018, at 5:42 AM, Gustaf Neumann <ne...@wu... 
>> <mailto:ne...@wu...>> wrote:
>>
>> Dear Thorpe,
>>
>> The function 'SSL_is_init_finished' (not found in your installation) 
>> is available in OpenSSL at least since 1.0.2, so it looks to me as if 
>> you compiled against recent .h files of OpenSSL, but you tried to 
>> link against an old version of OpenSSL, which does not have these 
>> functions.
>>
>> The see various versions of OpenSSL libraries on your system, and 
>> these containing the
>> function "OPENSSL_init_ssl", use a command like the following
>>
>>     for l in `locate libssl.a` ; do echo $l; nm $l | fgrep 
>> OPENSSL_init_ssl; done
>>
>> Assuming, you have installed OpenSSL-1.1.1 under /usr/local, then 
>> configure naviserver
>> with
>>      ./configure ... -with-openssl=/usr/local/
>>
>> This should add the proper "-L..." option to the linking command.
>>
>> Alternatively, you can install OpenSSL into /usr/local/ns. The 
>> mentioned script does similar with Tcl and tcllib. This keeps the the 
>> libraries used by NaviSever more independent of the packaged updates 
>> on your system.
>>
>> all the best
>> -gn
>>
>> On 22.09.18 18:19, THORPE MAYES via naviserver-devel wrote:
>>> Hi,
>>>
>>> I am trying to install Naviserver using this script:
>>>
>>> install-ns.sh
>>>
>>> I have installed Openssl-1.1.1.
>>>
>>> I am getting this error:
>>>
>>> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz 
>>> -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6 -lgcc_s  -lieee -lm  
>>> -Wl,--export-dynamic -L/usr/local/ns/lib -lssl -lcrypto 
>>> -Wl,-rpath,:/usr/local/ns/lib
>>> libnsd.so: undefined reference to `EVP_MD_CTX_new'
>>> libnsd.so: undefined reference to `HMAC_CTX_free'
>>> libnsd.so: undefined reference to `SSL_is_init_finished'
>>> libnsd.so: undefined reference to `OPENSSL_init_ssl'
>>> libnsd.so: undefined reference to `OpenSSL_version'
>>> libnsd.so: undefined reference to `OPENSSL_init_crypto'
>>> libnsd.so: undefined reference to `TLS_client_method'
>>> libnsd.so: undefined reference to `EVP_MD_CTX_free'
>>> libnsd.so: undefined reference to `HMAC_CTX_new'
>>> libnsd.so: undefined reference to `TLS_server_method'
>>> collect2: error: ld returned 1 exit status
>>> make[1]: *** [nsd] Error 1
>>> make[1]: Leaving directory `/usr/local/package/naviserver-4.99.16/nsd'
>>> make: *** [all] Error 1
>>>
>>> I am not sure where to go from here. Any help will be appreciated.
>>>
>>> Thank you,
>>>
>>> Thorpe
>>>
>>

Re: [naviserver-devel] Install Error via install-ns.sh

[Iuri S. <iu...@iu...>]

Are you using Debian/Ubuntu?

If so, you might run apt-get install gcc and related packages. Then you  can run install-ns.sh again,


> On Sep 22, 2018, at 13:19, THORPE MAYES via naviserver-devel <nav...@li...> wrote:
> 
> Hi,
> 
> I am trying to install Naviserver using this script:
> 
> install-ns.sh
> 
> I have installed Openssl-1.1.1.
> 
> I am getting this error:
> 
> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
> libnsd.so: undefined reference to `EVP_MD_CTX_new'
> libnsd.so: undefined reference to `HMAC_CTX_free'
> libnsd.so: undefined reference to `SSL_is_init_finished'
> libnsd.so: undefined reference to `OPENSSL_init_ssl'
> libnsd.so: undefined reference to `OpenSSL_version'
> libnsd.so: undefined reference to `OPENSSL_init_crypto'
> libnsd.so: undefined reference to `TLS_client_method'
> libnsd.so: undefined reference to `EVP_MD_CTX_free'
> libnsd.so: undefined reference to `HMAC_CTX_new'
> libnsd.so: undefined reference to `TLS_server_method'
> collect2: error: ld returned 1 exit status
> make[1]: *** [nsd] Error 1
> make[1]: Leaving directory `/usr/local/package/naviserver-4.99.16/nsd'
> make: *** [all] Error 1
> 
> I am not sure where to go from here. Any help will be appreciated.
> 
> Thank you,
> 
> Thorpe
> 
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Install Error via install-ns.sh

[THORPE M. <ta...@me...>]

Dear Gustav,

Thank you very much for your help.

Here is what I ended up doing:

1. I installed OpenSSL in its standard location (/usr/local)

2. I added -with-openssl=/usr/local/ to the ./config command in your bash script per your direction

3. I added these symlinks to /usr/local/ns/lib:

[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libssl.so.1.1 libssl.so.1.1
[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libcrypto.so.1.1 libcrypto.so.1.1
[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libssl.so libssl.so
[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libcrypto.so libcrytpo.so
[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libcrypto.a libcrytpo.a
[root@centos-s-1vcpu-1gb-sfo2-01-d2 lib]# ln -s /usr/local/lib64/libssl.a libssl.a

I am not sure that I needed them all, but no harm.

Those changes resulted in naviserver being installed via the script. 

Again, thank you.

Thorpe



> On Sep 23, 2018, at 5:42 AM, Gustaf Neumann <ne...@wu...> wrote:
> 
> Dear Thorpe,
> 
> The function 'SSL_is_init_finished' (not found in your installation) is available in OpenSSL at least since 1.0.2, so it looks to me as if you compiled against recent .h files of OpenSSL, but you tried to link against an old version of OpenSSL, which does not have these functions.
> 
> The see various versions of OpenSSL libraries on your system, and these containing the 
> function "OPENSSL_init_ssl", use a command like the following
> 
>     for l in `locate libssl.a` ; do echo $l; nm $l | fgrep OPENSSL_init_ssl; done
> 
> Assuming, you have installed OpenSSL-1.1.1 under /usr/local, then configure naviserver
> with 
>      ./configure ... -with-openssl=/usr/local/
> 
> This should add the proper "-L..." option to the linking command.
> 
> Alternatively, you can install OpenSSL into /usr/local/ns. The mentioned script does similar with Tcl and tcllib. This keeps the the libraries used by NaviSever more independent of the packaged updates on your system.
> 
> all the best
> -gn
> 
> On 22.09.18 18:19, THORPE MAYES via naviserver-devel wrote:
>> Hi,
>> 
>> I am trying to install Naviserver using this script:
>> 
>> install-ns.sh
>> 
>> I have installed Openssl-1.1.1.
>> 
>> I am getting this error:
>> 
>> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
>> libnsd.so: undefined reference to `EVP_MD_CTX_new'
>> libnsd.so: undefined reference to `HMAC_CTX_free'
>> libnsd.so: undefined reference to `SSL_is_init_finished'
>> libnsd.so: undefined reference to `OPENSSL_init_ssl'
>> libnsd.so: undefined reference to `OpenSSL_version'
>> libnsd.so: undefined reference to `OPENSSL_init_crypto'
>> libnsd.so: undefined reference to `TLS_client_method'
>> libnsd.so: undefined reference to `EVP_MD_CTX_free'
>> libnsd.so: undefined reference to `HMAC_CTX_new'
>> libnsd.so: undefined reference to `TLS_server_method'
>> collect2: error: ld returned 1 exit status
>> make[1]: *** [nsd] Error 1
>> make[1]: Leaving directory `/usr/local/package/naviserver-4.99.16/nsd'
>> make: *** [all] Error 1
>> 
>> I am not sure where to go from here. Any help will be appreciated.
>> 
>> Thank you,
>> 
>> Thorpe
>> 
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Install Error via install-ns.sh

[Gustaf N. <ne...@wu...>]

Dear Thorpe,

The function 'SSL_is_init_finished' (not found in your installation) is 
available in OpenSSL at least since 1.0.2, so it looks to me as if you 
compiled against recent .h files of OpenSSL, but you tried to link 
against an old version of OpenSSL, which does not have these functions.

The see various versions of OpenSSL libraries on your system, and these 
containing the
function "OPENSSL_init_ssl", use a command like the following

     for l in `locate libssl.a` ; do echo $l; nm $l | fgrep 
OPENSSL_init_ssl; done

Assuming, you have installed OpenSSL-1.1.1 under /usr/local, then 
configure naviserver
with
      ./configure ... -with-openssl=/usr/local/

This should add the proper "-L..." option to the linking command.

Alternatively, you can install OpenSSL into /usr/local/ns. The mentioned 
script does similar with Tcl and tcllib. This keeps the the libraries 
used by NaviSever more independent of the packaged updates on your system.

all the best
-gn

On 22.09.18 18:19, THORPE MAYES via naviserver-devel wrote:
> Hi,
>
> I am trying to install Naviserver using this script:
>
> install-ns.sh
>
> I have installed Openssl-1.1.1.
>
> I am getting this error:
>
> gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz 
> -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm 
> -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto 
> -Wl,-rpath,:/usr/local/ns/lib
> libnsd.so: undefined reference to `EVP_MD_CTX_new'
> libnsd.so: undefined reference to `HMAC_CTX_free'
> libnsd.so: undefined reference to `SSL_is_init_finished'
> libnsd.so: undefined reference to `OPENSSL_init_ssl'
> libnsd.so: undefined reference to `OpenSSL_version'
> libnsd.so: undefined reference to `OPENSSL_init_crypto'
> libnsd.so: undefined reference to `TLS_client_method'
> libnsd.so: undefined reference to `EVP_MD_CTX_free'
> libnsd.so: undefined reference to `HMAC_CTX_new'
> libnsd.so: undefined reference to `TLS_server_method'
> collect2: error: ld returned 1 exit status
> make[1]: *** [nsd] Error 1
> make[1]: Leaving directory `/usr/local/package/naviserver-4.99.16/nsd'
> make: *** [all] Error 1
>
> I am not sure where to go from here. Any help will be appreciated.
>
> Thank you,
>
> Thorpe
>

[naviserver-devel] Install Error via install-ns.sh

[THORPE M. <ta...@me...>]

Hi,

I am trying to install Naviserver using this script:

install-ns.sh

I have installed Openssl-1.1.1.

I am getting this error:

gcc -L../nsthread -L../nsd -L../nsdb -o nsd main.o libnsd.so -lz -lcrypt -lnsthread -L/usr/local/ns/lib -ltcl8.6  -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/ns/lib -lssl -lcrypto   -Wl,-rpath,:/usr/local/ns/lib
libnsd.so: undefined reference to `EVP_MD_CTX_new'
libnsd.so: undefined reference to `HMAC_CTX_free'
libnsd.so: undefined reference to `SSL_is_init_finished'
libnsd.so: undefined reference to `OPENSSL_init_ssl'
libnsd.so: undefined reference to `OpenSSL_version'
libnsd.so: undefined reference to `OPENSSL_init_crypto'
libnsd.so: undefined reference to `TLS_client_method'
libnsd.so: undefined reference to `EVP_MD_CTX_free'
libnsd.so: undefined reference to `HMAC_CTX_new'
libnsd.so: undefined reference to `TLS_server_method'
collect2: error: ld returned 1 exit status
make[1]: *** [nsd] Error 1
make[1]: Leaving directory `/usr/local/package/naviserver-4.99.16/nsd'
make: *** [all] Error 1

I am not sure where to go from here. Any help will be appreciated.

Thank you,

Thorpe

[naviserver-devel] NaviServer module for Web Push Notifications

[Gustaf N. <ne...@wu...>]

Dear all,

On bitbucket [1] is a new Tcl-only module for NaviServer (requires 
current tip)
implementing Web Push using the W3C Push API, which enables sending of
push messages to a "webapp" via a push service (e.g. Mozilla, Google). When
the client (e.g. a mobile phone) is currently not reachable the message is
stored for a specified time to live (TTL) until it can be delivered as a 
notification.

On [3] is a small demo application, which is as well included in the 
repository.
The technical details are summarized on [4]. Note, that the basic 
functionality
of the demo could as well be implemented browser notification API. Web
Push supports delivery of notifications triggered by an application 
server via
third-party push servers [5], even when the user is not online or browsing
the pushing site.

all the best

-gustaf neumann

[1] https://bitbucket.org/naviserver/nswebpush/src/default/
[2] https://www.w3.org/TR/push-api/
[3] https://openacs.org/webpush-demo/webpush-demo.tcl
[4] https://openacs.org/webpush-demo/report.html
[5] 
https://developers.google.com/web/fundamentals/push-notifications/how-push-works

[naviserver-devel] nsodbc for NaviServer

[Gustaf N. <ne...@wu...>]

Dear all,

since a couple of days, there is a new driver for NaviServer on 
bitbucket, namely nsodbc [1].
This is a port of an earlier driver for AOLserver [2], which was 
adjusted to the NaviServer
infrastructure. Most of the work was done by Brendan Graves from mindoka.

all the best

-gustaf neumann

[1] https://bitbucket.org/naviserver/nsodbc/src/default/
[2] https://sourceforge.net/projects/aolserver/files/nsodbc/

Re: [naviserver-devel] ns_column derivative

[Iuri S. <iu...@iu...>]

I’ve written the following palliative.


  db_with_handle db {
            set trigger_column_list [list]
            for {set i 0} {$i < [db_string select_fields { SELECT COUNT(*) FROM ec_custom_product_field_values} ] } {incr i} {
                lappend trigger_column_list [lindex [db_list select_name { SELECT name FROM ec_custom_product_field_values } ] $i]
            }
        }


Too ugly! Would there be a beautiful code?

Best wishes,
I


> On Jul 6, 2018, at 21:07, Iuri Sampaio <iu...@iu...> wrote:
> 
> Hi there,
> 
> What would be a derivative proc, which substitutes ns_column ?
> 
> The piece of context is right bellow.
> 
> Best wishes,
> I
> 
> 
>  db_with_handle db {
>             set trigger_column_list [list]
>             for {set i 0} {$i < [ns_column count $db ec_custom_product_field_values]} {incr i} {
>                 lappend trigger_column_list [ns_column name $db ec_custom_product_field_values $i]
>             }
>         }
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_column derivative

[Iuri S. <iu...@iu...>]

Hi there,

What would be a derivative proc, which substitutes ns_column ?

The piece of context is right bellow.

Best wishes,
I


 db_with_handle db {
            set trigger_column_list [list]
            for {set i 0} {$i < [ns_column count $db ec_custom_product_field_values]} {incr i} {
                lappend trigger_column_list [ns_column name $db ec_custom_product_field_values $i]
            }
        }

Re: [naviserver-devel] nsphp and php version

[Gustaf N. <ne...@wu...>]

Dear Ian,

In order to try compilation of nsphp from PHP rpms, you need
a) an rpm with zts enabled (php-zts), and
b) the development libraries (php-devel).

both have to be from a PHP 5.6.* version. Another weak point
is the question, how many of the configure options of PHP
were used in that rpm build, but that can be figured out via
the following command

  $ php -i | grep zts

You should see "--enable-maintainer-zts" in the output and the
list of options used to configure PHP. These options can be compared
with these in nsphp/Makefile under the php target.

My first guess is that the errors you are seeing are due to an interface
change between PHP 5.4 and 5.6.

-g
PS: i have never tried this endeavor.

On 18.06.18 23:28, Ian Harding wrote:
> I am trying to build nsphp without resorting to the "download and 
> build against" option in the Makefile.  I am having trouble getting 
> that to Do The Right Thing in the context of rpmbuild.
>
> I have php 5.4.16 installed but am getting errors, the scariest of 
> which being
>
> error: too many arguments to function 'call_user_function'
>
> (It's in a vm and hard to copy/paste) but will it build with 5.4.16?  
> Anyone got it to work in an RPM?

[naviserver-devel] nsphp and php version

[Ian H. <har...@gm...>]

I am trying to build nsphp without resorting to the "download and build
against" option in the Makefile.  I am having trouble getting that to Do
The Right Thing in the context of rpmbuild.

I have php 5.4.16 installed but am getting errors, the scariest of which
being

error: too many arguments to function 'call_user_function'

(It's in a vm and hard to copy/paste) but will it build with 5.4.16?
Anyone got it to work in an RPM?

Thanks!

- Ian

[naviserver-devel] Improved nsstats mutex statistics

[Gustaf N. <ne...@wu...>]

Dear all

some of might find the updated nscache mutex statistics of the "nsstats" 
module
interesting, which brings locking times into correlation with number of 
locks per page.

The screen shot-below is from openacs.org based on data from 182 mio locks.
Every page has on that site on average 314 mutex locks (from NaviServer 
and its modules),
taking on average ~2 micro seconds (not counting purely background 
related related locks).
Just by looking at the locks, this means a peak rate of that 
installation of 1.6K requests/second.
This is certainly way less what on can measure using synthetic 
benchmarks on e.g. static pages,
but at the same time a order of magnitude more of what is needed on this 
site max (less then
100 requests per seconds) or on several orders of magnitudes higher than 
average values.
The number of locks are mostly application specific (e.g. how an 
application uses
OpenACS and how OpenACS uses NaviServer).

The interesting part is that these figures are independent of database 
requests
or other slow calls, and these won't improve, if the site is run on a 
processor with
e.g. more cores.

As we can see from the table below, the high number of locks per request
originate on this site from nsvs. With more digging, one can find via 
nsstats "nsv locks"
that the highest numbers come from the internationalization  of message 
keys,
which is performed with nsvs in OpenACS ("lang_message_cache", "lang 
message_en_US").

The number of locks on particular mutexes can be improved via 
partitioning, concrete for
nsvs by increasing the nsvbuckets configuration variable. Avoiding these 
locks is even
better by moving such values into per-thread caches (e.g. Tcl variables).

The values of this tables are already the result of several improvements 
in OpenACS,
which used to call very often results from the util_memoize cache (now 
it is already
down to ~7 locks/page) On other OpenACS sites, these values differ 
(util_memoize
is still top).

Note that these statistics makes no distinction between what locks are 
caused
from connection threads or from background jobs, and the "potential 
requests/second"
assume that these are caused from connection threads.

However, these statistics might reveal on sites some interesting insights,
since application developers seem to often think "if this values is 
cache, i don't
have to care, i can stop to think".  When cache accesses happens in a tight
loop, may be though an API, statistics like the one above can help to 
improve
on hard to explain scalability bottlenecks.

all the best

-g