naviserver-devel — Developer discussion

Re: [naviserver-devel] Updated Online Documentation & Styling – Uniform, Responsive Design for NaviServer 5.0

[Brian F. <bri...@ai...>]

This looks fantastic! Great job.

Brian
________________________________
From: Gustaf Neumann (sslmail) <ne...@wu...>
Sent: Friday 28 March 2025 5:49 pm
To: nav...@li... <nav...@li...>
Subject: [naviserver-devel] Updated Online Documentation & Styling – Uniform, Responsive Design for NaviServer 5.0

Dear NaviServer Developers,

Over the past few days, I’ve been working on revamping our online documentation and styling. I’m pleased to share that we now have a uniform, responsive design for:

  - The plain NaviServer start page

  - The nsstats module

  - The manual pages

Notably, the new design no longer depends on external libraries (we previously relied on Bootstrap 5 and, in part, on W3.CSS). While the current implementation is responsive to a high degree, there’s always room for further refinement.

You can preview the updated pages via the following links (please note that the man pages on SourceForge may take a short while to propagate to all regions – if something appears off, try reloading your browser):

    Main Table of Contents:
    https://naviserver.sourceforge.io/5.0/toc.html

    Keyword Index:
    https://naviserver.sourceforge.io/5.0/index.html

    Command List:
    https://naviserver.sourceforge.io/5.0/naviserver/files/commandlist.html

    Sample Command Man Page (ns_conn):
    https://naviserver.sourceforge.io/5.0/naviserver/files/ns_conn.html

Styling the documentation, which is produced by the Tcl doctools, is quite challenging, since the produced markup is widely old-style, and it allows only little influence on the output.

I welcome any feedback or suggestions you might have

All the best
-g




[cid:879...@EU...]

_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Updated Online Documentation & Styling – Uniform, Responsive Design for NaviServer 5.0

[Gustaf N. (sslmail) <ne...@wu...>]

Dear NaviServer Developers,

Over the past few days, I’ve been working on revamping our online documentation and styling. I’m pleased to share that we now have a uniform, responsive design for:

  - The plain NaviServer start page

  - The nsstats module

  - The manual pages

Notably, the new design no longer depends on external libraries (we previously relied on Bootstrap 5 and, in part, on W3.CSS). While the current implementation is responsive to a high degree, there’s always room for further refinement.

You can preview the updated pages via the following links (please note that the man pages on SourceForge may take a short while to propagate to all regions – if something appears off, try reloading your browser):

    Main Table of Contents:
    https://naviserver.sourceforge.io/5.0/toc.html

    Keyword Index:
    https://naviserver.sourceforge.io/5.0/index.html

    Command List:
    https://naviserver.sourceforge.io/5.0/naviserver/files/commandlist.html

    Sample Command Man Page (ns_conn):
    https://naviserver.sourceforge.io/5.0/naviserver/files/ns_conn.html

Styling the documentation, which is produced by the Tcl doctools, is quite challenging, since the produced markup is widely old-style, and it allows only little influence on the output.

I welcome any feedback or suggestions you might have

All the best
-g

Re: [naviserver-devel] HTTP client: security by default

[Gustaf N. (sslmail) <ne...@wu...>]

Dear all,

A stable version of the certificate validation management for outgoing HTTPS requests (ns_http, ns_connchan) for NaviServer 5 is now in the NaviServer repository on GitHub. The current documentation is also online:

   https://naviserver.sourceforge.io/5.0/manual/files/admin-config.html#subsection13

Let me know, if you have additional requirements in this regard.

The newest version of NaviServer runs since this morning on openacs.org <http://openacs.org/>.

All the best
-gn

> On 23.01.2025, at 18:43, Gustaf Neumann (sslmail) <ne...@wu...> wrote:
> 
> Dear all,
> 
> This mail is a warning, that after an upgrade to the newest version from git, some of your configuration files/applications might need adjustments,
> 
> The newest commit to the main repository brings more security, by validating in ns_http client requests always the server certificate. While this is common for browsers, this is not necessarily the case for automated web operations (web services etc.), opening a large attack vector for man-in-the-middle attacks (see below). So far, checking the certificates was per default deactivated, which has the consequence, that in practice, many (most?) application developers just used with the defaults, leading to an unpleasant situation. Now, certificated checking is activated, it can be deactivated by adding the parameter “-insecure” to an “ns_http” request (there are other means, see below).
> 
> The regression test (including self-signed certificates for HTTPS, revproxy, …) works fine.
> 
> Next steps for me:
> - check consequences for letsencrypt  
> - add similar changes to “ns_connchan open|connect"
> - documentation of certificate management in detail (covering ns_http, ns_connchan, admin-config)
> 
> We are getting closer to the release candiate...
> 
> all the best
> -gn
> 
> 
> Security by default for HTTP client requests (ns_http)
> 
> Background: Authenticating the peer is a critical part of SSL connection setup when a client connects to a server. In this process, the server presents its public-key certificate. If the results of this authentication is missing, a server might be vulnerable to man-in-the-middle attacks. Protection against this is one of the intentions of HTTPS, and becomes more important when backend transactions are performed via HTTPS (administration of servers, cloud infrastructure, payments, ...), establishing secure network tunnels, or when sending confidential information via HTTPs
> 
> As the paper mentioned below points out, the validation of peer certificates is usually in place for web communication over the browser, but it is often not active for web service interactions.  One reason for this is that application developers choose very often the default setup.
> 
> The key instrument of NaviServer for performing HTTP client operations is ns_http, which certainly has the ability for certificate validation, but so far it was per default deactivated. This changes now with NaviServer 5, where the default is now including validation.
> 
> One of the consequences of validating peer certificates is that the administrator has to care about certificates (what certificates are accepted), including management of self-signed certificates.
> 
> We try to make the transition as smooth a possible by providing a reasonable default setup including established root certificates (ca-bundle.crt), providing simple means to add your own trusted certificates (including self-signed certificates). This default setup can be altered for a server via the configuration file, or for every single request via parameters.
> 
> The most important changes are:
> 
> - new parameter "-insecure" for "ns_http run" and "ns_http queue"
>  This parameter turns off certificate validation for the target HTTPS server. The name follows the naming convention of curl.
> 
> - The old parameter "-validate" is a now a no-op.
>  Per default, all ns_http requests to HTTPS servers are now validated
> 
> - Store certificates in directory ${NSHOME}/certificates instead of ${NSHOME}/etc
> 
> - New configuration parameter "CApath" and "CAfile" for the "httpclient" section of a server to specify default locations for certificate validation. It is possible to override these values via parameters to "ns_http run" and "ns_http queue".
> 
>  Default configuration:
> 
>     ns/${server}/httpclient {
>        ns_param CAfile ... ;# default ${NSHOME}/ca-bundle.crt
>        ns_param CApath ... ;# default ${NSHOME}/certificates
>     }
> 
>  * "CAfile" points to a .pem file containing established root certificates, often named "ca-bundle.crt". This file contains multiple of these certificates. The version shipped with NaviServer is based on Mozilla's root certificates. Also, many operating systems provide these certificates (e.g., on Ubuntu /etc/ssl/certs/ca-certificates.crt). One can certainly use various sources, e.g. via symbolic links or configuration parameters. The file can be manually updated e.g. via 
> 
>          curl https://curl.se/ca/cacert.pem -o /usr/local/ns/ca-bundle.crt
> 
>  * "CApath" points to a directory containing CA certificates in PEM
>     format. Each of the files must contain exactly one CA certificate (OpenSSL requirement).
> 
>     It is possible to add self-signed certificates to this directory to connect via ns_http with an HTTPS URL to this server, after running
> 
>         openssl rehash ${NSHOME}/certificates
> 
>     The rehash operation is performed automatically in the installation process.
> 
> - New configuration parameter "insecure" for the "httpclient" section of a server
> 
>     The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software
>     https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf 
> 
> - Added certificate settings to sample configuration files nsd-config.tcl and openacs-config.tcl
> 
> - Minor cleanup of Makefiles (improve consistency and configurability)
> 
> - similar changes for "ns_connchan connect|open" will follow
> 

[naviserver-devel] HTTP client: security by default

[Gustaf N. (sslmail) <ne...@wu...>]

Dear all,

This mail is a warning, that after an upgrade to the newest version from git, some of your configuration files/applications might need adjustments,

The newest commit to the main repository brings more security, by validating in ns_http client requests always the server certificate. While this is common for browsers, this is not necessarily the case for automated web operations (web services etc.), opening a large attack vector for man-in-the-middle attacks (see below). So far, checking the certificates was per default deactivated, which has the consequence, that in practice, many (most?) application developers just used with the defaults, leading to an unpleasant situation. Now, certificated checking is activated, it can be deactivated by adding the parameter “-insecure” to an “ns_http” request (there are other means, see below).

The regression test (including self-signed certificates for HTTPS, revproxy, …) works fine.

Next steps for me:
 - check consequences for letsencrypt  
 - add similar changes to “ns_connchan open|connect"
 - documentation of certificate management in detail (covering ns_http, ns_connchan, admin-config)

We are getting closer to the release candiate...

all the best
-gn


Security by default for HTTP client requests (ns_http)

Background: Authenticating the peer is a critical part of SSL connection setup when a client connects to a server. In this process, the server presents its public-key certificate. If the results of this authentication is missing, a server might be vulnerable to man-in-the-middle attacks. Protection against this is one of the intentions of HTTPS, and becomes more important when backend transactions are performed via HTTPS (administration of servers, cloud infrastructure, payments, ...), establishing secure network tunnels, or when sending confidential information via HTTPs

As the paper mentioned below points out, the validation of peer certificates is usually in place for web communication over the browser, but it is often not active for web service interactions.  One reason for this is that application developers choose very often the default setup.

The key instrument of NaviServer for performing HTTP client operations is ns_http, which certainly has the ability for certificate validation, but so far it was per default deactivated. This changes now with NaviServer 5, where the default is now including validation.

One of the consequences of validating peer certificates is that the administrator has to care about certificates (what certificates are accepted), including management of self-signed certificates.

We try to make the transition as smooth a possible by providing a reasonable default setup including established root certificates (ca-bundle.crt), providing simple means to add your own trusted certificates (including self-signed certificates). This default setup can be altered for a server via the configuration file, or for every single request via parameters.

The most important changes are:

- new parameter "-insecure" for "ns_http run" and "ns_http queue"
  This parameter turns off certificate validation for the target HTTPS server. The name follows the naming convention of curl.
  
- The old parameter "-validate" is a now a no-op.
  Per default, all ns_http requests to HTTPS servers are now validated

- Store certificates in directory ${NSHOME}/certificates instead of ${NSHOME}/etc

- New configuration parameter "CApath" and "CAfile" for the "httpclient" section of a server to specify default locations for certificate validation. It is possible to override these values via parameters to "ns_http run" and "ns_http queue".

  Default configuration:

     ns/${server}/httpclient {
        ns_param CAfile ... ;# default ${NSHOME}/ca-bundle.crt
        ns_param CApath ... ;# default ${NSHOME}/certificates
     }

  * "CAfile" points to a .pem file containing established root certificates, often named "ca-bundle.crt". This file contains multiple of these certificates. The version shipped with NaviServer is based on Mozilla's root certificates. Also, many operating systems provide these certificates (e.g., on Ubuntu /etc/ssl/certs/ca-certificates.crt). One can certainly use various sources, e.g. via symbolic links or configuration parameters. The file can be manually updated e.g. via 

          curl https://curl.se/ca/cacert.pem -o /usr/local/ns/ca-bundle.crt
  
  * "CApath" points to a directory containing CA certificates in PEM
     format. Each of the files must contain exactly one CA certificate (OpenSSL requirement).
  
     It is possible to add self-signed certificates to this directory to connect via ns_http with an HTTPS URL to this server, after running
  
         openssl rehash ${NSHOME}/certificates

     The rehash operation is performed automatically in the installation process.

- New configuration parameter "insecure" for the "httpclient" section of a server

     The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software
     https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf 

- Added certificate settings to sample configuration files nsd-config.tcl and openacs-config.tcl

- Minor cleanup of Makefiles (improve consistency and configurability)

- similar changes for "ns_connchan connect|open" will follow

Re: [naviserver-devel] Appending to an multi-element dictonary

[Not S. <un...@cr...>]

Hi Gustaf,

It does indeed. All those emails from me can be ignored.

Thank You,
David F


-----Original Message-----
From: Gustaf <ne...@wu...>
To: naviserver-devel <nav...@li...>
Date: Thursday, 16 January 2025 8:48 AM GMT
Subject: Re: [naviserver-devel] Appending to an multi-element dictonary


Dear David, 

nsv_dict has mostly the same interface as the Tcl dict.
Wasn’t the question answered sufficiently by Harald in 

    https://core.tcl-lang.org/tcl/tktview/47fd91176d  ?

All the best
-g

> On 15.01.2025, at 20:27, Not Spam <un...@cr...> wrote:
> 
> Good evening,
>  
> This may be more a coding issue but I'm not sure if this is by design, limited or programmed to, or that if you shouldn't be able to.
> However I've started heavily using dictionaries within as a in-house data-store and taken it out of context of the standard dictionary key-value design. 
> On count, I am currently creating an dictionary of seven elements.
>  
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "1" "Value 1"
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2" "Value 2"
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "3" "Value 3"
>  
> nsv_dict get "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2"
>     Result: Value 2
>  
> and full store:
> nsv_dict get "MyDictonary" "Store"
>     Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}}
>  
> but if I was to append a forth value to this:
>    nsv_dict append "MyDictonary" "Store" "Set" "Block" "Key" "Field" "4" "Value 4"
>  
> this sets up the next result as:
> Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}BlockKeyField4Value 4}
>  
> And that then anything afterwards throws the following error:
> 
> [15/Jan/2025:19:12:11][17646.3ae1efc15100][-conn:default:default:1:1-] Error: GET /dictTest.ix, PeerAddress: 10.2.1.77
> :    dict element in braces followed by "BlockKeyField4Value" instead of space
>  
> My workaround which is of obtaining the result and than appending the new data to that and then setting the appended string as the new value to the same dictionary but it feels klunky.
> I'm not sure how to combat this apart from above. 
>  
> Should appending suppose to work?
>  
> P.S: however as just testing after writing this email, It appears to occur in TCL too.
>   % dict set MyDictonary store set block key field 1 "hello"
> store {set {block {key {field {1 hello}}}}}
>   % dict set MyDictonary store set block key field 2 "hello"
> store {set {block {key {field {1 hello 2 hello}}}}}
>   % dict append MyDictonary store set block key field 3 "hello"
> store {set {block {key {field {1 hello 2 hello}}}}setblockkeyfield3hello}
>  
> Many Thanks,
> David F
>  
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Appending to an multi-element dictonary

[Gustaf N. (sslmail) <ne...@wu...>]

Dear David, 

nsv_dict has mostly the same interface as the Tcl dict.
Wasn’t the question answered sufficiently by Harald in 

    https://core.tcl-lang.org/tcl/tktview/47fd91176d  ?

All the best
-g

> On 15.01.2025, at 20:27, Not Spam <un...@cr...> wrote:
> 
> Good evening,
>  
> This may be more a coding issue but I'm not sure if this is by design, limited or programmed to, or that if you shouldn't be able to.
> However I've started heavily using dictionaries within as a in-house data-store and taken it out of context of the standard dictionary key-value design. 
> On count, I am currently creating an dictionary of seven elements.
>  
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "1" "Value 1"
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2" "Value 2"
>     nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "3" "Value 3"
>  
> nsv_dict get "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2"
>     Result: Value 2
>  
> and full store:
> nsv_dict get "MyDictonary" "Store"
>     Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}}
>  
> but if I was to append a forth value to this:
>    nsv_dict append "MyDictonary" "Store" "Set" "Block" "Key" "Field" "4" "Value 4"
>  
> this sets up the next result as:
> Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}BlockKeyField4Value 4}
>  
> And that then anything afterwards throws the following error:
> 
> [15/Jan/2025:19:12:11][17646.3ae1efc15100][-conn:default:default:1:1-] Error: GET /dictTest.ix, PeerAddress: 10.2.1.77
> :    dict element in braces followed by "BlockKeyField4Value" instead of space
>  
> My workaround which is of obtaining the result and than appending the new data to that and then setting the appended string as the new value to the same dictionary but it feels klunky.
> I'm not sure how to combat this apart from above. 
>  
> Should appending suppose to work?
>  
> P.S: however as just testing after writing this email, It appears to occur in TCL too.
>   % dict set MyDictonary store set block key field 1 "hello"
> store {set {block {key {field {1 hello}}}}}
>   % dict set MyDictonary store set block key field 2 "hello"
> store {set {block {key {field {1 hello 2 hello}}}}}
>   % dict append MyDictonary store set block key field 3 "hello"
> store {set {block {key {field {1 hello 2 hello}}}}setblockkeyfield3hello}
>  
> Many Thanks,
> David F
>  
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Appending to an multi-element dictonary

[Not S. <un...@cr...>]

Sorry, I mis-explained, what I mean I wish to append to value 4 to index 2
nsv_dict append "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2" "Value 4" 

Set {Block {Key {Field {1 {Value 1} 2 {Value 2 Value 4} 3 {Value 3}}}}} 

-----Original Message-----
From: Not <un...@cr...>
To: naviserver-devel <nav...@li...>
Date: Wednesday, 15 January 2025 7:46 PM GMT
Subject: [naviserver-devel] Appending to an multi-element dictonary


 
Good evening, 
  
This may be more a coding issue but I'm not sure if this is by design, limited or programmed to, or that if you shouldn't be able to. 
However I've started heavily using dictionaries within as a in-house data-store and taken it out of context of the standard dictionary key-value design.  
On count, I am currently creating an dictionary of seven elements. 
  
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "1" "Value 1" 
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2" "Value 2" 
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "3" "Value 3" 
  nsv_dict get "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2"
 
  
    Result: Value 2 
  
and full store: 
nsv_dict get "MyDictonary" "Store" 
    Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}} 
  
but if I was to append a forth value to this: 
   nsv_dict append "MyDictonary" "Store" "Set" "Block" "Key" "Field" "4" "Value 4" 
  
this sets up the next result as: 
Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}BlockKeyField4Value 4} 
  
And that then anything afterwards throws the following error: 

[15/Jan/2025:19:12:11][17646.3ae1efc15100][-conn:default:default:1:1-] Error: GET /dictTest.ix, PeerAddress: 10.2.1.77
:    dict element in braces followed by "BlockKeyField4Value" instead of space 
  
My workaround which is of obtaining the result and than appending the new data to that and then setting the appended string as the new value to the same dictionary but it feels klunky. 
I'm not sure how to combat this apart from above.  
  
Should appending suppose to work? 
  
 
P.S: however as just testing after writing this email, It appears to occur in TCL too. 
  % dict set MyDictonary store set block key field 1 "hello"
store {set {block {key {field {1 hello}}}}}
  % dict set MyDictonary store set block key field 2 "hello"
store {set {block {key {field {1 hello 2 hello}}}}}
  % dict append MyDictonary store set block key field 3 "hello"
store {set {block {key {field {1 hello 2 hello}}}}setblockkeyfield3hello}  
  
Many Thanks, 
David F 
 
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Appending to an multi-element dictonary

[Not S. <un...@cr...>]

 
Good evening, 
  
This may be more a coding issue but I'm not sure if this is by design, limited or programmed to, or that if you shouldn't be able to. 
However I've started heavily using dictionaries within as a in-house data-store and taken it out of context of the standard dictionary key-value design.  
On count, I am currently creating an dictionary of seven elements. 
  
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "1" "Value 1" 
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2" "Value 2" 
    nsv_dict set "MyDictonary" "Store" "Set" "Block" "Key" "Field" "3" "Value 3" 
  nsv_dict get "MyDictonary" "Store" "Set" "Block" "Key" "Field" "2"
 
  
    Result: Value 2 
  
and full store: 
nsv_dict get "MyDictonary" "Store" 
    Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}} 
  
but if I was to append a forth value to this: 
   nsv_dict append "MyDictonary" "Store" "Set" "Block" "Key" "Field" "4" "Value 4" 
  
this sets up the next result as: 
Set {Block {Key {Field {1 {Value 1} 2 {Value 2} 3 {Value 3}}}}BlockKeyField4Value 4} 
  
And that then anything afterwards throws the following error: 

[15/Jan/2025:19:12:11][17646.3ae1efc15100][-conn:default:default:1:1-] Error: GET /dictTest.ix, PeerAddress: 10.2.1.77
:    dict element in braces followed by "BlockKeyField4Value" instead of space 
  
My workaround which is of obtaining the result and than appending the new data to that and then setting the appended string as the new value to the same dictionary but it feels klunky. 
I'm not sure how to combat this apart from above.  
  
Should appending suppose to work? 
  
 
P.S: however as just testing after writing this email, It appears to occur in TCL too. 
  % dict set MyDictonary store set block key field 1 "hello"
store {set {block {key {field {1 hello}}}}}
  % dict set MyDictonary store set block key field 2 "hello"
store {set {block {key {field {1 hello 2 hello}}}}}
  % dict append MyDictonary store set block key field 3 "hello"
store {set {block {key {field {1 hello 2 hello}}}}setblockkeyfield3hello}  
  
Many Thanks, 
David F 
 

Re: [naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Wolfgang W. <wol...@di...>]

We will still need some testing, but it shouldn't take too long.

The code depends on our session handling, but it can be refactored to 
include more generic procedures and an example using NSV as session storage.

The result can be made available on GitHub.

Cheers, Wolfgang

Am 03.01.25 um 12:49 schrieb Gustaf Neumann (sslmail):
>
>
>> Wow, this was fast.
> A good architecture pays off in such cases!
>>
>> If anyone is interested in Passkeys and WebAuthn: I can provide the 
>> Javascript/Tcl/Naviserver details. Integration in any kind of 
>> framework should be quite easy.
>>
>
> Yes, please! I think, this would be very helpful for many.
> If this is appropriate, i could create a modules in the naviserver 
> space on github and provide you with the necessary rights…
>
> all the best -g
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
-- 

Re: [naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Gustaf N. (sslmail) <ne...@wu...>]

> Wow, this was fast.
A good architecture pays off in such cases!
> If anyone is interested in Passkeys and WebAuthn: I can provide the Javascript/Tcl/Naviserver details. Integration in any kind of framework should be quite easy.
> 

Yes, please! I think, this would be very helpful for many.
If this is appropriate, i could create a modules in the naviserver space on github and provide you with the necessary rights…

all the best -g

Re: [naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Wolfgang W. <wol...@di...>]

Wow, this was fast. It was meant as a question but it's really nice to have.

If anyone is interested in Passkeys and WebAuthn: I can provide the 
Javascript/Tcl/Naviserver details. Integration in any kind of framework 
should be quite easy.

Thank you,

Wolfgang Winkler

Am 03.01.25 um 10:43 schrieb Gustaf Neumann (sslmail):
>
>>> For this I currently write the .pem file to disk (after conversion from .der to .pem). Is it possible to circumvent this? We store the public key data in the database and it would be nice to use it as string.
>> If this is meant as a feature requests, ….
> In the updated version of NaviServer, one can now provide PEM content as strings, wherever PEM content is used for reading. Examples are in the regression test.
> All the best
> .g
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Gustaf N. (sslmail) <ne...@wu...>]

>> For this I currently write the .pem file to disk (after conversion from .der to .pem). Is it possible to circumvent this? We store the public key data in the database and it would be nice to use it as string.
> 

> If this is meant as a feature requests, ….

In the updated version of NaviServer, one can now provide PEM content as strings, wherever PEM content is used for reading. Examples are in the regression test.
All the best 
.g

Re: [naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Gustaf N. (sslmail) <ne...@wu...>]

> For this I currently write the .pem file to disk (after conversion from .der to .pem). Is it possible to circumvent this? We store the public key data in the database and it would be nice to use it as string.

if this is meant as a question, the answer is no, currently NaviServer is using the file-based OpenSSL API.
If this is meant as a feature requests, …. i had this as well on my agenda but no urge for it, since writing a PEM to the file is pretty fast and can be done on the Tcl layer as well.

-g

[naviserver-devel] WebAuthn for NaviServer - ns_crypto::md with PEM as string

[Wolfgang W. <wol...@di...>]

Dear all,

I am currently writing a client for integrating webauthn into 
naviserver. It's a bunch of Javascript, a lot of encoding/decoding and 
some crypthography.

The various kinks are ironed out, and I can register users and verify 
webauthn.get responses.

Here's my question: I am using

::ns_crypto::md string -digest sha256 -verify ${req}.pem -signature 
$signature -binary $data_to_sign

to verify the signature of the data I get from the webauthn provider.

For this I currently write the .pem file to disk (after conversion from 
.der to .pem). Is it possible to circumvent this? We store the public 
key data in the database and it would be nice to use it as string.

Thanks for your input,

Wolfgang Winkler

-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] Proposal: config file boolean parameter "showserverheader" to show/hide "Server:" response header

[Brian F. <bri...@ai...>]

Hello

This is still an issue for us. Would this be a possible feature to add?

thanks
Brian

________________________________
From: Brian Fenton <bri...@ai...>
Sent: Friday 27 October 2023 5:56 pm
To: nav...@li... <nav...@li...>
Subject: [naviserver-devel] Proposal: config file boolean parameter "showserverheader" to show/hide "Server:" response header

Hi all

Recently during a client security audit, the "Server: NaviServer/4.99.28" response header was flagged as an issue. The client has asked us to remove the header, if possible.
The RFC suggests that the "Server: " header is optional, so I believe this should be OK to remove. https://www.rfc-editor.org/rfc/rfc7231#section-7.4.2

We would like to propose a new config file boolean parameter "showserverheader" with default true.

Ns_ConnConstructHeaders in return.c could then check this parameter before outputting the "Server: " header e.g. something like this:

    if (Ns_ConfigBool(path, "showserverheader", NS_TRUE) == NS_TRUE) {
      Ns_DStringVarAppend(dsPtr, "Server: ", Ns_InfoServerName(), "/", Ns_InfoServerVersion(), "\r\n");
    }

Thoughts? Alternatives?

thanks
Brian
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Cleanup progress report

[Gustaf N. (sslmail) <ne...@wu...>]

I have addressed the work items below, all changes are in the repository. The “keyl*” commands are now deprecated.

Furthermore, i added a section to admin-tuning explaining the spooling threads (in essence the driver and writer threads) and worked as well through the other sections, removed some SGI stuff, etc.
https://naviserver.sourceforge.io/5.0/manual/files/admin-tuning.html

Altogether, we have 90 commands/subcommands deprecated. I am still planning for the release to add an option to build NaviServer without the deprecated commands (default is with deprecated commands). Why should every NaviServer binary carry these around? Esp. for constraint environments, this is not desired.

All the best
-g

> The statistics refer just to the commands starting with “ns” to leave out some of the internals, which are not too many.
> We have 6 C implemented commands not starting with the usual NaviServer prefix, and one command which was intentionally not documented:
> 
> - I will document “ns_crash”, If no one objects.
> 
> - the keyl* commands are inherited from TclX. From my point of view, there is little reason to use in new programs, since it resembles in essence the Tcl “dict” functionality with a different syntax.
>    (correct me, if I'm wrong am wrong). 
> 
>    I see the following options
>     * deprecate it 
>     * put it into a separate module
>     * drop it, since tclx is still maintained by flightaware (https://github.com/flightaware/tclx/), so people can load it via “package require”.
> 
>   I am in favor of deprecating it in NaviServer 5, such that users get warned about its usage.
> 
>  - _ns_adp_include: In my understanding, the only reason for having a command “ns_adp_include” and a separate command “_ns_adp_include” is the proc call-frame on the top-level to keep local variables. It should be possible to create this call-frame via the public API of Tcl.  I will look into this. 
> 
> As always, feedback is welcome.
> 
> -g

Re: [naviserver-devel] Cleanup progress report

[Gustaf N. (sslmail) <ne...@wu...>]

> 
> ------------> 48 occurrences of deprecated commands in documentation
> 
> so, cleanup item 6 is
> 
> 6) documentation should not be based on deprecated commands
> 


The mentioned issues are fixed.

Here is the summary of the current state of the cleanup:

Tested commands
------------> 709 tested commands, 86 deprecated, 639 documented, 0 NOT documented
------------> 0 usage message differ from doc, 639 are identical

Documented commands
------------> 639 documented commands, 639 tested, 0 NOT tested

Deprecated commands
------------> 0 deprecated commands are documented
------------> 0 occurrences of deprecated commands in documentation
------------> 38 deprecated commands are tested

C-implemented commands:
C-implemented command is not documented: _ns_adp_include
C-implemented command is not documented: exit
C-implemented command is not documented: chilled
C-implemented command is not documented: keylget
C-implemented command is not documented: keylkeys
C-implemented command is not documented: keylset
C-implemented command is not documented: ns_crash

The statistics refer just to the commands starting with “ns” to leave out some of the internals, which are not too many.
We have 6 C implemented commands not starting with the usual NaviServer prefix, and one command which was intentionally not documented:

- I will document “ns_crash”, If no one objects.

- the keyl* commands are inherited from TclX. From my point of view, there is little reason to use in new programs, since it resembles in essence the Tcl “dict” functionality with a different syntax.
   (correct me, if I'm wrong am wrong). 

   I see the following options
    * deprecate it 
    * put it into a separate module
    * drop it, since tclx is still maintained by flightaware (https://github.com/flightaware/tclx/), so people can load it via “package require”.

  I am in favor of deprecating it in NaviServer 5, such that users get warned about its usage.

 - _ns_adp_include: In my understanding, the only reason for having a command “ns_adp_include” and a separate command “_ns_adp_include” is the proc call-frame on the top-level to keep local variables. It should be possible to create this call-frame via the public API of Tcl.  I will look into this. 

As always, feedback is welcome.

-g

Re: [naviserver-devel] Cleanup progress report

[Gustaf N. (sslmail) <ne...@wu...>]

Many thanks for the feedback. There are actually multiple forks around. i’ve tried to summarize the situation about the availability in the new man page and added basic support for NaviServer 5. Based on the implementation, NaviServer sever supported the TclPro Debugger (function missing, the leftovers from AOLserver were broken, etc.). 

Details for the state and installation is now in the man page:

https://naviserver.sourceforge.io/5.0/naviserver/files/ns_adp_debug.html

all the best
-g

> On 07.12.2024, at 22:40, Georg Lehner <jor...@ma...> wrote:
> FWIW: At some moment Flighaware issued a bounty on reactivating the TclPro Debugger. I found an email from Steve Bennet from 3rd of July of 2017, when he announces that they made it available on github in the repo flightaware/TclProDebug. That's all I can add. I don't use debuggers a lot and have not used TclProDebug.
> 
> Best Regards,
> 
>   Georg
> 

Re: [naviserver-devel] Cleanup progress report

[Georg L. <jor...@ma...>]

Hello Gustaf,

Your dedication is impressive, thank you!

FWIW: At some moment Flighaware issued a bounty on reactivating the 
TclPro Debugger. I found an email from Steve Bennet from 3rd of July of 
2017, when he announces that they made it available on github in the 
repo flightaware/TclProDebug. That's all I can add. I don't use 
debuggers a lot and have not used TclProDebug.

Best Regards,

   Georg


On 12/4/24 13:57, Gustaf Neumann (sslmail) wrote:
> Dear fellow NaviServer users and developers,
>
> this is a progress report on the cleanup. Points 1-5 are done, no all 
> the parameters implemented are also documented in the manual pages, 
> the last days I fixed the documentation of 330 API calls, where the 
> implementation and the documentation differed.
>
>> >   1) all commands tested in the regression test should be documented
>> >   2) all documented commands should be tested (at least syntax-tested)
>> >   3) all implemented commands should be tested
>> >   4) the parameters of the implemented commands should be 
>> documented and vice versa.
>> >   5) deprecated commands should be listed in the deprecated section 
>> of the command listing in the documentation, but not advertised in 
>> the documentation
>
> While going through the man pages, I had to rewrite several of these, 
> since it was describing what the documentation does, and sometimes, it 
> was not clear to me, what the commands were exactly about, and how it 
> can/should be used. Therefore, I added several examples that should 
> help users. The mostly rewritten man pages include:
>
>     https://naviserver.sourceforge.io/5.0/naviserver/files/ns_adp_register.html
>     https://naviserver.sourceforge.io/5.0/naviserver/files/ns_set.html
>     https://naviserver.sourceforge.io/5.0/naviserver/files/ns_upload_stats.html
>     https://naviserver.sourceforge.io/5.0/naviserver/files/ns_urlspace.html
>
>
> These are links to the new NaviServer 5 man pages. Please check.
> In case, you are aware of more unclear documentation, or you would 
> like to see usage examples in the manual please, please contact me. I 
> will see what i can do.
>
> As discussed before, I have marked the former TDB calls mentioned 
> before as deprecated:
>
>         ns_browsermatch
>         ns_choosecharset
>         ns_cookiecharset
>         ns_formfieldcharset
>         ns_formvalueput
>         ns_paren
>         ns_tagelement
>         ns_tagelementset
>
> Based on the no overwhelming feedback concerning
>
>     ns_requestauthorize
>     ns_checkurl
>
>
> My plan is to mark the latter one as deprecated and update the 
> documentation of the first one.
>
> The commands which are deprecated are not advertised as commands in 
> man pages anymore. However, I realized still some long-time burden: 
> Some  parts of the documentation are still referring to deprecated 
> commands.
>
>     ------------> 48 occurrences of deprecated commands in documentation
>
>
> so, cleanup item 6 is
>
>     6) documentation should not be based on deprecated commands
>
>
> After having modified a few hundred man pages, these figures don’t 
> shock me anymore, … but it will need again some more time.
>
> One more API call requiring decisions is
>
>     ns_adp_debug
>
>
> The call provides an interface to the TclPro debugger. I don’t have 
> it, so i can’t test it. Has anyone experiences with this?
>
> After digging around the state is:
> - After Scriptics went down the TclPro suite of tools became the 
> ActiveState TclDevKit suite of tools
> - After ActiveState stopped managing the TDK, all the sources became 
> open https://github.com/activestate/tdk
> - There are a few forks including 
> https://github.com/puremourning/TclProDebug and 
> https://github.com/apnadkarni/TclProDebug
>
> The newer forks show some activity, the fork of Ashok has Tcl9 support 
> (but Ashok is not using it).
>
> My current state of mind is that we keep maintaining the interface, we 
> improve on the documentation and i’ll check, if the interfaces are 
> still the same.
>
> All the best
> -g
>
> PS: I got several other mails concerning NaviServer. These are not 
> ignored, i will come back to you on these.
>
>>
>> At least 1 and 2 are finished by today. This week, i have added over 
>> 700 tests to the regression test, which was hard work and took all my 
>> resources. We have now 2608 tests in the NaviServer 5 regression test 
>> vs. 1893 in the newest 4.99 branch:
>>
>>      % egrep -r --include=*.test '^test ' /usr/local/src/naviserver/
>>     |wc -l
>>      2608
>>
>>      % egrep -r --include=*.test '^test '
>>     /usr/local/src/naviserver-4.99/ |wc -l
>>      1893
>>
>>
>> The current state is not perfect, but a progress.
>>
>>     Tested commands not documented
>>     ------------> 706 tested commands, 72 deprecated,  648
>>     documented, 0 NOT documented
>>
>>     Documented commands not tested
>>     ------------> 648 documented commands, 648 tested, 0 NOT tested
>>
>>     Deprecated commands which are documented
>>     ------------> 0 deprecated commands are documented
>>
>>
>> The next goal is to check the alignment of the argument lists of the 
>> implementation with the documentation.
>>
>> On my wishlist is also a non-deprecated mode switchable via 
>> configuration file, where NaviServer offers only non-deprecated commands.
>>
>> In the meanwhile, something to think about for you: We have still a 
>> bunch of commands, which have a questionable future. These commands 
>> are in a "TBD" state for 19 years.
>>
>> I think, we should deprecate these commands (see below for the list) 
>> in the NaviServer 5 release.
>> Please speak up, if your applications need these:
>>
>>     ns_browsermatch
>>     ns_choosecharset
>>     ns_cookiecharset
>>     ns_formfieldcharset
>>     ns_formvalueput
>>     ns_paren
>>     ns_tagelement
>>     ns_tagelementset
>>
>> All the best
>> -g
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Cleanup progress report

[Gustaf N. (sslmail) <ne...@wu...>]

Dear fellow NaviServer users and developers,

this is a progress report on the cleanup. Points 1-5 are done, no all the parameters implemented are also documented in the manual pages, the last days I fixed the documentation of 330 API calls, where the implementation and the documentation differed.

> >   1) all commands tested in the regression test should be documented
> >   2) all documented commands should be tested (at least syntax-tested)
> >   3) all implemented commands should be tested
> >   4) the parameters of the implemented commands should be documented and vice versa.
> >   5) deprecated commands should be listed in the deprecated section of the command listing in the documentation, but not advertised in the documentation

While going through the man pages, I had to rewrite several of these, since it was describing what the documentation does, and sometimes, it was not clear to me, what the commands were exactly about, and how it can/should be used. Therefore, I added several examples that should help users. The mostly rewritten man pages include:

https://naviserver.sourceforge.io/5.0/naviserver/files/ns_adp_register.html
https://naviserver.sourceforge.io/5.0/naviserver/files/ns_set.html
https://naviserver.sourceforge.io/5.0/naviserver/files/ns_upload_stats.html
https://naviserver.sourceforge.io/5.0/naviserver/files/ns_urlspace.html

These are links to the new NaviServer 5 man pages. Please check. 
In case, you are aware of more unclear documentation, or you would like to see usage examples in the manual please, please contact me. I will see what i can do.

As discussed before, I have marked the former TDB calls mentioned before as deprecated:

        ns_browsermatch
        ns_choosecharset
        ns_cookiecharset
        ns_formfieldcharset
        ns_formvalueput
        ns_paren
        ns_tagelement
        ns_tagelementset

Based on the no overwhelming feedback concerning

ns_requestauthorize
ns_checkurl

My plan is to mark the latter one as deprecated and update the documentation of the first one.

The commands which are deprecated are not advertised as commands in man pages anymore. However, I realized still some long-time burden: Some  parts of the documentation are still referring to deprecated commands.

------------> 48 occurrences of deprecated commands in documentation

so, cleanup item 6 is

6) documentation should not be based on deprecated commands

After having modified a few hundred man pages, these figures don’t shock me anymore, … but it will need again some more time.

One more API call requiring decisions is 

ns_adp_debug

The call provides an interface to the TclPro debugger. I don’t have it, so i can’t test it. Has anyone experiences with this?

After digging around the state is:
- After Scriptics went down the TclPro suite of tools became the ActiveState TclDevKit suite of tools
- After ActiveState stopped managing the TDK, all the sources became open https://github.com/activestate/tdk
- There are a few forks including https://github.com/puremourning/TclProDebug and https://github.com/apnadkarni/TclProDebug

The newer forks show some activity, the fork of Ashok has Tcl9 support (but Ashok is not using it).

My current state of mind is that we keep maintaining the interface, we improve on the documentation and i’ll check, if the interfaces are still the same.

All the best
-g

PS: I got several other mails concerning NaviServer. These are not ignored, i will come back to you on these.

> 
> At least 1 and 2 are finished by today. This week, i have added over 700 tests to the regression test, which was hard work and took all my resources. We have now 2608 tests in the NaviServer 5 regression test vs. 1893 in the newest 4.99 branch:
> 
>  % egrep -r --include=*.test '^test ' /usr/local/src/naviserver/ |wc -l
>  2608
> 
>  % egrep -r --include=*.test '^test ' /usr/local/src/naviserver-4.99/ |wc -l
>  1893
> 
> The current state is not perfect, but a progress.
> 
> Tested commands not documented
> ------------> 706 tested commands, 72 deprecated,  648 documented, 0 NOT documented
> 
> Documented commands not tested
> ------------> 648 documented commands, 648 tested, 0 NOT tested
> 
> Deprecated commands which are documented
> ------------> 0 deprecated commands are documented
> 
> The next goal is to check the alignment of the argument lists of the implementation with the documentation.
> 
> On my wishlist is also a non-deprecated mode switchable via configuration file, where NaviServer offers only non-deprecated commands.
> 
> In the meanwhile, something to think about for you: We have still a bunch of commands, which have a questionable future. These commands are in a "TBD" state for 19 years.
> 
> I think, we should deprecate these commands (see below for the list) in the NaviServer 5 release.
> Please speak up, if your applications need these:
> 
>     ns_browsermatch
>     ns_choosecharset
>     ns_cookiecharset
>     ns_formfieldcharset
>     ns_formvalueput
>     ns_paren
>     ns_tagelement
>     ns_tagelementset
> 
> All the best
> -g

Re: [naviserver-devel] Issue with Log file in version 4.99.31-1

[Gustaf N. (sslmail) <ne...@wu...>]

> On 29.11.2024, at 10:09, Sassy Natan <sa...@gm...> wrote:
> 
> Hi Group,
> 
> I'm facing an issue with the way my log file is present with the new version.
> In the previous version the log file didn't include TAB or other special characters.


What was your previous version? The logfile sanitation is in place since 2019.

One can specify the behavior via the global parameter sanitizelogfiles.
The value of 0 means that no sanitize should happen.

The value 1 means that full sanitizing is in place, which means that non-printable characters are written as hex values.

The sanitize value of 2 means that sanitizing happens in a human friendly fashion: newlines are preserved, but sanitizing makes sure that multiline content cannot not confused with different log file entries. After a new-line character a continuation character starts the new line followed by 4 spaces.


The reason for printing hex representation for non-printable characters into the log file were security audits, there exist several nasty attacks (but probably not with the tab character).

An easy strategy without changing the logic in NaviServer is to remove tabs from the source, or using a postprocessing filter (a sed one-liner).

-g

[naviserver-devel] Issue with Log file in version 4.99.31-1

[Sassy N. <sa...@gm...>]

Hi Group,

I'm facing an issue with the way my log file is present with the new
version.
In the previous version the log file didn't include TAB or other special
characters.

See for example the following:

[29/Nov/2024:03:19:23][41803.75671fe006c0][-sched:0:3275:3-] Debug:
nsdbpg(log): call PQexec with <
:            delete from sec_sessions
:                  where  user_id is null or session_id in (
:                  select session_id  from sec_sessions s
:         * \x09   *     join wt_users_new fr on (s.user_id =
fr.employee_id)
:                  join wt_login_manager m using (sid)
:                  where   1732850363 - last_hit > case when
 status_code=101 then coalesce(min_session_timeout,600) else 14400 end );>
[29/Nov/2024:03:19:23][41803.75671fe006c0][-sched:0:3275:3-] Debug(sql):
pool log duration 0.023254 secs: '
:            delete from sec_sessions
:                  where  user_id is null or session_id in (
:                  select session_id  from sec_sessions s
:        *  \x09 *       join wt_users_new fr on (s.user_id =
fr.employee_id)
:                  join wt_login_manager m using (sid)
:                  where   1732850363 - last_hit > case when
 status_code=101 then coalesce(min_session_timeout,600) else 14400 end )
:        '
Is there any way to remove this? via configuration?
The logs are being collected by another system we have, and it breaks the
phrasing.

Maybe there is some config magic?




-- 
Regards,

Sassy Natan
972-(0)54-2203702

Re: [naviserver-devel] ns_set reform

[Sassy N. <sa...@gm...>]

Thank you

Regards,

Sassy Natan
972-(0)54-2203702


On Wed, 27 Nov 2024 at 10:00 Gustaf Neumann (sslmail) <ne...@wu...>
wrote:

> Progress report from the documentation cleanup;
>
> >   4) the parameters of the implemented commands should be documented and
> vice versa.
>
> Some first numbers, worse than expected:
>
> From the currently 464 documented commands, the arguments in the source
> code differ in more than 50 % of the cases
> ------------> 330 usage messages differ from doc, 316 are identical
>
>
> Some of the changes are minimal (like different argument names), some of
> these are not (missing or too many options/arguments/…).
> One has to go through this case by case, this will take some time, …
>
> I started already to work on this. The strategy is that In case of doubt,
> implementation has higher weight.
>
> > When is the plan to release version 5.0?
>
> The plan is this year or early next year
>
> -g
>
>
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] ns_set reform

[Gustaf N. (sslmail) <ne...@wu...>]

Progress report from the documentation cleanup;

>   4) the parameters of the implemented commands should be documented and vice versa.

Some first numbers, worse than expected:

From the currently 464 documented commands, the arguments in the source code differ in more than 50 % of the cases
------------> 330 usage messages differ from doc, 316 are identical

Some of the changes are minimal (like different argument names), some of these are not (missing or too many options/arguments/…).
One has to go through this case by case, this will take some time, … 

I started already to work on this. The strategy is that In case of doubt, implementation has higher weight.

> When is the plan to release version 5.0?

The plan is this year or early next year

-g

[naviserver-devel] Deprecation of Legacy Functions and Introduction of Modernization Script

[Gustaf N. (sslmail) <ne...@wu...>]

> I see that we are still using ns_formvalueput in some very old, but still used, code. 

I noticed that OpenACS is still using "ns_formvalueput" in some legacy packages within the OpenACS repository.

To help identify and manage outdated functions, I have created a small utility script available on my GitHub:

    https://github.com/gustafn/ns-modernizer/
    
How the Script Works:

The script recursively scans .tcl files in a directory tree, flags deprecated commands, and optionally replaces some of the simpler ones automatically. Here’s an example of its output:

----- /usr/local/oacs-head/openacs-4/packages/dotlrn-ecommerce/tcl/dotlrn-ecommerce-init.tcl
use of deprecated command: 'ns_register_adptag'
... updating ./dotlrn-ecommerce/tcl/dotlrn-ecommerce-init.tcl (1 changes)

----- /usr/local/oacs-head/openacs-4/packages/imsld/www/admin/imsld-export-xo.tcl
use of deprecated command: 'ns_tmpnam'
... updating ./imsld/www/admin/imsld-export-xo.tcl (1 changes)

The full README on GitHub provides additional details, including how to enable the "change mode" for automatic replacements.


Next Steps for Deprecation

Based on feedback, I plan to mark some TBD functions as deprecated. However, this does not imply their immediate removal; they will remain available for the foreseeable future.

Additionally, I identified two more functions with unclear distinctions that I’ve added to the TBD section:

    ns_requestauthorize
    ns_checkurl

Both functions are implemented using the same core method (NsTclRequestAuthorizeObjCmd) but have different manual pages [1, 2]. My current understanding is:

    NaviServer allows authentication procs to be registered at the C level, used primarily by the nsperm module.
    These two noted Tcl functions allow manual invocation of this functionality, which can be useful for testing or specific scenarios.
    However, having two separate functions seems unnecessary.

Proposal

I suggest keeping "ns_requestauthorize" because its name better reflects its purpose and marking "ns_checkurl" as deprecated. However, I propose preserving the more concise documentation from "ns_checkurl" for clarity with the other name.

Since OpenACS uses its own database-driven permission and authority management system and does not rely on "nsperm", I lack direct experience with these commands.

I welcome your insights, especially from anyone more familiar with nsperm or related functionality.

all the best
-g

[1] https://naviserver.sourceforge.io/n/naviserver/files/ns_requestauthorize.html
[2] https://naviserver.sourceforge.io/n/naviserver/files/ns_checkurl.html