naviserver-devel — Developer discussion

Re: [naviserver-devel] Lots of ns:mutex entries

[Gustaf N. <ne...@wu...>]

Great. Glad that the new version helped to locate issues. -g

On 23.05.18 15:11, Wolfgang Winkler wrote:
>
> Hello!
>
> We fixed the issues. The mutexes were indeed created without 
> destroying them.
>
> This version is our development server, where we normally use the main 
> hg branch to catch problems early and to test new features.
>
> Thanks,
>
> wiwo
>

Re: [naviserver-devel] Lots of ns:mutex entries

[Wolfgang W. <Wol...@di...>]

Hello!

We fixed the issues. The mutexes were indeed created without destroying
them.

This version is our development server, where we normally use the main
hg branch to catch problems early and to test new features.

Thanks,

wiwo


Am 2018-05-11 um 14:49 schrieb Gustaf Neumann:
> We are not observing this on our (busy) sites (running a week or longer).
>
> Can it be that you your application creates mutex objects without
> destroying it?
> In 4.99.12 (Aug 2016) there was a change in NaviServer to provide a more
> complete naming of mutexes. Maybe these mutexes just show up because
> of this?
>
> Is there any reason for not using the released version (4.99.16)?
>
> -g
>
> On 11.05.18 12:40, Wolfgang Winkler wrote:
>>
>> Hi!
>>
>> "ns_info locks" returns a lot of entries with names like these:
>>
>> syncobj:ns:mutex:tcl:151
>> syncobj:ns:mutex:tcl:150
>> syncobj:ns:mutex:tcl:149
>> syncobj:ns:mutex:tcl:148
>> syncobj:ns:mutex:tcl:147
>>
>> If the server has been running for a few days, we have millions of
>> entries. A single entry looks like this:
>>
>> syncobj:ns:mutex:tcl:1383 {} 1903 1 0 0.000000 0.000000 0.000084
>>
>> ns_info patchlevel: 4.99.16d9
>> info patchlevel: 8.6.6
>>
>> regards,
>>
>> Wolfgang
>>
>>
>> -- 
> ´
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] nsldap: Installation and setup

[Gustaf N. <ne...@wu...>]

the term "host" is an abbreviation of "hostname", so no NO URL 
scheme-prefix, port, path etc. From your last postings, my guess is that 
the following should fit your installation.

   ns_param host "ldap.litli.net <http://litli.net/>"

You could specify the port via

    ns_param port 389

but this is apparently the default. the ldap module - as it stands - 
does not use SSL/TLS.
-gn

On 17.05.18 04:54, Iuri Sampaio wrote:
> Yeah, that’s what I thought so. However, when I switched the host 
> within config.tcl (i) to the host that literally appears in the 
> processe's display (ii),  I got (iii).
>
> i. #config.tcl
> ns_section ns/ldap/pool/ldap
> ns_param user "cn=admin,dc=ldap,dc=litli,dc=net"
> ns_param password “*****"
>         ns_param host "ldap:///:389"
>         ns_param connections 1
>         ns_param verbose On

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Yeah, that’s what I thought so. However, when I switched the host within config.tcl (i) to the host that literally appears in the processe's display (ii),  I got (iii).

I wonder if running LDAP without SSL could be the cause. However, I found nothing in the documentation that would reference so.




i. #config.tcl
ns_section ns/ldap/pool/ldap
        ns_param user "cn=admin,dc=ldap,dc=litli,dc=net"
	ns_param password “*****"
        ns_param host "ldap:///:389"
        ns_param connections 1
        ns_param verbose On


iii. caltek   22931 22675  0 22:42 pts/1    00:00:00 ps -ef
openldap 23829     1  0 May14 ?        00:00:00 /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -F /etc/ldap/slapd.d


iii. [16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Error: nsldap: could not open connection to server ldap:///:389 on port 389: Inappropriate ioctl for device



[16/May/2018:22:39:27][22732.7f914effd700][-sched:8-] Debug: ns:interptrace[caltek]: deallocate nsproxy:cleanup a:(nil)
[16/May/2018:22:39:27][22732.7f914effd700][-sched:8-] Debug: ns:interptrace[caltek]: deallocate nsdb:releasehandles a:(nil)
[16/May/2018:22:39:27][22732.7f914effd700][-sched:8-] Debug: ns:interptrace[caltek]: deallocate ns:tcltrace ns_cleanup
[16/May/2018:22:39:29][22732.7f915cd36700][-driver:nssock:0-] Debug: Ns_SockAccept returns sock 5, err NONE
[16/May/2018:22:39:29][22732.7f915cd36700][-driver:nssock:0-] Debug: Ns_SockAccept returns sock -1, err Resource temporarily unavailable
[16/May/2018:22:39:29][22732.7f915cd36700][-driver:nssock:0-] Debug: [0] dequeue thread connPtr 0x555f13d233e0 idle 2 state 4 create 0
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: ns:interptrace[caltek]: allocate ns:tcltrace ns_init
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: user agent is Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: OACS= sec_handler: enter
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: ad_get_signed_cookie: Got signed cookie ad_session_id with value 350001,0,0,1526524659, signature 237 1526525859 B427AD47A039C798E4B2E29725B5456831F66EF3.
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: __ad_verify_signature: Getting token_id 237, value 02627536A2D3BDF38A7049D9AC1555DFB219A281 ; 
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: __ad_verify_signature: Expire_Time is 1526525859 (compare to 1526524769), hash is B427AD47A039C798E4B2E29725B5456831F66EF3
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: __ad_verify_signature: Hash matches - Hash check OK
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: __ad_verify_signature: Expiration time (1526525859) greater than current time (1526524769) - Expiration check OK
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: ad_get_signed_cookie: Verification of cookie ad_session_id OK
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: Security: Insecure session OK: session_id 350001, untrusted_user_id 0, auth_level none, user_id 0
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.acs-tcl.tcl.acs-permissions-procs.permission::permission_p_not_cached.select_permission_p --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f915900c670 REUSE sql
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Notice: Running first LDAP script ...
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: nsldap: getting a handle for thread 0x7f915dd38700
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: nsldap: connecting handle from pool ldap
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Error: nsldap: could not open connection to server ldap:///:389 on port 389: Inappropriate ioctl for device
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.acs-tcl.tcl.00-database-procs.db_nextval.nextval --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f91590db110 convert type none to sql <select nextval('t_acs_object_id_seq')>
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.acs-tcl.tcl.acs-permissions-procs.permission::permission_p_not_cached.select_permission_p --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f915900c670 REUSE sql
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.acs-tcl.tcl.acs-permissions-procs.permission::permission_p_not_cached.select_permission_p --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f915900c670 REUSE sql
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.dotlrn.tcl.dotlrn-security-procs.dotlrn::user_p.select_count --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f91590d8a70 REUSE sql
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: NO FULLQUERY FOR dbqd.acs-tcl.tcl.acs-permissions-procs.permission::permission_p_not_cached.select_permission_p --> using default SQL
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: 0x7f915900c670 REUSE sql
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: we have the following editors registered: 
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: Ns_ConnClose 0x555f13d233e0 stream 000000 chunk 000000 via writer 000000 sockPtr 0x7f91540012c0
[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Error: could not allocate 1 handle from pool "ldap"
    while executing
"ns_ldap gethandle "ldap""
    ("uplevel" body line 7)
    invoked from within
"uplevel {
    ad_page_contract {}

ns_log Notice "Running first LDAP script ..."


set lh [ns_ldap gethandle "ldap"]
doc_return 200 text/html "[ns_lda..."
    (procedure "code::tcl::/var/www/caltek//www/test-ldap" line 2)
    invoked from within
"code::tcl::$__adp_stub"
    ("uplevel" body line 12)
    invoked from within
"uplevel {

        if { [file exists $__adp_stub.tcl] } {

            # ensure that data source preparation procedure exists and is up-to-date
      ..."
    (procedure "adp_prepare" line 2)
    invoked from within
"adp_prepare"
    invoked from within
"template::adp_parse $themed_template {}"
    (procedure "adp_parse_ad_conn_file" line 14)
    invoked from within
"$handler"
    ("uplevel" body line 2)
    invoked from within
"uplevel $code"
    invoked from within
"ad_try {
                $handler
            } ad_script_abort val {
                # do nothing
            }"
    invoked from within
"rp_serve_concrete_file [ad_conn file]"
    (procedure "::nsf::procs::rp_serve_abstract_file" line 60)
    invoked from within
"rp_serve_abstract_file "$root/$extra_url""
    ("uplevel" body line 2)
    invoked from within
"uplevel $code"
    invoked from within
"ad_try {
                rp_serve_abstract_file "$root/$extra_url"
                set ::tcl_url2file([ad_conn url]) [ad_conn file]
                se..."
    called from rp_handler
        GET http://127.0.0.1:8080/test-ldap? referred by '' peer 127.0.0.1 user_id 0

 X-Forwarded-For:	201.50.61.208
 Host:	127.0.0.1:8080
 Connection:	close
 Cache-Control:	max-age=0
 Upgrade-Insecure-Requests:	1
 User-Agent:	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
 Accept:	text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
 Accept-Encoding:	gzip, deflate
 Accept-Language:	pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7
 Cookie:	ad_session_id="350001%2c0%2c0%2c1526524659%20{237%201526525859%20B427AD47A039C798E4B2E29725B5456831F66EF3}"; style=null

[16/May/2018:22:39:29][22732.7f915dd38700][-conn:caltek:0:27-] Debug: ns:interptrace[caltek]: deallocate nsproxy:cleanup a:(nil)
[16/May/2018:22:39:29]






> On May 16, 2018, at 09:51, Gustaf Neumann <ne...@wu...> wrote:
> 
> i am probably not the best person to ask, since we do not use here ldap.
> 
> however, my guess is that you have most likely a configuration problem:
> under "host" you specify "ldap://...." <ldap://....>, when i look into the source i see:
>     Ns_DStringPrintf(&ds, "ldap://%s:%d", handlePtr->host, handlePtr->port );
> 
> so , i would think that "host " is really what it implies, and not a ldap URL.
> 
> -gn
> 
> On 14.05.18 17:56, Iuri Sampaio wrote:
>> Hello Gustaf and all,
>> 
>> So far, I’ve installed and configured all code required to run LDAP together with Naviserver. Plus, I’ve added and amended all necessary parameters in order to enable LDAP authentication. 
>> 
>> However, I still haven’t figured out what is blocking the connection to the server. However parametrization seems fine, I believe the error is because I've assigned wrong formats to the parameter within config.tcl. 
>> 
>> LDAP and NS instances are running in the same server. I have tried localhost, w/ and w/o declaring ports explicitly. 
>> 
>> 
>> ns_section ns/ldap/pool/ldap
>>         ns_param user “cn=web,dc=ldap,dc=litli,dc=net"
>>         ns_param password “*****"
>> 	ns_param host "ldap://ldap.litli.net <http://litli.net/>"
>>         ns_param connections 1
>>         ns_param verbose On
>> 
>> 
>> 
>> Bellow you can see the logs of a successful boot up of NS.
>> 
>> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: modload: loading module nsldap from file /usr/local/ns/bin/nsldap.so
>> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: allowing * -> pool ldap
>> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: adding pool ldap to the list of allowed pools
>> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: Registering LDAPCheckPools (600)
>> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: nsldap: version 0.9 loaded
>> ...
>> [14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loading packages/auth-ldap/tcl/auth-ldap-procs.tcl...
>> [14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loaded packages/auth-ldap/tcl/auth-ldap-procs.tcl.
>> ...
>> 
>> 
>> 
>> Plus, LDAP instance is running just fine at http://ldap.litli.net <http://ldap.litli.net/>. Login is successful trough http and directly through command line:
>> 
>> 1)
>> ldapsearch -n -x -H ldap://ldap.litli.net <ldap://ldap.litli.net> -D “cn=web,dc=ldap,dc=litli,dc=net" "uid=iuri" -w ****
>> # extended LDIF
>> #
>> # LDAPv3
>> # base <dc=ldap,dc=litli,dc=net> (default) with scope subtree
>> # filter: uid=iuri
>> # requesting: ALL
>> #
>> 
>> 
>> 2) 
>> ldapsearch -x -W -D ‘cn=web,dc=ldap,dc=litli,dc=net' -b "" -s base
>> Enter LDAP Password: 
>> # extended LDIF
>> #
>> # LDAPv3
>> # base <> with scope baseObject
>> # filter: (objectclass=*)
>> # requesting: ALL
>> #
>> 
>> #
>> dn:
>> objectClass: top
>> objectClass: OpenLDAProotDSE
>> 
>> # search result
>> search: 2
>> result: 0 Success
>> 
>> # numResponses: 2
>> # numEntries: 1
>> 
>> 
>> ##############
>> 
>> 
>> However, the connection failed in attempt to allocate handle from ldap pool, when I run the following TCL command:
>> 
>> set lh [ns_ldap gethandle "ldap"]
>> 
>> 
>> See logs:
>> 
>> 
>> ###
>> # NS ERROR logs: /log/error.log
>> ###
>> 
>> ...
>> 
>> [14/May/2018:10:50:04][22834.7f4bd89ce700][-conn:caltek:0:1-] Error: could not allocate 1 handle from pool "ldap"
>>     while executing
>> "ns_ldap gethandle "ldap""
>>     ("uplevel" body line 2)
>>     invoked from within
>> "uplevel {
>>     set lh [ns_ldap gethandle "ldap"]
>> doc_return 200 text/html "[ns_ldap host $lh]"
>> 
>> 
>> }"
>> 
>> 
>> …
>> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: nsldap: getting a handle for thread 0x7f1653d03700
>> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: nsldap: connecting handle from pool ldap
>> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Error: nsldap: could not open connection to server ldap://ldap.litli.net <ldap://ldap.litli.net> on port 389: No such file or directory
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nsldap: Installation and setup

[Gustaf N. <ne...@wu...>]

i am probably not the best person to ask, since we do not use here ldap.

however, my guess is that you have most likely a configuration problem:
under "host" you specify "ldap://....", when i look into the source i see:

     Ns_DStringPrintf(&ds, "ldap://%s:%d", handlePtr->host, handlePtr->port );

so , i would think that "host " is really what it implies, and not a 
ldap URL.

-gn

On 14.05.18 17:56, Iuri Sampaio wrote:
> Hello Gustaf and all,
>
> So far, I’ve installed and configured all code required to run LDAP 
> together with Naviserver. Plus, I’ve added and amended all necessary 
> parameters in order to enable LDAP authentication.
>
> However, I still haven’t figured out what is blocking the connection 
> to the server. However parametrization seems fine, I believe the error 
> is because I've assigned wrong formats to the parameter within 
> config.tcl.
>
> LDAP and NS instances are running in the same server. I have tried 
> localhost, w/ and w/o declaring ports explicitly.
>
>
> ns_section ns/ldap/pool/ldap
> ns_param user “cn=web,dc=ldap,dc=litli,dc=net"
>         ns_param password “*****"
> ns_param host "ldap://ldap.litli.net <http://litli.net>"
>         ns_param connections 1
>         ns_param verbose On
>
>
>
> Bellow you can see the logs of a successful boot up of NS.
>
> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: modload: 
> loading module nsldap from file /usr/local/ns/bin/nsldap.so
> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: 
> allowing * -> pool ldap
> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: 
> adding pool ldap to the list of allowed pools
> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: 
> Registering LDAPCheckPools (600)
> [14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: nsldap: 
> version 0.9 loaded
> ...
> [14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loading 
> packages/auth-ldap/tcl/auth-ldap-procs.tcl...
> [14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loaded 
> packages/auth-ldap/tcl/auth-ldap-procs.tcl.
> ...
>
>
>
> Plus, LDAP instance is running just fine at http://ldap.litli.net. 
> Login is successful trough http and directly through command line:
>
> 1)
> ldapsearch -n -x -H ldap://ldap.litli.net -D 
> “cn=web,dc=ldap,dc=litli,dc=net" "uid=iuri" -w ****
> # extended LDIF
> #
> # LDAPv3
> # base <dc=ldap,dc=litli,dc=net> (default) with scope subtree
> # filter: uid=iuri
> # requesting: ALL
> #
>
>
> 2)
> ldapsearch -x -W -D ‘cn=web,dc=ldap,dc=litli,dc=net' -b "" -s base
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <> with scope baseObject
> # filter: (objectclass=*)
> # requesting: ALL
> #
>
> #
> dn:
> objectClass: top
> objectClass: OpenLDAProotDSE
>
> # search result
> search: 2
> result: 0 Success
>
> # numResponses: 2
> # numEntries: 1
>
>
> ##############
>
>
> However, the connection failed in attempt to allocate handle from ldap 
> pool, when I run the following TCL command:
>
> setlh[ns_ldap gethandle "ldap"]
>
>
> See logs:
>
>
> ###
> # NS ERROR logs: /log/error.log
> ###
>
> ...
>
> [14/May/2018:10:50:04][22834.7f4bd89ce700][-conn:caltek:0:1-] Error: 
> could not allocate 1 handle from pool "ldap"
>     while executing
> "ns_ldapgethandle "ldap""
>     ("uplevel" body line 2)
>     invoked from within
> "uplevel {
>     set lh [ns_ldapgethandle "ldap"]
> doc_return 200 text/html "[ns_ldaphost $lh]"
>
>
> }"
>
>
> …
> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: 
> nsldap: getting a handle for thread 0x7f1653d03700
> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: 
> nsldap: connecting handle from pool ldap
> [14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Error: 
> nsldap: could not open connection to server ldap://ldap.litli.net on 
> port 389: No such file or directory

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Hello Gustaf and all,

So far, I’ve installed and configured all code required to run LDAP together with Naviserver. Plus, I’ve added and amended all necessary parameters in order to enable LDAP authentication. 

However, I still haven’t figured out what is blocking the connection to the server. However parametrization seems fine, I believe the error is because I've assigned wrong formats to the parameter within config.tcl. 

LDAP and NS instances are running in the same server. I have tried localhost, w/ and w/o declaring ports explicitly. 


ns_section ns/ldap/pool/ldap
        ns_param user “cn=web,dc=ldap,dc=litli,dc=net"
        ns_param password “*****"
	ns_param host "ldap://ldap.litli.net"
        ns_param connections 1
        ns_param verbose On



Bellow you can see the logs of a successful boot up of NS.

[14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: modload: loading module nsldap from file /usr/local/ns/bin/nsldap.so
[14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: allowing * -> pool ldap
[14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: adding pool ldap to the list of allowed pools
[14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Debug: nsldap: Registering LDAPCheckPools (600)
[14/May/2018:10:44:25][22834.7f4be4abf700][-main-] Notice: nsldap: version 0.9 loaded
...
[14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loading packages/auth-ldap/tcl/auth-ldap-procs.tcl...
[14/May/2018:10:44:36][22834.7f4be4abf700][-main-] Notice: Loaded packages/auth-ldap/tcl/auth-ldap-procs.tcl.
...



Plus, LDAP instance is running just fine at http://ldap.litli.net <http://ldap.litli.net/>. Login is successful trough http and directly through command line:

1)
ldapsearch -n -x -H ldap://ldap.litli.net -D “cn=web,dc=ldap,dc=litli,dc=net" "uid=iuri" -w ****
# extended LDIF
#
# LDAPv3
# base <dc=ldap,dc=litli,dc=net> (default) with scope subtree
# filter: uid=iuri
# requesting: ALL
#


2) 
ldapsearch -x -W -D ‘cn=web,dc=ldap,dc=litli,dc=net' -b "" -s base
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectclass=*)
# requesting: ALL
#

#
dn:
objectClass: top
objectClass: OpenLDAProotDSE

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1


##############


However, the connection failed in attempt to allocate handle from ldap pool, when I run the following TCL command:

set lh [ns_ldap gethandle "ldap"]


See logs:


###
# NS ERROR logs: /log/error.log
###

...

[14/May/2018:10:50:04][22834.7f4bd89ce700][-conn:caltek:0:1-] Error: could not allocate 1 handle from pool "ldap"
    while executing
"ns_ldap gethandle "ldap""
    ("uplevel" body line 2)
    invoked from within
"uplevel {
    set lh [ns_ldap gethandle "ldap"]
doc_return 200 text/html "[ns_ldap host $lh]"


}"


…
[14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: nsldap: getting a handle for thread 0x7f1653d03700
[14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Debug: nsldap: connecting handle from pool ldap
[14/May/2018:11:37:17][23479.7f1653d03700][-conn:caltek:1:0-] Error: nsldap: could not open connection to server ldap://ldap.litli.net on port 389: No such file or directory





 


> On May 13, 2018, at 16:42, Gustaf Neumann <ne...@wu...> wrote:
> 
> On 13.05.18 18:36, Iuri Sampaio wrote:
>> Btw,
>> Is there any reason to .o and .so be missing on bitbucket repository (i.e. https://bitbucket.org/naviserver/nsldap/src/default/ ) ?
>> If not, do upload them there, please!
> 
> the extension .o stands for an "object" file, .so stands for "shared object".  The .o files are dependent on the machine architecture and are generated by a compiler such as "gcc", "clang" or others. The repository on bitbucket is a *source code* repository, this is no place for *binaries*.
> 
> Get some tutorials concerning C and make, such as e.g. [1] which i found after 1min of googling around.
> 
> -gn
> 
> [1] https://www.thegeekstuff.com/2010/08/make-utility/
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nsldap: Installation and setup

[Gustaf N. <ne...@wu...>]

On 13.05.18 18:36, Iuri Sampaio wrote:
> Btw,
> Is there any reason to .o and .so be missing on bitbucket repository 
> (i.e. https://bitbucket.org/naviserver/nsldap/src/default/ ) ?
> If not, do upload them there, please!

the extension .o stands for an "object" file, .so stands for "shared 
object".  The .o files are dependent on the machine architecture and are 
generated by a compiler such as "gcc", "clang" or others. The repository 
on bitbucket is a *source code* repository, this is no place for *binaries*.

Get some tutorials concerning C and make, such as e.g. [1] which i found 
after 1min of googling around.

-gn

[1] https://www.thegeekstuff.com/2010/08/make-utility/

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Gustaf,

You clearly got me wrong.

My question was based on the fact that .o and .so aren’t available on bitbucket repos. As I previously pointed out.
https://bitbucket.org/naviserver/nsldap/src/default/ <https://bitbucket.org/naviserver/nsldap/src/default/>


Thus, in case I find these sort of scenarios (without source forge repository), I was wondering a way out to compile nsldap.c, generating .o and .so myself. 

My apologies for asking. I’ll try to quiet myself. 

Best wishes,
I


> On May 13, 2018, at 06:29, Gustaf Neumann <ne...@wu...> wrote:
> 
> On 12.05.18 18:14, Iuri Sampaio wrote:
>> how would I generate .o and .so from .c?
>> 
>> May I use gcc? What are the necessary parameters ? (if needed)
>> 
>> i.e. gcc -g -O -c hello.c -fPIC -DPIC -o .libs/hello.o
>> 
> Are you writing your first C program? Please get a C book or find
> appropriate online materials. With C it is quite easy shooting yourself
> in the foot.
> The parameters depend on many things, the C compiler, the program,
> the libraries, the operating system, the level of optimization 
> or debugging, kind of linking, etc. In the context of NaviServer, 
> use the provided Makefiles unlessyou know what you do.
> 
> Check out stackoverflow etc. for such kind of question and 
> please send only questions concerning naviserver to this list.
> 
> -gn
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Btw,
Is there any reason to .o and .so be missing on bitbucket repository (i.e.  https://bitbucket.org/naviserver/nsldap/src/default/ <https://bitbucket.org/naviserver/nsldap/src/default/> ) ?
If not, do upload them there, please! 

Best wishes,
I

> On May 13, 2018, at 13:30, Iuri Sampaio <iu...@iu...> wrote:
> 
> Gustaf,
> 
> You clearly got me wrong.
> 
> My question was based on the fact that .o and .so aren’t available on bitbucket repos. As I previously pointed out.
> https://bitbucket.org/naviserver/nsldap/src/default/ <https://bitbucket.org/naviserver/nsldap/src/default/>
> 
> 
> Thus, in case I find these sort of scenarios (without source forge repository), I was wondering a way out to compile nsldap.c, generating .o and .so myself. 
> 
> My apologies for asking. I’ll try to quiet myself. 
> 
> Best wishes,
> I
> 
> 
>> On May 13, 2018, at 06:29, Gustaf Neumann <ne...@wu... <mailto:ne...@wu...>> wrote:
>> 
>> On 12.05.18 18:14, Iuri Sampaio wrote:
>>> how would I generate .o and .so from .c?
>>> 
>>> May I use gcc? What are the necessary parameters ? (if needed)
>>> 
>>> i.e. gcc -g -O -c hello.c -fPIC -DPIC -o .libs/hello.o
>>> 
>> Are you writing your first C program? Please get a C book or find
>> appropriate online materials. With C it is quite easy shooting yourself
>> in the foot.
>> The parameters depend on many things, the C compiler, the program,
>> the libraries, the operating system, the level of optimization 
>> or debugging, kind of linking, etc. In the context of NaviServer, 
>> use the provided Makefiles unlessyou know what you do.
>> 
>> Check out stackoverflow etc. for such kind of question and 
>> please send only questions concerning naviserver to this list.
>> 
>> -gn
>> 
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
>> naviserver-devel mailing list
>> nav...@li... <mailto:nav...@li...>
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] nsldap: Installation and setup

[Gustaf N. <ne...@wu...>]

On 12.05.18 18:14, Iuri Sampaio wrote:
> how would I generate .o and .so from .c?
>
> May I use gcc? What are the necessary parameters ? (if needed)
>
> i.e. gcc -g -O -c hello.c -fPIC -DPIC -o .libs/hello.o
>
Are you writing your first C program? Please get a C book or find
appropriate online materials. With C it is quite easy shooting yourself
in the foot.

The parameters depend on many things, the C compiler, the program,
the libraries, the operating system, the level of optimization
or debugging, kind of linking, etc. In the context of NaviServer,
use the provided Makefiles unlessyou know what you do.

Check out stackoverflow etc. for such kind of question and
please send only questions concerning naviserver to this list.

-gn

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Thanks Gustaf, 

Yes, it helped. I was blindly looking at the source, available on Bitbucket, which has only Makefile (i.e. there was no nsldap.o neither nsldap.so). Then I started to wonder how I would compile it, or if there was another method to do so.  

I should have thought out of the box a bit (i.e. after I noticed the files were missing), and gone straight ahead to source forge file repository, which is know by myself already.

I’ve installed nsldap successfully. Thanks a lot! 

One quick thing, I ran into the error bellow. lber.h was missing on my Debian distro. 

nsldap.c:38:18: fatal error: lber.h: No such file or directory
 
I went to the file nsldap to debug it and I quickly verified that lber.h is required and called in the respective line 38.  Nothing much, that a few more packages installed through apt-get to solve the problem.

 apt-get install libsasl2-dev libldap2-dev libssl-dev


Best wishes,
Iuri
  
 

> On May 12, 2018, at 06:16, Gustaf Neumann <ne...@wu...> wrote:
> 
> On 10.05.18 04:01, Iuri Sampaio wrote:
>> Hi there,
>> 
>> In attempt to to install and setup nsldap, I have changed install-ns.sh script to look to HEAD repos but When running install-ns build it returns 
>> 
>> ------------------------ Downloading sources ----------------------------
>> abort: no repository found in '/usr/local/src/modules/nsdbbdb' (.hg not found)!
> the error message means: your changes to install-ns are incorrect, you are trying to use mercurial on a non-mercurial directory (mercurial checks the .hg directory). You can can install the modules via source code management system (mercurial) or via the packaged tar file (recommended for beginners).
>> Then I realized that recompiling NS from HEAD could dangerous. rsrs!
>> 
>> In another front, I've downloaded NSLDAP, available on bitbucket https://bitbucket.org/naviserver/nsldap <https://bitbucket.org/naviserver/nsldap>, but I don’t know how to install and enable into my current Naviserver installation (i.e. generating .so file in the proper location and so on). 
>> 
>> Do I need to recompile NS? Re-Installing it from scratch?  
>> Where should I place nsldap source files?
>> 
>> The information is not available on README.txt
> the cooking recipe for all modules is essentially the same. 
> a) first install NaviServer (preferably on the standard place in /usr/local/ns, otherwise
>    you have to specify for the compilation of a module "NAVISERVER=...")
> 
> b) obtain the modules source code, either via the modules tar file from sourceforge
>    (naviserver-*-modules.tar.gz) - this is preferred, or via mercurial (clone every single
>    module you need)
> 
> c) change to the directory of the modules and do
> 
>       make
>       make install
> 
>    Sometimes, module specific configurations are need (e.g. paths to includes
>    or libraries) as indicated in the README file of the module.
> the place, where you put the source files, does not matter.
> there is no need to recompile NaviServer, when adding modules.
> 
> Hope, this helps
> -gn
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Btw,

how would I generate .o and .so from .c?

May I use gcc? What are the necessary parameters ? (if needed)

i.e. gcc -g -O -c hello.c -fPIC -DPIC -o .libs/hello.o


Best wishes,
Iuri
 
> On May 12, 2018, at 13:05, Iuri Sampaio <iu...@iu...> wrote:
> 
> Thanks Gustaf, 
> 
> Yes, it helped. I was blindly looking at the source, available on Bitbucket, which has only Makefile (i.e. there was no nsldap.o neither nsldap.so). Then I started to wonder how I would compile it, or if there was another method to do so.  
> 
> I should have thought out of the box a bit (i.e. after I noticed the files were missing), and gone straight ahead to source forge file repository, which is know by myself already.
> 
> I’ve installed nsldap successfully. Thanks a lot! 
> 
> One quick thing, I ran into the error bellow. lber.h was missing on my Debian distro. 
> 
> nsldap.c:38:18: fatal error: lber.h: No such file or directory
>  
> I went to the file nsldap to debug it and I quickly verified that lber.h is required and called in the respective line 38.  Nothing much, that a few more packages installed through apt-get to solve the problem.
> 
>  apt-get install libsasl2-dev libldap2-dev libssl-dev
> 
> 
> Best wishes,
> Iuri
>   
>  
> 
>> On May 12, 2018, at 06:16, Gustaf Neumann <ne...@wu... <mailto:ne...@wu...>> wrote:
>> 
>> On 10.05.18 04:01, Iuri Sampaio wrote:
>>> Hi there,
>>> 
>>> In attempt to to install and setup nsldap, I have changed install-ns.sh script to look to HEAD repos but When running install-ns build it returns 
>>> 
>>> ------------------------ Downloading sources ----------------------------
>>> abort: no repository found in '/usr/local/src/modules/nsdbbdb' (.hg not found)!
>> the error message means: your changes to install-ns are incorrect, you are trying to use mercurial on a non-mercurial directory (mercurial checks the .hg directory). You can can install the modules via source code management system (mercurial) or via the packaged tar file (recommended for beginners).
>>> Then I realized that recompiling NS from HEAD could dangerous. rsrs!
>>> 
>>> In another front, I've downloaded NSLDAP, available on bitbucket https://bitbucket.org/naviserver/nsldap <https://bitbucket.org/naviserver/nsldap>, but I don’t know how to install and enable into my current Naviserver installation (i.e. generating .so file in the proper location and so on). 
>>> 
>>> Do I need to recompile NS? Re-Installing it from scratch?  
>>> Where should I place nsldap source files?
>>> 
>>> The information is not available on README.txt
>> the cooking recipe for all modules is essentially the same. 
>> a) first install NaviServer (preferably on the standard place in /usr/local/ns, otherwise
>>    you have to specify for the compilation of a module "NAVISERVER=...")
>> 
>> b) obtain the modules source code, either via the modules tar file from sourceforge
>>    (naviserver-*-modules.tar.gz) - this is preferred, or via mercurial (clone every single
>>    module you need)
>> 
>> c) change to the directory of the modules and do
>> 
>>       make
>>       make install
>> 
>>    Sometimes, module specific configurations are need (e.g. paths to includes
>>    or libraries) as indicated in the README file of the module.
>> the place, where you put the source files, does not matter.
>> there is no need to recompile NaviServer, when adding modules.
>> 
>> Hope, this helps
>> -gn
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
>> naviserver-devel mailing list
>> nav...@li... <mailto:nav...@li...>
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] nsldap: Installation and setup

[Gustaf N. <ne...@wu...>]

On 10.05.18 04:01, Iuri Sampaio wrote:
> Hi there,
>
> In attempt to to install and setup nsldap, I have changed 
> install-ns.sh script to look to HEAD repos but When running install-ns 
> build it returns
>
> ------------------------ Downloading sources ----------------------------
> abort: no repository found in '/usr/local/src/modules/nsdbbdb' (.hg 
> not found)!

the error message means: your changes to install-ns are incorrect, you 
are trying to use mercurial on a non-mercurial directory (mercurial 
checks the .hg directory). You can can install the modules via source 
code management system (mercurial) or via the packaged tar file 
(recommended for beginners).

> Then I realized that recompiling NS from HEAD could dangerous. rsrs!
>
> In another front, I've downloaded NSLDAP, available on bitbucket 
> https://bitbucket.org/naviserver/nsldap, but I don’t know how to 
> install and enable into my current Naviserver installation (i.e. 
> generating .so file in the proper location and so on).
>
> Do I need to recompile NS? Re-Installing it from scratch?
> Where should I place nsldap source files?
>
> The information is not available on README.txt

the cooking recipe for all modules is essentially the same.

a) first install NaviServer (preferably on the standard place in 
/usr/local/ns, otherwise
    you have to specify for the compilation of a module "NAVISERVER=...")

b) obtain the modules source code, either via the modules tar file from 
sourceforge
    (naviserver-*-modules.tar.gz) - this is preferred, or via mercurial 
(clone every single
    module you need)

c) change to the directory of the modules and do

       make
       make install

    Sometimes, module specific configurations are need (e.g. paths to 
includes
    or libraries) as indicated in the README file of the module.

the place, where you put the source files, does not matter.
there is no need to recompile NaviServer, when adding modules.

Hope, this helps

-gn

Re: [naviserver-devel] Lots of ns:mutex entries

[Gustaf N. <ne...@wu...>]

We are not observing this on our (busy) sites (running a week or longer).

Can it be that you your application creates mutex objects without 
destroying it?
In 4.99.12 (Aug 2016) there was a change in NaviServer to provide a more
complete naming of mutexes. Maybe these mutexes just show up because of 
this?

Is there any reason for not using the released version (4.99.16)?

-g

On 11.05.18 12:40, Wolfgang Winkler wrote:
>
> Hi!
>
> "ns_info locks" returns a lot of entries with names like these:
>
> syncobj:ns:mutex:tcl:151
> syncobj:ns:mutex:tcl:150
> syncobj:ns:mutex:tcl:149
> syncobj:ns:mutex:tcl:148
> syncobj:ns:mutex:tcl:147
>
> If the server has been running for a few days, we have millions of 
> entries. A single entry looks like this:
>
> syncobj:ns:mutex:tcl:1383 {} 1903 1 0 0.000000 0.000000 0.000084
>
> ns_info patchlevel: 4.99.16d9
> info patchlevel: 8.6.6
>
> regards,
>
> Wolfgang
>
>
> -- 
´

[naviserver-devel] Lots of ns:mutex entries

[Wolfgang W. <Wol...@di...>]

Hi!

"ns_info locks" returns a lot of entries with names like these:

syncobj:ns:mutex:tcl:151
syncobj:ns:mutex:tcl:150
syncobj:ns:mutex:tcl:149
syncobj:ns:mutex:tcl:148
syncobj:ns:mutex:tcl:147

If the server has been running for a few days, we have millions of
entries. A single entry looks like this:

syncobj:ns:mutex:tcl:1383 {} 1903 1 0 0.000000 0.000000 0.000084

ns_info patchlevel: 4.99.16d9
info patchlevel: 8.6.6

regards,

Wolfgang


-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz

Re: [naviserver-devel] ns_http and https timeout

[Ian H. <har...@gm...>]

On Tue, May 8, 2018 at 11:24 AM, Gustaf Neumann <ne...@wu...> wrote:

> On 08.05.18 20:10, Ian Harding wrote:
>
> I tried with 4.99.16 and I have the same problem.  Is the change you
> mention in that release or tip?
>
> The release of 4.99.16 was already on 2017-12-29. i would have used a
> different wording, when i would have referred to that! No, the change is on
> tip, as a response of your request...
>
Wow!  Thank you.  This works much better!

> -gn
>
>
> On Fri, May 4, 2018 at 11:52 AM, Gustaf Neumann <ne...@wu...> wrote:
>
>> On 04.05.18 18:44, Ian Harding wrote:
>>
>>
>>
>>> i will look into this in the near future
>>>
>>
>> Thank you!  I think I'll have to keep using exec curl for the time
>> being...
>>
>> There is an updated version of NaviServer at bitbucket, which improves
>> the situation by leading to a consistent behavior of https and http from
>> the user perspective.
>>
>> One can now as well distinguish between "errors" and "timeouts" by using
>> try:
>>
>>   try {
>>      set r [ns_http run -timeout 0.01 http://google.com/]
>>   } trap NS_TIMEOUT {msg} {
>>      ns_log notice TIMEOUT
>>   }
>>
>>
>> All the best
>>
>> -g
>>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

[naviserver-devel] nsldap: Installation and setup

[Iuri S. <iu...@iu...>]

Hi there,

In attempt to to install and setup nsldap, I have changed install-ns.sh script to look to HEAD repos but When running install-ns build it returns 

------------------------ Downloading sources ----------------------------
abort: no repository found in '/usr/local/src/modules/nsdbbdb' (.hg not found)!

Then I realized that recompiling NS from HEAD could dangerous. rsrs!

In another front, I've downloaded NSLDAP, available on bitbucket https://bitbucket.org/naviserver/nsldap <https://bitbucket.org/naviserver/nsldap>, but I don’t know how to install and enable into my current Naviserver installation (i.e. generating .so file in the proper location and so on). 

Do I need to recompile NS? Re-Installing it from scratch?  
Where should I place nsldap source files?

The information is not available on README.txt
 
Best wishes,
Iuri

Re: [naviserver-devel] ns_http and https timeout

[Gustaf N. <ne...@wu...>]

On 08.05.18 20:10, Ian Harding wrote:
> I tried with 4.99.16 and I have the same problem. Is the change you 
> mention in that release or tip?

The release of 4.99.16 was already on 2017-12-29. i would have used a 
different wording, when i would have referred to that! No, the change is 
on tip, as a response of your request...

-gn

>
> On Fri, May 4, 2018 at 11:52 AM, Gustaf Neumann <ne...@wu... 
> <mailto:ne...@wu...>> wrote:
>
>     On 04.05.18 18:44, Ian Harding wrote:
>>
>>
>>         i will look into this in the near future
>>
>>
>>     Thank you!  I think I'll have to keep using exec curl for the
>>     time being...
>
>     There is an updated version of NaviServer at bitbucket, which
>     improves the situation by leading to a consistent behavior of
>     https and http from the user perspective.
>
>     One can now as well distinguish between "errors" and "timeouts" by
>     using
>     try:
>
>        try {
>           set r [ns_http run -timeout 0.01http://google.com/]
>        } trap NS_TIMEOUT {msg} {
>           ns_log notice TIMEOUT
>        }
>
>     All the best
>
>     -g
>

Re: [naviserver-devel] ns_http and https timeout

[Ian H. <har...@gm...>]

I tried with 4.99.16 and I have the same problem.  Is the change you
mention in that release or tip?

On Fri, May 4, 2018 at 11:52 AM, Gustaf Neumann <ne...@wu...> wrote:

> On 04.05.18 18:44, Ian Harding wrote:
>
>
>
>> i will look into this in the near future
>>
>
> Thank you!  I think I'll have to keep using exec curl for the time being...
>
> There is an updated version of NaviServer at bitbucket, which improves the
> situation by leading to a consistent behavior of https and http from the
> user perspective.
>
> One can now as well distinguish between "errors" and "timeouts" by using
> try:
>
>   try {
>      set r [ns_http run -timeout 0.01 http://google.com/]
>   } trap NS_TIMEOUT {msg} {
>      ns_log notice TIMEOUT
>   }
>
>
> All the best
>
> -g
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] ns_http and https timeout

[Gustaf N. <ne...@wu...>]

On 04.05.18 18:44, Ian Harding wrote:
>
>
>     i will look into this in the near future
>
>
> Thank you!  I think I'll have to keep using exec curl for the time 
> being...

There is an updated version of NaviServer at bitbucket, which improves 
the situation by leading to a consistent behavior of https and http from 
the user perspective.

One can now as well distinguish between "errors" and "timeouts" by using
try:

   try {
      set r [ns_http run -timeout 0.01 http://google.com/]
   } trap NS_TIMEOUT {msg} {
      ns_log notice TIMEOUT
   }

All the best

-g

Re: [naviserver-devel] ns_http and https timeout

[Ian H. <har...@gm...>]

>
> i will look into this in the near future
>

Thank you!  I think I'll have to keep using exec curl for the time being...


>
> all the best
> -g
>
> [1] https://bitbucket.org/naviserver/nsdns/src/default/
>
> On 04.05.18 01:15, Ian Harding wrote:
>
> I am trying to use ns_http with an https uri and finding that the -timeout
> seems not to work if the host does not exist.  I get
>
> ssl connect failed: error:00000000:lib(0):func(0):reason(0)
>
> When I run it with a short timeout against an existing server I see:
>
> seattle-vm-1-0:nscp 1> catch [http_post https://someserver.com "" [list
> Content-type text/html" GET]] err
> ns_http failed: timeout
>
> When I run it with a short timeout against a server with a DNS entry but
> no actual listener I see the ssl connect failed error.
>
> Is there any way for the timeout to include the ssl handshake?
>
> Here's my proc fyi:
>
> proc http_post {hostname post_data {headerlist {Content-type
> application/x-www-form-urlencoded}} {method POST}} {
>
> # Simple script to stop using curl
> # arguments are
> #       hostname: URI to connect to
> #       post_data: url encoded post data stuck together with ampersands
> like a GET
> #       headers: optional list of header names and value (Optional
> defaults to post header)
> #       method: POST or GET.  LOL bad proc name (Optional defaults to POST)
> # returns
> #       the data returned
>
>         set queryHeaders [ns_set create]
>         set replyHeaders [ns_set create]
>         foreach {key value} $headerlist {
>                 ns_set put $queryHeaders $key $value
>         }
>
>         set h [ns_http queue -method $method -timeout 1:0 -headers
> $queryHeaders -body $post_data $hostname]
>         ns_http wait -elapsed E -timeout 1:0 -result returnval -headers
> $replyHeaders -status S $h
>
>         ns_log notice "[lindex [info level 0] 0] status code: $S elapsed:
> $E"
>         ns_log notice "[lindex [info level 0] 0]reply: $returnval"
>
>         return $returnval
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] ns_http and https timeout

[Gustaf N. <ne...@wu...>]

Hi Ian,

there are at least two problem areas. The first one is the DNS lookup-up:

- the standard DNS-lookup in NaviServer does not support timeouts.
    one approach is to use the nsdns module [1], where one can perform the
    lookup via

          ns_dns resolve hostname ?-type type? ?-server server? 
?-timeout timeout?

    and bypass the lookup in ns_http by providing it a resolved IP address.

However, as you pointed out, this is just a part of the equation.
If one connects to a host via HTTP, the timeout works as expected,
but the timeout via HTTPS leads to a long blocking request, when nobody
listens at the backend.

The situation with SSL is more complex, because the connect() operation
via OpenSSL (and friends) is more than just a unix connect(), since it
requires a handshake (read and write operations), where several
things can go wrong, etc..... but fixing this should bo big hassle.

i will look into this in the near future

all the best
-g

[1] https://bitbucket.org/naviserver/nsdns/src/default/

On 04.05.18 01:15, Ian Harding wrote:
> I am trying to use ns_http with an https uri and finding that the 
> -timeout seems not to work if the host does not exist.  I get
>
> ssl connect failed: error:00000000:lib(0):func(0):reason(0)
>
> When I run it with a short timeout against an existing server I see:
>
> seattle-vm-1-0:nscp 1> catch [http_post https://someserver.com "" 
> [list Content-type text/html" GET]] err
> ns_http failed: timeout
>
> When I run it with a short timeout against a server with a DNS entry 
> but no actual listener I see the ssl connect failed error.
>
> Is there any way for the timeout to include the ssl handshake?
>
> Here's my proc fyi:
>
> proc http_post {hostname post_data {headerlist {Content-type 
> application/x-www-form-urlencoded}} {method POST}} {
>
> # Simple script to stop using curl
> # arguments are
> #       hostname: URI to connect to
> #       post_data: url encoded post data stuck together with 
> ampersands like a GET
> #       headers: optional list of header names and value (Optional 
> defaults to post header)
> #       method: POST or GET.  LOL bad proc name (Optional defaults to 
> POST)
> # returns
> #       the data returned
>
>         set queryHeaders [ns_set create]
>         set replyHeaders [ns_set create]
>         foreach {key value} $headerlist {
>                 ns_set put $queryHeaders $key $value
>         }
>
>         set h [ns_http queue -method $method -timeout 1:0 -headers 
> $queryHeaders -body $post_data $hostname]
>         ns_http wait -elapsed E -timeout 1:0 -result returnval 
> -headers $replyHeaders -status S $h
>
>         ns_log notice "[lindex [info level 0] 0] status code: $S 
> elapsed: $E"
>         ns_log notice "[lindex [info level 0] 0]reply: $returnval"
>
>         return $returnval
>

[naviserver-devel] ns_http and https timeout

[Ian H. <har...@gm...>]

I am trying to use ns_http with an https uri and finding that the -timeout
seems not to work if the host does not exist.  I get

ssl connect failed: error:00000000:lib(0):func(0):reason(0)

When I run it with a short timeout against an existing server I see:

seattle-vm-1-0:nscp 1> catch [http_post https://someserver.com "" [list
Content-type text/html" GET]] err
ns_http failed: timeout

When I run it with a short timeout against a server with a DNS entry but no
actual listener I see the ssl connect failed error.

Is there any way for the timeout to include the ssl handshake?

Here's my proc fyi:

proc http_post {hostname post_data {headerlist {Content-type
application/x-www-form-urlencoded}} {method POST}} {

# Simple script to stop using curl
# arguments are
#       hostname: URI to connect to
#       post_data: url encoded post data stuck together with ampersands
like a GET
#       headers: optional list of header names and value (Optional defaults
to post header)
#       method: POST or GET.  LOL bad proc name (Optional defaults to POST)
# returns
#       the data returned

        set queryHeaders [ns_set create]
        set replyHeaders [ns_set create]
        foreach {key value} $headerlist {
                ns_set put $queryHeaders $key $value
        }

        set h [ns_http queue -method $method -timeout 1:0 -headers
$queryHeaders -body $post_data $hostname]
        ns_http wait -elapsed E -timeout 1:0 -result returnval -headers
$replyHeaders -status S $h

        ns_log notice "[lindex [info level 0] 0] status code: $S elapsed:
$E"
        ns_log notice "[lindex [info level 0] 0]reply: $returnval"

        return $returnval

}

Re: [naviserver-devel] Fwd: NaviServer

[Sassy N. <sa...@gm...>]

Thank you!!! very much!

On Tue, Apr 17, 2018 at 3:05 PM Gustaf Neumann <ne...@wu...> wrote:

> ns_share is outdated since ages. the workaround in [2] is not fixinig
> ns_share, but replacing ns_share via nsv commands. Have you looked at [1]?
>
> -gn
> PS: i've fixed just now the nsshare module on bitbucket to make it compile
> with actual compilers, against actual version of Tcl and NaviServer. ...
> but this is no recommendation to use it.
>
> [1] https://naviserver.sourceforge.io/n/naviserver/files/nsv.html#section3
> [2] https://openacs.org/forums/message-view?message_id=4263411
>
> On 17.04.18 12:32, Sassy Natan wrote:
>
> I'm doing some work for a customer of mine, and he is still running
> aolserver 3!!!
> It is 15 years system running on fedora server! really old one with
> postgres 8.
>
> In my test I have manage to move the server to work with postgres 9.6 and
> aolserver 4.5, but I still on the process for moving it to run under
> NaviSever.
>
> I want it to be run with the latest navisever and postgres 10.
>
> The problem is with the ns_share command.
>
> The server use some old tcl script from ArsDigita-Community-System-3.2.3
> which use ns_share command many times. I would like to avoid fixing the
> code here.
>
> I saw there is some module name that implements some long-depreciated old
> AOLserver commands
> <https://bitbucket.org/naviserver/nsshare/src/099864af6816?at=default>.
> However I never managed to compile it.
>
> I get some error like this:
>
> root@devplatfrom:/tmp/1# make NAVISERVER=/usr/lib/naviserver
> DESTDIR=/tmp/121212/
> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC -g -O2
> -fdebug-prefix-map=/build/tcl8.6-1Mwawt/tcl8.6-8.6.8+dfsg=.
> -fstack-protector-strong -Wformat -Werror=format-security
> -fno-unit-at-a-time -pipe -Wdate-time -D_FORTIFY_SOURCE=2
> -I/usr/lib/naviserver/include -I"/usr/include/tcl8.6"  -DHAVE_CONFIG_H
> -c -o nsshare.o nsshare.c
> nsshare.c:93:1: error: conflicting types for ‘InitInterp’
>  InitInterp(Tcl_Interp *interp, void *arg)
>  ^~~~~~~~~~
> nsshare.c:46:24: note: previous declaration of ‘InitInterp’ was here
>  static Ns_TclTraceProc InitInterp;
>                         ^~~~~~~~~~
> nsshare.c:46:24: warning: ‘InitInterp’ used but never defined
> nsshare.c:93:1: warning: ‘InitInterp’ defined but not used
> [-Wunused-function]
>  InitInterp(Tcl_Interp *interp, void *arg)
>  ^~~~~~~~~~
> <builtin>: recipe for target 'nsshare.o' failed
> make: *** [nsshare.o] Error 1
>
>
> Looking in the code, it seems to be like all other modules which I manage
> to build, (nsaccess for example or nsdbpg etc..)
>
> What do I missing? as to do  refactoring  to the
> ArsDigita-Community-System-3.2.3
> <https://github.com/MikeSisk/ArsDigita-Community-System-3.2.3/tree/master/tcl>
> might be a huge risk.
>
> I saw this https://openacs.org/forums/message-view?message_id=4263411, so
> I guess there is a work around, and it might be even better to work with
> nsv commands, But maybe we can make the module works?
>
> Thank You!
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> naviserver-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
> --
> Univ.Prof. Dr. Gustaf Neumann
> WU Vienna
> Institute of Information Systems and New Media
> Welthandelsplatz 1, A-1020 Vienna, Austria
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>


-- 
Regards,

Sassy Natan
972-(05)54-2203702

Re: [naviserver-devel] Fwd: NaviServer

[Gustaf N. <ne...@wu...>]

ns_share is outdated since ages. the workaround in [2] is not fixinig 
ns_share, but replacing ns_share via nsv commands. Have you looked at [1]?

-gn
PS: i've fixed just now the nsshare module on bitbucket to make it 
compile with actual compilers, against actual version of Tcl and 
NaviServer. ... but this is no recommendation to use it.

[1] https://naviserver.sourceforge.io/n/naviserver/files/nsv.html#section3
[2] https://openacs.org/forums/message-view?message_id=4263411

On 17.04.18 12:32, Sassy Natan wrote:
> I'm doing some work for a customer of mine, and he is still running 
> aolserver 3!!!
> It is 15 years system running on fedora server! really old one with 
> postgres 8.
>
> In my test I have manage to move the server to work with postgres 9.6 
> and aolserver 4.5, but I still on the process for moving it to run 
> under NaviSever.
>
> I want it to be run with the latest navisever and postgres 10.
>
> The problem is with the ns_share command.
>
> The server use some old tcl script from 
> ArsDigita-Community-System-3.2.3 which use ns_share command many 
> times. I would like to avoid fixing the code here.
>
> I saw there is some module name that implements some long-depreciated 
> old AOLserver commands 
> <https://bitbucket.org/naviserver/nsshare/src/099864af6816?at=default>. 
> However I never managed to compile it.
>
> I get some error like this:
>
> root@devplatfrom:/tmp/1# make NAVISERVER=/usr/lib/naviserver 
> DESTDIR=/tmp/121212/
> gcc -O2 -fomit-frame-pointer -DNDEBUG -Wall -fPIC -g -O2 
> -fdebug-prefix-map=/build/tcl8.6-1Mwawt/tcl8.6-8.6.8+dfsg=. 
> -fstack-protector-strong -Wformat -Werror=format-security 
> -fno-unit-at-a-time -pipe -Wdate-time -D_FORTIFY_SOURCE=2 
> -I/usr/lib/naviserver/include -I"/usr/include/tcl8.6" -DHAVE_CONFIG_H  
>   -c -o nsshare.o nsshare.c
> nsshare.c:93:1: error: conflicting types for ‘InitInterp’
>  InitInterp(Tcl_Interp *interp, void *arg)
>  ^~~~~~~~~~
> nsshare.c:46:24: note: previous declaration of ‘InitInterp’ was here
>  static Ns_TclTraceProc InitInterp;
>                         ^~~~~~~~~~
> nsshare.c:46:24: warning: ‘InitInterp’ used but never defined
> nsshare.c:93:1: warning: ‘InitInterp’ defined but not used 
> [-Wunused-function]
>  InitInterp(Tcl_Interp *interp, void *arg)
>  ^~~~~~~~~~
> <builtin>: recipe for target 'nsshare.o' failed
> make: *** [nsshare.o] Error 1
>
>
> Looking in the code, it seems to be like all other modules which I 
> manage to build, (nsaccess for example or nsdbpg etc..)
>
> What do I missing? as to do refactoring  to the 
> ArsDigita-Community-System-3.2.3 
> <https://github.com/MikeSisk/ArsDigita-Community-System-3.2.3/tree/master/tcl> 
> might be a huge risk.
>
> I saw this https://openacs.org/forums/message-view?message_id=4263411, 
> so I guess there is a work around, and it might be even better to work 
> with nsv commands, But maybe we can make the module works?
>
> Thank You!
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria