naviserver-devel — Developer discussion

[naviserver-devel] NaviServer 4-99.7 available

[Gustaf N. <ne...@wu...>]

Dear all,

below is the updated snipped of the news file.
The release 4.99.7 of NaviServer is tagged and uploaded
together with the modules to sourceforge, the documentation
pages on sourceforge are as well updated.

all the best
-gn


======================================
NaviServer 4.99.7, released 2015-02-23
======================================

Changes relative to 4.99.6
     216 files changed, 20967 insertions(+), 13287 deletions(-)

New Features:

   * Added Interface to NaviServer to read/write directly
     via driver routines. The new command "ns_connchan"
     provides a higher level alternative to the idiom
     "ns_conn channel" + Tcl-commands. The new command
     uses the internal socketcallback infrastructure and allows to
     
       - detach the currently open connection channel from a
         connection thread, to
       - read/write/close the channel from other threads, and to
       - query the states to then currently detached channels
         (similar to "ns_writer list")
       
     By accessing the driver infrastructure, "ns_connchan" makes it
     possible to read/write from/to SSL/TLS channels. The new command
     makes it easy e.g.  to implement WebSockets over SSL/TLS
     (ws://... and wss://...) which is not possible with the
     pre-existing infrastructure.

     Summary of Advantages:
       * use the same interface for http and https (and other drivers)
       * no (potential) problems with select() in Tcl when using
         many (>1024) file descriptors
       * less potential problems with Tcl-Sockets (buffering, translations)
       * lower resource consumption

     Implemented interface:
        ns_connchan detach
        ns_connchan close channel
        ns_connchan list ?-server server?
        ns_connchan callback ?-timeout s? ?-receivetimeout s? ?-sendtimeout s? channel command when
        ns_connchan read channel
        ns_connchan write channel string

   * Improved ns_log handling
     - New subcommand [ns_log stats] returns the statistics
       from calls to ns_log by severity-level. This allows
       to write monitoring tools to provide rough overviews
       at which time at which server what kind of
       servers/warnings/... were produced
       
     - Allow to modify logging of sql statements via
          ns_logctl severity Debug(sql) on
       at runtime
       
     - Add debugging severity for tasks "Debug(task)"

   * Added ability to load a driver module without listening on a
     port: This is useful in cases, where the driver registers
     client commands (such as nssock or nsssl) and only these
     are wanted. Loading the driver without listening can be
     achieved by specifying "0" for the port.

   * Improved support for windows compilation with native compilers
     for 32 and 64 bit (Many thanks to Andreas Piskorski)


Performance Improvements:
   - Added more compiler hints likely/unlikely
   - Made Ns_Encrypt threadsafe under Linux (reduce mutex)
   - Use getpwnam_r, getpwuid_r, getgrnam_r, and getgrgid_r instead of
     their thread-unsafe counterparts. This change allow these functions
     to run without locking.
   - Reduced size of compressed requests in several situations (empty io_vecs)


Bug Fixes:
   * Fixed encoding in [ns_conn content]
   * Fixed incorrect result of "ns_conn protocol" (returned empty)
   * Fixed crash when a thread was created with
     "ns_thread begindetached" and an error occurred
     during startup
   * Fixed memory management for TclThread by moving TclThreadArg into
     thread local storage
   * Fixed race condition in TLS cleanup
   * Fixed bug in argument parsing of "ns_cond"
   * Fixed setting of private/shared in ns_module
   * Fixed bug with "ns_adp_ctl stream on"
   * Moved network module filtering to the front to avoid double loading
     of Tcl modules (many thanks to William Scott Jordan for the fix)
   * ns_http:
     - Fixed trailing slashes in some urls
     - Fixed possible race condition when time gap between "ns_http queue"
       and "ns_http wait" is larger
   * Minor fixes in nscgi
   * Windows fixes
     - Fixed weird bugs when Tcl-functions are called during
       load of .dll files (showed up as hanging/crashing functions
       returning time from Tcl)
     - Fixed Ns_NormalizePath() for //foo/bar paths on Win32 (bug #1162890)
     - Use distinct bit-patterns for signal detection under windows.
     - Fixed array unset bug
   * Fixed a potential race condition in writer threads
   * [ns_perm delperm]: Fixed ignoring of argument "-noinherit"
   * Fixed reply headers in HEAD requests


Documentation improvements:
   - Improved typesetting, wording
   - Removed more obsolete content (e.g. concerning CVS, old dummy parameters)
   - Use examples from existing code basis
   - Improved comment density in source code
   - Documented missing subcommands, such as "ns_time format"


Tcl API Changes:
   * Marked "ns_geturl" as deprecated, use ns_http instead
   * Marked redundant names of subcommands in "ns_thread" as deprecated
     - ns_thread begin -> ns_thread create
     - ns_thread getid -> ns_thread id
     - ns_thread join -> ns_thread wait
     - ns_thread get -> ns_thread handle
   * Allow time interval of "-timeout" for "ns_db gethandle" to be
     specified with second factions (in the usual "ns_time" format)


C API Changes:
   - [ns_conn content]: new flag "-binary" to return always the raw bytes
   - [ns_thread begin|begindetached]: added flag "-name" for naming
     thread explicit


Configuration Changes:
   - Updated Makefiles as recommended in documentation


Code Changes:

   * Many of the changes are the result of a (sometimes over-picky)
     code-checkers on unix and windows systems, in particular pc-lint,
     cppcheck and the clang static checkers. Many thanks to
     Maurizio Martignano for setting up the testing framework
     based on SonarQube and for eagerly pull in updates.
   
   * Function prototypes and API definition
     - add missing function prototypes
     - removed occurrences of old-style prototypes (when functions are defined without arguments)
     - removed unneeded external references and symbols and unreferenced enumeration values
     - aligned argument names in function prototypes with function definitions
     - added non-null/returns non-null assertions for prototypes
     - added asserts to check non-null properties
     - added "const" when appropriate
     - added missing "static" declarations

   * Improved and more consistent macro definitions
     - use INT2PTR and PTR2INT consistently when converting between
       integers and pointers
     - use NS_TRUE/NS_FALSE instead of 1/0
     - use NS_INVALID_FD instead of -1
     - use NS_SOCKET and NS_INVALID_SOCKET instead of int/-1
     - use NS_INVALID_PID for invalid pids
     - change from usage of -1 or INVALID_SOCKET (windows predefined constant)
       to NS_INVALID_SOCKET due to confusions
     - use UNUSED consistently
     - character properties and conversions
        . new macro CHARTYPE for testing character properties
        . new macro CHARCONV for converting characters
     - added a new macros PTR2NSSOCK and NSSOCK2PTR in analogy to PTR2INT
       and INT2PTR to mask more win incompatibilities
     - prefer "const" over "CONST" unless in Tcl interface
     - improved consistency of variable and macro naming
     - put macro values into parenthesis

   * Improved general code quality
     - reduced variable scopes
     - remove cases of undefined C behavior
     - prefer leaving a switch statement via "break", and not via "return"
     - move assignments out of if-conditions
     - brace body of "if" statements and "for" loops
     - make ignoring of function results explicit
     - avoid shadowing of variable names
     - fix potentially uninitialized local and static variables

   * Used more precise and consistent types
     - prefer type "bool" over "int" when appropriate
     - use unsigned types for bit operations
     - prefer size_t, ssize_t, socklen_t, pid_t, nfds_t over int
     - use consistently uintptr_t for nextids
     - use enumerations instead of plain defines on some occasions
     - prefer boolean expression
     - reduce implicit conversions
       (signed/unsigned, signed constants, loss of precision, ...)
     - removed many cases of casts of complex expressions
     - reduce prohibited operations with plain (signed) chars
     - recast results of <<
     - standardize type definitions of objv in ObjCmds to the
       variant recommended by Tcl man pages

   * Improved C-standard compliance
     - silence "-pedantic" compilation with -stc=c99 under linux on most cases
     - avoid c90 identifiers starting with str[a-z], to[a-z], E[A-Z], _*
     - avoid c90 reserved identifier "signal", "free", "remove", "time"
     - added missing printf macros

   * Improved usage of c-library functions and OS abstraction
      - use arc4random() when available
      - use crypt_r() when available
      - prefer memcpy() over strncpy(), since it has in most cases the
        appropriate semantics and it is faster
     - new/fixed OS abstractions functions/macros:
        ns_lseek, ns_dup, ns_dup2, use ns_open, ns_close, ns_lseek, ns_dup,
        ns_dup2, ns_write, ns_read, ns_mkstemp

   * Removed potential crashes from buffer overflows
      - use memory bounded functions instead of potentially buffer
        overflowing functions
      - replace unsafe atof() by strtod()
      - replace unsafe atoi() by strtol()
      - replace call to deprecated function asctime() by a call to
        asctime_s()
      - remove hard-coded sizes of ip-addresses and replace it by NS_IPADDR_SIZE
      - don't hardcode sizes for string arrays

   * Use of modern-style argument parser Ns_ParseObjv() on several places
     instead of manual argv parsing
	ns_job interface, ns_truncate, ns_ftruncate,
	ns_addrbyhost, ns_hostbyaddr
	
   * Make mutex timings configurable via #define NS_NO_MUTEX_TIMING

   * Improved code formatting
     - add emacs editor hints to to keep formatting conventions
     - prefer spaces over tabs
     
   * Extended regression test


Changes in Modules worth to be noted:

   * websocket:
     New module implementing a WebSocket interface.  This is a Tcl-only
     module defined with two sample applications (chat and a file
     watcher for e.g error.log and access.log).  It support ws://
     (WebSocket over http) and wss:// (WebSocket over https).  For
     details and configuration, see the README file of the module.
     
   * nszlib:
     Added commands:

       ns_zlib deflate /data/
       ns_zlib inflate /data/

     which are compatible with the same-named php functions.
     [ns_zlib deflate ...] returns the raw compressed binary data.
     These functions are e.g. needed e.g. when implementing
     as sasl interface for shibboleth.

  * nsssl:
     - Ability to load the module without listening to socket by
      specifying 0 as port.  This is useful for e.g. just using the HTTPS
      client command ns_ssl.
    - Update configuration recommendations

  * nsoracle:
    Backport of the nsoracle driver from AOLserver to NaviServer with
    code cleanup.

Re: [naviserver-devel] NaviServer 4.99.7 ante portas

[David O. <da...@qc...>]

Great stuff. Thanks Gustaf!

On 20 February 2015 at 09:55, Gustaf Neumann <ne...@wu...> wrote:

> Dear Friends of NaviServer,
>
> the last release of NaviServer was more than half a year or more than
> 730 commits ago, so it becomes time for a release of 4.99.7 if this
> is fine with everybody.
>
> Below is the preliminary summary of the changes since the release
> of 4.99.6 (excluding the changes in the modules). If you have
> pending commits please let me know.
>
> i'll hope to be able to do the usual testing, updates of the
> versions numbers in the documentation, tagging, summarizing
> of the module changes etc. over the weekend.
>
> all the best
> -gustaf neumann
>
> PS: sooner or later a release 5.0 seems unavoidable, but are coming closer.
>
>
>
>
> ======================================
> NaviServer 4.99.7, released 2015-XX-XX
> ======================================
>
> Changes relative to 4.99.6
>      215 files changed, 20782 insertions(+), 13282 deletions(-)
>
> New Features:
>
>    * Added Interface to NaviServer to read/write directly
>      via driver routines. The new command "ns_connchan"
>      provides a higher level alternative to the idiom
>      "ns_conn channel" + Tcl-commands. The new command
>      uses the internal socketcallback infrastructure and allows to
>
>        - detach the currently open connection channel from a
>          connection thread, to
>        - read/write/close the channel from other threads, and to
>        - query the states to then currently detached channels
>          (similar to "ns_writer list")
>
>      By accessing the driver infrastructure, "ns_connchan" makes it
>      possible to read/write from/to SSL/TLS channels. The new command
>      makes it easy e.g.  to implement WebSockets over SSL/TLS
>      (ws://... and wss://...) which is not possible with the
>      pre-existing infrastructure.
>
>      Summary of Advantages:
>        * use the same interface for http and https (and other drivers)
>        * no (potential) problems with select() in Tcl when using
>          many (>1024) file descriptors
>        * less potential problems with Tcl-Sockets (buffering, translations)
>        * lower resource consumption
>
>      Implemented interface:
>         ns_connchan detach
>         ns_connchan close channel
>         ns_connchan list ?-server server?
>         ns_connchan callback ?-timeout s? ?-receivetimeout s?
> ?-sendtimeout s? channel command when
>         ns_connchan read channel
>         ns_connchan write channel string
>
>    * Improved ns_log handling
>      - new subcommand [ns_log stats] returns the statistics
>        from calls to ns_log by severity-level. This allows
>        to write monitoring tools to provide rough overviews
>        at which time at which server what kind of
>        servers/warnings/... were produced
>
>      - Allow to modify logging of sql statements via
>           ns_logctl severity Debug(sql) on
>        at runtime
>
>      - Add debugging severity for tasks "Debug(task)"
>
>    * Added ability to load a driver module without listening on a
>      port: This is useful in cases, where the driver registers
>      client commands (such as nssock or nsssl) and only these
>      are wanted. Loading the driver without listening can be
>      achieved by specifying "0" for the port.
>
>    * Improved support for windows compilation with native compilers
>      for 32 and 64 bit (Many thanks to Andreas Piskorski)
>
>
> Performance Improvements:
>    - add compiler hints likely/unlikely
>    - make Ns_Encrypt threadsafe under Linux (reduce mutex)
>    - use getpwnam_r, getpwuid_r, getgrnam_r, and getgrgid_r instead of
>      their thread-unsafe counterparts. This change allow these functions
>      to run without locking.
>    - reduce size of compressed requests in several situations (empty
> io_vecs)
>
>
> Bug Fixes:
>    * fixed encoding in [ns_conn content]
>    * fixed incorrect result of "ns_conn protocol" (returned empty)
>    * fixed crash when a thread was created with
>      "ns_thread begindetached" and an error occurred
>      during startup
>    * fixed memory management for TclThread by moving TclThreadArg into
>      thread local storage
>    * fixed race condition in TLS cleanup
>    * fixed bug in argument parsing of "ns_cond"
>    * fixed setting of private/shared in ns_module
>    * fixed bug with "ns_adp_ctl stream on"
>    * move network module filtering to the front to avoid double loading
>      of tcl modules (many thanks to William Scott Jordan for the fix)
>    * ns_http:
>      - fix trailing slashes in some urls
>      - fix possible race condition when time gap between "ns_http queue"
>        and "ns_http wait" is larger
>    * minor fixes in nscgi
>    * windows
>      - fixed Ns_NormalizePath() for //foo/bar paths on Win32 (bug #1162890)
>      - use distinct bit-patterns for signal detection under windows.
>       - fix array unset bug
>    * fixed a potential race condition in writer threads
>    * [ns_perm delperm] fixed ignoring of argument "-noinherit"
>    * fixed reply headers in HEAD requests
>
>
> Documentation improvements:
>    - improve typesetting, wording
>    - remove more obsolete content (e.g. concerning CVS, old dummy
> parameters)
>    - use examples from existing code basis
>    - improve comment density in source code
>    - document missing subcommands, such as "ns_time format"
>
>
> Tcl API Changes:
>    * mark "ns_geturl" as deprecated, use ns_http instead
>    * marked redundant names of subcommands in "ns_thread" as deprecated
>      - ns_thread begin -> ns_thread create
>      - ns_thread getid -> ns_thread id
>      - ns_thread join -> ns_thread wait
>      - ns_thread get -> ns_thread handle
>    * allow time interval of "-timeout" for "ns_db gethandle" to be
>      specified with second factions (in the usual "ns_time" format)
>
>
> C API Changes:
>    - [ns_conn content]: new flag "-binary" to return always the raw bytes
>    - [ns_thread begin|begindetached]: added flag "-name" for naming
>      thread explicit
>
>
> Configuration Changes:
>    - update Makefiles as recommended in documentation
>
>
> Code Changes:
>
>    * Many of the changes are the result of a (sometimes over-picky)
>      code-checkers on unix and windows systems, in particular pc-lint,
>      cppcheck and the clang static checkers. Many thanks to
>      Maurizio Martignano for setting up the testing framework
>      based on SonarQube.
>
>    * Function prototypes and API definition
>      - add missing function prototypes
>      - removed occurrences of old-style prototypes (when functions are
> defined without arguments)
>      - removed unneeded external references and symbols and unreferenced
> enumeration values
>      - aligned argument names in function prototypes with function
> definitions
>      - added non-null/returns non-null assertions for prototypes
>      - added asserts to check non-null properties
>      - added "const" when appropriate
>      - added missing "static" declarations
>
>    * Improved and more consistent macro definitions
>      - use INT2PTR and PTR2INT consistently when converting between
>        integers and pointers
>      - use NS_TRUE/NS_FALSE instead of 1/0
>      - use NS_INVALID_FD instead of -1
>      - use NS_SOCKET and NS_INVALID_SOCKET instead of int/-1
>      - use NS_INVALID_PID for invalid pids
>      - change from usage of -1 or INVALID_SOCKET (windows predefined
> constant)
>        to NS_INVALID_SOCKET due to confusions
>      - use UNUSED consistently
>      - character properties and conversions
>         . new macro CHARTYPE for testing character properties
>         . new macro CHARCONV for converting characters
>      - added a new macros PTR2NSSOCK and NSSOCK2PTR in analogy to PTR2INT
>        and INT2PTR to mask more win incompatibilities
>      - prefer "const" over "CONST" unless in Tcl interface
>      - improved consistency of variable and macro naming
>      - put macro values into parenthesis
>
>    * Improved general code quality
>      - reduced variable scopes
>      - remove cases of undefined C behavior
>      - prefer leaving a switch statement via "break", and not via "return"
>      - move assignments out of if-conditions
>      - brace body of "if" statements and "for" loops
>      - make ignoring of function results explicit
>      - avoid shadowing of variable names
>      - fix potentially uninitialized local and static variables
>
>    * Used more precise and consistent types
>      - prefer type "bool" over "int" when appropriate
>      - use unsigned types for bit operations
>      - prefer size_t, ssize_t, socklen_t, pid_t, nfds_t over int
>      - use consistently uintptr_t for nextids
>      - use enumerations instead of plain defines on some occasions
>      - prefer boolean expression
>      - reduce implicit conversions
>        (signed/unsigned, signed constants, loss of precision, ...)
>      - removed many cases of casts of complex expressions
>      - reduce prohibited operations with plain (signed) chars
>      - recast results of <<
>      - standardize type definitions of objv in ObjCmds to the
>        variant recommended by Tcl man pages
>
>    * Improved C-standard compliance
>      - silence "-pedantic" compilation with -stc=c99 under linux on most
> cases
>      - avoid c90 identifiers starting with str[a-z], to[a-z], E[A-Z], _*
>      - avoid c90 reserved identifier "signal", "free", "remove", "time"
>      - added missing printf macros
>
>    * Improved usage of c-library functions and OS abstraction
>       - use arc4random() when available
>       - use crypt_r() when available
>       - prefer memcpy() over strncpy(), since it has in most cases the
>         appropriate semantics and it is faster
>      - new/fixed OS abstractions functions/macros:
>         ns_lseek, ns_dup, ns_dup2, use ns_open, ns_close, ns_lseek, ns_dup,
>         ns_dup2, ns_write, ns_read, ns_mkstemp
>
>    * Removed potential crashes from buffer overflows
>       - use memory bounded functions instead of potentially buffer
>         overflowing functions
>       - replace unsafe atof() by strtod()
>       - replace unsafe atoi() by strtol()
>       - replace call to deprecated function asctime() by a call to
>         asctime_s()
>       - remove hard-coded sizes of ip-addresses and replace it
>         by NS_IPADDR_SIZE
>       - don't hardcode sizes for string arrays
>
>    * Use of modern-style argument parser Ns_ParseObjv() on several places
>      instead of manual argv parsing
>         ns_job interface, ns_truncate, ns_ftruncate,
>         ns_addrbyhost, ns_hostbyaddr
>
>    * Make mutex timings configurable via #define NS_NO_MUTEX_TIMING
>
>    * Improved code formatting
>      - add emacs editor hints to to keep formatting conventions
>      - prefer spaces over tabs
>
>    * Extended regression test
>
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>



-- 
David Osborne
Qcode Software Limited
http://www.qcode.co.uk

Re: [naviserver-devel] NaviServer 4.99.7 ante portas

[Cesáreo G. R. <ce...@ce...>]

Hi Gustaf

Many thanks. Really a great job!

Thanks
Cesáreo

El 20/febrero/15 a las 4:55, Gustaf Neumann escribió:
> Dear Friends of NaviServer,
>
> the last release of NaviServer was more than half a year or more than
> 730 commits ago, so it becomes time for a release of 4.99.7 if this
> is fine with everybody.
>
> Below is the preliminary summary of the changes since the release
> of 4.99.6 (excluding the changes in the modules). If you have
> pending commits please let me know.
>
> i'll hope to be able to do the usual testing, updates of the
> versions numbers in the documentation, tagging, summarizing
> of the module changes etc. over the weekend.
>
> all the best
> -gustaf neumann
>
> PS: sooner or later a release 5.0 seems unavoidable, but are coming closer.
>
>
>
>
> ======================================
> NaviServer 4.99.7, released 2015-XX-XX
> ======================================
>
> Changes relative to 4.99.6
>       215 files changed, 20782 insertions(+), 13282 deletions(-)
>
> New Features:
>
>     * Added Interface to NaviServer to read/write directly
>       via driver routines. The new command "ns_connchan"
>       provides a higher level alternative to the idiom
>       "ns_conn channel" + Tcl-commands. The new command
>       uses the internal socketcallback infrastructure and allows to
>
>         - detach the currently open connection channel from a
>           connection thread, to
>         - read/write/close the channel from other threads, and to
>         - query the states to then currently detached channels
>           (similar to "ns_writer list")
>
>       By accessing the driver infrastructure, "ns_connchan" makes it
>       possible to read/write from/to SSL/TLS channels. The new command
>       makes it easy e.g.  to implement WebSockets over SSL/TLS
>       (ws://... and wss://...) which is not possible with the
>       pre-existing infrastructure.
>
>       Summary of Advantages:
>         * use the same interface for http and https (and other drivers)
>         * no (potential) problems with select() in Tcl when using
>           many (>1024) file descriptors
>         * less potential problems with Tcl-Sockets (buffering, translations)
>         * lower resource consumption
>
>       Implemented interface:
>          ns_connchan detach
>          ns_connchan close channel
>          ns_connchan list ?-server server?
>          ns_connchan callback ?-timeout s? ?-receivetimeout s? ?-sendtimeout s? channel command when
>          ns_connchan read channel
>          ns_connchan write channel string
>
>     * Improved ns_log handling
>       - new subcommand [ns_log stats] returns the statistics
>         from calls to ns_log by severity-level. This allows
>         to write monitoring tools to provide rough overviews
>         at which time at which server what kind of
>         servers/warnings/... were produced
>
>       - Allow to modify logging of sql statements via
>            ns_logctl severity Debug(sql) on
>         at runtime
>
>       - Add debugging severity for tasks "Debug(task)"
>
>     * Added ability to load a driver module without listening on a
>       port: This is useful in cases, where the driver registers
>       client commands (such as nssock or nsssl) and only these
>       are wanted. Loading the driver without listening can be
>       achieved by specifying "0" for the port.
>
>     * Improved support for windows compilation with native compilers
>       for 32 and 64 bit (Many thanks to Andreas Piskorski)
>
>
> Performance Improvements:
>     - add compiler hints likely/unlikely
>     - make Ns_Encrypt threadsafe under Linux (reduce mutex)
>     - use getpwnam_r, getpwuid_r, getgrnam_r, and getgrgid_r instead of
>       their thread-unsafe counterparts. This change allow these functions
>       to run without locking.
>     - reduce size of compressed requests in several situations (empty io_vecs)
>
>
> Bug Fixes:
>     * fixed encoding in [ns_conn content]
>     * fixed incorrect result of "ns_conn protocol" (returned empty)
>     * fixed crash when a thread was created with
>       "ns_thread begindetached" and an error occurred
>       during startup
>     * fixed memory management for TclThread by moving TclThreadArg into
>       thread local storage
>     * fixed race condition in TLS cleanup
>     * fixed bug in argument parsing of "ns_cond"
>     * fixed setting of private/shared in ns_module
>     * fixed bug with "ns_adp_ctl stream on"
>     * move network module filtering to the front to avoid double loading
>       of tcl modules (many thanks to William Scott Jordan for the fix)
>     * ns_http:
>       - fix trailing slashes in some urls
>       - fix possible race condition when time gap between "ns_http queue"
>         and "ns_http wait" is larger
>     * minor fixes in nscgi
>     * windows
>       - fixed Ns_NormalizePath() for //foo/bar paths on Win32 (bug #1162890)
>       - use distinct bit-patterns for signal detection under windows.
>        - fix array unset bug
>     * fixed a potential race condition in writer threads
>     * [ns_perm delperm] fixed ignoring of argument "-noinherit"
>     * fixed reply headers in HEAD requests
>
>
> Documentation improvements:
>     - improve typesetting, wording
>     - remove more obsolete content (e.g. concerning CVS, old dummy parameters)
>     - use examples from existing code basis
>     - improve comment density in source code
>     - document missing subcommands, such as "ns_time format"
>
>
> Tcl API Changes:
>     * mark "ns_geturl" as deprecated, use ns_http instead
>     * marked redundant names of subcommands in "ns_thread" as deprecated
>       - ns_thread begin -> ns_thread create
>       - ns_thread getid -> ns_thread id
>       - ns_thread join -> ns_thread wait
>       - ns_thread get -> ns_thread handle
>     * allow time interval of "-timeout" for "ns_db gethandle" to be
>       specified with second factions (in the usual "ns_time" format)
>
>
> C API Changes:
>     - [ns_conn content]: new flag "-binary" to return always the raw bytes
>     - [ns_thread begin|begindetached]: added flag "-name" for naming
>       thread explicit
>
>
> Configuration Changes:
>     - update Makefiles as recommended in documentation
>
>
> Code Changes:
>
>     * Many of the changes are the result of a (sometimes over-picky)
>       code-checkers on unix and windows systems, in particular pc-lint,
>       cppcheck and the clang static checkers. Many thanks to
>       Maurizio Martignano for setting up the testing framework
>       based on SonarQube.
>
>     * Function prototypes and API definition
>       - add missing function prototypes
>       - removed occurrences of old-style prototypes (when functions are defined without arguments)
>       - removed unneeded external references and symbols and unreferenced enumeration values
>       - aligned argument names in function prototypes with function definitions
>       - added non-null/returns non-null assertions for prototypes
>       - added asserts to check non-null properties
>       - added "const" when appropriate
>       - added missing "static" declarations
>
>     * Improved and more consistent macro definitions
>       - use INT2PTR and PTR2INT consistently when converting between
>         integers and pointers
>       - use NS_TRUE/NS_FALSE instead of 1/0
>       - use NS_INVALID_FD instead of -1
>       - use NS_SOCKET and NS_INVALID_SOCKET instead of int/-1
>       - use NS_INVALID_PID for invalid pids
>       - change from usage of -1 or INVALID_SOCKET (windows predefined constant)
>         to NS_INVALID_SOCKET due to confusions
>       - use UNUSED consistently
>       - character properties and conversions
>          . new macro CHARTYPE for testing character properties
>          . new macro CHARCONV for converting characters
>       - added a new macros PTR2NSSOCK and NSSOCK2PTR in analogy to PTR2INT
>         and INT2PTR to mask more win incompatibilities
>       - prefer "const" over "CONST" unless in Tcl interface
>       - improved consistency of variable and macro naming
>       - put macro values into parenthesis
>
>     * Improved general code quality
>       - reduced variable scopes
>       - remove cases of undefined C behavior
>       - prefer leaving a switch statement via "break", and not via "return"
>       - move assignments out of if-conditions
>       - brace body of "if" statements and "for" loops
>       - make ignoring of function results explicit
>       - avoid shadowing of variable names
>       - fix potentially uninitialized local and static variables
>
>     * Used more precise and consistent types
>       - prefer type "bool" over "int" when appropriate
>       - use unsigned types for bit operations
>       - prefer size_t, ssize_t, socklen_t, pid_t, nfds_t over int
>       - use consistently uintptr_t for nextids
>       - use enumerations instead of plain defines on some occasions
>       - prefer boolean expression
>       - reduce implicit conversions
>         (signed/unsigned, signed constants, loss of precision, ...)
>       - removed many cases of casts of complex expressions
>       - reduce prohibited operations with plain (signed) chars
>       - recast results of <<
>       - standardize type definitions of objv in ObjCmds to the
>         variant recommended by Tcl man pages
>
>     * Improved C-standard compliance
>       - silence "-pedantic" compilation with -stc=c99 under linux on most cases
>       - avoid c90 identifiers starting with str[a-z], to[a-z], E[A-Z], _*
>       - avoid c90 reserved identifier "signal", "free", "remove", "time"
>       - added missing printf macros
>
>     * Improved usage of c-library functions and OS abstraction
>        - use arc4random() when available
>        - use crypt_r() when available
>        - prefer memcpy() over strncpy(), since it has in most cases the
>          appropriate semantics and it is faster
>       - new/fixed OS abstractions functions/macros:
>          ns_lseek, ns_dup, ns_dup2, use ns_open, ns_close, ns_lseek, ns_dup,
>          ns_dup2, ns_write, ns_read, ns_mkstemp
>
>     * Removed potential crashes from buffer overflows
>        - use memory bounded functions instead of potentially buffer
>          overflowing functions
>        - replace unsafe atof() by strtod()
>        - replace unsafe atoi() by strtol()
>        - replace call to deprecated function asctime() by a call to
>          asctime_s()
>        - remove hard-coded sizes of ip-addresses and replace it
>          by NS_IPADDR_SIZE
>        - don't hardcode sizes for string arrays
>
>     * Use of modern-style argument parser Ns_ParseObjv() on several places
>       instead of manual argv parsing
> 	ns_job interface, ns_truncate, ns_ftruncate,
> 	ns_addrbyhost, ns_hostbyaddr
> 	
>     * Make mutex timings configurable via #define NS_NO_MUTEX_TIMING
>
>     * Improved code formatting
>       - add emacs editor hints to to keep formatting conventions
>       - prefer spaces over tabs
>
>     * Extended regression test
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] NaviServer 4.99.7 ante portas

[Zoran V. <zv...@ar...>]

On 20 Feb 2015, at 10:55, Gustaf Neumann <ne...@wu...> wrote:

> Changes relative to 4.99.6
>     215 files changed, 20782 insertions(+), 13282 deletions(-)

I'm impressed!
Execellent work, as usual :-)

Cheers,
Zoran

[naviserver-devel] NaviServer 4.99.7 ante portas

[Gustaf N. <ne...@wu...>]

Dear Friends of NaviServer,

the last release of NaviServer was more than half a year or more than
730 commits ago, so it becomes time for a release of 4.99.7 if this
is fine with everybody.

Below is the preliminary summary of the changes since the release
of 4.99.6 (excluding the changes in the modules). If you have
pending commits please let me know.

i'll hope to be able to do the usual testing, updates of the
versions numbers in the documentation, tagging, summarizing
of the module changes etc. over the weekend.

all the best
-gustaf neumann

PS: sooner or later a release 5.0 seems unavoidable, but are coming closer.




======================================
NaviServer 4.99.7, released 2015-XX-XX
======================================

Changes relative to 4.99.6
     215 files changed, 20782 insertions(+), 13282 deletions(-)

New Features:

   * Added Interface to NaviServer to read/write directly
     via driver routines. The new command "ns_connchan"
     provides a higher level alternative to the idiom
     "ns_conn channel" + Tcl-commands. The new command
     uses the internal socketcallback infrastructure and allows to
     
       - detach the currently open connection channel from a
         connection thread, to
       - read/write/close the channel from other threads, and to
       - query the states to then currently detached channels
         (similar to "ns_writer list")
       
     By accessing the driver infrastructure, "ns_connchan" makes it
     possible to read/write from/to SSL/TLS channels. The new command
     makes it easy e.g.  to implement WebSockets over SSL/TLS
     (ws://... and wss://...) which is not possible with the
     pre-existing infrastructure.

     Summary of Advantages:
       * use the same interface for http and https (and other drivers)
       * no (potential) problems with select() in Tcl when using
         many (>1024) file descriptors
       * less potential problems with Tcl-Sockets (buffering, translations)
       * lower resource consumption

     Implemented interface:
        ns_connchan detach
        ns_connchan close channel
        ns_connchan list ?-server server?
        ns_connchan callback ?-timeout s? ?-receivetimeout s? ?-sendtimeout s? channel command when
        ns_connchan read channel
        ns_connchan write channel string

   * Improved ns_log handling
     - new subcommand [ns_log stats] returns the statistics
       from calls to ns_log by severity-level. This allows
       to write monitoring tools to provide rough overviews
       at which time at which server what kind of
       servers/warnings/... were produced
       
     - Allow to modify logging of sql statements via
          ns_logctl severity Debug(sql) on
       at runtime
       
     - Add debugging severity for tasks "Debug(task)"

   * Added ability to load a driver module without listening on a
     port: This is useful in cases, where the driver registers
     client commands (such as nssock or nsssl) and only these
     are wanted. Loading the driver without listening can be
     achieved by specifying "0" for the port.

   * Improved support for windows compilation with native compilers
     for 32 and 64 bit (Many thanks to Andreas Piskorski)


Performance Improvements:
   - add compiler hints likely/unlikely
   - make Ns_Encrypt threadsafe under Linux (reduce mutex)
   - use getpwnam_r, getpwuid_r, getgrnam_r, and getgrgid_r instead of
     their thread-unsafe counterparts. This change allow these functions
     to run without locking.
   - reduce size of compressed requests in several situations (empty io_vecs)


Bug Fixes:
   * fixed encoding in [ns_conn content]
   * fixed incorrect result of "ns_conn protocol" (returned empty)
   * fixed crash when a thread was created with
     "ns_thread begindetached" and an error occurred
     during startup
   * fixed memory management for TclThread by moving TclThreadArg into
     thread local storage
   * fixed race condition in TLS cleanup
   * fixed bug in argument parsing of "ns_cond"
   * fixed setting of private/shared in ns_module
   * fixed bug with "ns_adp_ctl stream on"
   * move network module filtering to the front to avoid double loading
     of tcl modules (many thanks to William Scott Jordan for the fix)
   * ns_http:
     - fix trailing slashes in some urls
     - fix possible race condition when time gap between "ns_http queue"
       and "ns_http wait" is larger
   * minor fixes in nscgi
   * windows
     - fixed Ns_NormalizePath() for //foo/bar paths on Win32 (bug #1162890)
     - use distinct bit-patterns for signal detection under windows.
      - fix array unset bug
   * fixed a potential race condition in writer threads
   * [ns_perm delperm] fixed ignoring of argument "-noinherit"
   * fixed reply headers in HEAD requests


Documentation improvements:
   - improve typesetting, wording
   - remove more obsolete content (e.g. concerning CVS, old dummy parameters)
   - use examples from existing code basis
   - improve comment density in source code
   - document missing subcommands, such as "ns_time format"


Tcl API Changes:
   * mark "ns_geturl" as deprecated, use ns_http instead
   * marked redundant names of subcommands in "ns_thread" as deprecated
     - ns_thread begin -> ns_thread create
     - ns_thread getid -> ns_thread id
     - ns_thread join -> ns_thread wait
     - ns_thread get -> ns_thread handle
   * allow time interval of "-timeout" for "ns_db gethandle" to be
     specified with second factions (in the usual "ns_time" format)


C API Changes:
   - [ns_conn content]: new flag "-binary" to return always the raw bytes
   - [ns_thread begin|begindetached]: added flag "-name" for naming
     thread explicit


Configuration Changes:
   - update Makefiles as recommended in documentation


Code Changes:

   * Many of the changes are the result of a (sometimes over-picky)
     code-checkers on unix and windows systems, in particular pc-lint,
     cppcheck and the clang static checkers. Many thanks to
     Maurizio Martignano for setting up the testing framework
     based on SonarQube.
   
   * Function prototypes and API definition
     - add missing function prototypes
     - removed occurrences of old-style prototypes (when functions are defined without arguments)
     - removed unneeded external references and symbols and unreferenced enumeration values
     - aligned argument names in function prototypes with function definitions
     - added non-null/returns non-null assertions for prototypes
     - added asserts to check non-null properties
     - added "const" when appropriate
     - added missing "static" declarations

   * Improved and more consistent macro definitions
     - use INT2PTR and PTR2INT consistently when converting between
       integers and pointers
     - use NS_TRUE/NS_FALSE instead of 1/0
     - use NS_INVALID_FD instead of -1
     - use NS_SOCKET and NS_INVALID_SOCKET instead of int/-1
     - use NS_INVALID_PID for invalid pids
     - change from usage of -1 or INVALID_SOCKET (windows predefined constant)
       to NS_INVALID_SOCKET due to confusions
     - use UNUSED consistently
     - character properties and conversions
        . new macro CHARTYPE for testing character properties
        . new macro CHARCONV for converting characters
     - added a new macros PTR2NSSOCK and NSSOCK2PTR in analogy to PTR2INT
       and INT2PTR to mask more win incompatibilities
     - prefer "const" over "CONST" unless in Tcl interface
     - improved consistency of variable and macro naming
     - put macro values into parenthesis

   * Improved general code quality
     - reduced variable scopes
     - remove cases of undefined C behavior
     - prefer leaving a switch statement via "break", and not via "return"
     - move assignments out of if-conditions
     - brace body of "if" statements and "for" loops
     - make ignoring of function results explicit
     - avoid shadowing of variable names
     - fix potentially uninitialized local and static variables

   * Used more precise and consistent types
     - prefer type "bool" over "int" when appropriate
     - use unsigned types for bit operations
     - prefer size_t, ssize_t, socklen_t, pid_t, nfds_t over int
     - use consistently uintptr_t for nextids
     - use enumerations instead of plain defines on some occasions
     - prefer boolean expression
     - reduce implicit conversions
       (signed/unsigned, signed constants, loss of precision, ...)
     - removed many cases of casts of complex expressions
     - reduce prohibited operations with plain (signed) chars
     - recast results of <<
     - standardize type definitions of objv in ObjCmds to the
       variant recommended by Tcl man pages

   * Improved C-standard compliance
     - silence "-pedantic" compilation with -stc=c99 under linux on most cases
     - avoid c90 identifiers starting with str[a-z], to[a-z], E[A-Z], _*
     - avoid c90 reserved identifier "signal", "free", "remove", "time"
     - added missing printf macros

   * Improved usage of c-library functions and OS abstraction
      - use arc4random() when available
      - use crypt_r() when available
      - prefer memcpy() over strncpy(), since it has in most cases the
        appropriate semantics and it is faster
     - new/fixed OS abstractions functions/macros:
        ns_lseek, ns_dup, ns_dup2, use ns_open, ns_close, ns_lseek, ns_dup,
        ns_dup2, ns_write, ns_read, ns_mkstemp

   * Removed potential crashes from buffer overflows
      - use memory bounded functions instead of potentially buffer
        overflowing functions
      - replace unsafe atof() by strtod()
      - replace unsafe atoi() by strtol()
      - replace call to deprecated function asctime() by a call to
        asctime_s()
      - remove hard-coded sizes of ip-addresses and replace it
        by NS_IPADDR_SIZE
      - don't hardcode sizes for string arrays

   * Use of modern-style argument parser Ns_ParseObjv() on several places
     instead of manual argv parsing
	ns_job interface, ns_truncate, ns_ftruncate,
	ns_addrbyhost, ns_hostbyaddr
	
   * Make mutex timings configurable via #define NS_NO_MUTEX_TIMING

   * Improved code formatting
     - add emacs editor hints to to keep formatting conventions
     - prefer spaces over tabs
     
   * Extended regression test

     

Re: [naviserver-devel] Some Questions about NaviServer development

[Gustaf N. <ne...@wu...>]

Am 19.01.15 um 18:37 schrieb Clinciu Andrei:
> First I'd like to wish you all a happy new year.
>
> Soon it will be almost one year from the first time I started using 
> and learning  NaviServer.
hi and welcome to the club!
> In the meantime I've started developing a web framework as an exercise 
> (it's taking almost an year) to learn everything I can about 
> NaviServer. During this time I;ve had the time to dive in many cool 
> things but also faced many problems having to rewrite a lot of code.
> I've developed a short list of questions I hope someone has an answer 
> or knows of an alternative. I had many more but I prefer to RTM and 
> try everything before I ask.
> If anyone has similar experiences in production/development or  has 
> any "best tuning/usage" advice I'd be more than happy to read it.
>
> I've started using NSSSL , however, is there a possibility to make it 
> work on a per virtual host? *That it can load the .pem file from a 
> location within the virtual host location? *
If you have just a few server then the following setup should work:

ns_section ns/servers
         ns_param        $serverA       $serverAName
         ns_param        $serverB      $serverBName

ns_section ns/server/${serverA}/modules
         ns_param   nssslA             nsssl.so

ns_section    "ns/server/${serverA}/module/nssslA"
        ns_param      certificate        $serverroot/etc/cert.pem

im am not sure, how well this will work with mass hosting, since every 
server will probably need it own driver instance.

> The framework I'm working with is in TclOO (i was using namespaces but 
> I had problems generating the pages, as somethings wouldn't render 
> correctly so had to make a new object for each connection) however it 
> seems not to work when preloading files so I have to load everything 
> each time a user hits the page. I've tried to make it work and even 
> trying to serialize everything but I did not seem to succeed. Did 
> anyone make it work? *Or is it a wiser choice to switch everything to 
> NX 2.0? *
The problem with TclOO in this respect is that some features of XOTcl 
were dropped by design. It has e.g. no support for serialization and 
recreation of objects/classes, which are part of XOTcl 2.0 and NX. My 
recommendation is certainly to use NX, but i am certainly biased.
>
> Same as above, is there a way to reload files into the init for all 
> threads once the server is started so there will not be a need to 
> restart it?
There is no full built-in support in NaviServer, but it can be done. 
OpenACS has a mechanism for hot-reloading of Tcl/Objects/Classes, we use 
this all the time.
>
> I''ve noted a problem with keep-alive on most of the machines where 
> I've tested it. When accessing any domain form outside the LAN or 
> localhost it seems to take a long time to complete the load of images 
> or bigger files. Tests in Chrome and Firefox with the network analyzer 
> show that it loads the file in under one second but it keeps the 
> connection open for the file untill the max keep-alive seconds have 
> elapsed. So If i have a keep-alive of 10-20 seconds the page won't 
> load untill it's closed. I tried to put it to 5 seconds but it seems 
> way too long. The only good keep-alive time that seems to be adequate 
> is 2 seconds. /*By disabling it, will the server be more burdened or 
> doesn't it affect anything?
> */
a keep-alive of 10-20 seconds is way to long.
next-scripting.org has a closewait of 2 and a keepwait of 5, which are 
the default values, with that it works quite ok.
Reducing the value does not harm, but every new request implies a new 
tcp-connections which implies a new 3-way handshake in the standard 
case. other than this, there is no bad side effect.
> Is using the fastpath  for css, img, js files 100-200 kb a good choice 
> or should I think of a accelerator/proxy?
there should be no need for this. depending on your loads, configure a 
few writer threads and compressed static deliveries ( gzip_static in the 
fastpath section).

> I've tried many tuning options for the server to make it server many 
> .adp requests. However they seem to never go above 100 requests/sec 
> (with big latency) with simple writing/reading from db. Is this 
> normal? (Quad-Core and 16-core tests..) Compared to apache with a PHP 
> yii framework, the framework in PHP has faster response time 120 r/s.
there seems to be something wrong in your setup. processing .adp files 
can be quite fast. just now, i've checked the delivery of the default 
index.adp of a server, where server A (intel machine) requests this page 
over the local network from server B (as well an intel machine). I've 
measured 6594 requests per seconds (the default configuration with 
maxthreads increased).

an .adp page can certainly perform complex operations. Do you see this 
bad performance for all requests?
> When using mmap on linux the loading time decreases substantially of 
> any web page naviserver serves, sometimes 30 seconds or more...
when testing with 5k sized on the same configuration as above, i see no 
big differences, the configuration without mmap is slightly faster.
serving 5k.html with mmap: 5781 req/sec
serving 5k.html without mmap: 5847 req/sec
> Also sometimes when using MySQL  with ns_dbi without embeding the 
> database. It can happen that it gives an error "could not open handle" 
> and then the server crashes once in a while without any information. 
> Or it won't reconnect without a server restart. This isn't such a 
> problem since I use postgresql but some clients still have data  and 
> require extraction or manipulation from mysql and errors like these 
> are weird.
i can't comment on mysql, we use mostly postgres, in connection with 
OpenACS, nsdbpg is still recommended. We have seen siginficant slowdowns 
with nsdbipg when it runs out of the prepared case, openacs has many 
"... in (...)" queries, which are bad for preparing.

all the best
-gn

[naviserver-devel] Some Questions about NaviServer development

[Clinciu A. <the...@ya...>]

First I'd like to wish you all a happy new year.
Soon it will be almost one year from the first time I started using and learning  NaviServer. 
In the meantime I've started developing a web framework as an exercise (it's taking almost an year) to learn everything I can about NaviServer. During this time I;ve had the time to dive in many cool things but also faced many   problems having to rewrite a lot of code.
I've developed a short list of questions I hope someone has an answer or knows of an alternative. I had many more but I prefer to RTM and try everything before I ask.If anyone has similar experiences in production/development or  has any "best tuning/usage" advice I'd be more than happy to read it.

I've started using NSSSL , however, is there a possibility to make it work on a per virtual host? That it can load the .pem file from a location within the virtual host location? 

The framework I'm working with is in TclOO (i was using namespaces but I had problems generating the pages, as somethings wouldn't render correctly so had to make a new object for each connection) however it seems not to work when preloading files so I have to load everything each time a user hits the page. I've tried to make it work and even trying to serialize everything but I did not seem to succeed. Did anyone make it work? Or is it a wiser choice to switch everything to NX 2.0? 

Same as above, is there a way to reload files into the init for all threads once the server is started so there will not be a need to restart it?
I''ve noted a problem with keep-alive on most of the machines where I've tested it. When accessing any domain form outside the LAN or localhost it seems to take a long time to complete the load of images or bigger files. Tests in Chrome and Firefox with the network analyzer show that it loads the file in under one second but it keeps the connection open for the file untill the max keep-alive seconds have elapsed. So If i have a keep-alive of 10-20 seconds the page won't load untill it's closed. I tried to put it to 5 seconds but it seems way too long. The only good keep-alive time that seems to be adequate is 2 seconds. By disabling it, will the server be more burdened or doesn't it affect anything? 
Is using the fastpath  for css, img, js files 100-200 kb a good choice or should I think of a accelerator/proxy? 

I've tried many tuning options for the server to make it server many .adp requests. However they seem to never go above 100 requests/sec (with big latency) with simple writing/reading from db. Is this normal? (Quad-Core and 16-core tests..) Compared to apache with a PHP yii framework, the framework in PHP has faster response time 120 r/s.

When using mmap on linux the loading time decreases substantially of any web page naviserver serves, sometimes 30 seconds or more...

Also sometimes when using MySQL  with ns_dbi without embeding the database. It can happen that it gives an error "could not open handle" and then the server crashes once in a while without any information. Or it won't reconnect without a server restart. This isn't such a problem since I use postgresql but some clients still have data  and require extraction or manipulation from mysql and errors like these are weird.

Tests have been done in both 4.99.5 and 4.99.6 and the bitbucket versions.

Thanks for the continuing support you guys provide. We now have a very powerful webserver with integrated TCL support that has many wonderful features others need a dozen of tools to implement it. I'm amazed at each new release.Good work! 
 With regards,
Clinciu Andrei George

"Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui rasfrante in sufletul omului."
"A good word, a smile and a good deed are just like rays of the sun reflected in man's soul."   by Nicolae Iorga

[naviserver-devel] Job Opening: Naviserver Software Engineer, Operations

[Joel R. <jo...@ru...>]

Ruby Lane is an exciting company to work for with benefits that rival
any start-up.  We're financially stable and continue to show steady
growth.  Additionally, we have never taken outside investment.  We're
looking for long-term employees who want rewarding work and also time
to focus on their personal lives.

You should probably do some basic research on us to see if you're
interested.  I'll let you do that instead of feeding you statistics.
First, Ruby Lane is almost exclusively telecommute.  Our main office
is in San Francisco.  We have employees working from all over the US,
Europe, Africa, and Asia.  If you'd like to relocate--this might be
the job for you.  If you prefer to work from home, we're a great place
to be.  We provide a company laptop--Apple or Lenovo--but your work
environment is up to you.  I live in Asia and we have software
engineers in South Africa and the UK, so we really do mean work from
anywhere.  Telecommuting is great, but it requires discipline to be
reliable and productive.

Our codebase is over 18 years old, but we regularly modernize our
toolset to take advantage of new innovations.  Right now our main
webapp is written in TCL using the Naviserver web server.  We use
Percona MySQL for our database and Bootstrap/jQuery for UI design.  We
use git.  We're also entirely hosted on Amazon's Cloud.  We've adopted
some of Amazon's technologies, but sometimes prefer to do things
ourselves like our upcoming rollout of elasticsearch.  All of our
images are stored on S3 and we're refactoring code to scale out
instead of scale up. If you want to chat with one of our programmers
to answer questions, that's not a problem.

We’re looking for someone to provide dedicated support to our customer
support and marketing and sales teams.  You would help them with site
updates and bug fixes.  This would be a very visible position within
the company and take you all over our code base and systems.  Expect
enough day-to-day variety to never be bored.

We write a webapp--you're going to need to know some HTML and CSS.  We
deploy servers using Puppet and we deploy our software onto them using
Capistrano.  Your code will be visible to end users, so a little
design knowledge is a great thing.

We have an official company culture--mostly to be honest and
straightforward with each other and our clients.  To be good to each
other.  You'll communicate with your co-workers mostly via Skype,
email, and sometimes phone. Most of the time you won't deal with our
customers directly, but its not out of the question.  If we have to
work with a customer to diagnose and fix an issue, we will.

I love working for Ruby Lane because I'm treated like a professional.
We'll have eyes on you early on to make sure you're fitting in with
the company, but as you prove yourself you'll get latitude.  Because
you’ll be working with other groups internally, we’d prefer something
close to US Eastern hours.  We’re usually more open with the hours,
but this is a unique position.

So, our growth prospects are good and we need more developers to keep
up with the projects and to help serve our customers. If you think
you'd like to be on our team, I'd love to hear from you. I need a
resume (txt or PDF are best), and I'd appreciate a cover letter that
tells me something about you and your personality. We are not opposed
to hiring mature programmers--we'd love to have your experience on our
team if you've kept your skills fresh.  Self-taughts are also
encouraged--you had the motivation to learn it on your own and that is
respectable.

Your submissions will be read directly by me and need to be sent to
jo...@ru....

I will ask for references, and I do check them.

About Ruby Lane

Founded in 1998, Ruby Lane Inc is a closely held established
California Corporation and offers flexible work hours and location,
competitive salary, company paid medical insurance, 401(K) matching,
profit sharing, and paid vacation/holidays/sick days. Ruby Lane is an
equal opportunity employer.

-- 
Joel Roberts
CTO
Ruby Lane
Serving the Online Community Since 1998

Shop . Sell . Relax
Antiques & Art . Vintage Collectibles . Jewelry
www.rubylane.com
+1.415.362.7611

[naviserver-devel] ns_connchan

[Gustaf N. <ne...@wu...>]

Dear NaviServer-Friends,

as a small present to the community, i've developed an
additional ensemble named "ns_connchan" which provides
an interface to the driver-receive and driver-send calls
of NaviServer. ns_connchan provides a higher level
alternative to the idiom  "ns_conn channel" + Tcl-commands
which accesses the raw channel

The new command uses the socketcallback infrastructure and allows to
- detach the currently open connection channel from a connection thread, to
- read/write/close the channel from other threads, and to
- query the states to then currently detached channels (similar to 
"ns_writer list")

Advantages:
    * use the same interface for http and https (and other drivers)
    * no (potential) problems with select in Tcl when using
       many (>1024) file descriptors
    * less potential problems with Tcl-Sockets (buffering, translations)
    * lower resource consumption

Implemented interface:

     ns_connchan detach
     ns_connchan close channel
     ns_connchan list ?-server server?
     ns_connchan callback ?-timeout s? ?-receivetimeout s? ?-sendtimeout 
s? channel command when
     ns_connchan read channel
     ns_connchan write channel string

The biggest advantage of "ns_connchan" is that it makes to possible
to read/write to SSL/TLS channels. The new command makes it easy e.g.
- to implement websockets over SSL/TLS (ws://... and wss://...)
- it can be used to implement OpenACS background delivery
   over SSL/TLS (although ns_writer provides already an infrastructure 
for most of it)
- it can be used to implement h264 streaming over SSL/TLS
and many more.

The most important usage scenario for us is currently a wider
usable websocket support. Websockets is facing nowadays
more popularity via  iPython/Jupyter Notebooks and nbviewer.
I've started from an websocket implementation developed by
wolfgang winkler and changed it to the new interfaces.
The new implementation works already nicely for us in
OpenACS. I'll do some more polishing and think
how to provide this as a NaviServer module or similar.

The bitbucket repository contains the source, man page and
test basic test cases for ns_connchan.

All the best in the new year!
-gustaf neumann

PS: A useful extension might be to provide support for multiple
worker threads for socketcallback infrastructure in the future.

Re: [naviserver-devel] tcl modules

[Jeff R. <dv...@di...>]

Gustaf,

Thanks.  I also just pushed up a commit to fix the handling of ns_module 
private/shared, so that while loading a module they return the correct 
path.  (They were reversed).

-J

Gustaf Neumann wrote:
> Dear all,
>
> i've commited a change for that issue, that
> William Jordan sent to me. I hope, this fixes the
> issue.
>
> -g
>
> Am 14.12.14 10:27, schrieb Gustaf Neumann:
>> Loading the same module twice is not intentional.
>> Loading modules is probably more complicated as it
>> has to be, but it has to care about global
>> modules (for all servers) and per-server modules, and that
>> network modules are loaded at the end.
>>
>> -g
>>
>> Am 10.12.14 02:54, schrieb Jeff Rogers:
>>> Hi all,
>>>
>>> It looks like a tcl-only module declared in the conf file will get
>>> loaded twice by init.tcl (as a network and a non-network module).
>>> There's a mention in init.tcl about needing to do a 2-phase loading of
>>> the network modules, but loading modules twice seems wrong.
>>>
>>> ----
>>> ns_section ns/server/server1/modules
>>>
>>> ns_param mymodule tcl
>>> ----
>>>
>>> Before I go ahead and fix this, is there a particular reason for the
>>> behavior as it is?
>>>
>>> -J
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] tcl modules

[Gustaf N. <ne...@wu...>]

Dear all,

i've commited a change for that issue, that
William Jordan sent to me. I hope, this fixes the
issue.

-g

Am 14.12.14 10:27, schrieb Gustaf Neumann:
> Loading the same module twice is not intentional.
> Loading modules is probably more complicated as it
> has to be, but it has to care about global
> modules (for all servers) and per-server modules, and that
> network modules are loaded at the end.
>
> -g
>
> Am 10.12.14 02:54, schrieb Jeff Rogers:
>> Hi all,
>>
>> It looks like a tcl-only module declared in the conf file will get
>> loaded twice by init.tcl (as a network and a non-network module).
>> There's a mention in init.tcl about needing to do a 2-phase loading of
>> the network modules, but loading modules twice seems wrong.
>>
>> ----
>> ns_section ns/server/server1/modules
>>
>> ns_param mymodule tcl
>> ----
>>
>> Before I go ahead and fix this, is there a particular reason for the
>> behavior as it is?
>>
>> -J

Re: [naviserver-devel] MinGW Compile error

[Andrew P. <at...@pi...>]

On Mon, Dec 15, 2014 at 05:39:57PM -0100, Maksym Zinchenko wrote:
> Hello, I'm trying to compile Naviserver 4.99.6 with MinGw/Msys but getting
> an error while run make command:

As far as I know, no one has worked on the MinGw/Msys build
system in many years, so it probably doesn't currently work.

For Windows, I recommend using the Microsoft's free (no cost) compiler
via Naviserver's nmake build system.  That's what I use, and it
definitely works.  For what Microsoft software to install, see these
posts:

  http://article.gmane.org/gmane.comp.web.naviserver.devel/3102/match=recommended+compiler
  http://www.mail-archive.com/naviserver-devel%40lists.sourceforge.net/msg03095.html
  Subject: Windows 7 recommended compiler and build tools?
  Date: 2014-08-10 10:45:43 GMT

  http://article.gmane.org/gmane.comp.web.naviserver.devel/3272/match=windows+compilers+2013+vs+2010
  http://www.mail-archive.com/naviserver-devel%40lists.sourceforge.net/msg03265.html
  Subject: Windows compilers, 2013 vs. 2010, etc.
  Date: 2014-11-17 14:23:11 GMT

You will also want the latest Naviserver source code from Bitbucket.

-- 
Andrew Piskorski <at...@pi...>

[naviserver-devel] MinGW Compile error

[Maksym Z. <siq...@gm...>]

Hello, I'm trying to compile Naviserver 4.99.6 with MinGw/Msys but getting
an error while run make command:


c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe:
cannot find -ltcl86
collect2.exe: error: ld returned 1 exit status
make[1]:*** [libnsthread.dll] Error 1


I'm running Windows7 32-bit, I have compiled and installed Tcl8.6.3 and
Tk8.6.3 with MinGW under C:/Tcl8.6.3

Sorry for lame question I'm not very good with computers and compiling
things under Windows.


Thank you.

Re: [naviserver-devel] tcl modules

[Gustaf N. <ne...@wu...>]

Loading the same module twice is not intentional.
Loading modules is probably more complicated as it
has to be, but it has to care about global
modules (for all servers) and per-server modules, and that
network modules are loaded at the end.

-g

Am 10.12.14 02:54, schrieb Jeff Rogers:
> Hi all,
>
> It looks like a tcl-only module declared in the conf file will get
> loaded twice by init.tcl (as a network and a non-network module).
> There's a mention in init.tcl about needing to do a 2-phase loading of
> the network modules, but loading modules twice seems wrong.
>
> ----
> ns_section ns/server/server1/modules
>
> ns_param mymodule tcl
> ----
>
> Before I go ahead and fix this, is there a particular reason for the
> behavior as it is?
>
> -J

Re: [naviserver-devel] Code Quality

[Gustaf N. <ne...@wu...>]

Dear all,

we are now at less than 1000 issues in the
c-code-quality results, which is already pretty
good (see graphic below).

NaviServer compiles as well pretty clean with the
newest gcc5/clang-3.6 and "--std=c99 -pedantic",
except for one problem:
conversion between object and function pointers.
In general, C does not guarantee that a function pointer
has the same size as an (data) object pointer, so casts
between these are dangerous. On the common
architectures this is not a problem, but still, the issue shows
up as a warning. It was not hard to fix the issue on most
occasions, but i have not touched the following area:

NaviServer (and AOLserver) allows to store information
per function pointer via
    Ns_RegisterProcInfo() / Ns_GetProcInfo()

However, it uses Tcl's hash tables, which use (void *) or
strings as keys. void pointers are object pointers, so
one cannot pass it a function pointer. The solution
is to look probably into custom hash types/coding
of the pointers/different lookups.

all the best
-g




> We started with 16.000 (or 18.000?) issues in
> NaviServer, and now we are well below 3.000. There are still
> a couple of things, i want to address...
>
> NaviServer compares already quite well with other projects.
> PostgreSQL shows in 0.28 issues per LOC [1],
> AOLserver 0.73, and NaviServer is now at 0.19.
>
> 	     PostgreSQL	Aolserver	NaviServer
> LOC	     487895	18743		14081
> Issues	     140262	13712		2664
> Issues/LOC   0,2875	0,7316		0,1892

Re: [naviserver-devel] Performance comparison of base64 encoders

[Jeff R. <dv...@di...>]

I just checked, and ns_base64encode is also faster than [binary encode 
base64] by about a factor of 2.

-J

Gustaf Neumann wrote:
> Dear friends,
>
> Maybe, someone finds this information useful:
>
> just now, i did a quick comparison of the various
> options for base64 codecs. It shows that NaviServer's
> built-in ns_base64encode  is very fast:
> it is up to a factor of 1.000 faster than the Tcl-only
> version of base64::encode, and even a factor of 10
> faster than base64::encode based on Trf (Trf 2.1.4).
> While a big difference relative to the plain Tcl version
> is expected, the improvement over the C-based Trf
> is remarkable.
>
> The results are measured with the latest version of
> NaviServer from bitbucket under Mac OS X 10.9.5
>
> Using ns_base64encode reduces the external
> dependencies and is by far the fastest version.
>
> all the best
> -g
>
>
>
> size=100
> 		tcllib:	      134.713824 microseconds
> 		tcllib+trf:     4.561536 microseconds
> 		ns:		0.541651 microseconds
>
> size=1000
> 		tcllib:	     1281.819696 microseconds
> 		tcllib+trf:    16.93683  microseconds
> 		ns:		1.517516 microseconds
> 			
> size=10000
> 		tcllib:	    12638.85578 microseconds
> 		tcllib+trf:   143.11120 microseconds		
> 		ns:	       11.08016 microseconds
>
> size=100000
> 		tcllib:	   127344.794553 microseconds
> 		tcllib+trf:  1423.021879 microseconds
> 		ns:	      113.504248 microseconds
>
> ------
>
> foreach i {10 100 1000 10000} {
>     set s [string repeat 0123456789 $i]
>     lappend _ size=[string length $s]
>     lappend _ tcllib:[time {base64::encode $s} 1000]
>     lappend _ ns:[time {ns_base64encode $s} 1000]
> }
> join $_ \n
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

[naviserver-devel] tcl modules

[Jeff R. <dv...@di...>]

Hi all,

It looks like a tcl-only module declared in the conf file will get 
loaded twice by init.tcl (as a network and a non-network module). 
There's a mention in init.tcl about needing to do a 2-phase loading of 
the network modules, but loading modules twice seems wrong.

----
ns_section ns/server/server1/modules

ns_param mymodule tcl
----

Before I go ahead and fix this, is there a particular reason for the 
behavior as it is?

-J

Re: [naviserver-devel] Performance comparison of base64 encoders

[Bernhard v. W. <ber...@qc...>]

Thanks, that will be very useful especially for large files.

Regards
Bernhard

On 7 December 2014 at 13:01, Gustaf Neumann <ne...@wu...> wrote:

>  Dear friends,
>
> Maybe, someone finds this information useful:
>
> just now, i did a quick comparison of the various
> options for base64 codecs. It shows that NaviServer's
> built-in ns_base64encode  is very fast:
> it is up to a factor of 1.000 faster than the Tcl-only
> version of base64::encode, and even a factor of 10
> faster than base64::encode based on Trf (Trf 2.1.4).
> While a big difference relative to the plain Tcl version
> is expected, the improvement over the C-based Trf
> is remarkable.
>
> The results are measured with the latest version of
> NaviServer from bitbucket under Mac OS X 10.9.5
> Using ns_base64encode reduces the external
> dependencies and is by far the fastest version.
>
> all the best
> -g
>
>
> size=100
> 		tcllib:	      134.713824 microseconds
> 		tcllib+trf:     4.561536 microseconds
> 		ns:		0.541651 microseconds
>
> size=1000
> 		tcllib:	     1281.819696 microseconds
> 		tcllib+trf:    16.93683  microseconds
> 		ns:		1.517516 microseconds
> 			
> size=10000
> 		tcllib:	    12638.85578 microseconds
> 		tcllib+trf:   143.11120 microseconds		
> 		ns:	       11.08016 microseconds
>
> size=100000
> 		tcllib:	   127344.794553 microseconds
> 		tcllib+trf:  1423.021879 microseconds
> 		ns:	      113.504248 microseconds
>
> ------
>
> foreach i {10 100 1000 10000} {
>    set s [string repeat 0123456789 $i]
>    lappend _ size=[string length $s]
>    lappend _ tcllib:[time {base64::encode $s} 1000]
>    lappend _ ns:[time {ns_base64encode $s} 1000]
> }
> join $_ \n
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

[naviserver-devel] Performance comparison of base64 encoders

[Gustaf N. <ne...@wu...>]

Dear friends,

Maybe, someone finds this information useful:

just now, i did a quick comparison of the various
options for base64 codecs. It shows that NaviServer's
built-in ns_base64encode  is very fast:
it is up to a factor of 1.000 faster than the Tcl-only
version of base64::encode, and even a factor of 10
faster than base64::encode based on Trf (Trf 2.1.4).
While a big difference relative to the plain Tcl version
is expected, the improvement over the C-based Trf
is remarkable.

The results are measured with the latest version of
NaviServer from bitbucket under Mac OS X 10.9.5

Using ns_base64encode reduces the external
dependencies and is by far the fastest version.

all the best
-g



size=100
		tcllib:	      134.713824 microseconds
		tcllib+trf:     4.561536 microseconds
		ns:		0.541651 microseconds

size=1000
		tcllib:	     1281.819696 microseconds
		tcllib+trf:    16.93683  microseconds
		ns:		1.517516 microseconds
			
size=10000
		tcllib:	    12638.85578 microseconds
		tcllib+trf:   143.11120 microseconds		
		ns:	       11.08016 microseconds

size=100000
		tcllib:	   127344.794553 microseconds
		tcllib+trf:  1423.021879 microseconds
		ns:	      113.504248 microseconds

------

foreach i {10 100 1000 10000} {
    set s [string repeat 0123456789 $i]
    lappend _ size=[string length $s]
    lappend _ tcllib:[time {base64::encode $s} 1000]
    lappend _ ns:[time {ns_base64encode $s} 1000]
}
join $_ \n

Re: [naviserver-devel] PDB debug symbols fixed on Windows

[Andrew P. <at...@pi...>]

On Mon, Nov 03, 2014 at 12:10:44AM -0500, Andrew Piskorski wrote:
> I finally figured out why my debug symbols on Windows were messed up.

> To fix, I simply changed the (Windows only) makefiles to name the
> library files "libnsd.dll" and "libnsd.lib", here:
> 
>   https://bitbucket.org/apiskors/naviserver/commits/55797002913598dd2152d2bea4dba7bbeef589bf

By itself, that change had an unfortunate side effect on the Unix
build; to fix it you need this:

  https://bitbucket.org/apiskors/naviserver/commits/ac1bd35ff47bdab4f687c2587b1c5e49fd8cea92

With that, to the best of my knowledge everything in my fork is now
suitable for merging into the main Naviserver head.

-- 
Andrew Piskorski <at...@pi...>

[naviserver-devel] FW: more logging problems on Windows

[Maurizio M. <Mau...@sp...>]

Dear Andrew,
	My distribution of Windows-OpenACS
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/).
Contains the sources and the various makefiles. It uses "nmake" and has been
compiled with Visual Studio 2013.

It does work, it is reliable and is used by various production sites.

BTW: I did compile with the same options not only Aolserver but also TCL.

Maurizio




-----Original Message-----
From: Andrew Piskorski [mailto:at...@pi...] 
Sent: 17 November 2014 16:41
To: nav...@li...
Cc: it...@ar...
Subject: Re: [naviserver-devel] more logging problems on Windows

On Mon, Nov 17, 2014 at 10:36:48AM +0100, Gustaf Neumann wrote:

> The problem is not a argument-passing or memory problem, but seems to 
> be a linking problem. When a file is opened in a module (here nslog) 
> only this module can write successfully to this file. Under Windows 
> other dlls can't write to it, even when they write to the same fd. It 
> seems to me as if the nslog dll is linked to a private runtime 
> environment.

Very interesting!  Good detective work, Gustaf.

> Andrew, can you compare the linking options for nslog with these from 
> Maurizio?

I don't know what link options Maurizio is using.

On Mon, Nov 17, 2014 at 01:05:28PM +0100, Gustaf Neumann wrote:

> that's a very good hint: AFIKT, all the modules used already the same 
> /M* options. However, i've changed the runtime library option from 
> /MT* to MD* (multi-threaded version) and a quick test (fildes to 
> windows handle conversion) indicates that the strange behavior is 
> gone.

We've been using /MTd (or plain /MT for non-debug builds) for everything,
because that's what was in the include/Makefile.win32 Ibrahim sent me, which
was identical to the version in Mercurial last touched by Vlad Seryakov back
in 2007.

>From Microsoft's docs:

  http://msdn.microsoft.com/en-us/library/2kzt1wy3%28v=vs.110%29.aspx
  http://support.microsoft.com/kb/140584/en-us

  "/MT" is the "multithread, static version"
  "/MD" is the "multithread- and DLL-specific version"

Microsoft warns about mixing two different versions of the run-time library,
but says nothing about the mysterious file IO behavior we saw with /MTd.  I
looked through my three hardcopy books on Windows system programming and
found nothing relevant there either.

The most useful discussion I found of /MT vs. /MD (although still not at all
complete) was here:

  http://stackoverflow.com/questions/757418/should-i-compile-with-md-or-mt

It sounds like /MDd is the right default for Naviserver's Windows builds.
I haven't tried that or updated to Gustaf's latest code yet, but will do so
soon.

I'm compiling and running Naviserver on the same machine anyway, so /MD
should be completely fine for my use.  From the discussion above, it sounds
like someone distributing compiled binaries to run on other machines MIGHT
prefer to use /MT to simplify their application install process, but even
there /MD is still probably the correct default.

--
Andrew Piskorski <at...@pi...>

----------------------------------------------------------------------------
--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from
Actuate! Instantly Supercharge Your Business Reports and Dashboards with
Interactivity, Sharing, Native Excel Exports, App Integration & more Get
technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] more logging problems on Windows

[Andrew P. <at...@pi...>]

On Mon, Nov 17, 2014 at 10:36:48AM +0100, Gustaf Neumann wrote:

> The problem is not a argument-passing or memory problem,
> but seems to be a linking problem. When a file is opened in a
> module (here nslog) only this module can write successfully
> to this file. Under Windows other dlls can't write to it, even
> when they write to the same fd. It seems to me as if the
> nslog dll is linked to a private runtime environment.

Very interesting!  Good detective work, Gustaf.

> Andrew, can you compare the linking options for nslog with these
> from Maurizio?

I don't know what link options Maurizio is using.

On Mon, Nov 17, 2014 at 01:05:28PM +0100, Gustaf Neumann wrote:

> that's a very good hint: AFIKT, all the modules used already the
> same /M* options. However, i've changed the runtime library
> option from /MT* to MD* (multi-threaded version) and a quick
> test (fildes to windows handle conversion) indicates that
> the strange behavior is gone.

We've been using /MTd (or plain /MT for non-debug builds) for
everything, because that's what was in the include/Makefile.win32
Ibrahim sent me, which was identical to the version in Mercurial last
touched by Vlad Seryakov back in 2007.

>From Microsoft's docs:

  http://msdn.microsoft.com/en-us/library/2kzt1wy3%28v=vs.110%29.aspx
  http://support.microsoft.com/kb/140584/en-us

  "/MT" is the "multithread, static version"
  "/MD" is the "multithread- and DLL-specific version"

Microsoft warns about mixing two different versions of the run-time
library, but says nothing about the mysterious file IO behavior we saw
with /MTd.  I looked through my three hardcopy books on Windows system
programming and found nothing relevant there either.

The most useful discussion I found of /MT vs. /MD (although still not
at all complete) was here:

  http://stackoverflow.com/questions/757418/should-i-compile-with-md-or-mt

It sounds like /MDd is the right default for Naviserver's Windows builds.
I haven't tried that or updated to Gustaf's latest code yet, but will
do so soon.

I'm compiling and running Naviserver on the same machine anyway, so
/MD should be completely fine for my use.  From the discussion above,
it sounds like someone distributing compiled binaries to run on other
machines MIGHT prefer to use /MT to simplify their application install
process, but even there /MD is still probably the correct default.

-- 
Andrew Piskorski <at...@pi...>

[naviserver-devel] Windows compilers, 2013 vs. 2010, etc.

[Andrew P. <at...@pi...>]

On Fri, Nov 14, 2014 at 12:00:47PM +0100, Maurizio Martignano wrote:
> Subject: Re: [naviserver-devel] more logging problems on Windows

> Dear Andrew,
> 	I would like to draw your attention to the availability of Visual
> Studio 2013 Community Edition.
> 
> http://msdn.microsoft.com/en-us/visual-studio-community-vs.aspx

So the new thing here seems to be that with this latest version, the
free Visual Studio GUI now includes both 32 and 64 bit support.

In older versions with the 2010 and/or 2012 compilers, the GUI was
only integrated with the 32-bit compiler.  The 64-bit compilers were
ALSO free, in the separate "Windows SDK 7.1 for Windows 7" package,
but only command line, no GUI.

The 2010 compilers can build for versions of Windows as old as Windows
XP, while it looks like the VS 2013 Community Edition supports ONLY
Windows 7 and newer.  That doesn't matter to me as I'm building solely
for Windows 7 anyway, but it might to some.

If I was starting from scratch, I'd probably go with the 2013 stuff,
but I don't currently see any reason to upgrade from the older
compiler I'm currently using.

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] more logging problems on Windows

[Gustaf N. <ne...@wu...>]

Hi Ibrahim,

that's a very good hint: AFIKT, all the modules used already 
the
same /M* options. However, i've changed the runtime library
option from /MT* to MD* (multi-threaded version) and a quick
test (fildes to windows handle conversion) indicates that
the strange behavior is gone. Nevertheless, using the ns_
prefix for the OS-interface is still beneficial for handling of
different prototypes on different operating systems
and leads to a better abstraction than the macros, which
were a mere measure to reduce windows compiler warning
(to use ISO names _open, _close etc.).

This small change will certainly help people to keep their 
hair on their
head, when they try to adapt more naviserver modules for 
windows....

all the best
-g

[1] 
https://bitbucket.org/naviserver/naviserver/commits/08680db31a71a6c8d512a6f7e7de64303c5ac262

On 17.11.14 10:52, Ibrahim Tannir wrote:
> Hi Gustaf,
>
> Just a hint: it is absolutely essential that all the 
> modules on Windows be compiled with the same model. I use 
> -MD or -MDd. Otherwise, each of the modules compiled with 
> a different model will have it's own global variables, 
> environment and memory segment and the two modules will 
> not interact even if they share the same global variables. 
> It goes even further with whether the modules use MFC and 
> common runtime or not, which Microsoft libraries they will 
> include. That is why I avoid using make files and let 
> Visual Studio do all the work and I make sure that all the 
> modules have the same setup (see attached file). (I have 
> shared my project files, all of which I created myself, 
> with someone from the community recently).
>
> Hope this helps,
> Ibrahim
>
>
>
> On 17-Nov-14 10:36, Gustaf Neumann wrote:
>> Dear Andrew and all,
>>
>> Over the weekend, I invested some more time into the windows
>> port and got a better understanding of the logging  issue 
>> (a write()
>> operation succeeds inside nslog.c, but the identically 
>> operations
>> inside driver.c does not work; "not work" means, that the 
>> write
>> operation succeeds (i.e. reports the written number of 
>> bytes),
>> but the file is always empty, even after a close()).
>>
>> The problem is not a argument-passing or memory problem,
>> but seems to be a linking problem. When a file is opened 
>> in a
>> module (here nslog) only this module can write successfully
>> to this file. Under Windows other dlls can't write to it, 
>> even
>> when they write to the same fd. It seems to me as if the
>> nslog dll is linked to a private runtime environment.
>> Looking up the windows handle same file descriptor
>> (e.g. 3) leads to a different results when performed in 
>> nslog.c
>> or in driver.c (or somewhere else in a function in libnsd).
>>
>> The current solution is to define for windows stub-functions
>> ns_open, ns_close, ns_write, ns_read which are defined in 
>> libnsd.
>> These functions are called from ns_log instead of the native
>> OS functions (under windows _open, _close, _write, _read),
>> which have btw different prototypes, so this abstraction
>> seems beneficial for several reasons.
>>
>> The essential change is in [1], which add the ns_* 
>> functions and
>> reverts the previous  _MSV_VER bypass in nslog.c. For
>> completeness, ns_tell should be added as well.
>>
>> Andrew, can you compare the linking options for nslog 
>> with these
>> from Maurizio? Although the problem for nslog seems to be 
>> solved,
>> i would not be surprised if other weirdnesses lurch around
>> from other modules.
>>
>> -g
>

Re: [naviserver-devel] more logging problems on Windows

[Ibrahim T. <it...@ar...>]

Hi Gustaf,

Just a hint: it is absolutely essential that all the modules on 
Windows be compiled with the same model. I use -MD or -MDd. 
Otherwise, each of the modules compiled with a different model 
will have it's own global variables, environment and memory 
segment and the two modules will not interact even if they share 
the same global variables. It goes even further with whether the 
modules use MFC and common runtime or not, which Microsoft 
libraries they will include. That is why I avoid using make files 
and let Visual Studio do all the work and I make sure that all 
the modules have the same setup (see attached file). (I have 
shared my project files, all of which I created myself, with 
someone from the community recently).

Hope this helps,
Ibrahim



On 17-Nov-14 10:36, Gustaf Neumann wrote:
> Dear Andrew and all,
>
> Over the weekend, I invested some more time into the windows
> port and got a better understanding of the logging  issue (a write()
> operation succeeds inside nslog.c, but the identically operations
> inside driver.c does not work; "not work" means, that the write
> operation succeeds (i.e. reports the written number of bytes),
> but the file is always empty, even after a close()).
>
> The problem is not a argument-passing or memory problem,
> but seems to be a linking problem. When a file is opened in a
> module (here nslog) only this module can write successfully
> to this file. Under Windows other dlls can't write to it, even
> when they write to the same fd. It seems to me as if the
> nslog dll is linked to a private runtime environment.
> Looking up the windows handle same file descriptor
> (e.g. 3) leads to a different results when performed in nslog.c
> or in driver.c (or somewhere else in a function in libnsd).
>
> The current solution is to define for windows stub-functions
> ns_open, ns_close, ns_write, ns_read which are defined in libnsd.
> These functions are called from ns_log instead of the native
> OS functions (under windows _open, _close, _write, _read),
> which have btw different prototypes, so this abstraction
> seems beneficial for several reasons.
>
> The essential change is in [1], which add the ns_* functions and
> reverts the previous  _MSV_VER bypass in nslog.c. For
> completeness, ns_tell should be added as well.
>
> Andrew, can you compare the linking options for nslog with these
> from Maurizio? Although the problem for nslog seems to be solved,
> i would not be surprised if other weirdnesses lurch around
> from other modules.
>
> -g