naviserver-devel — Developer discussion

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Cesáreo G. R. <ce...@ce...>]

Hi

I already did it with control port with same results[1]. As you notice,
it has to be something with env. HOME is /root although start script was
/usr/local/ns/bin/nsd -u openacs -g web ...

It seems it doesn't load all env variables of USER (openacs).

Regards,
Cesáreo




[1]
openacs:nscp 6> array get ::env
LD_LIBRARY_PATH :/usr/local/pgsql/lib HOME /root LANG es_ES.UTF-8 PWD
/root MAIL /var/mail/root LOGNAME root  SHELL /bin/bash TERM xterm-color
SHLVL 1 LS_OPTIONS -l PS1 {\h:\w\$ } SSH_CONNECTION {* *} PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/pgsql/bin:/usr/local/pgsql/bin
SSH_TTY /dev/pts/0 USER openacs _ /etc/init.d/naviserver_ctl
openacs:nscp 7> exec /usr/bin/yui-compressor --type css
{*}/home/openacs/openacs/packages/acs-templating/www/resources/forms.css
.form-widget-error,.form-error{color:#c30000}.form-required- ( ... )
adding-bottom:10px}.margin-form-div h1{margin-left:13.5em}dpkg: aviso:
fallo al abrir el fichero de configuración `/root/.dpkg.cfg' para
lectura: Permiso denegado

El 08/junio/16 a las 2:28, Gustaf Neumann escribió:
> Am 08.06.16 um 00:29 schrieb Cesáreo García Rodicio:
>> Sorry, I don't have /ds/shell installed so I can not try now.
> Most openacs developers have the developer support installed...
> 
> Alternatively, you can use to the control port (nscp module), when 
> configured.
> With a configuration like the sample nsd-config.tcl, you can use "telnet 
> 4080"
> to connect to the server and debug there interactively.
> 
> -g
> 
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Gustaf N. <ne...@wu...>]

Am 08.06.16 um 00:29 schrieb Cesáreo García Rodicio:
> Sorry, I don't have /ds/shell installed so I can not try now.
Most openacs developers have the developer support installed...

Alternatively, you can use to the control port (nscp module), when 
configured.
With a configuration like the sample nsd-config.tcl, you can use "telnet 
4080"
to connect to the server and debug there interactively.

-g

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Cesáreo G. R. <ce...@ce...>]

Dear Gustaf,

Sorry, I don't have /ds/shell installed so I can not try now.

I'll debug env variables to see what happens

Thanks,
Cesáreo

El 07/junio/16 a las 11:46, Gustaf Neumann escribió:
> Dear Cesáreo,
> 
> What happens, when you run the following command in the developer shell 
> of OpenACS (/ds/shell)
> 
>     exec /usr/bin/yui-compressor --type css < /home/openacs/openacs/packages/acs-templating/www/resources/forms.css
> 
> Does the result depend on the current working directory?
> Are there any environment variables  to hint at /root (check in ds/shell 
> with "array get ::env").
> 
> -g
> 
> Am 07.06.16 um 17:15 schrieb Cesáreo García Rodicio:
>> Hi Gustaf
>>
>> It occurs when trying to minify a file (source file edited so it tries
>> to minify before gzipped it)
>>
>> As I see in ns_gzipfile it occurs in
>>   exec {*}$minifyCmd < $source | {*}$gzipCmd > $target [1]
>>
>> debugging I see in my setup equivalent to (when modifying forms.css):
>>
>> /usr/bin/yui-compressor --type css <
>> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
>> /bin/gzip -9 > /home/openacs/openacs/packages/ac\
>> s-templating/www/resources/forms.css.gz
>>
>> In sh console I didn't notice any error only in naviserver console[2].
>>
>> Anyway I'll try to dig a bit more :-)
>>
>> Thanks,
>> Cesáreo
>>
>> PD: Checking this I see two minor typos [3] [4]
>>
>> ---------------------------------------------
>>
>> [1]
>> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-192
>> [2]
>> openacs:nscp 1> exec /usr/bin/yui-compressor --type css <
>> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
>> /bin/gzip -9 > /home/openacs/openacs/packages/acs-t\
>> emplating/www/resources/forms.css.gz
>> exec /usr/bin/yui-compressor --type css <
>> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
>> /bin/gzip -9 > /home/openacs/openacs/packages/acs-templating/www/re\
>> sources/forms.css.gz
>> dpkg: aviso: fallo al abrir el fichero de configuración
>> `/root/.dpkg.cfg' para lectura: Permiso denegado
>>
>>
>> [3]
>> https://bitbucket.org/naviserver/naviserver/src/92c2d93bcaceee799faa8fe58f6f913328efdf01/doc/src/naviserver/ns_return.man?at=default&fileviewer=file-view-default#ns_return.man-272
>>
>>   -> js instead of css
>>
>> [4]
>> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-194
>> -> minify instead of minninfy
>>
>>
>> El 06/junio/16 a las 2:32, Gustaf Neumann escribió:
>>> Dear Cesáreo,
>>>> But, when naviserver try to minify a file I see some error[3] reading a
>>>> dpkg (debian package instaler) conf file (/root/.dpkg.cfg). And file
>>>> doesn't seem to be minified
>>> This is indeed strange. In exactly which situation do you get the error?
>>>
>>> The minify integration is purely on the tcl layer [1] and is based on
>>> input/output redirection and is just used for refreshing stale files.
>>> So, it will not try to compress/minify files that were not compressed
>>> before (yui-compressor fails sometimes to compress files) and is only
>>> triggered, when the time stamp of the plain file differs from the .gz file.
>>>
>>> Can it be, that the path has "funny characters"? It might help to add
>>> debug statements to proc ns_gzipfile (and restart the server).
>>>
>>> -g
>>>
>>> [1]
>>> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-182
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
>>> patterns at an interface-level. Reveals which users, apps, and protocols are
>>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>>> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> 
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Gustaf N. <ne...@wu...>]

Dear Cesáreo,

What happens, when you run the following command in the developer shell 
of OpenACS (/ds/shell)

    exec /usr/bin/yui-compressor --type css < /home/openacs/openacs/packages/acs-templating/www/resources/forms.css

Does the result depend on the current working directory?
Are there any environment variables  to hint at /root (check in ds/shell 
with "array get ::env").

-g

Am 07.06.16 um 17:15 schrieb Cesáreo García Rodicio:
> Hi Gustaf
>
> It occurs when trying to minify a file (source file edited so it tries
> to minify before gzipped it)
>
> As I see in ns_gzipfile it occurs in
>   exec {*}$minifyCmd < $source | {*}$gzipCmd > $target [1]
>
> debugging I see in my setup equivalent to (when modifying forms.css):
>
> /usr/bin/yui-compressor --type css <
> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
> /bin/gzip -9 > /home/openacs/openacs/packages/ac\
> s-templating/www/resources/forms.css.gz
>
> In sh console I didn't notice any error only in naviserver console[2].
>
> Anyway I'll try to dig a bit more :-)
>
> Thanks,
> Cesáreo
>
> PD: Checking this I see two minor typos [3] [4]
>
> ---------------------------------------------
>
> [1]
> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-192
> [2]
> openacs:nscp 1> exec /usr/bin/yui-compressor --type css <
> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
> /bin/gzip -9 > /home/openacs/openacs/packages/acs-t\
> emplating/www/resources/forms.css.gz
> exec /usr/bin/yui-compressor --type css <
> /home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
> /bin/gzip -9 > /home/openacs/openacs/packages/acs-templating/www/re\
> sources/forms.css.gz
> dpkg: aviso: fallo al abrir el fichero de configuración
> `/root/.dpkg.cfg' para lectura: Permiso denegado
>
>
> [3]
> https://bitbucket.org/naviserver/naviserver/src/92c2d93bcaceee799faa8fe58f6f913328efdf01/doc/src/naviserver/ns_return.man?at=default&fileviewer=file-view-default#ns_return.man-272
>
>   -> js instead of css
>
> [4]
> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-194
> -> minify instead of minninfy
>
>
> El 06/junio/16 a las 2:32, Gustaf Neumann escribió:
>> Dear Cesáreo,
>>> But, when naviserver try to minify a file I see some error[3] reading a
>>> dpkg (debian package instaler) conf file (/root/.dpkg.cfg). And file
>>> doesn't seem to be minified
>> This is indeed strange. In exactly which situation do you get the error?
>>
>> The minify integration is purely on the tcl layer [1] and is based on
>> input/output redirection and is just used for refreshing stale files.
>> So, it will not try to compress/minify files that were not compressed
>> before (yui-compressor fails sometimes to compress files) and is only
>> triggered, when the time stamp of the plain file differs from the .gz file.
>>
>> Can it be, that the path has "funny characters"? It might help to add
>> debug statements to proc ns_gzipfile (and restart the server).
>>
>> -g
>>
>> [1]
>> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-182
>>
>>
>> ------------------------------------------------------------------------------
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Cesáreo G. R. <ce...@ce...>]

Hi Gustaf

It occurs when trying to minify a file (source file edited so it tries
to minify before gzipped it)

As I see in ns_gzipfile it occurs in
 exec {*}$minifyCmd < $source | {*}$gzipCmd > $target [1]

debugging I see in my setup equivalent to (when modifying forms.css):

/usr/bin/yui-compressor --type css <
/home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
/bin/gzip -9 > /home/openacs/openacs/packages/ac\
s-templating/www/resources/forms.css.gz

In sh console I didn't notice any error only in naviserver console[2].

Anyway I'll try to dig a bit more :-)

Thanks,
Cesáreo

PD: Checking this I see two minor typos [3] [4]

---------------------------------------------

[1]
https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-192
[2]
openacs:nscp 1> exec /usr/bin/yui-compressor --type css <
/home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
/bin/gzip -9 > /home/openacs/openacs/packages/acs-t\
emplating/www/resources/forms.css.gz
exec /usr/bin/yui-compressor --type css <
/home/openacs/openacs/packages/acs-templating/www/resources/forms.css |
/bin/gzip -9 > /home/openacs/openacs/packages/acs-templating/www/re\
sources/forms.css.gz
dpkg: aviso: fallo al abrir el fichero de configuración
`/root/.dpkg.cfg' para lectura: Permiso denegado


[3]
https://bitbucket.org/naviserver/naviserver/src/92c2d93bcaceee799faa8fe58f6f913328efdf01/doc/src/naviserver/ns_return.man?at=default&fileviewer=file-view-default#ns_return.man-272

 -> js instead of css

[4]
https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-194
-> minify instead of minninfy


El 06/junio/16 a las 2:32, Gustaf Neumann escribió:
> Dear Cesáreo,
>> But, when naviserver try to minify a file I see some error[3] reading a
>> dpkg (debian package instaler) conf file (/root/.dpkg.cfg). And file
>> doesn't seem to be minified
> This is indeed strange. In exactly which situation do you get the error?
> 
> The minify integration is purely on the tcl layer [1] and is based on 
> input/output redirection and is just used for refreshing stale files. 
> So, it will not try to compress/minify files that were not compressed 
> before (yui-compressor fails sometimes to compress files) and is only 
> triggered, when the time stamp of the plain file differs from the .gz file.
> 
> Can it be, that the path has "funny characters"? It might help to add 
> debug statements to proc ns_gzipfile (and restart the server).
> 
> -g
> 
> [1] 
> https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-182
> 
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Gustaf N. <ne...@wu...>]

Dear Cesáreo,
> But, when naviserver try to minify a file I see some error[3] reading a
> dpkg (debian package instaler) conf file (/root/.dpkg.cfg). And file
> doesn't seem to be minified
This is indeed strange. In exactly which situation do you get the error?

The minify integration is purely on the tcl layer [1] and is based on 
input/output redirection and is just used for refreshing stale files. 
So, it will not try to compress/minify files that were not compressed 
before (yui-compressor fails sometimes to compress files) and is only 
triggered, when the time stamp of the plain file differs from the .gz file.

Can it be, that the path has "funny characters"? It might help to add 
debug statements to proc ns_gzipfile (and restart the server).

-g

[1] 
https://bitbucket.org/naviserver/naviserver/src/7373f7aecab04afe4ba0fdabd4899bf713882c55/tcl/fastpath.tcl?at=default&fileviewer=file-view-default#fastpath.tcl-182

[naviserver-devel] Error trying to minify files in Debian Server (dpkg error)

[Cesáreo G. R. <ce...@ce...>]

Hi

I'm testing minify options of fastpath[1]. I've installed yui-compressor
(Debian Linux: apt-get install yui-compressor) and I test it in command
shell with webserver user[2]. It seems to work fine.

But, when naviserver try to minify a file I see some error[3] reading a
dpkg (debian package instaler) conf file (/root/.dpkg.cfg). And file
doesn't seem to be minified

Any clue?

Thanks
Cesáreo



----------------

[1]
http://naviserver.sourceforge.net/n/naviserver/files/ns_return.html#subsection1

[2]
openacs@www:~$ /usr/bin/yui-compressor -s --type css
Usage: java -jar yuicompressor-x.y.z.jar [options] [input file]
Global Options
  -h, --help                Displays this information

[3]
Warning: mininfy returned error: dpkg: aviso: fallo al abrir el fichero
de configuración `/root/.dpkg.cfg' para lectura: Permiso denegado

[4] FastPath Config
# Naviserver Config
ns_section      "ns/fastpath"

ns_param        gzip_static         true
ns_param        gzip_refresh        true
ns_param        gzip_cmd            "/bin/gzip -9"
ns_param        minify_css_cmd   "/usr/bin/yui-compressor --type css"
ns_param        minify_js_cmd     "/usr/bin/yui-compressor --type js"

[naviserver-devel] module nsssl merged to main NaviServer repository

[Gustaf N. <ne...@wu...>]

Dear all,

the module "nsssl" is now part of the main NaviServer repository. I 
followed the merge instructions in [1], which lead to rewrite / merge of 
the history of the formerly separated repositories. Adding nsssl to the 
main repo eases the setup for many installations and allows us to use 
functionality from OpenSSL in various components. For example, the 
current version supports already to proivde https URLs to ns_http, 
leading as well to a reduction of duplicated code. The change makes it 
easier to support functionality such as STARTTLS in the future as well 
as more crypto functionality.

all the best

-gustaf neumann

[1] https://www.mercurial-scm.org/wiki/MergingUnrelatedRepositories

-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

[naviserver-devel] NaviServer 4.99.11 available

[Gustaf N. <ne...@wu...>]

Dear friends,

NaviServer 4.99.11 is available. The new version is tagged with
naviserver-4.99.11 in mercurial and is as well available at source-forge
(naviserver, modules, documentation pages). I'll prepare as well
an announcement for c.l.tcl and OpenACS.

Below is the section for 4.99.11 from the NEWS file.

all the best

-gustaf neumann


=======================================
NaviServer 4.99.11, released 2016-05-01
=======================================

Changes relative to 4.99.10

  156 files changed, 8613 insertions(+), 3142 deletions(-)

New Features:

   * IPv6 support: NaviServer works now with IPv6 and IPv4
     addresses.  It is e.g. possible to use "::" as address (to denote
     the unspecified address, roughly similar to 0.0.0.0 in IPv4) to
     accept any IPv6 or IPv4 address, when the OS supports IPv4-mapped
     addresses, or one can use arbitrary addresses. Note, that one can
     as well load the driver for a server multiple times, once with an
     IPv6 and once with an IPv4 address.  Adding this feature required
     changes in the drivers, since earlier versions had IPv4 structures
     compiled in. As a consequence, also drivers for additional
     protocols have to be adjusted (for this release, the IPv6-adjusted
     modules are all modules of the naviserver core, plus nsssl, nsudp,
     nssmtpd, nsdns)

   * Added HTTP/1.1 pipelining support: Pipelining is a (not so often
     used) feature of HTTP/1.1 where multiple requests are sent on a
     persistent connection without waiting for the results of the first
     request. Recently more (esp. mobile) clients started to use
     pipelining. Previous versions of NaviServer handled just the first
     request of the pipeline and swallowed (ignored) later requests of
     the pipeline.

   * Added support for HTML5 file attribute "multiple": HTML5 allows
     for uploading multiple files under the same INPUT name. Previous
     version of NaviServer just accepted the first file with the given
     name.

   * Reduced memory consumption:
   
     - Improved support from upload handling of large input. When
       uploads larger than "maxupload" is spooled to a file, the
       results are available via [ns_conn contentfile] rather than
       [ns_conn content].  There might be many places in application
       programs, where this distinction might be relevant. The new
       version gets now form content via [ns_getform] from both
       sources, fixes potential encoding problems.  Similarly, the new
       function [ns_getcontent] returns the unparsed content either
       binary or encoded as file or as string. These new features makes
       it easer to use upload spooling to avoid memory bloats.
       
     - Furthermore, (spooled) files can be sent from disk to some http
       server via ns_http (or ns_ssl) without the need to load the
       content first to memory by using the new parameter body_file)

   * Improved crypto-support: the new version supports SHA2 hash codes
     (command ns_sha2) and keyed message authentications based on SHA2
     (command ns_hmac_sha2). The new version uses the C implementation
     of the crypto functions by Olivier Gay <oli...@a3...>.

   * Reduced memory consumption: avoid memory bloat on uploads and in
     HTTP client operations with large contents (esp. useful for
     contents >1GB).  ns_getform an the new ns_getcontent work now on
     spooled and in-memory content in a uniform manner.
     
   * Improved debugging support: new severity "Debug(request)" can be
     used follow life-time of a request (useful especially for
     inspecting concurrency of long-running requests).


Bug Fixes:

  - Fixed potential lockup, when bind operation fails,
    e.g. when server is already running (old bug)

  - Keep query parameter after directory induced redirect in fastpath
    (bug report of David Osborne in the NaviServer list)
    
  - Don't swallow erroneous HTTP requests: previously, error replies
    generated from the driver (e.g. 400 invalid request) were
    not logged at all (neither in the access.log or in the error.log).
    This is due to the nature, how the nslog modules works. Now
    such conditions are reported in the error.log.

  - Make HTTP-request line parser more precise. Previously
    many invalid requests were treated as HTTP/0.9 requests leading
    to strange error messages.

  - Provide more (and more detailed) error messages to ease debugging.

  - Improve validity of HTML error messages generated by NaviServer

  - Resurrect handling of non-HTTP drivers (based on
    drvPtr->requestProc) such as nssmtpd.

  - apply url-encoded form parsing only when content is
    www-form-urlencoded.  Previously, it was attempted on every content
    which might lead to unexpected results.

Documentation improvements:

  - Align style and logo of default web server configuration with style
    of man pages
  - Improved validity of HTML man pages
  - Fixed documentation bug on ns_cookie
  - Improved documentation of configure parameters
   - various more man pages updated

Tcl API Changes:

   - new function: [ns_info ipv6] obtains information whether the binary
     supports IPv6 or not

   - As a consequence of "multiple file" support (see above) the
     following functions return for the specified name potentially a
     list of values
     
         [ns_conn fileoffset $name]
	[ns_conn filelength $name]
	[ns_conn fileheaders $name]
	
     and the ns_set returned by [ns_getform] will contain multiple
     entries for the file (and $name.tmpfile and $name.content-type).

   - new commands:
       * ns_sha2 ?-length ...? data
       * ns_hmac_sha2 ?-length ...? key message
       * ns_getcontent (obtain body of request either from memory
         of from spoolfile without memory bloat on huge files)

   - new option "-body_file" for ns_http and ns_ssl. By using this
     parameter one can send huge files without memory bloat
     (i.e. loading the full file into memory). The function is more or
     less the counterpart to the spoolfile for uploads when the content
     is larger than maxupload.

   - ns_getform processes content from [ns_conn content] or
     [ns_conn contentfile] (whatever is available).

   - interface change:
       * old interface of ns_sendmail based on positional arguments is deprecated
       * new interface of ns_sendmail is
           ns_sendmail ?-to ...?  ?-from ...? ?-subject ...? ?-body ...? \
                       ?-headers ...? ?-bcc ...? ?-cc ...?
       
C API Changes:
   - New functions:
   
     * Ns_HttpParseHost(): parses host and port number from a string.
       Reason: IPv4 address conventions use simply a colon for
       separating host and port, but IPv6 address contain as well
       various colons. The new function supports IP-literal notation
       according to RFC 3986 section 3.2.2 (IP address between square
       brackets), which has to be used as well for IPv6.
       
     * Similarly, the new function Ns_HttpLocationString() to unifies
       now generation of host and location strings

     * New IP version agnostic functions for manipulating sockets,
       addresses and ports:
	Ns_LogSockaddr(), Ns_SockaddrGetPort(), Ns_SockaddrGetSockLen(),
	Ns_SockaddrMask(), Ns_SockaddrMaskBits(), Ns_SockaddrSetPort(),
	Ns_SockaddrSameIP(), ns_inet_ntop(), ns_inet_pton()
	
     * New function Ns_ConnReturnInvalidMethod() to report that the provided
       method is invalid (was returning "not found" before)

     * New function Ns_DStringAppendPrintable() to append a string which might
       contain non-printable characters in a printable fashion to a DString
       (useful for e.g. debugging)

     * Ns_Stat() is the former function FastStat() from fastpath made
       reusable in other modules.

   - Changed C-API functions (incompatible with earlier versions):
     * The following function accept now generic (version agnostic) sockaddr:
       Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
       Ns_SockBind(), Ns_SockBindUdp()
     * Ns_CtxString() renamed to Ns_HexString() to make its intention
       more clear and added a parameter to control lower and upper case



Configuration Changes:
   * New configure flag --disable-ipv6 to build NaviServer just with IPv4 support,
     even when IPv6 support is available.
   * Updated sample config files for IPv6
   * Added makefile target: "lldb-sample" lets one running nsd with
     the sample config file "nsd-config.tcl" under lldb

Command Line Changes:
   * Extended prebind options to work with IPv6 addresses

Code Changes:

   * All built-in MIME types are updated and completed with the IANA
     media type definitions. Warning: This change might return now
     different results from ns_guesstype unless the mime type is
     specified in the config file.

   * Improve reusability: The new version exports the low-level I/O functions
     of nssock to make it reusable in other driver modules. The following
     functions are now exported to a library:
   	Ns_DriverSockAccept(), Ns_DriverSockRecv(), Ns_DriverSockSend(),
	Ns_DriverSockClose()

   * Code cleanup:
     - used clean approach to obtain hex string from function pointer
     - use boolean types on more occasions
     - use more often typedef-ed function definitions
     - make NS_TCL_SET_STATIC and NS_TCL_SET_DYNAMIC an enum
       (Ns_TclSetType), since the values are mutual exclusive
     
   * Regression tests
     - extended test (we have now 1084 test cases)
     - remove deprecated calls
  

[naviserver-devel] Towards NaviServer 4.99.11

[Gustaf N. <ne...@wu...>]

Dear friends of NaviServer,

I have uploaded a release candidate of NaviServer 4.99.11 to
sourceforge (as 4.99.11b2 [1]).  In case nobody objects, or there are some other 
urgent changes from you, I'll finish a release of 4.99.11 the next days.
Below is a list of changes between NaviServer 4.99.10 and 4.99.11.

  155 files changed, 8357 insertions(+), 3106 deletions(-)

all the best
-gustaf neumann

[1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.11/


=======================================
NaviServer 4.99.11, released 2016-XX-XX
=======================================

New Features:

   * IPv6 support: NaviServer works now with IPv6 and IPv4
     addresses.  It is e.g. possible to use "::" as address (to denote
     the unspecified address, roughly similar to 0.0.0.0 in IPv4) to
     accept any IPv6 or IPv4 address, when the OS supports IPv4-mapped
     addresses, or one can use arbitrary addresses. Note, that one can
     as well load the driver for a server multiple times, once with an
     IPv6 and once with an IPv4 address.  Adding this feature required
     changes in the drivers, since earlier versions had IPv4 structures
     compiled in. As a consequence, also drivers for additional
     protocols have to be adjusted (for this release, the IPv6-adjusted
     modules are all modules of the naviserver core, plus nsssl, nsudp,
     nssmtpd, nsdns)

   * Added HTTP/1.1 pipelining support: Pipelining is a (not so often
     used) feature of HTTP/1.1 where multiple requests are sent on a
     persistent connection without waiting for the results of the first
     request. Recently more (esp. mobile) clients started to use
     pipelining. Previous versions of NaviServer handled just the first
     request of the pipeline and swallowed (ignored) later requests of
     the pipeline.

   * Added support for HTML5 file attribute "multiple": HTML5 allows
     for uploading multiple files under the same INPUT name. Previous
     version of NaviServer just accepted the first file with the given
     name.

   * Reduced memory consumption:
   
     - Improved support from upload handling of large input. When
       uploads larger than "maxupload" is spooled to a file, the
       results are available via [ns_conn contentfile] rather than
       [ns_conn content].  There might be many places in application
       programs, where this distinction might be relevant. The new
       version gets now form content via [ns_getform] from both
       sources, fixes potential encoding problems.  Similarly, the new
       function [ns_getcontent] returns the unparsed content either
       binary or encoded as file or as string. These new features makes
       it easer to use upload spooling to avoid memory bloats.
       
     - Furthermore, (spooled) files can be sent from disk to some http
       server via ns_http (or ns_ssl) without the need to load the
       content first to memory by using the new parameter body_file)

   * Improved crypto-support: the new version supports SHA2 hashcodes
     (command ns_sha2) and keyed message authentications based on SHA2
     (command ns_hmac_sha2). The new version uses the C implementation
     of the crypto functions by Olivier Gay <oli...@a3...>.

   * Reduced memory consumption: avoid memory bloat on uploads and in
     HTTP client operations with large contents (esp. useful for
     contents >1GB).  ns_getform an the new ns_getcontent work now on
     spooled and in-memory content in a uniform manner.
     
   * Improved debugging support: new severity "Debug(request)" can be
     used follow life-time of a request (useful especially for
     inspecting concurrency of long-running requests).


Performance Improvements:

Bug Fixes:

  - Fixed potential lockup, when bind operation fails,
    e.g. when server is already running (old bug)

  - Keep query parameter after directory induced redirect in fastpath
    (bug report of David Osborne in the NaviServer list)
    
  - Don't swallow erroneous HTTP requests: previously, error replies
    generated from the driver (e.g. 400 invalid request) were
    not logged at all (neither in the access.log or in the error.log).
    This is due to the nature, how the nslog modules works. Now
    such conditions are reported in the error.log.

  - Make HTTP-request line parser more precise. Previously
    many invalid requests were treated as HTTP/0.9 requests leading
    to strange error messages.

  - Provide more (and more detailed) error messages to ease debugging.

  - Improve validity of HTML error messages generated by NaviServer

  - Resurrect handling of non-HTTP drivers (based on
    drvPtr->requestProc) such as nssmtpd.

Documentation improvements:

  - Align style and logo of default web server configuration with style
    of man pages
  - Improved validity of HTML man pages
  - Fixed documentation bug on ns_cookie
  - Improved documentation of configure parameters
   - various more man pages updated

Tcl API Changes:

   - new function: [ns_info ipv6] obtains information whether the binary
     supports IPv6 or not

   - As a consequence of "multiple file" support (see above) the
     following functions return for the specified name potentially a
     list of values
     
         [ns_conn fileoffset $name]
	[ns_conn filelength $name]
	[ns_conn fileheaders $name]
	
     and the ns_set returned by [ns_getform] will contain multiple
     entries for the file (and $name.tmpfile and $name.content-type).

   - new commands:
       * ns_sha2 ?-length ...? data
       * ns_hmac_sha2 ?-length ...? key message
       * ns_getcontent (obtain body of request either from memory
         of from spoolfile without memory bloat on huge files)

   - new option "-body_file" for ns_http and ns_ssl. By using this
     parameter one can send huge files without memory bloat
     (i.e. loading the full file into memory). The function is more or
     less the counterpart to the spoolfile for uploads when the content
     is larger than maxupload.

   - ns_getform processes content from [ns_conn content] or
     [ns_conn contentfile] (whatever is available).

   - interface change:
       * old interface of ns_sendmail based on positional arguments is deprecated
       * new interface of ns_sendmail is
           ns_sendmail ?-to ...?  ?-from ...? ?-subject ...? ?-body ...? \
                       ?-headers ...? ?-bcc ...? ?-cc ...?
       
C API Changes:
   - New functions:
   
     * Ns_HttpParseHost(): parses host and port number from a string.
       Reason: IPv4 address conventions use simply a colon for
       separating host and port, but IPv6 address contain as well
       various colons. The new function supports IP-literal notation
       according to RFC 3986 section 3.2.2 (IP address between square
       brackets), which has to be used as well for IPv6.
       
     * Similarly, the new function Ns_HttpLocationString() to unifies
       now generation of host and location strings

     * New IP version agnostic functions for manipulating sockets,
       addresses and ports:
	Ns_LogSockaddr(), Ns_SockaddrGetPort(), Ns_SockaddrGetSockLen(),
	Ns_SockaddrMask(), Ns_SockaddrMaskBits(), Ns_SockaddrSetPort(),
	Ns_SockaddrSameIP(), ns_inet_ntop(), ns_inet_pton()
	
     * New function Ns_ConnReturnInvalidMethod() to report that the provided
       method is invalid (was returning "not found" before)

     * New function Ns_DStringAppendPrintable() to append a string which might
       contain non-printable characters in a printable fashion to a DString
       (useful for e.g. debugging)

     * Ns_Stat() is the former function FastStat() from fastpath made
       reusable in other modules.

   - Changed C-API functions (incompatible with earlier versions):
     * The following function accept now generic (version agnostic) sockaddr:
       Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
       Ns_SockBind(), Ns_SockBindUdp()
     * Ns_CtxString() renamed to Ns_HexString() to make its intention
       more clear and added a parameter to control lower and upper case



Configuration Changes:
   * New configure flag --disable-ipv6 to build NaviServer just with IPv4 support,
     even when IPv6 support is available.
   * Updated sample config files for IPv6
   * Added makefile target: "lldb-sample" lets one running nsd with
     the sample config file "nsd-config.tcl" under lldb

Command Line Changes:
   * Extended prebind options to work with IPv6 addresses

Code Changes:

   * All built-in MIME types are updated and completed with the IANA
     media type definitions. Warning: This change might return now
     different results from ns_guesstype unless the mime type is
     specified in the config file.

   * Improve reusability: The new version exports the low-level I/O functions
     of nssock to make it reusable in other driver modules. The following
     functions are now exported to a library:
   	Ns_DriverSockAccept(), Ns_DriverSockRecv(), Ns_DriverSockSend(),
	Ns_DriverSockClose()

   * Code cleanup:
     - used clean approach to obtain hex string from function pointer
     - use boolean types on more occasions
     - use more often typedef-ed function definitions
     - make NS_TCL_SET_STATIC and NS_TCL_SET_DYNAMIC an enum
       (Ns_TclSetType), since the values are mutual exclusive
     
   * Regression tests
     - extended test (we have now 1084 test cases)
     - remove deprecated calls
     

Re: [naviserver-devel] Form vars + 302 redirect with trailing slash

[Gustaf N. <ne...@wu...>]

Dear David,

this issue is addressed by the following commit on bitbucket:

https://bitbucket.org/naviserver/naviserver/commits/052ba1d24e6061c03fe1a09735bb29031cf12a61

all the best and happy easter!
-g

Am 17.03.16 um 20:57 schrieb Gustaf Neumann:
> Dear David,
>
> Although i do wonder, why this was not noticed before, i think this is 
> a bug.
> The code is quite old and is the same as in aolserver. The intention 
> is of this
> code block is to restart the query, there is no reason, why in such case
> the query variables should be omitted.
>
> -g
>
> Am 17.03.16 um 11:56 schrieb David Osborne:
>> HI,
>>
>> Quick question.
>> We've noticed that when fastpath does an internal 302 redirect for 
>> URLs without a trailing slash, it seems to drop the query string.
>> Is this intended behaviour?
>>
>> eg.
>> http://site.com/news?a=1
>> 302 redirects to
>> http://site.com/news/
>>
>> I think it maybe this redirect that's in action:
>> https://bitbucket.org/naviserver/naviserver/src/0aa51bd108e09f34131231b1e80446d614b943eb/nsd/fastpath.c?at=default&fileviewer=file-view-default#fastpath.c-245
>>
>> Regards,
>> -- David

Re: [naviserver-devel] Additional mime types in nsd/mimetypes.c

[Wolfgang W. <wol...@di...>]

Dear Gustaf!

Thanks for you effort. Lower-casing the mime-types should not pose a 
problem.

best regards,

Wolfgang Winkler

Am 2016-03-18 um 14:32 schrieb Gustaf Neumann:
> Small followup:
>
> since mime-types are case insensitive (rfc 2045), mixed case mime 
> types can cause confusions,
> Candidates are the newly added "*.macroEnabled.*" mime types (see e.g. 
> [1]).
> To avoid run-time overhead, i've lower-cased all built-in mime types.
>
> -g
>
> [1] https://lists.freedesktop.org/archives/xdg/2013-March/012711.html
>
> Am 17.03.16 um 20:27 schrieb Gustaf Neumann:
>> Dear Wolfgang,
>>
>> Commit [1] updates the media types according to the IANA specs.
>> In the commit i have completed the references for the definitions where
>> possible. Several extensions are uses unfortunately for several media
>> types, so one has to stick to one definition. Altogether, it was a 
>> larger
>> change than expected:
>>
>>  1 file changed, 607 insertions(+), 99 deletions(-)
>>
>> Please double check, if everything is fine now
>>
>> best regards
>> -gustaf neumann
>>
>> [1] 
>> https://bitbucket.org/naviserver/naviserver/commits/c438c3ba27821dcce3f10057c25824c4a9595ae8
>> [2] http://www.iana.org/assignments/media-types/media-types.xhtml
>>
>> Am 17.03.16 um 08:01 schrieb Wolfgang Winkler:
>>> Hi!
>>>
>>> We are using the following additional mime types in our naviserver 
>>> configuration:
>>>
>>> ns_section "ns/mimetypes"
>>>   ns_param  ".docx" 
>>> "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
>>>   ns_param  ".dotx" 
>>> "application/vnd.openxmlformats-officedocument.wordprocessingml.template"
>>>   ns_param  ".docm" "application/vnd.ms-word.document.macroEnabled.12"
>>>   ns_param  ".dotm" "application/vnd.ms-word.template.macroEnabled.12"
>>>
>>>   ns_param  ".xlsx" 
>>> "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
>>>   ns_param  ".xltx" 
>>> "application/vnd.openxmlformats-officedocument.spreadsheetml.template"
>>>   ns_param  ".xlsm" "application/vnd.ms-excel.sheet.macroEnabled.12"
>>>   ns_param  ".xltm" "application/vnd.ms-excel.template.macroEnabled.12"
>>>   ns_param  ".xlam" "application/vnd.ms-excel.addin.macroEnabled.12"
>>>   ns_param  ".xlsb" 
>>> "application/vnd.ms-excel.sheet.binary.macroEnabled.12"
>>>
>>>   ns_param  ".pptx" 
>>> "application/vnd.openxmlformats-officedocument.presentationml.presentation"
>>>   ns_param  ".potx" 
>>> "application/vnd.openxmlformats-officedocument.presentationml.template"
>>>   ns_param  ".ppsx" 
>>> "application/vnd.openxmlformats-officedocument.presentationml.slideshow"
>>>   ns_param  ".ppam" 
>>> "application/vnd.ms-powerpoint.addin.macroEnabled.12"
>>>   ns_param  ".pptm" 
>>> "application/vnd.ms-powerpoint.presentation.macroEnabled.12"
>>>   ns_param  ".potm" 
>>> "application/vnd.ms-powerpoint.template.macroEnabled.12"
>>>   ns_param  ".ppsm" 
>>> "application/vnd.ms-powerpoint.slideshow.macroEnabled.12"
>>>
>>> These are used mostly for downloading attachments and it works fine. 
>>> But I think these file types are common enough to justify an 
>>> inclusion in nsd/mimetypes.c.
>>>
>>> Regards,
>>>
>>> Wolfgang Winkler
>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz



etailment Wien – www.wien.etailment.at <http://wien.etailment.at/>

Wir sind persönlich am 23. und 24. Februar mit einem Messestand 
vertreten, besuchen Sie uns!

Re: [naviserver-devel] Additional mime types in nsd/mimetypes.c

[Gustaf N. <ne...@wu...>]

Small followup:

since mime-types are case insensitive (rfc 2045), mixed case mime types 
can cause confusions,
Candidates are the newly added "*.macroEnabled.*" mime types (see e.g. 
[1]).
To avoid run-time overhead, i've lower-cased all built-in mime types.

-g

[1] https://lists.freedesktop.org/archives/xdg/2013-March/012711.html

Am 17.03.16 um 20:27 schrieb Gustaf Neumann:
> Dear Wolfgang,
>
> Commit [1] updates the media types according to the IANA specs.
> In the commit i have completed the references for the definitions where
> possible. Several extensions are uses unfortunately for several media
> types, so one has to stick to one definition. Altogether, it was a larger
> change than expected:
>
>  1 file changed, 607 insertions(+), 99 deletions(-)
>
> Please double check, if everything is fine now
>
> best regards
> -gustaf neumann
>
> [1] 
> https://bitbucket.org/naviserver/naviserver/commits/c438c3ba27821dcce3f10057c25824c4a9595ae8
> [2] http://www.iana.org/assignments/media-types/media-types.xhtml
>
> Am 17.03.16 um 08:01 schrieb Wolfgang Winkler:
>> Hi!
>>
>> We are using the following additional mime types in our naviserver 
>> configuration:
>>
>> ns_section "ns/mimetypes"
>>   ns_param  ".docx" 
>> "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
>>   ns_param  ".dotx" 
>> "application/vnd.openxmlformats-officedocument.wordprocessingml.template"
>>   ns_param  ".docm" "application/vnd.ms-word.document.macroEnabled.12"
>>   ns_param  ".dotm" "application/vnd.ms-word.template.macroEnabled.12"
>>
>>   ns_param  ".xlsx" 
>> "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
>>   ns_param  ".xltx" 
>> "application/vnd.openxmlformats-officedocument.spreadsheetml.template"
>>   ns_param  ".xlsm" "application/vnd.ms-excel.sheet.macroEnabled.12"
>>   ns_param  ".xltm" "application/vnd.ms-excel.template.macroEnabled.12"
>>   ns_param  ".xlam" "application/vnd.ms-excel.addin.macroEnabled.12"
>>   ns_param  ".xlsb" 
>> "application/vnd.ms-excel.sheet.binary.macroEnabled.12"
>>
>>   ns_param  ".pptx" 
>> "application/vnd.openxmlformats-officedocument.presentationml.presentation"
>>   ns_param  ".potx" 
>> "application/vnd.openxmlformats-officedocument.presentationml.template"
>>   ns_param  ".ppsx" 
>> "application/vnd.openxmlformats-officedocument.presentationml.slideshow"
>>   ns_param  ".ppam" "application/vnd.ms-powerpoint.addin.macroEnabled.12"
>>   ns_param  ".pptm" 
>> "application/vnd.ms-powerpoint.presentation.macroEnabled.12"
>>   ns_param  ".potm" 
>> "application/vnd.ms-powerpoint.template.macroEnabled.12"
>>   ns_param  ".ppsm" 
>> "application/vnd.ms-powerpoint.slideshow.macroEnabled.12"
>>
>> These are used mostly for downloading attachments and it works fine. 
>> But I think these file types are common enough to justify an 
>> inclusion in nsd/mimetypes.c.
>>
>> Regards,
>>
>> Wolfgang Winkler

Re: [naviserver-devel] Form vars + 302 redirect with trailing slash

[Gustaf N. <ne...@wu...>]

Dear David,

Although i do wonder, why this was not noticed before, i think this is a 
bug.
The code is quite old and is the same as in aolserver. The intention is 
of this
code block is to restart the query, there is no reason, why in such case
the query variables should be omitted.

-g

Am 17.03.16 um 11:56 schrieb David Osborne:
> HI,
>
> Quick question.
> We've noticed that when fastpath does an internal 302 redirect for 
> URLs without a trailing slash, it seems to drop the query string.
> Is this intended behaviour?
>
> eg.
> http://site.com/news?a=1
> 302 redirects to
> http://site.com/news/
>
> I think it maybe this redirect that's in action:
> https://bitbucket.org/naviserver/naviserver/src/0aa51bd108e09f34131231b1e80446d614b943eb/nsd/fastpath.c?at=default&fileviewer=file-view-default#fastpath.c-245
>
> Regards,
> -- David

Re: [naviserver-devel] Additional mime types in nsd/mimetypes.c

[Gustaf N. <ne...@wu...>]

Dear Wolfgang,

Commit [1] updates the media types according to the IANA specs.
In the commit i have completed the references for the definitions where
possible. Several extensions are uses unfortunately for several media
types, so one has to stick to one definition. Altogether, it was a larger
change than expected:

  1 file changed, 607 insertions(+), 99 deletions(-)

Please double check, if everything is fine now

best regards
-gustaf neumann

[1] 
https://bitbucket.org/naviserver/naviserver/commits/c438c3ba27821dcce3f10057c25824c4a9595ae8
[2] http://www.iana.org/assignments/media-types/media-types.xhtml

Am 17.03.16 um 08:01 schrieb Wolfgang Winkler:
> Hi!
>
> We are using the following additional mime types in our naviserver 
> configuration:
>
> ns_section "ns/mimetypes"
>   ns_param  ".docx" 
> "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
>   ns_param  ".dotx" 
> "application/vnd.openxmlformats-officedocument.wordprocessingml.template"
>   ns_param  ".docm" "application/vnd.ms-word.document.macroEnabled.12"
>   ns_param  ".dotm" "application/vnd.ms-word.template.macroEnabled.12"
>
>   ns_param  ".xlsx" 
> "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
>   ns_param  ".xltx" 
> "application/vnd.openxmlformats-officedocument.spreadsheetml.template"
>   ns_param  ".xlsm" "application/vnd.ms-excel.sheet.macroEnabled.12"
>   ns_param  ".xltm" "application/vnd.ms-excel.template.macroEnabled.12"
>   ns_param  ".xlam" "application/vnd.ms-excel.addin.macroEnabled.12"
>   ns_param  ".xlsb" 
> "application/vnd.ms-excel.sheet.binary.macroEnabled.12"
>
>   ns_param  ".pptx" 
> "application/vnd.openxmlformats-officedocument.presentationml.presentation"
>   ns_param  ".potx" 
> "application/vnd.openxmlformats-officedocument.presentationml.template"
>   ns_param  ".ppsx" 
> "application/vnd.openxmlformats-officedocument.presentationml.slideshow"
>   ns_param  ".ppam" "application/vnd.ms-powerpoint.addin.macroEnabled.12"
>   ns_param  ".pptm" 
> "application/vnd.ms-powerpoint.presentation.macroEnabled.12"
>   ns_param  ".potm" 
> "application/vnd.ms-powerpoint.template.macroEnabled.12"
>   ns_param  ".ppsm" 
> "application/vnd.ms-powerpoint.slideshow.macroEnabled.12"
>
> These are used mostly for downloading attachments and it works fine. 
> But I think these file types are common enough to justify an inclusion 
> in nsd/mimetypes.c.
>
> Regards,
>
> Wolfgang Winkler

[naviserver-devel] Form vars + 302 redirect with trailing slash

[David O. <da...@qc...>]

HI,

Quick question.
We've noticed that when fastpath does an internal 302 redirect for URLs
without a trailing slash, it seems to drop the query string.
Is this intended behaviour?

eg.
http://site.com/news?a=1
302 redirects to
http://site.com/news/

I think it maybe this redirect that's in action:
https://bitbucket.org/naviserver/naviserver/src/0aa51bd108e09f34131231b1e80446d614b943eb/nsd/fastpath.c?at=default&fileviewer=file-view-default#fastpath.c-245

Regards,
-- David

Re: [naviserver-devel] Additional mime types in nsd/mimetypes.c

[Gustaf N. <ne...@wu...>]

Dear Wolfgang,

in 2013 i've updated the compiled-in mimetypes with the complete list of 
mimetypes from
RFCs, W3C and IANA listings. One has to redo this. Unfortunately, the 
IANA has reorganized
some of their materials (the "mime types" became "media types", the 
detail information is
in separate files) and included as well materials from RFCs, etc.

Since there are many versions of mime-type mappings out there, i prefer
to stick with IANA to provide a solid reference. I will look into this 
over the weekend.

all the best
-g

Am 17.03.16 um 08:01 schrieb Wolfgang Winkler:
> Hi!
>
> We are using the following additional mime types in our naviserver 
> configuration:
>
> ns_section "ns/mimetypes"
>   ns_param  ".docx" 
> "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
>   ns_param  ".dotx" 
> "application/vnd.openxmlformats-officedocument.wordprocessingml.template"
>   ns_param  ".docm" "application/vnd.ms-word.document.macroEnabled.12"
>   ns_param  ".dotm" "application/vnd.ms-word.template.macroEnabled.12"
>
>   ns_param  ".xlsx" 
> "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
>   ns_param  ".xltx" 
> "application/vnd.openxmlformats-officedocument.spreadsheetml.template"
>   ns_param  ".xlsm" "application/vnd.ms-excel.sheet.macroEnabled.12"
>   ns_param  ".xltm" "application/vnd.ms-excel.template.macroEnabled.12"
>   ns_param  ".xlam" "application/vnd.ms-excel.addin.macroEnabled.12"
>   ns_param  ".xlsb" 
> "application/vnd.ms-excel.sheet.binary.macroEnabled.12"
>
>   ns_param  ".pptx" 
> "application/vnd.openxmlformats-officedocument.presentationml.presentation"
>   ns_param  ".potx" 
> "application/vnd.openxmlformats-officedocument.presentationml.template"
>   ns_param  ".ppsx" 
> "application/vnd.openxmlformats-officedocument.presentationml.slideshow"
>   ns_param  ".ppam" "application/vnd.ms-powerpoint.addin.macroEnabled.12"
>   ns_param  ".pptm" 
> "application/vnd.ms-powerpoint.presentation.macroEnabled.12"
>   ns_param  ".potm" 
> "application/vnd.ms-powerpoint.template.macroEnabled.12"
>   ns_param  ".ppsm" 
> "application/vnd.ms-powerpoint.slideshow.macroEnabled.12"
>
> These are used mostly for downloading attachments and it works fine. 
> But I think these file types are common enough to justify an inclusion 
> in nsd/mimetypes.c.
>
> Regards,
>
> Wolfgang Winkler
>
> -- 
>
> *Wolfgang Winkler*
> Geschäftsführung
> wol...@di...
> mobil +43.699.19971172
>
> dc:*büro*
> digital concepts Novak Winkler OG
> Software & Design
> Landstraße 68, 5. Stock, 4020 Linz
> www.digital-concepts.com <http://www.digital-concepts.com>
> tel +43.732.997117.72
> tel +43.699.1997117.2
>
> Firmenbuchnummer: 192003h
> Firmenbuchgericht: Landesgericht Linz
>
>
>
> etailment Wien – www.wien.etailment.at <http://wien.etailment.at/>
>
> Wir sind persönlich am 23. und 24. Februar mit einem Messestand 
> vertreten, besuchen Sie uns!
>

[naviserver-devel] Additional mime types in nsd/mimetypes.c

[Wolfgang W. <wol...@di...>]

Hi!

We are using the following additional mime types in our naviserver 
configuration:

ns_section "ns/mimetypes"
   ns_param  ".docx" 
"application/vnd.openxmlformats-officedocument.wordprocessingml.document"
   ns_param  ".dotx" 
"application/vnd.openxmlformats-officedocument.wordprocessingml.template"
   ns_param  ".docm" "application/vnd.ms-word.document.macroEnabled.12"
   ns_param  ".dotm" "application/vnd.ms-word.template.macroEnabled.12"

   ns_param  ".xlsx" 
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
   ns_param  ".xltx" 
"application/vnd.openxmlformats-officedocument.spreadsheetml.template"
   ns_param  ".xlsm" "application/vnd.ms-excel.sheet.macroEnabled.12"
   ns_param  ".xltm" "application/vnd.ms-excel.template.macroEnabled.12"
   ns_param  ".xlam" "application/vnd.ms-excel.addin.macroEnabled.12"
   ns_param  ".xlsb" "application/vnd.ms-excel.sheet.binary.macroEnabled.12"

   ns_param  ".pptx" 
"application/vnd.openxmlformats-officedocument.presentationml.presentation"
   ns_param  ".potx" 
"application/vnd.openxmlformats-officedocument.presentationml.template"
   ns_param  ".ppsx" 
"application/vnd.openxmlformats-officedocument.presentationml.slideshow"
   ns_param  ".ppam" "application/vnd.ms-powerpoint.addin.macroEnabled.12"
   ns_param  ".pptm" 
"application/vnd.ms-powerpoint.presentation.macroEnabled.12"
   ns_param  ".potm" 
"application/vnd.ms-powerpoint.template.macroEnabled.12"
   ns_param  ".ppsm" 
"application/vnd.ms-powerpoint.slideshow.macroEnabled.12"

These are used mostly for downloading attachments and it works fine. But 
I think these file types are common enough to justify an inclusion in 
nsd/mimetypes.c.

Regards,

Wolfgang Winkler

-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz



etailment Wien – www.wien.etailment.at <http://wien.etailment.at/>

Wir sind persönlich am 23. und 24. Februar mit einem Messestand 
vertreten, besuchen Sie uns!

[naviserver-devel] spnego for NaviServer

[Gustaf N. <ne...@wu...>]

Dear all,

a few minutes ago, i've added the knspnego module to the NaviServer
modules on bitbucket, which can be used for single-sign (SSO)
via (mostly) windows clients.

SPNEGO [1] is an implementation of Simple and Protected GSSAPI
Negotiation Mechanism. SPNEGO is used in Microsoft's "HTTP Negotiate"
authentication extension. SPNEGO was first implemented in Internet
Explorer 5.01 and IIS 5.0 and provided single sign-on capability later
marketed as Integrated Windows Authentication.

The knspnego module is a port of the aolserver module [2] to NaviServer.
The implementation of the module is based on the Apache module
mod_spnego [3] and supports Kerberos.

Codewise, it is from formatting/comments/... not on the same
level as other NaviServer modules, but apparently it works fine.
I have just helped porting and compiling the module for a user,
who says it works fine for their customers.

Maybe this can reduce work for other people having similar
requirements

-gn

[1] https://en.wikipedia.org/wiki/SPNEGO
[2] http://aolserver.cvs.sourceforge.net/viewvc/aolserver/knspnego/
[3] https://sourceforge.net/projects/modspnego/

Re: [naviserver-devel] IPv6 for NaviServer

[Gustaf N. <ne...@wu...>]

Great news!

Today, i've installed the tip version of naviserver also on openacs.org.
-g

Am 26.02.16 um 16:44 schrieb Maurizio Martignano:
>
> Dear Gustaf,
>
> Thanks to your last changes, now Naviserver supports IPV6 also when 
> compiled with Visual Studio 2015 and targeting Windows 64 bit.
>
> I’ve updated consequently the Windows-Open distribution 
> (http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/).
>
> Thank you,
>
> Maurizio
>
> *From:*Gustaf Neumann [mailto:ne...@wu...]
> *Sent:* 26 February 2016 09:22
> *To:* nav...@li...
> *Subject:* Re: [naviserver-devel] IPv6 for NaviServer
>
> Great,
>
> this means, we do not have to "emulate" it - as done in previous versions.
> I've done some more cleanup for MSVC and activated IPV6 per default
> (everything untested)
>
> -g
> [1] 
> https://bitbucket.org/naviserver/naviserver/commits/432a251c4e8ee567327b1228cee8e25a726af5bc
>
>
> Am 25.02.16 um 18:04 schrieb Maurizio Martignano:
>
>     Dear Gustaf,
>
>     Thank you for your inputs.
>
>     I implemented all your changes but the system was still not working.
>
>     I had a careful look at it and eventually I found the cause (at
>     least in my configuration):
>
>     The problem in caused by the function inet_ntop in nswin32.c.
>     After removing it, everything works fine.
>
>     The system seems now OK to me.
>
>     Thank you,
>
>     Maurizio
>
>     *From:*Gustaf Neumann [mailto:ne...@wu...]
>     *Sent:* 25 February 2016 12:33
>     *To:* nav...@li...
>     <mailto:nav...@li...>
>     *Subject:* Re: [naviserver-devel] IPv6 for NaviServer
>
>     Hi Maurizio,
>
>     thanks for the fixes. as such, they could not work, but they point
>     out things, where one has to watch out. I have fixed these
>     things in a way it could work under windows.
>
>     By looking at the differences, it seems as if you compiled just a
>     IPv4 version (no IPv6 enabled). i would have expected this
>     to go more smoothly. In case you tried the sample
>     nsd-config.tcl script, that could not work with IPv4, since
>     IPv6 addresses were hard-coded.
>
>     all the best
>     -g
>
>     Am 24.02.16 um 18:00 schrieb Maurizio Martignano:
>
>         Dear Gustaf,
>
>            Thank you as always for all your work.
>
>           
>
>         About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
>
>         Visual Studio 2015 with 64 bit as target.
>
>         With this configuration I tried to modify the file sockaddr.c (to make it
>
>         compile), but I'm not sure about the changes I made.
>
>         I also had to modify the file nswin32.c as I could not find anywhere the
>
>         function "NsSockGetPort".
>
>         Well the resulting system compiles... but nothing works.
>
>         I need much more time to do a proper testing/tuning and I can't do that at
>
>         the moment.
>
>         As soon as I discover more, I'll let you know.
>
>           
>
>         Back to your delivery of the 16th of February.
>
>           
>
>         Once again thank you,
>
>         Maurizio
>
>           
>
>           
>
>         -----Original Message-----
>
>         From: Gustaf Neumann [mailto:ne...@wu...]
>
>         Sent: 24 February 2016 14:01
>
>         To: Navidevel<nav...@li...>
>         <mailto:nav...@li...>
>
>         Subject: [naviserver-devel] IPv6 for NaviServer
>
>           
>
>         Dear all,
>
>           
>
>         over the last week, i've worked on IPv6 support for NaviServer.
>
>         For this task, i took the following path:
>
>         1) build a version of NaviServer for IPv6
>
>         2) build a single code source for compiling either for IPv4 or IPv6
>
>         3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
>
>         4) Run regression test suite under IPv4 and/IPv6
>
>         5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
>
>              (requires some interface changes to handle generic sockaddrs)
>
>           
>
>         Everything seems to work fine, one can work with the same config files as
>
>         before, and everything works as expected, but one can as well listen on IPv6
>
>         addresses (e.g.
>
>         the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
>
>         kernel allows to deliver as well
>
>         IPv4 traffic to the IPv6 interface, then one can talk also multiple
>
>         protocols. The mileage may vary depending on the OS and kernel.
>
>           
>
>         It is as well possible to deactivate IPv6 support at compile time via
>
>         --disable-ipv6.
>
>           
>
>         Here is a summary of the necessary changes:
>
>         - Added a amall Abstraction API for IP-version agnostic calls:
>
>             Ns_LogSockaddr(), Ns_SockaddrGetPort(),
>
>             Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
>
>             Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
>
>           
>
>         - Replaced in interface more places where "sockaddr_in" (the IPv4
>
>             version of the sockaddr) with the generic (version agnostic)
>
>             sockaddr (also in IPv4 code to obtain API consistency when no IPv6
>
>             is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
>
>             Ns_SockBind(), Ns_SockBindUdp()
>
>           
>
>         - The Ns_Sock structure contained sockaddr_in, which is not sufficient
>
>             to store IPv6 addresses. Therefore it was necessary to change
>
>             that to "sockaddr_storage" which guarantees to be large enough
>
>             to keep all kind of addresses. This affects as well driver modules
>
>             which communicate via the NaviServer managed sockets. Old drivers
>
>             might crash with the new structure. Therefore, I've bumped
>
>             NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
>
>             can communicate that they support IPv6. In IPv6 mode, NaviServer
>
>             refuses to work with versions <3.
>
>           
>
>         - Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
>
>             (except for legacy an minimal library support)
>
>           
>
>         - Added log file warnings on various places, where NaviServer
>
>             was ignoring error states in socket communications silently.
>
>           
>
>         - Transformed hash table for open listen ports to string keys
>
>             (the old code used 32bit IPv4-addresses as keys in hash tables)
>
>         - added test for ns_listencallback
>
>           
>
>         - Ns_HttpParseHost: new abstraction for parsing host and port number.
>
>             Reason: IPv6 literal notation contains many colons, parsing
>
>             IP-literal notation according to RFC 3986 section 3.2.2 is necessary
>
>             (removed places where code was searching for ':' to expect the port)
>
>           
>
>         - Added [ns_info ipv6] to obtain information whether the binary
>
>             supports IPv6 or not
>
>           
>
>         - Added configure flag --disable-ipv6 to build NaviServer just with
>
>             IPv4 support.
>
>           
>
>         - provide defines for core differences between IPv4 and IPv6 #ifdef
>
>         HAVE_IPV6
>
>         # define NS_IP_LOOPBACK     "::1"
>
>         # define NS_IP_UNSPECIFIED  "::"
>
>         # define NS_SOCKADDR_IN     sockaddr_storage
>
>         # define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
>
>         #else
>
>         # define NS_IP_LOOPBACK     "127.0.0.1"
>
>         # define NS_IP_UNSPECIFIED  "0.0.0.0"
>
>         # define NS_SOCKADDR_IN     sockaddr_in
>
>         # define NS_IPADDR_SIZE     INET_ADDRSTRLEN
>
>         #endif
>
>           
>
>            - nsperm:
>
>             * Aligned implementation with the documentation
>
>               (hosts.allow/deny have comma-separated entries)
>
>             * Added submask specification in the form /xxx
>
>               (e.g. ::1/64)
>
>             * Use binary IPv4 and IPv6 values for hash and mask lookup
>
>             * Generalizing masking functions
>
>             * Refactor code to avoid memory leaks
>
>             * Updated documentation
>
>           
>
>         - modules:
>
>             * nsudp and nsssl are now IPv6 aware/compliant
>
>           
>
>         TODOs:
>
>         - more platform testing
>
>             (probably changes for Windows necessary, help is appreciated)
>
>         - remove commented out debug statements
>
>           
>
>         - make more modules IPv6 compliant
>
>         - documentation updates
>
>           
>
>         I've tested the generic and IPv4 only version via regression test and random
>
>         browsing on Mac OS X and Linux. If you have other platforms, testing would
>
>         be appreciated. There are some adjustments for windows probably necessary
>
>         (in the absence of "configure", one has to add manually "HAVE_IPV4" to the
>
>         compile flags). Any help is welcome.
>
>           
>
>         best regards
>
>         -gustaf neumann
>
>           
>
>           
>

Re: [naviserver-devel] IPv6 for NaviServer

[Maurizio M. <Mau...@sp...>]

Dear Gustaf,

               Thanks to your last changes, now Naviserver supports IPV6
also when compiled with Visual Studio 2015 and targeting Windows 64 bit.

 

I've updated consequently the Windows-Open distribution
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/).

 

Thank you,

Maurizio

 

From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 26 February 2016 09:22
To: nav...@li...
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Great, 

this means, we do not have to "emulate" it - as done in previous versions.
I've done some more cleanup for MSVC and activated IPV6 per default 
(everything untested)

-g
[1]
https://bitbucket.org/naviserver/naviserver/commits/432a251c4e8ee567327b1228
cee8e25a726af5bc


Am 25.02.16 um 18:04 schrieb Maurizio Martignano:

Dear Gustaf,

               Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 25 February 2016 12:33
To: nav...@li...
<mailto:nav...@li...> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-----Original Message-----
From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:nav...@li...>
<nav...@li...>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
    (requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various places, where NaviServer
   was ignoring error states in socket communications silently.
 
- Transformed hash table for open listen ports to string keys
   (the old code used 32bit IPv4-addresses as keys in hash tables)
- added test for ns_listencallback
 
- Ns_HttpParseHost: new abstraction for parsing host and port number.
   Reason: IPv6 literal notation contains many colons, parsing
   IP-literal notation according to RFC 3986 section 3.2.2 is necessary
   (removed places where code was searching for ':' to expect the port)
 
- Added [ns_info ipv6] to obtain information whether the binary
   supports IPv6 or not
 
- Added configure flag --disable-ipv6 to build NaviServer just with
   IPv4 support.
 
- provide defines for core differences between IPv4 and IPv6 #ifdef
HAVE_IPV6
# define NS_IP_LOOPBACK     "::1"
# define NS_IP_UNSPECIFIED  "::"
# define NS_SOCKADDR_IN     sockaddr_storage
# define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
#else
# define NS_IP_LOOPBACK     "127.0.0.1"
# define NS_IP_UNSPECIFIED  "0.0.0.0"
# define NS_SOCKADDR_IN     sockaddr_in
# define NS_IPADDR_SIZE     INET_ADDRSTRLEN
#endif
 
  - nsperm:
   * Aligned implementation with the documentation
     (hosts.allow/deny have comma-separated entries)
   * Added submask specification in the form /xxx
     (e.g. ::1/64)
   * Use binary IPv4 and IPv6 values for hash and mask lookup
   * Generalizing masking functions
   * Refactor code to avoid memory leaks
   * Updated documentation
 
- modules:
   * nsudp and nsssl are now IPv6 aware/compliant
 
TODOs:
- more platform testing
   (probably changes for Windows necessary, help is appreciated)
- remove commented out debug statements
 
- make more modules IPv6 compliant
- documentation updates
 
I've tested the generic and IPv4 only version via regression test and random
browsing on Mac OS X and Linux. If you have other platforms, testing would
be appreciated. There are some adjustments for windows probably necessary
(in the absence of "configure", one has to add manually "HAVE_IPV4" to the
compile flags). Any help is welcome.
 
best regards
-gustaf neumann
 
 
----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM +
Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor
end-to-end web transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140
_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel







----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140







_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel







-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria






----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140






_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel






-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] IPv6 for NaviServer

[Gustaf N. <ne...@wu...>]

Great,

this means, we do not have to "emulate" it - as done in previous versions.
I've done some more cleanup for MSVC and activated IPV6 per default
(everything untested)

-g
[1] 
https://bitbucket.org/naviserver/naviserver/commits/432a251c4e8ee567327b1228cee8e25a726af5bc


Am 25.02.16 um 18:04 schrieb Maurizio Martignano:
>
> Dear Gustaf,
>
> Thank you for your inputs.
>
> I implemented all your changes but the system was still not working.
>
> I had a careful look at it and eventually I found the cause (at least 
> in my configuration):
>
> The problem in caused by the function inet_ntop in nswin32.c. After 
> removing it, everything works fine.
>
> The system seems now OK to me.
>
> Thank you,
>
> Maurizio
>
> *From:*Gustaf Neumann [mailto:ne...@wu...]
> *Sent:* 25 February 2016 12:33
> *To:* nav...@li...
> *Subject:* Re: [naviserver-devel] IPv6 for NaviServer
>
> Hi Maurizio,
>
> thanks for the fixes. as such, they could not work, but they point
> out things, where one has to watch out. I have fixed these
> things in a way it could work under windows.
>
> By looking at the differences, it seems as if you compiled just a
> IPv4 version (no IPv6 enabled). i would have expected this
> to go more smoothly. In case you tried the sample
> nsd-config.tcl script, that could not work with IPv4, since
> IPv6 addresses were hard-coded.
>
> all the best
> -g
>
> Am 24.02.16 um 18:00 schrieb Maurizio Martignano:
>
>     Dear Gustaf,
>
>        Thank you as always for all your work.
>
>     About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
>
>     Visual Studio 2015 with 64 bit as target.
>
>     With this configuration I tried to modify the file sockaddr.c (to make it
>
>     compile), but I'm not sure about the changes I made.
>
>     I also had to modify the file nswin32.c as I could not find anywhere the
>
>     function "NsSockGetPort".
>
>     Well the resulting system compiles... but nothing works.
>
>     I need much more time to do a proper testing/tuning and I can't do that at
>
>     the moment.
>
>     As soon as I discover more, I'll let you know.
>
>     Back to your delivery of the 16th of February.
>
>     Once again thank you,
>
>     Maurizio
>
>     -----Original Message-----
>
>     From: Gustaf Neumann [mailto:ne...@wu...]
>
>     Sent: 24 February 2016 14:01
>
>     To: Navidevel<nav...@li...>
>     <mailto:nav...@li...>
>
>     Subject: [naviserver-devel] IPv6 for NaviServer
>
>     Dear all,
>
>     over the last week, i've worked on IPv6 support for NaviServer.
>
>     For this task, i took the following path:
>
>     1) build a version of NaviServer for IPv6
>
>     2) build a single code source for compiling either for IPv4 or IPv6
>
>     3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
>
>     4) Run regression test suite under IPv4 and/IPv6
>
>     5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
>
>          (requires some interface changes to handle generic sockaddrs)
>
>     Everything seems to work fine, one can work with the same config files as
>
>     before, and everything works as expected, but one can as well listen on IPv6
>
>     addresses (e.g.
>
>     the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
>
>     kernel allows to deliver as well
>
>     IPv4 traffic to the IPv6 interface, then one can talk also multiple
>
>     protocols. The mileage may vary depending on the OS and kernel.
>
>     It is as well possible to deactivate IPv6 support at compile time via
>
>     --disable-ipv6.
>
>     Here is a summary of the necessary changes:
>
>     - Added a amall Abstraction API for IP-version agnostic calls:
>
>         Ns_LogSockaddr(), Ns_SockaddrGetPort(),
>
>         Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
>
>         Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
>
>     - Replaced in interface more places where "sockaddr_in" (the IPv4
>
>         version of the sockaddr) with the generic (version agnostic)
>
>         sockaddr (also in IPv4 code to obtain API consistency when no IPv6
>
>         is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
>
>         Ns_SockBind(), Ns_SockBindUdp()
>
>     - The Ns_Sock structure contained sockaddr_in, which is not sufficient
>
>         to store IPv6 addresses. Therefore it was necessary to change
>
>         that to "sockaddr_storage" which guarantees to be large enough
>
>         to keep all kind of addresses. This affects as well driver modules
>
>         which communicate via the NaviServer managed sockets. Old drivers
>
>         might crash with the new structure. Therefore, I've bumped
>
>         NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
>
>         can communicate that they support IPv6. In IPv6 mode, NaviServer
>
>         refuses to work with versions <3.
>
>     - Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
>
>         (except for legacy an minimal library support)
>
>     - Added log file warnings on various places, where NaviServer
>
>         was ignoring error states in socket communications silently.
>
>     - Transformed hash table for open listen ports to string keys
>
>         (the old code used 32bit IPv4-addresses as keys in hash tables)
>
>     - added test for ns_listencallback
>
>     - Ns_HttpParseHost: new abstraction for parsing host and port number.
>
>         Reason: IPv6 literal notation contains many colons, parsing
>
>         IP-literal notation according to RFC 3986 section 3.2.2 is necessary
>
>         (removed places where code was searching for ':' to expect the port)
>
>     - Added [ns_info ipv6] to obtain information whether the binary
>
>         supports IPv6 or not
>
>     - Added configure flag --disable-ipv6 to build NaviServer just with
>
>         IPv4 support.
>
>     - provide defines for core differences between IPv4 and IPv6 #ifdef
>
>     HAVE_IPV6
>
>     # define NS_IP_LOOPBACK     "::1"
>
>     # define NS_IP_UNSPECIFIED  "::"
>
>     # define NS_SOCKADDR_IN     sockaddr_storage
>
>     # define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
>
>     #else
>
>     # define NS_IP_LOOPBACK     "127.0.0.1"
>
>     # define NS_IP_UNSPECIFIED  "0.0.0.0"
>
>     # define NS_SOCKADDR_IN     sockaddr_in
>
>     # define NS_IPADDR_SIZE     INET_ADDRSTRLEN
>
>     #endif
>
>        - nsperm:
>
>         * Aligned implementation with the documentation
>
>           (hosts.allow/deny have comma-separated entries)
>
>         * Added submask specification in the form /xxx
>
>           (e.g. ::1/64)
>
>         * Use binary IPv4 and IPv6 values for hash and mask lookup
>
>         * Generalizing masking functions
>
>         * Refactor code to avoid memory leaks
>
>         * Updated documentation
>
>     - modules:
>
>         * nsudp and nsssl are now IPv6 aware/compliant
>
>     TODOs:
>
>     - more platform testing
>
>         (probably changes for Windows necessary, help is appreciated)
>
>     - remove commented out debug statements
>
>     - make more modules IPv6 compliant
>
>     - documentation updates
>
>     I've tested the generic and IPv4 only version via regression test and random
>
>     browsing on Mac OS X and Linux. If you have other platforms, testing would
>
>     be appreciated. There are some adjustments for windows probably necessary
>
>     (in the absence of "configure", one has to add manually "HAVE_IPV4" to the
>
>     compile flags). Any help is welcome.
>
>     best regards
>
>     -gustaf neumann
>
>     ----------------------------------------------------------------------------
>
>     --
>
>     Site24x7 APM Insight: Get Deep Visibility into Application Performance APM +
>
>     Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor
>
>     end-to-end web transactions and take corrective actions now Troubleshoot
>
>     faster and improve end-user experience. Signup Now!
>
>     http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>
>     _______________________________________________
>
>     naviserver-devel mailing list
>
>     nav...@li...
>     <mailto:nav...@li...>
>
>     https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
>
>
>     ------------------------------------------------------------------------------
>
>     Site24x7 APM Insight: Get Deep Visibility into Application Performance
>
>     APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>
>     Monitor end-to-end web transactions and take corrective actions now
>
>     Troubleshoot faster and improve end-user experience. Signup Now!
>
>     http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>
>
>
>
>     _______________________________________________
>
>     naviserver-devel mailing list
>
>     nav...@li...
>     <mailto:nav...@li...>
>
>     https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
>
>
> -- 
> Univ.Prof. Dr. Gustaf Neumann
> WU Vienna
> Institute of Information Systems and New Media
> Welthandelsplatz 1, A-1020 Vienna, Austria
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] IPv6 for NaviServer

[Maurizio M. <Mau...@sp...>]

Dear all,

               To complement my previous message, I have observed that the
system compiled with HAVE_IPV4 works.

On the contrary, if compiled with HAVE_IPV6, still presents some few
problems.

So the Windows-OpenACS distribution will have at the moment Naviserver
compiled with HAVE_IPV4.

In the meantime and with no rush I will try to investigate the problems of
the HAVE_IPV6 version.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:Mau...@sp...] 
Sent: 25 February 2016 22:20
To: nav...@li...
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear all,

               This is a comment to my previous message.

 

Unfortunately, the compiled system is still not working properly. I need to
investigate more in details what the problems are.

 

In the mean time I put back in the Windows-OpenACS, the Naviserver commit of
the 24th of February, the one without IPV6 Support.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:Mau...@sp...] 
Sent: 25 February 2016 18:05
To: nav...@li...
<mailto:nav...@li...> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear Gustaf,

               Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 25 February 2016 12:33
To: nav...@li...
<mailto:nav...@li...> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-----Original Message-----
From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:nav...@li...>
<nav...@li...>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
    (requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various places, where NaviServer
   was ignoring error states in socket communications silently.
 
- Transformed hash table for open listen ports to string keys
   (the old code used 32bit IPv4-addresses as keys in hash tables)
- added test for ns_listencallback
 
- Ns_HttpParseHost: new abstraction for parsing host and port number.
   Reason: IPv6 literal notation contains many colons, parsing
   IP-literal notation according to RFC 3986 section 3.2.2 is necessary
   (removed places where code was searching for ':' to expect the port)
 
- Added [ns_info ipv6] to obtain information whether the binary
   supports IPv6 or not
 
- Added configure flag --disable-ipv6 to build NaviServer just with
   IPv4 support.
 
- provide defines for core differences between IPv4 and IPv6 #ifdef
HAVE_IPV6
# define NS_IP_LOOPBACK     "::1"
# define NS_IP_UNSPECIFIED  "::"
# define NS_SOCKADDR_IN     sockaddr_storage
# define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
#else
# define NS_IP_LOOPBACK     "127.0.0.1"
# define NS_IP_UNSPECIFIED  "0.0.0.0"
# define NS_SOCKADDR_IN     sockaddr_in
# define NS_IPADDR_SIZE     INET_ADDRSTRLEN
#endif
 
  - nsperm:
   * Aligned implementation with the documentation
     (hosts.allow/deny have comma-separated entries)
   * Added submask specification in the form /xxx
     (e.g. ::1/64)
   * Use binary IPv4 and IPv6 values for hash and mask lookup
   * Generalizing masking functions
   * Refactor code to avoid memory leaks
   * Updated documentation
 
- modules:
   * nsudp and nsssl are now IPv6 aware/compliant
 
TODOs:
- more platform testing
   (probably changes for Windows necessary, help is appreciated)
- remove commented out debug statements
 
- make more modules IPv6 compliant
- documentation updates
 
I've tested the generic and IPv4 only version via regression test and random
browsing on Mac OS X and Linux. If you have other platforms, testing would
be appreciated. There are some adjustments for windows probably necessary
(in the absence of "configure", one has to add manually "HAVE_IPV4" to the
compile flags). Any help is welcome.
 
best regards
-gustaf neumann
 
 
----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM +
Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor
end-to-end web transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140
_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

 

----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140

 

_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

 

-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] IPv6 for NaviServer

[Maurizio M. <Mau...@sp...>]

Dear all,

               This is a comment to my previous message.

 

Unfortunately, the compiled system is still not working properly. I need to
investigate more in details what the problems are.

 

In the mean time I put back in the Windows-OpenACS, the Naviserver commit of
the 24th of February, the one without IPV6 Support.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:Mau...@sp...] 
Sent: 25 February 2016 18:05
To: nav...@li...
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear Gustaf,

               Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 25 February 2016 12:33
To: nav...@li...
<mailto:nav...@li...> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-----Original Message-----
From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:nav...@li...>
<nav...@li...>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
    (requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various places, where NaviServer
   was ignoring error states in socket communications silently.
 
- Transformed hash table for open listen ports to string keys
   (the old code used 32bit IPv4-addresses as keys in hash tables)
- added test for ns_listencallback
 
- Ns_HttpParseHost: new abstraction for parsing host and port number.
   Reason: IPv6 literal notation contains many colons, parsing
   IP-literal notation according to RFC 3986 section 3.2.2 is necessary
   (removed places where code was searching for ':' to expect the port)
 
- Added [ns_info ipv6] to obtain information whether the binary
   supports IPv6 or not
 
- Added configure flag --disable-ipv6 to build NaviServer just with
   IPv4 support.
 
- provide defines for core differences between IPv4 and IPv6 #ifdef
HAVE_IPV6
# define NS_IP_LOOPBACK     "::1"
# define NS_IP_UNSPECIFIED  "::"
# define NS_SOCKADDR_IN     sockaddr_storage
# define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
#else
# define NS_IP_LOOPBACK     "127.0.0.1"
# define NS_IP_UNSPECIFIED  "0.0.0.0"
# define NS_SOCKADDR_IN     sockaddr_in
# define NS_IPADDR_SIZE     INET_ADDRSTRLEN
#endif
 
  - nsperm:
   * Aligned implementation with the documentation
     (hosts.allow/deny have comma-separated entries)
   * Added submask specification in the form /xxx
     (e.g. ::1/64)
   * Use binary IPv4 and IPv6 values for hash and mask lookup
   * Generalizing masking functions
   * Refactor code to avoid memory leaks
   * Updated documentation
 
- modules:
   * nsudp and nsssl are now IPv6 aware/compliant
 
TODOs:
- more platform testing
   (probably changes for Windows necessary, help is appreciated)
- remove commented out debug statements
 
- make more modules IPv6 compliant
- documentation updates
 
I've tested the generic and IPv4 only version via regression test and random
browsing on Mac OS X and Linux. If you have other platforms, testing would
be appreciated. There are some adjustments for windows probably necessary
(in the absence of "configure", one has to add manually "HAVE_IPV4" to the
compile flags). Any help is welcome.
 
best regards
-gustaf neumann
 
 
----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM +
Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor
end-to-end web transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140
_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel





----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140





_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel





-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] IPv6 for NaviServer

[Maurizio M. <Mau...@sp...>]

Dear Gustaf,

               Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 25 February 2016 12:33
To: nav...@li...
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-----Original Message-----
From: Gustaf Neumann [mailto:ne...@wu...] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:nav...@li...>
<nav...@li...>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
    (requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various places, where NaviServer
   was ignoring error states in socket communications silently.
 
- Transformed hash table for open listen ports to string keys
   (the old code used 32bit IPv4-addresses as keys in hash tables)
- added test for ns_listencallback
 
- Ns_HttpParseHost: new abstraction for parsing host and port number.
   Reason: IPv6 literal notation contains many colons, parsing
   IP-literal notation according to RFC 3986 section 3.2.2 is necessary
   (removed places where code was searching for ':' to expect the port)
 
- Added [ns_info ipv6] to obtain information whether the binary
   supports IPv6 or not
 
- Added configure flag --disable-ipv6 to build NaviServer just with
   IPv4 support.
 
- provide defines for core differences between IPv4 and IPv6 #ifdef
HAVE_IPV6
# define NS_IP_LOOPBACK     "::1"
# define NS_IP_UNSPECIFIED  "::"
# define NS_SOCKADDR_IN     sockaddr_storage
# define NS_IPADDR_SIZE     INET6_ADDRSTRLEN
#else
# define NS_IP_LOOPBACK     "127.0.0.1"
# define NS_IP_UNSPECIFIED  "0.0.0.0"
# define NS_SOCKADDR_IN     sockaddr_in
# define NS_IPADDR_SIZE     INET_ADDRSTRLEN
#endif
 
  - nsperm:
   * Aligned implementation with the documentation
     (hosts.allow/deny have comma-separated entries)
   * Added submask specification in the form /xxx
     (e.g. ::1/64)
   * Use binary IPv4 and IPv6 values for hash and mask lookup
   * Generalizing masking functions
   * Refactor code to avoid memory leaks
   * Updated documentation
 
- modules:
   * nsudp and nsssl are now IPv6 aware/compliant
 
TODOs:
- more platform testing
   (probably changes for Windows necessary, help is appreciated)
- remove commented out debug statements
 
- make more modules IPv6 compliant
- documentation updates
 
I've tested the generic and IPv4 only version via regression test and random
browsing on Mac OS X and Linux. If you have other platforms, testing would
be appreciated. There are some adjustments for windows probably necessary
(in the absence of "configure", one has to add manually "HAVE_IPV4" to the
compile flags). Any help is welcome.
 
best regards
-gustaf neumann
 
 
----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM +
Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor
end-to-end web transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140
_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel






----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151
<http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140> &iu=/4140






_______________________________________________
naviserver-devel mailing list
nav...@li...
<mailto:nav...@li...> 
https://lists.sourceforge.net/lists/listinfo/naviserver-devel






-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria