naviserver-devel — Developer discussion

Re: [naviserver-devel] unknown type name off64_t

[Andrew P. <at...@pi...>]

On Mon, Sep 01, 2014 at 09:36:31AM +0200, Gustaf Neumann wrote:

> you have probably configured naviserver using --with-lib=....

No, I configured Naviserver like so (which I also just ran again now):

  $ export inst_dir=/usr/local/nsd-4.99.6-dtk-1 
  $ AUTOCONF=autoconf2.50 ./autogen.sh --enable-symbols --with-tcl=/usr/lib/tcl8.5 --prefix=$inst_dir 

> this should be fixed in bitbucket. please update, reconfigure and reinstall.

With all the latest sources, I still get the same error when building
nsoracle, no change.  I then redid my configure using "--with-zlib=/usr/lib",
but that seemed to make no difference, same error when using the stock ns.h.

In all cases, adding this line to my installed include/ns.h fixes my nsoracle
build:

  #undef HAVE_ZLIB_H 

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Gustaf N. <ne...@wu...>]

Am 31.08.14 21:51, schrieb Andrew Piskorski:
> I'm still learning how to actually use Mercurial.  E.g., I originally
> cloned my local repository and working copy (on my Linux box) from the
> official Naviserver repository above.  My repo remembers that, and so
> defaults to trying to push changes back to there, but of course that
> fails because I don't have write access.  So instead I explicitly
> pushed to my own newly-forked https://bitbucket.org/apiskors/nsoracle
> repo, which worked fine.
The steps with cloning + pull request are perfectly fine. I've 
incorporated your
changes to the repository, and gave you write access to naviserver/nsoracle
(which i can't test beyond compilation).
I've added some more changes to nsoracle (removing unused variables,
get rid of two potential NULL pointer exceptions, removed the necessity
to specify INST for NaviServer.

all the best
-gustaf neumann

Re: [naviserver-devel] unknown type name off64_t

[Gustaf N. <ne...@wu...>]

Am 31.08.14 21:06, schrieb Andrew Piskorski:
> The error (below) is coming from zlib.h, but oddly, I already
> successfully built both the Naviserver core and the nszlib module.
> Does this suggest some sort of double include or other build setup
> problem?
>
you have probably configured naviserver using --with-lib=.... There was 
a bug
in the Makefile infrastructure not passing the includeflag for zlib 
around. It is
needed for the datastructure of Ns_Compress*() infrastructure (one can
get rid of the dependency, by using opaque structures, but i have left this
for the time being, keeping the interface stable).

This should be fixed in bitbucket. Please update, reconfigure and reinstall.

-gustaf neumann

Re: [naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Andrew P. <at...@pi...>]

On Sat, Aug 30, 2014 at 10:04:05AM +0200, Gustaf Neumann wrote:

> Here you see the commits:
> 
>     https://bitbucket.org/naviserver/nsoracle/commits/all

Ok, I made some changes to the nsoracle/Makefile, pushed my changes to
my own fork on bitbucket.org, and created a pull request.  You can see
that stuff here:

  https://bitbucket.org/apiskors/nsoracle/commits/12d73fdae3ac0e0703ff237f77003b441d3ddfd9
  https://bitbucket.org/naviserver/nsoracle/pull-requests

I'm still learning how to actually use Mercurial.  E.g., I originally
cloned my local repository and working copy (on my Linux box) from the
official Naviserver repository above.  My repo remembers that, and so
defaults to trying to push changes back to there, but of course that
fails because I don't have write access.  So instead I explicitly
pushed to my own newly-forked https://bitbucket.org/apiskors/nsoracle
repo, which worked fine.

But should I change my local repo somehow to instead point to my own
fork on Bitbucket?  I'm not sure what the best workflow is with these
tools yet...  Any major recommendations?

One minor annoyance is that "hg log" does not show my entire commit
message.  No big deal, I just have to use "hg -v log" instead, but
that did confuse me for a couple minutes.

-- 
Andrew Piskorski <at...@pi...>

[naviserver-devel] unknown type name off64_t

[Andrew P. <at...@pi...>]

Using Gustaf's latest version of nsoracle plus some local Makefile
changes to add back 10g compatibility, I'm getting a compile error,
but it seems to have nothing to do with nsoracle per se!

The error (below) is coming from zlib.h, but oddly, I already
successfully built both the Naviserver core and the nszlib module.
Does this suggest some sort of double include or other build setup
problem?

If I manually undefine HAVE_ZLIB_H in ns.h, then zlib.h does NOT get
included, and building nsoracle.so works!  But what's the right way
for me to fix the problem?


$ hg summ 
parent: 59:7a129b2649ec tip 
 - get rid of signed-problems (following prototypes from the oracle oci interface) 

gcc -I/usr/local/oracle-client/product/10.2.0/sdk/include -I/usr/local/oracle-client/product/10.2.0/rdbms/demo -I/usr/local/oracle-client/product/10.2.0/rdbms/public -I/usr/local/oracle-client/product/10.2.0/network/public -I/usr/local/oracle-client/product/10.2.0/plsql/public -g -O2 -fomit-frame-pointer -Wall -fPIC -g -O2 -fno-unit-at-a-time -pipe -D_FORTIFY_SOURCE=2 -I/usr/local/nsd-4.99.6-dtk-1/include -I"/usr/local/include" -DHAVE_CONFIG_H   -c -o nsoracle.o nsoracle.c 
In file included from /usr/local/nsd-4.99.6-dtk-1/include/ns.h:44:0, 
                 from nsoracle.h:26, 
                 from nsoracle.c:16: 
/usr/include/zlib.h:1448:4: error: unknown type name 'off64_t' 
/usr/include/zlib.h:1448:39: error: unknown type name 'off64_t' 
/usr/include/zlib.h:1449:4: error: unknown type name 'off64_t' 
/usr/include/zlib.h:1450:44: error: unknown type name 'off64_t' 
/usr/include/zlib.h:1451:42: error: unknown type name 'off64_t' 
make: *** [nsoracle.o] Error 1 

$ grep DESCRIP /etc/lsb-release  
DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS" 

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Gustaf N. <ne...@wu...>]

Am 29.08.14 18:45, schrieb Andrew Piskorski:
> On Wed, Aug 27, 2014 at 10:14:26PM +0200, Gustaf Neumann wrote:
>
>> in order to give my guesswork more substance, i took the oracle driver from
>> https://github.com/aolserver/nsoracle
> Is that the same as the current CVS head at SourceForge?
the version in github has the mentioned fixes for jcollins and asteets
(the last one from aug25) plus three changes from jwolfe (newest
version from may 2011, memleak + threading fixes). So, the github version
seems newer.
> How did you folks import the AOLserver CVS history into Mercurial?
This is actually quite simple. On you local machine:

    hg clone git://github.com/aolserver/nsoracle.git
    cd nsoracle
    hg push https://YO...@bi.../PATHTOREPOSITORY


You can see the result of the import at

    https://bitbucket.org/naviserver/nsoracle/

Here you see the commits:

    https://bitbucket.org/naviserver/nsoracle/commits/all

The repository contains the patch of my last mail,
and - for the time being - a short NaviServer hint
to the README file. This is probably easier to
handle. If you have some patches, i can give
you the necessary permissions to this repository
if convenient.

all the best
-gustaf

Re: [naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Andrew P. <at...@pi...>]

On Wed, Aug 27, 2014 at 10:14:26PM +0200, Gustaf Neumann wrote:

> in order to give my guesswork more substance, i took the oracle driver from
> https://github.com/aolserver/nsoracle

Is that the same as the current CVS head at SourceForge?

How did you folks import the AOLserver CVS history into Mercurial?
I'd heard that cvs-fast-export is supposed to be good but haven't
tried it yet:

  http://www.catb.org/~esr/cvs-fast-export/
  https://gitorious.org/cvs-fast-export/

Is a backup of the full SourceForge CVS repository available somewhere?
Or does that not matter, and export tools do just as good a job talking
remotely to SourceForge CVS?

If no one else gets to it first, I will learn how and put nsoracle
into Mercurial on Bitbucket.  (No promises on when I get to that
though, it might be a while.)

> However, this version of the nsoracle is NOT in a good state.

Hm.  Thanks for that patch, I will try using it soon.

Checking the CVS log, the last people to touch the nsoracle codebase
were "asteets" in 2010 and "jcollins" in 2006.  (Anybody know who
those folks are?)  It looks like jcollins made the vast majority of
changes from 2004 - 2006.

> Andrew, what version of nsoracle are you using?

I have a note from 2006-03-30 saying:

  nsoracle 2.2 (-r v2_7) is the stable version, and works fine for
  Oracle 8i and 9i.  However, it does not build properly against the 10g
  client libraries.  The nsoracle Head, which also inclues MANY other
  changes, builds fine with 10g.

So prior to 2006, I was using the truly ancient "v2_7" tagged version
of nsoracle.  I suspect that was the last version of the Oracle driver
released by ArsDigita and the first imported into SourceForge CVS as
"nsoracle".

Then sometime in 2006-03 I switched to using the then-current nsoracle
head.  I have two builds still sitting around, from 2006-06-18 and
2007-01-04, and am still (!) running that 2007-01-04 build today, with
an equally old AOLserver 4.0.10.x from 2006-03-28.  I'm using that for
an older and customized OpenACS installation, so I can't guarantee
that we're exercising all the Oracle and nsoracle features other
OpenACS sites are using, but it seems to have been working fine for us
since then.

I believe I can replicate the same sources used in that build with a
date tag, e.g. "cvs update -dP -D 2007-01-04".  Comparing the
2007-01-04 code to the current CVS head, I see only one file that has
changed, nsoracle.c, which was on rev 1.25 in 2007 and 1.28 now:


RCS file: /cvsroot/aolserver/nsoracle/nsoracle.c,v 
---------------------------- 
revision 1.28 
date: 2010/08/25 20:38:48;  author: asteets;  state: Exp;  lines: +2 -2 
Don't pass uninitialized pointers into OCI.  Fixes potential crash bug. 
---------------------------- 
revision 1.27 
date: 2008/07/14 22:01:09;  author: asteets;  state: Exp;  lines: +2 -2 
Fix typo in error message. 
---------------------------- 
revision 1.26 
date: 2007/04/19 19:33:33;  author: asteets;  state: Exp;  lines: +13 -7 
Prevent ORA-01405 when stored function returns NULL.  Fixes memory leak. 
---------------------------- 
revision 1.25 
date: 2006/02/22 16:14:58;  author: jcollins;  state: Exp;  lines: +3 -2 
Fix for bug 1435192.  The code to handle fetching LONG data 
types piecewise wasn't correctly using the null indicator 
variables. 
---------------------------- 

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Gustaf N. <ne...@wu...>]

Dear Andrew,

you are right, nssha1 and nscache are built-in, no extra module is 
needed for these.
Currently, there is no separate oracle driver, but up to my knowlege, 
the aolserver driver
for oracle should work.

in order to give my guesswork more substance, i took the oracle driver from
https://github.com/aolserver/nsoracle

and modified it slightly to compile it for naviserver. With the 
modifications below,
it compiles and links fine with naviserver (tested on mac os x 10.9.4 with
Oracle client libs Version 11.2.0.4.0 (instant client package and 
instant client package
SDK, 64-bit).

However, this version of the nsoracle is NOT in a good state. Most of 
the changes
were necessary to compile the driver at all, the libraries referenced 
form the original
Makefile do not exist anymore, the compilation ends with 51 warnings 
(missing
qualifiers, converisons of pointers to integer, signedness problems, etc.).

Andrew, what version of nsoracle are you using?

-gustaf

================================
export ORACLE_HOME=/usr/local/src/oracle/instantclient_11_2/

# includes: under $ORACLE_HOME/sdk/include/
# libaries: $ORACLE_HOME/lib/

make CC=clang INST=/usr/local/ns

================================

diff --git a/Makefile b/Makefile
index b3d11d8..dab6b3e 100644
--- a/Makefile
+++ b/Makefile
@@ -48,11 +48,8 @@ OCI_VERSION=$(shell strings $(ORACLE_HOME)/lib/libclntsh.so | grep "^Version.[0-
  OCI_MAJOR_VERSION=$(shell echo $(OCI_VERSION) | cut -d ' ' -f2 | cut -d '.' -f1)
  NS_VERSION=$(shell grep NS_VERSION $(NSHOME)/include/ns.h)
  
-MODLIBS  =  -L$(ORACLE_HOME)/lib -lclntsh \
-           -lcore$(OCI_MAJOR_VERSION) \
-           -lcommon$(OCI_MAJOR_VERSION) \
-           -lgeneric$(OCI_MAJOR_VERSION) \
-           -lclient$(OCI_MAJOR_VERSION)
+MODLIBS  +=  -L$(ORACLE_HOME)/lib -lclntsh -locci -lnsthread -lnsd -lnsdb -ltcl8.5
+
  
  ifneq (,$(findstring NS_VERSION,$(NS_VERSION)))
  MODLIBS  +=  -lnsdb
@@ -69,10 +66,8 @@ include $(NSHOME)/include/Makefile.global
  
  # Tack on the oracle includes after Makefile.global stomps CFLAGS
  CFLAGS := \
-    -I$(ORACLE_HOME)/rdbms/demo \
-    -I$(ORACLE_HOME)/rdbms/public \
-    -I$(ORACLE_HOME)/network/public \
-    -I$(ORACLE_HOME)/plsql/public $(filter-out -Wconversion,$(CFLAGS))
+    -I$(ORACLE_HOME)/sdk/include \
+    $(filter-out -Wconversion,$(CFLAGS))
  
  all: $(MOD) $(MODCASS)
  
diff --git a/nsoracle.c b/nsoracle.c
index fd4b4af..94af40e 100644
--- a/nsoracle.c
+++ b/nsoracle.c
@@ -37,7 +37,7 @@ dvoid *Ns_OracleRealloc(dvoid *cxt, dvoid *buf, size_t size) {
  
  /*{{{ Ns_OracleFree */
  void Ns_OracleFree(dvoid *cxt, dvoid *buf) {
-    return Ns_Free(buf);
+    Ns_Free(buf);
  }
  /*}}}*/
  
@@ -5651,7 +5651,7 @@ ora_get_column_index(Tcl_Interp * interp, Ns_DbTableInfo * tinfo,
  /* re-implement the ns_column command */
  int
  ora_column_command(ClientData dummy, Tcl_Interp * interp,
-        int argc, char *argv[])
+        int argc, CONST84 char *argv[])
  {
      int result = TCL_ERROR;
      Ns_DbHandle *handle;
@@ -5776,7 +5776,7 @@ ora_column_command(ClientData dummy, Tcl_Interp * interp,
  /* re-implement the ns_table command */
  int
  ora_table_command(ClientData dummy, Tcl_Interp * interp,
-        int argc, char *argv[])
+        int argc, CONST84 char *argv[])
  {
      int result = TCL_ERROR;
      Ns_DString tables_string;
diff --git a/nsoracle.h b/nsoracle.h
index 1afe644..08e46c6 100644
--- a/nsoracle.h
+++ b/nsoracle.h
@@ -25,6 +25,14 @@
  #include <sys/stat.h>
  #include <ns.h>
  
+#if defined(NS_VERSION)
+# define NS_AOLSERVER_3_PLUS 1
+# define Ns_Free ns_free
+# define Ns_Malloc ns_malloc
+# define Ns_Realloc ns_realloc
+# define Ns_StrDup ns_strdup
+#endif
+
  #ifndef NS_DML
  #include <nsdb.h>
  #endif


Am 27.08.14 15:53, schrieb Andrew Piskorski:
> In my ancient AOLserver installations, I've been using the add-on
> modules nsoracle, nssha1, and nscache, none of which are in the main
> Naviserver Mercurial repository here:
>
>    https://bitbucket.org/naviserver/
>
> Can anyone tell me the status of those modules with Naviserver?
>
> I suspect the nssha1 and nscache functionality is now built into the
> core Naviserver, but what about the Oracle db driver?  Is its latest
> version still the code in the AOLserver SourceForge CVS, or is there a
> newer/better version around somewhere?
>

[naviserver-devel] status of nsoracle, nssha1, nscache modules?

[Andrew P. <at...@pi...>]

In my ancient AOLserver installations, I've been using the add-on
modules nsoracle, nssha1, and nscache, none of which are in the main
Naviserver Mercurial repository here:

  https://bitbucket.org/naviserver/

Can anyone tell me the status of those modules with Naviserver?

I suspect the nssha1 and nscache functionality is now built into the
core Naviserver, but what about the Oracle db driver?  Is its latest
version still the code in the AOLserver SourceForge CVS, or is there a
newer/better version around somewhere?

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Ibrahim T. <it...@ar...>]

Hi Andrew,

Here are my sources along with all the VC10 projects and 
makefiles for a 64-bit build of Naviserver 4.99.2 with TCL 8.4.18.

http://s3.amazonaws.com/ARCHIWARE/mynavi.tar.gz

Unfortunately, I don't have the time to fidget with the makefiles 
and VC projects to make them universally usable.

As a starting point use the VC10 mynavi/win32/navi.sln for 
building the server.

Everything is unedited the we use it for our own purposes. You'll 
probably run into some post-build batch script errors. Simply 
remove the post-build tasks from the projects.

Hope this helps.

Cheers,
Ibrahim

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Maurizio M. <Mau...@sp...>]

Going 32 bit nowadays is a no go. Full stop.

Wish you luck.

Maurizio


-----Original Message-----
From: Andrew Piskorski [mailto:at...@pi...] 
Sent: 25 August 2014 19:46
To: nav...@li...
Cc: Maurizio Martignano
Subject: Re: [naviserver-devel] Windows 7 recommended compiler and build
tools?

On Sun, Aug 10, 2014 at 05:18:57PM +0200, Maurizio Martignano wrote:

> As build environment I would select your option #2, that is " 
> Microsoft's compiler, using nmake from the command-line."

So, just how are you folks actually making that work?

I imagine the build process should be something roughly along these lines:

  "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Debug /x64
/win7
  ./configure --enable-symbols --with-tcl=C:/P/Tcl85/lib
--prefix=C:/web/nsd499-dtk
  nmake -f Makefile all 

But Naviserver's "configure" is of course a Bourne shell script and cannot
run from the Windows command prompt.  So what should I do instead?  Install
Msys just to run configure?  Or perhaps skip configure and just pass various
options to nmake?  Is the Makefile that ships with Naviserver even known to
work on Windows with nmake and the MS SDK compiler?

I do NOT have 64-bit zlib installed yet.  Hopefully I can build without it,
and just turn the zlib features in Naviserver?

Btw, I have ActiveTcl (which is threaded) installed in C:\P\Tcl85\, so I
figured I'd try linking against that first before compiling my own Tcl.

The stock Tcl win/README file says that they build with "Visual C++
6.0 and the TEA Makefile", but VC++ 6.0 is 32-bit only, so the ActiveState
folks must be doing something slightly different for their 64-bit builds.
Anybody know offhand what they really use or where their 64-bit Windows
build scripts are located?

--
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Andrew P. <at...@pi...>]

On Mon, Aug 25, 2014 at 01:45:58PM -0400, Andrew Piskorski wrote:

> The stock Tcl win/README file says that they build with "Visual C++
> 6.0 and the TEA Makefile", but VC++ 6.0 is 32-bit only, so the
> ActiveState folks must be doing something slightly different for
> their 64-bit builds.  Anybody know offhand what they really use or
> where their 64-bit Windows build scripts are located?

Well, that question at least seems simple:  The Tcl win/makefile.vc has
lots of comments about turning on 64-bit support, etc.  So presumably
64-bit ActiveTcl uses the stock Tcl build scripts on Windows.

So I guess my fallback option is to use Tcl's makefile.vc as a guide
in setting up a Naviserver Windows build environment from scratch.
But do I actually need to do that?  Maybe Naviserver already has some
better support for Windows command-line builds and I just don't know
how to use it correctly.

If I do need to add something, of course I'm happy to contribute it
back, so how would you folks like to see this stuff work?  In general,
what's the right way to do it?  Is it actually feasible to have one
set of makefiles support both Unix and Windows, or is the existence of
Tcl's makefile.vc a big hint that that doesn't really work?

Btw, the Fossil version history of Tcl's win/makefile.vc is here:

  http://core.tcl.tk/tcl/artifact/dafbf950b8cca6b71a1cc3b8fef858f32a6aeacc
  http://core.tcl.tk/tcl/finfo?name=win/makefile.vc

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Andrew P. <at...@pi...>]

On Sun, Aug 10, 2014 at 05:18:57PM +0200, Maurizio Martignano wrote:

> As build environment I would select your option #2, that is " Microsoft's
> compiler, using nmake from the command-line."

So, just how are you folks actually making that work?

I imagine the build process should be something roughly along these lines:

  "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Debug /x64 /win7 
  ./configure --enable-symbols --with-tcl=C:/P/Tcl85/lib --prefix=C:/web/nsd499-dtk 
  nmake -f Makefile all 

But Naviserver's "configure" is of course a Bourne shell script and
cannot run from the Windows command prompt.  So what should I do
instead?  Install Msys just to run configure?  Or perhaps skip
configure and just pass various options to nmake?  Is the Makefile
that ships with Naviserver even known to work on Windows with nmake
and the MS SDK compiler?

I do NOT have 64-bit zlib installed yet.  Hopefully I can build
without it, and just turn the zlib features in Naviserver?

Btw, I have ActiveTcl (which is threaded) installed in C:\P\Tcl85\, so
I figured I'd try linking against that first before compiling my own Tcl.

The stock Tcl win/README file says that they build with "Visual C++
6.0 and the TEA Makefile", but VC++ 6.0 is 32-bit only, so the
ActiveState folks must be doing something slightly different for their
64-bit builds.  Anybody know offhand what they really use or where
their 64-bit Windows build scripts are located?

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] Naviserver needs autoconf 2.69

[Gustaf N. <ne...@wu...>]

Dear Andrew,

autoconf is only needed for developers working directly on the source 
repository (mercurial).
The .tar release from bitbucket of 4.99.6 comes with a generated 
"configure" script and
has no such dependency.

Autoconf 2.69 is not really brand-new, it was released more than two 
years ago,
it is strange, that it has not made it yet to 12.04, but maybe the 
ubuntu people
don't want the change anything in the build system. I've on all my 
development
systems 2.69 installed. The AC_PREREQ is a safety measure, since everything
is tested with this release. Autoconf is a moving target, where 2.69 is 
in many
details more picky.

all the best
-gustaf

Am 19.08.14 00:07, schrieb Andrew Piskorski:
> I pulled a copy of the Naviserver 4.99.6 source release from BitBucket
> with Mercurial.  But running ./autogen.sh tells me:
>
>    configure.ac:36: error: Autoconf version 2.69 or higher is required
>
> I'm on Ubuntu 12.04.3 LTS, and its newest (normally) available
> autoconf appears to be version 2.68 in the autoconf2.50 package.
> So I installed the autoconf backport here, which gives me 2.69 and
> fixes things on Linux:
>
>    https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/autoconf
>
> But does Naviserver really need such a specific version of autoconf?
> Maybe it would be best to just check the output configure file into
> the version control system?
>


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

[naviserver-devel] Naviserver needs autoconf 2.69

[Andrew P. <at...@pi...>]

I pulled a copy of the Naviserver 4.99.6 source release from BitBucket
with Mercurial.  But running ./autogen.sh tells me:

  configure.ac:36: error: Autoconf version 2.69 or higher is required

I'm on Ubuntu 12.04.3 LTS, and its newest (normally) available
autoconf appears to be version 2.68 in the autoconf2.50 package.
So I installed the autoconf backport here, which gives me 2.69 and
fixes things on Linux:

  https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/autoconf

But does Naviserver really need such a specific version of autoconf?
Maybe it would be best to just check the output configure file into
the version control system?

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Maurizio M. <Mau...@sp...>]

Dear all,
	Using different C/C++ compilers for different modules of the same
binary/executable is always a bad idea and not only in Windows.

Different compilers can be used to create different binaries, not really
integrated with each other's. In the Windows-OpenACS distribution
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) the core
system (i.e. Aolserver and its modules, Tcl/Tk, xotcl, Thread,  are all
built with one single compiler (Microsoft Visual Studio 2013 Professional
Editions). All other tools, utilities are built with what is more
convenient, PostgreSQL 8.3.23 binaries with MinGW and PostgreSQL 9.3.1 with
MinGW64.

As build environment I would select your option #2, that is " Microsoft's
compiler, using nmake from the command-line."

In terms of target I would use at the moment Windows Server 2012 R2. Windows
7 is rather peculiar, but I do not know the actual user requirements.

Hope this helps,
Maurizio Martignano

-----Original Message-----
From: Andrew Piskorski [mailto:at...@pi...] 
Sent: 10 August 2014 12:46
To: nav...@li...
Subject: [naviserver-devel] Windows 7 recommended compiler and build tools?

I just started looking into running NaviServer 4.99.6 (the latest
release) on Windows 7 (64-bit), and am trying to figure out the proper
approach to building it (plus my own custom code) from source.

>From prior experience with ancient versions of Tcl and AOLserver, I
know of at least three build approaches on Microsoft Windows:

1. Microsoft Visual Studio, using (brittle, ugly) GUI-based project
   files to control the build.
2. Microsoft's compiler, using nmake from the command-line.
3. Msys + MinGW, using Gnu make.

Which of those do you recommend for NaviServer?  Also, do I need to use the
same compiler for everything, or is it feasible to say build NaviServer with
MinGW, but Tcl or my own custom code with Microsoft's
2013 compiler?

In NaviServer 4.99.6, the README gives a download link for msys_mingw8.zip,
which dates from c. 2004.  Is that old link just an oversight, or is there
some reason I should use such an ancient version of MinGW rather than a
current one?

In the win and win32 subdirectories, I see *.sln and *.dsw files for use
with Visual Studio 12.0.  Do those work better/worse than MinGW?
Also, is the command-line make-based build infrastructure set up to work
with Microsoft's compilers, or is it only for use with MinGW?

Does MinGW still only support 32-bit, and for 64-bit I need to instead use
the "MinGW-w64" fork?  Since MinGW-w64 handles both 32 and 64 bit, is there
any reason to prefer the original MinGW over it?

Is there any reason I should stick with 32-bit NaviServer, Tcl, etc. on
Windows 7, rather than moving to 64-bit?

Thanks in advance for your advice and tips!

Btw, Microsoft's current compilers are apparently available free of charge,
but just figuring out what you actually need to download and install is
non-trivial.  I found this advice from Jan. 2012 very
helpful:

  http://www.willus.com/ccomp.shtml?p11

And here's the Microsoft stuff I actually downloaded and installed:

- Windows SDK 7.1 for Windows 7 and .NET Framework 4:
  Rather than using the online installer (although it would work fine):
    http://www.microsoft.com/en-us/download/details.aspx?id=8279
  I downloaded the GRMSDKX_EN_DVD.iso x86-64:
    http://www.microsoft.com/download/en/details.aspx?id=8442
 
http://download.microsoft.com/download/F/1/0/F10113F5-B750-4969-A255-274341A
C6BCE/GRMSDKX_EN_DVD.iso
  As far as I could tell that's still the newest version of the SDK.
  From the Release Notes:
    http://go.microsoft.com/fwlink/?LinkID=186826
  it's clear that it includes both the Visual C++ 2010 compilers for both 32
and 64 bit
  ("the same compilers that ship with Visual Studio 2010"),
  the MSBuild 4.0 command line build environment, etc.

- The SDK does not ship with the necessary .NET Framework:
    http://msdn.microsoft.com/en-us/vstudio/aa496123
  So I used the web installer to install .Net Framework 4.5.1 from
2013-10-12:
    http://www.microsoft.com/en-us/download/details.aspx?id=40773
  NOT the older .Net Framework 4 from 2011-02-21:
    http://www.microsoft.com/download/en/details.aspx?id=17851

- VS Express 2013 for Windows Desktop:
  To get the GUI, I also installed the Express for Desktop version of
  Visual Studio, via the VS2013_RTM_DskExp_ENU.iso file:
 
http://www.visualstudio.com/downloads/download-visual-studio-vs#d-express-wi
ndows-desktop
    http://www.visualstudio.com/en-US/products/visual-studio-express-vs

--
Andrew Piskorski <at...@pi...>

----------------------------------------------------------------------------
--
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

[Andrew P. <at...@pi...>]

On Sun, Aug 10, 2014 at 06:45:43AM -0400, Andrew Piskorski wrote:

> Also, do I need to use the same compiler for everything, or is it
> feasible to say build NaviServer with MinGW, but Tcl or my own
> custom code with Microsoft's 2013 compiler?

I found the message below, basically saying that mixing compilers on
Windows is a Bad Idea...

  From: Ibrahim Tannir
  Subject: Re: [AOLSERVER] Naviserver Win-64 Sources
  Date: 2012-10-16 10:10:35 GMT

  http://www.mail-archive.com/naviserver-devel%40lists.sourceforge.net/msg02833.html
  http://article.gmane.org/gmane.comp.web.naviserver.devel/2837/match=win+64

-- 
Andrew Piskorski <at...@pi...>

[naviserver-devel] Windows 7 recommended compiler and build tools?

[Andrew P. <at...@pi...>]

I just started looking into running NaviServer 4.99.6 (the latest
release) on Windows 7 (64-bit), and am trying to figure out the proper
approach to building it (plus my own custom code) from source.

>From prior experience with ancient versions of Tcl and AOLserver, I
know of at least three build approaches on Microsoft Windows:

1. Microsoft Visual Studio, using (brittle, ugly) GUI-based project
   files to control the build.
2. Microsoft's compiler, using nmake from the command-line.
3. Msys + MinGW, using Gnu make.

Which of those do you recommend for NaviServer?  Also, do I need to
use the same compiler for everything, or is it feasible to say build
NaviServer with MinGW, but Tcl or my own custom code with Microsoft's
2013 compiler?

In NaviServer 4.99.6, the README gives a download link for
msys_mingw8.zip, which dates from c. 2004.  Is that old link just an
oversight, or is there some reason I should use such an ancient
version of MinGW rather than a current one?

In the win and win32 subdirectories, I see *.sln and *.dsw files for
use with Visual Studio 12.0.  Do those work better/worse than MinGW?
Also, is the command-line make-based build infrastructure set up to
work with Microsoft's compilers, or is it only for use with MinGW?

Does MinGW still only support 32-bit, and for 64-bit I need to instead
use the "MinGW-w64" fork?  Since MinGW-w64 handles both 32 and 64 bit,
is there any reason to prefer the original MinGW over it?

Is there any reason I should stick with 32-bit NaviServer, Tcl,
etc. on Windows 7, rather than moving to 64-bit?

Thanks in advance for your advice and tips!

Btw, Microsoft's current compilers are apparently available free of
charge, but just figuring out what you actually need to download and
install is non-trivial.  I found this advice from Jan. 2012 very
helpful:

  http://www.willus.com/ccomp.shtml?p11

And here's the Microsoft stuff I actually downloaded and installed:

- Windows SDK 7.1 for Windows 7 and .NET Framework 4:
  Rather than using the online installer (although it would work fine):
    http://www.microsoft.com/en-us/download/details.aspx?id=8279
  I downloaded the GRMSDKX_EN_DVD.iso x86-64:
    http://www.microsoft.com/download/en/details.aspx?id=8442
    http://download.microsoft.com/download/F/1/0/F10113F5-B750-4969-A255-274341AC6BCE/GRMSDKX_EN_DVD.iso
  As far as I could tell that's still the newest version of the SDK.
  From the Release Notes:
    http://go.microsoft.com/fwlink/?LinkID=186826
  it's clear that it includes both the Visual C++ 2010 compilers for both 32 and 64 bit
  ("the same compilers that ship with Visual Studio 2010"),
  the MSBuild 4.0 command line build environment, etc.

- The SDK does not ship with the necessary .NET Framework:
    http://msdn.microsoft.com/en-us/vstudio/aa496123
  So I used the web installer to install .Net Framework 4.5.1 from 2013-10-12:
    http://www.microsoft.com/en-us/download/details.aspx?id=40773
  NOT the older .Net Framework 4 from 2011-02-21:
    http://www.microsoft.com/download/en/details.aspx?id=17851

- VS Express 2013 for Windows Desktop:
  To get the GUI, I also installed the Express for Desktop version of
  Visual Studio, via the VS2013_RTM_DskExp_ENU.iso file:
    http://www.visualstudio.com/downloads/download-visual-studio-vs#d-express-windows-desktop
    http://www.visualstudio.com/en-US/products/visual-studio-express-vs

-- 
Andrew Piskorski <at...@pi...>

Re: [naviserver-devel] default filename extension

[John B. <jo...@ma...>]

Wow, Jeff, it's incredibly kind of you to have written this extension in response to my query.

I did find a small bug with the code you sent, namely that it also appended .adp to urls that ended with /, so you'd get a request for "http://localhost/.adp" when you tried to load "http://localhost" 

Below is how I fixed the code to avoid that small bug (a check for a trailing /).  Otherwise, your code works beautifully!  

Do you want to go ahead and check in to bitbucket that small change I've made?

-john


int
DefExtProc(void *arg, Ns_Conn *conn, int why)
{
	char *url;
	char **urlv;
	int urlc;
	Tcl_DString ds;

	url=conn->request->url;
	urlv=conn->request->urlv;
	urlc=conn->request->urlc;

	if (url[strlen(url) - 1] != '/') {
		if (!Tcl_StringMatch(urlv[urlc-1],"*.*")) {
			Ns_Log(Debug,"Adding trailing .adp to url");
			Tcl_DStringInit(&ds);
			Tcl_DStringAppend(&ds,url,-1);
			Tcl_DStringAppend(&ds,".adp",4);
			Ns_SetRequestUrl(conn->request, Tcl_DStringValue(&ds));
			Tcl_DStringFree(&ds);
		}
	}

	return NS_OK;
}

Re: [naviserver-devel] default filename extension

[John B. <jo...@ma...>]

Thanks Gustaf, thanks for your thoughts:

1) I agree that a 404 handler is another way of doing this, and it has the advantage of only adding overhead when the file is not there, with no overhead added for other situations. I've used that approached previously on aolserver.

2) it's true that Apache's MultiViews can be eccentric. Since I'm controlling all the content on my web sites, I'm probably not as aware of this as people with public Apache servers.

3) the other approach I've taken to have clean urls is with:
ns_register_proc GET /top/* direct_topbooks 0

I'll put this topic to bed now...

-john

Re: [naviserver-devel] default filename extension

[Jeff R. <dv...@di...>]

John Buckman wrote:
>
> On Jul 9, 2014, at 3:00 PM, Jeff Rogers <dv...@DI...
> <mailto:dv...@DI...>> wrote:
>
>> There's no builtin way I know of to define a default extension.  I would
>> do this with a preauth filter, something like this:
>
> The preauth filter is a nice way to do it, albeit with a bit of file
> existence checking overhead for every request that I'd prefer avoid.
 >
 > The way my C code worked (see previous msg) was to look to see if the
 > request url has no extension, and no trailing / on it either, and then
 > to append .adp in that case.  That has the advantage of being very low
 > overhead.

I certainly understand the desire to minimize overhead.  The tcl filter 
I posted could easily enough be changed to only look at the string 
values and not worry about checking for file existence.

There is still the overhead of the filter call and the tcl code, however 
(my tests show this overhead in the 10-12 microsecond range).  You can 
eliminate most of this overhead by implementing a filter in C rather 
than tcl, and the naviserver C api makes this easy.  That would provide 
most of the performance boost you're looking for while being lots more 
flexible and easier to manage than by changing the guts of SetUrl. In 
particular, since it's a filter you can easily limit the functionality 
to one subdirectory.

I created a simple extension to show the idea; it's at
https://bitbucket.org/jeffr/nsdefext/src

To build, clone it to a subfolder of your naviserver build folder, "cd 
nsdefext", and "make install".

To use it, just add it to your modules in your conf file:

ns_section      "ns/server/default/modules"
ns_param         nssock          nssock.so
ns_param         nslog           nslog.so
ns_param         nsdefext           nsdefext.so

I haven't tested the code extensively, but the main body is about 10 
lines, so it should be pretty simple to understand and modify.

Admittedly it's been a while since I've written an apache extension, but 
I recall it being much more complex than this :)

-J

>
> As you noted, Apache has this with MultiViews, and I use that feature
> extensively on Magnatune.com <http://Magnatune.com> so that URLs look
> nice and clean, ie: http://magnatune.com/artists/linda_wood
>
> So yes, I'd vote for this to be a config level feature, since it's
> already in Apache that way.
>
> Looking at request.c it looks to me like the action happens in the
> "SetUrl" function.
>
> Specifically, there already is logic in here to deal with trailing
> slashes, so a default filename extension feature would slot in well at
> this point:
>
>      /*
>       * Append a trailing slash to the normalized URL if the original URL
>       * ended in slash that wasn't also the leading slash.
>       */
>
>      while (*url == '/') {
>          ++url;
>      }
>      if (*url != '\0' && url[strlen(url) - 1] == '/') {
>          Ns_DStringAppend(&ds2, "/");
>      }
>
> /*
>   john - check to see if a default_extension has been set in the config
> file, and if so, append it now if appropriate.
> */
>
>      request->url = ns_strdup(ds2.string);
>      Ns_DStringFree(&ds2);
>

Re: [naviserver-devel] default filename extension

[Gustaf N. <ne...@wu...>]

Dear John,

filters allow application developers to decide, how to handle incoming 
requests in application specific ways, and the needs differ certainly. 
One can modify the filter that jeff posted slightly to omit the checking 
overhead by adding the default extension based on a dot in the path as 
well.  I would actually prefer a file-checking variant, since when an 
default extension is added without checking, one is loosing the ability 
to download e.g. uploaded files without extensions, leading to probably 
confusing error messages.

Applications like e.g. OpenACS have quite a complex request processor, 
that i would like to shrink down (handling .tcl, .adp and .vuh (virtual 
url handlers per directory, somewhat similar to unknown handlers in 
tcl)), but such a simple handler as you suggested won't help here much.

Actually the content negotiation of Apache distinguishes between
- mime extensions (e.g. .html)
- language extensions (e.g. .en), and
- encoding extensions (e.g. .gz).
which can be provided in any order (e.g. foo.html.gz.en or foo.en.gz.html).

All this can be scripted in NaviServer, and i am pretty sure, that works 
in apache only
via multiple check operations in the file system (for the various orders 
that are supported).
But via googling for MultiView, one finds quickly many warnings about 
surprising results
etc., preferring sometimes 404-messages instead of running unwanted 
scripts, etc.

Another option for handling such situations could be a script-able unknown
handler for NaviServer, somewhat similar to the directory handler 
_ns_dirlist, but i am
as well not sure, whether this would be a big advantage over the filter, 
since one has
to trigger finally as well something like "file not found" in some 
situations (similar to the
return codes of filters)

-g

Am 11.07.14 19:30, schrieb John Buckman:
>
> On Jul 9, 2014, at 3:00 PM, Jeff Rogers <dv...@DI... 
> <mailto:dv...@DI...>> wrote:
>
>> There's no builtin way I know of to define a default extension.  I would
>> do this with a preauth filter, something like this:
>
> The preauth filter is a nice way to do it, albeit with a bit of file 
> existence checking overhead for every request that I'd prefer avoid.
>
> The way my C code worked (see previous msg) was to look to see if the 
> request url has no extension, and no trailing / on it either, and then 
> to append .adp in that case.  That has the advantage of being very low 
> overhead.
>
> As you noted, Apache has this with MultiViews, and I use that feature 
> extensively on Magnatune.com <http://Magnatune.com> so that URLs look 
> nice and clean, ie: http://magnatune.com/artists/linda_wood
>
> So yes, I'd vote for this to be a config level feature, since it's 
> already in Apache that way.
>
> Looking at request.c it looks to me like the action happens in the 
> "SetUrl" function.
>
> Specifically, there already is logic in here to deal with trailing 
> slashes, so a default filename extension feature would slot in well at 
> this point:
>
>     /*
>      * Append a trailing slash to the normalized URL if the original URL
>      * ended in slash that wasn't also the leading slash.
>      */
>
>     while (*url == '/') {
>         ++url;
>     }
>     if (*url != '\0' && url[strlen(url) - 1] == '/') {
>         Ns_DStringAppend(&ds2, "/");
>     }
>
> /*
>  john - check to see if a default_extension has been set in the config 
> file, and if so, append it now if appropriate.
> */
>
>     request->url = ns_strdup(ds2.string);
>     Ns_DStringFree(&ds2);
>
>

Re: [naviserver-devel] default filename extension

[John B. <jo...@ma...>]

On Jul 9, 2014, at 3:00 PM, Jeff Rogers <dv...@DI...> wrote:

> There's no builtin way I know of to define a default extension.  I would 
> do this with a preauth filter, something like this:

The preauth filter is a nice way to do it, albeit with a bit of file existence checking overhead for every request that I'd prefer avoid.

The way my C code worked (see previous msg) was to look to see if the request url has no extension, and no trailing / on it either, and then to append .adp in that case.  That has the advantage of being very low overhead.

As you noted, Apache has this with MultiViews, and I use that feature extensively on Magnatune.com so that URLs look nice and clean, ie: http://magnatune.com/artists/linda_wood

So yes, I'd vote for this to be a config level feature, since it's already in Apache that way.

Looking at request.c it looks to me like the action happens in the "SetUrl" function.

Specifically, there already is logic in here to deal with trailing slashes, so a default filename extension feature would slot in well at this point:

    /*
     * Append a trailing slash to the normalized URL if the original URL
     * ended in slash that wasn't also the leading slash.
     */

    while (*url == '/') {
        ++url;
    }
    if (*url != '\0' && url[strlen(url) - 1] == '/') {
        Ns_DStringAppend(&ds2, "/");
    }

/*
 john - check to see if a default_extension has been set in the config file, and if so, append it now if appropriate.
*/

    request->url = ns_strdup(ds2.string);
    Ns_DStringFree(&ds2);

Re: [naviserver-devel] possible range bug?

[Gustaf N. <ne...@wu...>]

Am 10.07.14 19:51, schrieb Jeff Rogers:
> Gustaf Neumann wrote:
>> Am 09.07.14 21:18, schrieb Jeff Rogers:
>>> If I understand correctly, this codepath if used for a non-file based
>>> return, so e.g., "ns_return -binary" should get here.
>> The code paths are more complex and depend also on the settings of the
>> configuration file (e.g. caching, mmap, ...);
>>
>> when i was working on the async writer i created the call-graph [1]
>> below (manually, might contain errors; just did if for the parts i was
>> interested in). ReturnRange() is reached for file-descriptor based and
>> for data based deliveries
> I started to make a similar diagram, but mine's only on a whiteboard so
> far and not as complete.  What did you use for this diagram (I'm hoping
> LibreOffice draw), and can you share the source file?
>
I've made this with graphviz, the source is here:
http://openacs.org/xowiki/download/file/writer.dot

use it e.g. like:
     dot -Tpng writer.dot > writer.png
     dot -Tsvg writer.dot > writer.svg

-g

Re: [naviserver-devel] possible range bug?

[Jeff R. <dv...@di...>]

Gustaf Neumann wrote:
> Am 09.07.14 21:18, schrieb Jeff Rogers:
>> If I understand correctly, this codepath if used for a non-file based
>> return, so e.g., "ns_return -binary" should get here.
> The code paths are more complex and depend also on the settings of the
> configuration file (e.g. caching, mmap, ...);
>
> when i was working on the async writer i created the call-graph [1]
> below (manually, might contain errors; just did if for the parts i was
> interested in). ReturnRange() is reached for file-descriptor based and
> for data based deliveries

I started to make a similar diagram, but mine's only on a whiteboard so 
far and not as complete.  What did you use for this diagram (I'm hoping 
LibreOffice draw), and can you share the source file?


>> Also, it looks like ranges aren't supported at all for character data
>> (e.g., adp responses or non-binary ns_return).  Is this intentional and
>> desirable?  It seems reasonable at first glance, since the most useful
>> use case for ranges is large binaries, but it seems a bit inconsistent.

> as mentioned a while earlier on the list, the call graph of the data
> delivery logic
> is quite complex and might be simplified. I've simplified it in some
> steps when
> working on the async writer, but still, more can be done.

Agreed here.  Maybe it's time to get rid of some of the old C apis, and 
fix some of the naming to make others clearer?  The use of "Send" vs 
"Write" is confusing,  especially in the case of Ns_ConnSend* vs 
Ns_ConnSend.

> yes, currently range requests are ignored for e.g. .adp requests, but they
> are -supposed to be  - fully supported on file-requests, which are in
> practice
> the most important cases (e.g. various pdf readers depend on this, i think
> i have seen this as well range requests on video formats).
>
> Seems that so far, no-one had needs for range requests on dynamic content,
> which is often useless (e.g. language settings lead to different message
> strings
> on systems like e.g. OpenACS), but there might certainly be use-cases
> for that.

I agree with the sentiment here.  I could come up with a use case but 
I'm not aware of any currently in use.  Remember too, that even dynamic 
content is frequently mostly static, or static for long enough periods 
of time to allow static treatment (i.e., caching) to be useful.

-J

>
> the best start is usually to add a test case. I've added a test case
> showing that
> range requests are ignored on .adp requests.
>
> -gustaf neumann
>
> [1] http://openacs.org/xowiki/file/writer.png
>
>
>
> ------------------------------------------------------------------------------
> Open source business process management suite built on Java and Eclipse
> Turn processes into business applications with Bonita BPM Community Edition
> Quickly connect people, data, and systems into organized workflows
> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> http://p.sf.net/sfu/Bonitasoft
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>