naviserver-devel — Developer discussion

Re: [naviserver-devel] compilation ?

[Vlad S. <vl...@cr...>]

I just fixed that in the CVS

Zoran Vasiljevic wrote:
> Hi!
> 
> I'm afraid the compilation is somehow broken:
> 
> zoran:~/sf/naviserver/nsd zoran$ make
> gcc -pipe -g -Wall -Wno-implicit-int -fno-common  -I../include -I"/ 
> usr/local/include"  -DNO_CONST -DHAVE_CONFIG_H   -c -o log.o log.c
> log.c: In function `LogTime':
> log.c:762: error: wrong type argument to unary minus
> log.c: At top level:
> log.c:37: warning: `RCSID' defined but not used
> make: *** [log.o] Error 1
> 
> I have re-run the configure with --enable-symbols --enable-threads
> but I somehow miss many of the usual CFLAGS normally defined when
> I do the compilation. This is normally what I'd expected to see:
> 
> gcc -pipe -DURLDECODE_RELAXED=1 -g -Wall -Wno-implicit-int -fno- 
> strict-aliasing -fPIC  -I../include -I/usr/local/include  -DNO_CONST   
> -DTCL_THREADS=1 -DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 - 
> DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 - 
> DHAVE_READDIR_R=1 -DHAVE_THREE_ARG_READDIR_R=1 -DPEEK_XCLOSEIM=1 - 
> D_LARGEFILE64_SOURCE=1 -DTCL_WIDE_INT_TYPE=long\ long - 
> DHAVE_STRUCT_STAT64=1 -DHAVE_OPEN64=1 -DHAVE_LSEEK64=1 - 
> DHAVE_TYPE_OFF64_T=1 -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRSTR=1  
> -DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 -DHAVE_TMPNAM=1 - 
> DHAVE_WAITPID=1 -DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 - 
> DHAVE_SYS_PARAM_H=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 - 
> DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 - 
> DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 - 
> DHAVE_ST_BLKSIZE=1 -DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 - 
> DHAVE_LANGINFO=1 -DHAVE_SYS_IOCTL_H=1   -DTCL_CFG_DEBUG=1 - 
> DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_POLL=1 -DHAVE_DRAND48=1 - 
> DHAVE_RANDOM=1 -DHAVE_CMMSG=1 -DHAVE_GETADDRINFO=1 - DHAVE_GETNAMEINFO=1 
> -DHAVE_ZLIB_H=1 -DHAVE_LIBZ=1  -DNsdInit=_init   - c -o set.o set.c
> 
> Where have all those defines gone?
> 
> Zoran
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: NEC IT Guy Games.  How far can you 
> shotput
> a projector? How fast can you ride your desk chair down the office luge 
> track?
> If you want to score the big prize, get to know the little guy.  Play to 
> win an NEC 61" plasma display: http://www.necitguy.com/?r=20
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] compilation ?

[Zoran V. <zv...@ar...>]

Hi!

I'm afraid the compilation is somehow broken:

zoran:~/sf/naviserver/nsd zoran$ make
gcc -pipe -g -Wall -Wno-implicit-int -fno-common  -I../include -I"/ 
usr/local/include"  -DNO_CONST -DHAVE_CONFIG_H   -c -o log.o log.c
log.c: In function `LogTime':
log.c:762: error: wrong type argument to unary minus
log.c: At top level:
log.c:37: warning: `RCSID' defined but not used
make: *** [log.o] Error 1

I have re-run the configure with --enable-symbols --enable-threads
but I somehow miss many of the usual CFLAGS normally defined when
I do the compilation. This is normally what I'd expected to see:

gcc -pipe -DURLDECODE_RELAXED=1 -g -Wall -Wno-implicit-int -fno- 
strict-aliasing -fPIC  -I../include -I/usr/local/include  -DNO_CONST   
-DTCL_THREADS=1 -DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 - 
DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 - 
DHAVE_READDIR_R=1 -DHAVE_THREE_ARG_READDIR_R=1 -DPEEK_XCLOSEIM=1 - 
D_LARGEFILE64_SOURCE=1 -DTCL_WIDE_INT_TYPE=long\ long - 
DHAVE_STRUCT_STAT64=1 -DHAVE_OPEN64=1 -DHAVE_LSEEK64=1 - 
DHAVE_TYPE_OFF64_T=1 -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRSTR=1  
-DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 -DHAVE_TMPNAM=1 - 
DHAVE_WAITPID=1 -DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 - 
DHAVE_SYS_PARAM_H=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 - 
DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 - 
DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 - 
DHAVE_ST_BLKSIZE=1 -DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 - 
DHAVE_LANGINFO=1 -DHAVE_SYS_IOCTL_H=1   -DTCL_CFG_DEBUG=1 - 
DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_POLL=1 -DHAVE_DRAND48=1 - 
DHAVE_RANDOM=1 -DHAVE_CMMSG=1 -DHAVE_GETADDRINFO=1 - 
DHAVE_GETNAMEINFO=1 -DHAVE_ZLIB_H=1 -DHAVE_LIBZ=1  -DNsdInit=_init   - 
c -o set.o set.c

Where have all those defines gone?

Zoran

Re: [naviserver-devel] release ?

[Bernd E. <eid...@we...>]

Hi Stephen,

> http://groks.org/naviserver-4.99.0.tar.gz
>
> If this looks good, I'll tag it and release it to the file download
> section of the site.  So, make sure it still builds ok and check the
> NEWS file.

The NEWS file is great, everything put together at a glance. I put it on the 
website.

Your naviserver-4.99.0.tar.gz configured,maked and compiled well on my SuSE 
9.2.

Bernd.

Re: [naviserver-devel] Re: [naviserver-commits] naviserver NEWS,1.2,1.3

[Vlad S. <vl...@cr...>]

Okay, i thought it is global for the whole project.
I am maintaning modules i imported but they need some work for adding 
all those files.

Stephen Deasey wrote:
> The NEWS file is just like the ChangeLog in the sense that it only
> refers to changes in the package it's contained in.  Each of the
> modules listed below needs it's own NEWS file.
> 
> Which reminds me, we need to make sure the modules are in good shape. 
> At a minumum, they each need a README, ChangeLog and LICENSE file. 
> The last one is most important.  There also needs to be the standard
> license terms at the top of each source file (for whatever license you
> choose), and appropriate copyright statements.  An AUTHORS file might
> also be a good idea.
> 
> Ideally we'd have the website list all modules.  We can make a news
> release whenever a new module is imported into CVS to keep people
> informed.
> 
> 
> On 6/8/05, Vlad Seryakov <ser...@us...> wrote:
> 
>>Update of /cvsroot/naviserver/naviserver
>>In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8033
>>
>>Modified Files:
>>        NEWS
>>Log Message:
>>added new modules section into NEWS
>>
>>
>>Index: NEWS
>>===================================================================
>>RCS file: /cvsroot/naviserver/naviserver/NEWS,v
>>retrieving revision 1.2
>>retrieving revision 1.3
>>diff -C2 -d -r1.2 -r1.3
>>*** NEWS        8 Jun 2005 16:14:03 -0000       1.2
>>--- NEWS        8 Jun 2005 16:41:53 -0000       1.3
>>***************
>>*** 47,48 ****
>>--- 47,67 ----
>>       * Modules nspd and nsext moved from core to external modules
>>       * Objectified TclX keyed lists
>>+
>>+ New Modules:
>>+
>>+     * New nsudp module to support HTTP over UDP
>>+     * New nstk module that implements templating and Tcl utilities
>>+     * New nsaspell module implements interface to GNU Aspell
>>+     * New nsberkeleydb module implements interface to BerkelyDB
>>+     * New nschartdir module implements charting interface to ChartDirector
>>+     * New nsclamav module implements interface to ClamAV anti-virus
>>+     * New nsdns module implements DNS client and server
>>+     * New nsfortune module implements interface to Unix fortune game
>>+     * New nsfreetds module implements database driver to FreeTDS
>>+     * New nsgdchart module implements charting using GD library
>>+     * New nsimap module implements IMAP client using UW imap client
>>+     * New nsocaml module implements interface to OCaml langage
>>+     * New nssavi module implements interface to Sophos anti-virus
>>+     * New nssnmp module implements SNMP client and SNMP trap server
>>+     * New nszlib module implements interface to zlib library
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: NEC IT Guy Games.  How far can you shotput
> a projector? How fast can you ride your desk chair down the office luge track?
> If you want to score the big prize, get to know the little guy.  
> Play to win an NEC 61" plasma display: http://www.necitguy.com/?r 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] Re: [naviserver-commits] naviserver NEWS,1.2,1.3

[Stephen D. <sd...@gm...>]

The NEWS file is just like the ChangeLog in the sense that it only
refers to changes in the package it's contained in.  Each of the
modules listed below needs it's own NEWS file.

Which reminds me, we need to make sure the modules are in good shape.=20
At a minumum, they each need a README, ChangeLog and LICENSE file.=20
The last one is most important.  There also needs to be the standard
license terms at the top of each source file (for whatever license you
choose), and appropriate copyright statements.  An AUTHORS file might
also be a good idea.

Ideally we'd have the website list all modules.  We can make a news
release whenever a new module is imported into CVS to keep people
informed.


On 6/8/05, Vlad Seryakov <ser...@us...> wrote:
> Update of /cvsroot/naviserver/naviserver
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8033
>=20
> Modified Files:
>         NEWS
> Log Message:
> added new modules section into NEWS
>=20
>=20
> Index: NEWS
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> RCS file: /cvsroot/naviserver/naviserver/NEWS,v
> retrieving revision 1.2
> retrieving revision 1.3
> diff -C2 -d -r1.2 -r1.3
> *** NEWS        8 Jun 2005 16:14:03 -0000       1.2
> --- NEWS        8 Jun 2005 16:41:53 -0000       1.3
> ***************
> *** 47,48 ****
> --- 47,67 ----
>        * Modules nspd and nsext moved from core to external modules
>        * Objectified TclX keyed lists
> +
> + New Modules:
> +
> +     * New nsudp module to support HTTP over UDP
> +     * New nstk module that implements templating and Tcl utilities
> +     * New nsaspell module implements interface to GNU Aspell
> +     * New nsberkeleydb module implements interface to BerkelyDB
> +     * New nschartdir module implements charting interface to ChartDirec=
tor
> +     * New nsclamav module implements interface to ClamAV anti-virus
> +     * New nsdns module implements DNS client and server
> +     * New nsfortune module implements interface to Unix fortune game
> +     * New nsfreetds module implements database driver to FreeTDS
> +     * New nsgdchart module implements charting using GD library
> +     * New nsimap module implements IMAP client using UW imap client
> +     * New nsocaml module implements interface to OCaml langage
> +     * New nssavi module implements interface to Sophos anti-virus
> +     * New nssnmp module implements SNMP client and SNMP trap server
> +     * New nszlib module implements interface to zlib library

Re: [naviserver-devel] release ?

[Stephen D. <sd...@gm...>]

On 5/4/05, Zoran Vasiljevic <zv...@ar...> wrote:
> Hi friends,
>
> ...
>
> I could use this occasion to raise the "are we about to make a release"
> question
> again. As it seems, it would be fine for us to tag the CVS and make a
> tarball.
> What do you think? If yes, which version? 4.99?
>=20
> Zoran


Hey all.  I've made some build chages lately but email notifications
were busted at the time so you might not have noticed.

Here's a tarball I made by cvs exporting HEAD and running make dist:

http://groks.org/naviserver-4.99.0.tar.gz

If this looks good, I'll tag it and release it to the file download
section of the site.  So, make sure it still builds ok and check the
NEWS file.

[naviserver-devel] website

[Bernd E. <eid...@we...>]

Hi,

_please_ excuse the delay for bringing up the homepage. There is a first 
version at www.servercult.com now. 

Feel free to bring up everything you like or dislike. This is just a proposal 
(and also work in progress).

In my opinion some very basic texts describing the 'why' and 'where to go' of 
the project should be made available for the community; also I agree with you 
to keep the content low to focus work on the more important areas!

Until direct login is possible for you please send me any content you like to 
add, see, change!

Regards,
Bernd.

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 22:50 schrieb Stephen Deasey:

>
> Because form input always needs to be checked.  You can't declare it
> debugged and turn off exception checking.  People will continue to
> feed your program broken/malicious input.
>

Ah, this is what you mean! What about fixing tcllib to allow two
types of asserts? The one with off/on switch and the one without?
I haven't looked into there, but this can't be rocket science.

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Stephen D. <sd...@gm...>]

On 6/1/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 01.06.2005 um 22:23 schrieb Stephen Deasey:
>=20
> >  That's a useful feature, but not when you're checking
> > form input!
>=20
> Why not?


Because form input always needs to be checked.  You can't declare it
debugged and turn off exception checking.  People will continue to
feed your program broken/malicious input.

Re: [naviserver-devel] Re: RFE #1202462

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 21:27 schrieb Stephen Deasey:

>
> Sounds good.
>

OK. I will start the work and give diffs for review when I come
to some point.

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 22:23 schrieb Stephen Deasey:

>  That's a useful feature, but not when you're checking
> form input!

Why not? 
  

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Stephen D. <sd...@gm...>]

On 6/1/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 01.06.2005 um 21:30 schrieb Stephen Deasey:
>=20
> >  For input
> > validation, some other mechanism should be used.
> >
>=20
> Input of what kind? From Tcl or from C?
> Do you mean form-input? Why you can't apply the
> tcllib assertion on form-input, if this is what you mean?


One of the features of tcllib assertions is that you can turn them
off, or into no-ops when in production mode, just like standard C
assertions.  That's a useful feature, but not when you're checking
form input!

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 21:30 schrieb Stephen Deasey:

>  For input
> validation, some other mechanism should be used.
>

Input of what kind? From Tcl or from C?
Do you mean form-input? Why you can't apply the
tcllib assertion on form-input, if this is what you mean?

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Stephen D. <sd...@gm...>]

On 6/1/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 01.06.2005 um 13:24 schrieb Stephen Deasey:
>=20
> >  So, how
> > about assertions?
> >
>=20
> Good.
>=20
> So, that means we scratch the idea of putting it in the
> proc/ns_proc syntax and keep it as is, i.e. we parse-out
> only the options from the argument list and do no extra
> type checking there. Type-checking would be implemented
> as a separate command then.
>=20
> I understand the blues with the boolean arguments and
> I am willing to scratch them from the specs. So, if you
> need "-dothat" you'd be actually saying "-dothat true"
> or "-dothat false".
>=20
> All that leaves us with the ns_parseargs pretty much as
> it is now, or?
>=20
> Apropos assertions:
>      http://www.tcl.tk/cgi-bin/tct/tip/53.html
>=20
> This was withdrawn because of the fact that it already
> exist in tcllib. So, there is very little to do except
> "package req tcllib" and use what's there.


Looks good.  For program bug detection, use tcllib assertions in
cooperation with the option parsing we have now.  For input
validation, some other mechanism should be used.

Re: [naviserver-devel] Re: RFE #1202462

[Stephen D. <sd...@gm...>]

On 6/1/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 01.06.2005 um 14:10 schrieb Stephen Deasey:
>=20
> > On 5/30/05, Zoran Vasiljevic <zv...@ar...> wrote:
> >
> >>
> >> Am 30.05.2005 um 12:01 schrieb Stephen Deasey:
> >>
> >>
> >>>
> >>> What's the return value of Ns_FSOpen?
> >>>
> >>>
> >>
> >> Something that can be fed to Ns_FSRead and/or Ns_FSClose
> >> for example. I haven't started the implementation yet
> >> but this is first what comes to mind.
> >>
> >> Presumably, it will be Tcl_Channel in VFS or a regular
> >> OS filehandle w/o VFS.
> >>
> >
> >
> > The return type would be unknown unless the caller also used #ifdef?
>=20
> The return value is of course opaque and should be returned
> to the FS abstraction layer. But I read on...
>=20
> >
> > I don't see anything wrong with using Tcl_VFS* directly to read the
> > config file, write to a log file etc., and the tcl commands in
> > nsd/tclfile.c should be replaced by simple Tcl wrappers anyway.  It's
> > the stuff in nsd/fastpath.c that's important.  File descriptors need
> > to be available for use with modern, high performance system calls
> > with a minimum of conditional compilation/execution.
> >
> > This also has the potential to be a never-ending bug hunt.  Absolutely
> > every call to the file system has to be converted to Tcl_VFS* or
> > Ns_VFS*, including all modules and the libraries they wrap, or people
> > will be surprised...
>=20
> > I'm not against this feature, but it's pretty esoteric and the
> > implementation is invasive so the quality bar needs to be set high.
> > I'm sure you can do it  :-)
> >
>=20
> Of course, all you say holds. If you need to go to the max when
> accesing files, then Tcl VFS will not be the fastest kid on the block.
>=20
> But what about this: we make a Ns_FS wrappers and put them in place,
> even in the fastpath and adp code. We also add Ns_FSGetOSHandle which
> will then return invalid handle to the caller if the file is not located
> on the native filesystem. This is trivial and costs no performance.
> Now, the caller can accomodate for that and decide what to do.
> A very good example is the actual fastpath code which does the same
> with the mmap. It works-around the fact that mmap is not defined in
> the server config and uses simple open/read/close sequence.
>=20
> This way you'd be free to add whatever fancy roaring implementation
> (kaio for example) to get the max-speed, if you like.
> We need only cover access to files, no sockets, pipes, etc.
> What we'd need is just a handful of calls:
>=20
>    open, close, stat, seek, read, write, opendir, readdir, closedir,
> getcwd
>=20
> (did I forgot something?)
>=20
> Most of them would use the opaque file-handle from which you can
> obtain the OS handle to make any private speed optimizations. If the
> handle cannot be obtained, then you provide the best-effort fallback.
>=20
> What would/could be wrong with that?


Sounds good.

Re: [naviserver-devel] Re: RFE #1202462

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 14:10 schrieb Stephen Deasey:

> On 5/30/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
>>
>> Am 30.05.2005 um 12:01 schrieb Stephen Deasey:
>>
>>
>>>
>>> What's the return value of Ns_FSOpen?
>>>
>>>
>>
>> Something that can be fed to Ns_FSRead and/or Ns_FSClose
>> for example. I haven't started the implementation yet
>> but this is first what comes to mind.
>>
>> Presumably, it will be Tcl_Channel in VFS or a regular
>> OS filehandle w/o VFS.
>>
>
>
> The return type would be unknown unless the caller also used #ifdef?

The return value is of course opaque and should be returned
to the FS abstraction layer. But I read on...

>
> I don't see anything wrong with using Tcl_VFS* directly to read the
> config file, write to a log file etc., and the tcl commands in
> nsd/tclfile.c should be replaced by simple Tcl wrappers anyway.  It's
> the stuff in nsd/fastpath.c that's important.  File descriptors need
> to be available for use with modern, high performance system calls
> with a minimum of conditional compilation/execution.
>
> This also has the potential to be a never-ending bug hunt.  Absolutely
> every call to the file system has to be converted to Tcl_VFS* or
> Ns_VFS*, including all modules and the libraries they wrap, or people
> will be surprised...

> I'm not against this feature, but it's pretty esoteric and the
> implementation is invasive so the quality bar needs to be set high.
> I'm sure you can do it  :-)
>

Of course, all you say holds. If you need to go to the max when
accesing files, then Tcl VFS will not be the fastest kid on the block.

But what about this: we make a Ns_FS wrappers and put them in place,
even in the fastpath and adp code. We also add Ns_FSGetOSHandle which
will then return invalid handle to the caller if the file is not located
on the native filesystem. This is trivial and costs no performance.
Now, the caller can accomodate for that and decide what to do.
A very good example is the actual fastpath code which does the same
with the mmap. It works-around the fact that mmap is not defined in
the server config and uses simple open/read/close sequence.

This way you'd be free to add whatever fancy roaring implementation
(kaio for example) to get the max-speed, if you like.
We need only cover access to files, no sockets, pipes, etc.
What we'd need is just a handful of calls:

   open, close, stat, seek, read, write, opendir, readdir, closedir,  
getcwd

(did I forgot something?)

Most of them would use the opaque file-handle from which you can
obtain the OS handle to make any private speed optimizations. If the
handle cannot be obtained, then you provide the best-effort fallback.

What would/could be wrong with that?

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 13:24 schrieb Stephen Deasey:

>  So, how
> about assertions?
>

Good.

So, that means we scratch the idea of putting it in the
proc/ns_proc syntax and keep it as is, i.e. we parse-out
only the options from the argument list and do no extra
type checking there. Type-checking would be implemented
as a separate command then.

I understand the blues with the boolean arguments and
I am willing to scratch them from the specs. So, if you
need "-dothat" you'd be actually saying "-dothat true"
or "-dothat false".

All that leaves us with the ns_parseargs pretty much as
it is now, or?

Apropos assertions:
     http://www.tcl.tk/cgi-bin/tct/tip/53.html

This was withdrawn because of the fact that it already
exist in tcllib. So, there is very little to do except
"package req tcllib" and use what's there.

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 01.06.2005 um 15:45 schrieb Vlad Seryakov:

> To be as much close to current Tcl convention in proc syntax we can  
> add optional type AFTER default value
>
> ns_proc test { {-flag 0 flag} {-list a oneof {a b c}} {-name ""}  
> { value "" } } {
> }
>
> This way i do not have to remember 2 different ways to provide  
> default value and if i want type checking, i will give optional  
> typoe, if not then no type cheking.

The only side-effect is of course that you'd need to give the default
argument if you need type-checking. I believe I can live with that.

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Vlad S. <vl...@cr...>]

To be as much close to current Tcl convention in proc syntax we can add 
optional type AFTER default value

ns_proc test { {-flag 0 flag} {-list a oneof {a b c}} {-name ""} { value 
"" } } {
}

This way i do not have to remember 2 different ways to provide default 
value and if i want type checking, i will give optional typoe, if not 
then no type cheking.

Stephen Deasey wrote:
> On 5/30/05, Zoran Vasiljevic <zv...@ar...> wrote:
> 
>>Am 30.05.2005 um 12:23 schrieb Stephen Deasey:
>>
>>
>>>The other option is -eightbit:flag syntax.  It's not perfect, but it's
>>>better than nested lists and has ~5 years of usage in the ACS behind
>>>it.
>>>
>>
>>Another try (I can't just let go) ...
>>
>>I do not see much sense in trying to keep the Tcl argument list
>>format under all costs. Give me a good reason why we should do that?
>>Anyways, we are about to define a new "proc"-type of command
>>(ns_proc) aren't we? What is the sense then to force ourselves
>>to the old format?
>>
>>Why don't we *extend* Tcl-proc arg-treatment like this:
>>
>>    {name typecheck ?default_value?}
>>
>>This is *not* Tcl-proc compatible but *is* in the spirit of
>>the language. The "typecheck" above can/will be definitely
>>a list of one or more elements, depending on the check itself.
>>The typecheck is *forced*. You must give a type. If you need
>>no checking at all use "string" which will cover all possiblities.
>>By *forcing* the type-check, many of the programming
>>errors and double-checks in the procedure itself can be saved.
>>
>>Also, all people needing no extra checking can always use plain Tcl-proc
>>calls. Others who'd need checking would use exteneded ns_proc syntax.
>>This is pretty clear and I see no problems with that at all.
>>
>>For example:
>>    ns_parseargs {{-retries int 0} {-blockcopy flag} {device string}
>>args}
>>
>>or
>>    ns_proc dosomething {{-retries int 0} {-blockcopy flag} {device
>>string}} {
>>       # ...
>>    }
>>
>>The most natural Tcl-way is to use lists. We've seen that the
>>desperate try to keep the Tcl-proc syntax leads to too complicated
>>nested lists which humans are not easily assembling.
>>So, this is a plain list which is easy to write and read by humans.
>>
>>We can evolve typecheck over time to include more elaborate
>>tests, but a handful of those (string, int, flag, bool, list) are most
>>useful and complete for about all the needs I can think of
>>at the moment.
>>
>>What is wrong with that (except that it perhaps does not follow
>>openacs model)?
> 
> 
> 
> ns_proc test {{-flag f}} {
>     if {$flag} {
>         ns_log notice ok
>     }
> }
> 
> test -flag t ;# ok
> test -flag true ;# ok
> test -flag yes ;# ok
> test -flag 1 ;# ok
> 
> The result of the expression passed to Tcl's if is converted to
> boolean so there's no need to use [string is true $x].   The
> difference between '-flag' and '-flag t' is two characters -- I don't
> think it's worth it.
> 
> As I mentioned before, in general I think it's a bad idea to create
> boolean flags as it makes it cumbersome to call such a proc where the
> flag is computed.  e.g.:
> 
> if {$x} {
>     foo -flag ...
> } else {
>   foo ...
> }
> 
> It gets really ugly when there are multiple flags.  I added boolean
> flags to the C implementation as there is already existing code which
> uses them which could benefit from the arg parsing.
> 
> 
> Regarding the 'oneof' type, maybe what's actually needed is some
> syntactic sugar?  e.g. something like:
> 
> proc assert_oneof {varname list} {
>     upvar $varname x
>     if {![lsearch -exact $list $x]} {
>         error "assertion failed: $varname must be one of $list"
>     }
> }
> 
> proc test {{x foo}} {
>     assert_oneof x {foo bar baz}
>     ...
> }
> 
> 
> The disadvantage of extending the proc syntax as you've suggested
> above is that users are then forced to always provide a type, even if
> it's just the 'null' type, string.  This kind of type checking is
> likely to be the exception rather than the rule, but I imagine option
> parsing will be quite common.  The ordering of the default argument
> has changed which is incompatible with what the programmer expects
> from standard proc usage, even if the mechanical incompatibility does
> not matter.
> 
> The oneof check shows that things will get more complicated than
> simple 'is int', 'is bool' checks.  Min and max range checks on
> integers would be nice, regexp checks on strings would be cool etc...
> 
> What about applying more than one check to an arg?  There would need
> to be syntax for it as you can't specify the arg more than once as you
> can specify multiple assertions.
> 
> Other languages solve this with assertions or pre-conditions and
> post-conditions, and that feels more like what we're doing here.  Tcl
> *does* have type checking: an integer argument passed to a proc which
> is then passed to the expr command will cause a conversion failure if
> the value is 'foo'.  The type checking on the proc argument will just
> cause the failure earlier (and will result in double-checking).
> 
> What we're catching here are programmer errors.  Checking is more
> urgently required for malicious attacks.  You really want to be able
> to say something like:
> 
>     ns_queryget -assert int -- life 42
> 
> It was always in the back of my mind that we needed some kind of
> unified checking mechanism to handle both procs and ns_queryget (and
> forms, as in Vlad's nstk).  Rather than shoe-horn the type checking of
> proc into the proc args, perhaps it would be better to use something
> more direct like the assertion example above?  You could imagine
> various type of syntactic sugar:
> 
>     assert x oneof {foo bar baz}
> 
>     assert {
>         x int
>         y digit
>     }
> 
>     ...
> 
> Type checking happens at compile time in many languages which makes
> them better than assertions, but that doesn't apply here.  So, how
> about assertions?
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by Yahoo.
> Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
> Search APIs Find out how you can build Yahoo! directly into your own
> Applications - visit http://developer.yahoo.net/?fr_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Re: RFE #1202462

[Stephen D. <sd...@gm...>]

On 5/30/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 30.05.2005 um 12:01 schrieb Stephen Deasey:
>=20
> >
> > What's the return value of Ns_FSOpen?
> >
>=20
> Something that can be fed to Ns_FSRead and/or Ns_FSClose
> for example. I haven't started the implementation yet
> but this is first what comes to mind.
>=20
> Presumably, it will be Tcl_Channel in VFS or a regular
> OS filehandle w/o VFS.


The return type would be unknown unless the caller also used #ifdef?

The issue is not preferring open over Tcl_VFSOpen, and it's not mmap
per se.  mmap is just one example of a function which requires a file
descriptor or real files in the file system.  sendfile is another, as
are readv/writev, epoll, kqueue, inotify, etc.

The Tcl VFS is a lowest common denominator abstraction over file
systems.  It provides the common things, but can't provide
implementations of everything, nor can it return a useful file
descriptor from a Tcl_Channel, at least in the case where everything
is contained within one big file.  I think Ns_TclGetOpenFd is going to
fail.

I don't see anything wrong with using Tcl_VFS* directly to read the
config file, write to a log file etc., and the tcl commands in
nsd/tclfile.c should be replaced by simple Tcl wrappers anyway.  It's
the stuff in nsd/fastpath.c that's important.  File descriptors need
to be available for use with modern, high performance system calls
with a minimum of conditional compilation/execution.

This also has the potential to be a never-ending bug hunt.  Absolutely
every call to the file system has to be converted to Tcl_VFS* or
Ns_VFS*, including all modules and the libraries they wrap, or people
will be surprised...


I'm not against this feature, but it's pretty esoteric and the
implementation is invasive so the quality bar needs to be set high.=20
I'm sure you can do it  :-)

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Stephen D. <sd...@gm...>]

On 5/30/05, Zoran Vasiljevic <zv...@ar...> wrote:
>=20
> Am 30.05.2005 um 12:23 schrieb Stephen Deasey:
>=20
> >
> > The other option is -eightbit:flag syntax.  It's not perfect, but it's
> > better than nested lists and has ~5 years of usage in the ACS behind
> > it.
> >
>=20
> Another try (I can't just let go) ...
>=20
> I do not see much sense in trying to keep the Tcl argument list
> format under all costs. Give me a good reason why we should do that?
> Anyways, we are about to define a new "proc"-type of command
> (ns_proc) aren't we? What is the sense then to force ourselves
> to the old format?
>=20
> Why don't we *extend* Tcl-proc arg-treatment like this:
>=20
>     {name typecheck ?default_value?}
>=20
> This is *not* Tcl-proc compatible but *is* in the spirit of
> the language. The "typecheck" above can/will be definitely
> a list of one or more elements, depending on the check itself.
> The typecheck is *forced*. You must give a type. If you need
> no checking at all use "string" which will cover all possiblities.
> By *forcing* the type-check, many of the programming
> errors and double-checks in the procedure itself can be saved.
>=20
> Also, all people needing no extra checking can always use plain Tcl-proc
> calls. Others who'd need checking would use exteneded ns_proc syntax.
> This is pretty clear and I see no problems with that at all.
>=20
> For example:
>     ns_parseargs {{-retries int 0} {-blockcopy flag} {device string}
> args}
>=20
> or
>     ns_proc dosomething {{-retries int 0} {-blockcopy flag} {device
> string}} {
>        # ...
>     }
>=20
> The most natural Tcl-way is to use lists. We've seen that the
> desperate try to keep the Tcl-proc syntax leads to too complicated
> nested lists which humans are not easily assembling.
> So, this is a plain list which is easy to write and read by humans.
>=20
> We can evolve typecheck over time to include more elaborate
> tests, but a handful of those (string, int, flag, bool, list) are most
> useful and complete for about all the needs I can think of
> at the moment.
>=20
> What is wrong with that (except that it perhaps does not follow
> openacs model)?


ns_proc test {{-flag f}} {
    if {$flag} {
        ns_log notice ok
    }
}

test -flag t ;# ok
test -flag true ;# ok
test -flag yes ;# ok
test -flag 1 ;# ok

The result of the expression passed to Tcl's if is converted to
boolean so there's no need to use [string is true $x].   The
difference between '-flag' and '-flag t' is two characters -- I don't
think it's worth it.

As I mentioned before, in general I think it's a bad idea to create
boolean flags as it makes it cumbersome to call such a proc where the
flag is computed.  e.g.:

if {$x} {
    foo -flag ...
} else {
  foo ...
}

It gets really ugly when there are multiple flags.  I added boolean
flags to the C implementation as there is already existing code which
uses them which could benefit from the arg parsing.


Regarding the 'oneof' type, maybe what's actually needed is some
syntactic sugar?  e.g. something like:

proc assert_oneof {varname list} {
    upvar $varname x
    if {![lsearch -exact $list $x]} {
        error "assertion failed: $varname must be one of $list"
    }
}

proc test {{x foo}} {
    assert_oneof x {foo bar baz}
    ...
}


The disadvantage of extending the proc syntax as you've suggested
above is that users are then forced to always provide a type, even if
it's just the 'null' type, string.  This kind of type checking is
likely to be the exception rather than the rule, but I imagine option
parsing will be quite common.  The ordering of the default argument
has changed which is incompatible with what the programmer expects
from standard proc usage, even if the mechanical incompatibility does
not matter.

The oneof check shows that things will get more complicated than
simple 'is int', 'is bool' checks.  Min and max range checks on
integers would be nice, regexp checks on strings would be cool etc...

What about applying more than one check to an arg?  There would need
to be syntax for it as you can't specify the arg more than once as you
can specify multiple assertions.

Other languages solve this with assertions or pre-conditions and
post-conditions, and that feels more like what we're doing here.  Tcl
*does* have type checking: an integer argument passed to a proc which
is then passed to the expr command will cause a conversion failure if
the value is 'foo'.  The type checking on the proc argument will just
cause the failure earlier (and will result in double-checking).

What we're catching here are programmer errors.  Checking is more
urgently required for malicious attacks.  You really want to be able
to say something like:

    ns_queryget -assert int -- life 42

It was always in the back of my mind that we needed some kind of
unified checking mechanism to handle both procs and ns_queryget (and
forms, as in Vlad's nstk).  Rather than shoe-horn the type checking of
proc into the proc args, perhaps it would be better to use something
more direct like the assertion example above?  You could imagine
various type of syntactic sugar:

    assert x oneof {foo bar baz}

    assert {
        x int
        y digit
    }

    ...

Type checking happens at compile time in many languages which makes
them better than assertions, but that doesn't apply here.  So, how
about assertions?

Re: [Re: Re: [Re: Re: [naviserver-devel] ns_parseargs example]]

[Zoran V. <zv...@ar...>]

Am 30.05.2005 um 19:21 schrieb vl...@cr...:

> The question is what ns_proc will do if i give wrong type?

Throw error.

>
> Catching all ns_proc calls for exceptions will make code not very  
> clear?
> How typechecks will be enforced?
>

That is the *exact* reason why you need checks: to throw error!
If you need no checks then "none" or "string" should be given
in which case no checking will be performed. But if you *need*
checks than Tcl offers nothing that will assist you.

Just to be clear: what I mean is *not* replace the "proc" call
with ns_proc. I think about augmeting the "proc" with ns_proc
so both could be used. So, only for new code this will make sense.

I *very* frequently run into the checking all arguments
using [string is...] type of checks and if one check fails
I generate error.  It is so often repeated that I find it
very annoying to write all those over and over.
That is the reason why I like what Stephen made and am
not giving up :-)

One of the major TCL strengts *and* weaknesses is: lack of type  
checking.
Now, to assert this on the variable level is impossible and I doubt that
this will ever happen. But enforcing it on the procedure level is very
simple and handy (for exmaple like this effort we're discussing now)
and we should eventually agree on some of the ways. Thanks to Stephens
work, we already have this on C-level. It would be great if we can
extend it to Tcl level as well in some meaningful way.

Zoran

[Re: Re: [Re: Re: [naviserver-devel] ns_parseargs example]]

[<vl...@cr...>]

The question is what ns_proc will do if i give wrong type?

Catching all ns_proc calls for exceptions will make code not very clear?
How typechecks will be enforced?

-------- Original Message --------
To: nav...@li...
From: Zoran Vasiljevic <zv...@ar...>
Subject: Re: [Re: Re: [naviserver-devel] ns_parseargs example]


Am 30.05.2005 um 12:23 schrieb Stephen Deasey:

>
> The other option is -eightbit:flag syntax.  It's not perfect, but it's
> better than nested lists and has ~5 years of usage in the ACS behind
> it.
>

Another try (I can't just let go) ...

I do not see much sense in trying to keep the Tcl argument list
format under all costs. Give me a good reason why we should do that?
Anyways, we are about to define a new "proc"-type of command
(ns_proc) aren't we? What is the sense then to force ourselves
to the old format?

Why don't we *extend* Tcl-proc arg-treatment like this:

    {name typecheck ?default_value?}

This is *not* Tcl-proc compatible but *is* in the spirit of
the language. The "typecheck" above can/will be definitely
a list of one or more elements, depending on the check itself.
The typecheck is *forced*. You must give a type. If you need
no checking at all use "string" which will cover all possiblities.
By *forcing* the type-check, many of the programming
errors and double-checks in the procedure itself can be saved.

Also, all people needing no extra checking can always use plain Tcl-proc
calls. Others who'd need checking would use exteneded ns_proc syntax.
This is pretty clear and I see no problems with that at all.

For example:
    ns_parseargs {{-retries int 0} {-blockcopy flag} {device string}  
args}

or
    ns_proc dosomething {{-retries int 0} {-blockcopy flag} {device  
string}} {
       # ...
    }

The most natural Tcl-way is to use lists. We've seen that the
desperate try to keep the Tcl-proc syntax leads to too complicated
nested lists which humans are not easily assembling.
So, this is a plain list which is easy to write and read by humans.

We can evolve typecheck over time to include more elaborate
tests, but a handful of those (string, int, flag, bool, list) are most
useful and complete for about all the needs I can think of
at the moment.

What is wrong with that (except that it perhaps does not follow
openacs model)?

Zoran





-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 30.05.2005 um 12:23 schrieb Stephen Deasey:

>
> The other option is -eightbit:flag syntax.  It's not perfect, but it's
> better than nested lists and has ~5 years of usage in the ACS behind
> it.
>

Another try (I can't just let go) ...

I do not see much sense in trying to keep the Tcl argument list
format under all costs. Give me a good reason why we should do that?
Anyways, we are about to define a new "proc"-type of command
(ns_proc) aren't we? What is the sense then to force ourselves
to the old format?

Why don't we *extend* Tcl-proc arg-treatment like this:

    {name typecheck ?default_value?}

This is *not* Tcl-proc compatible but *is* in the spirit of
the language. The "typecheck" above can/will be definitely
a list of one or more elements, depending on the check itself.
The typecheck is *forced*. You must give a type. If you need
no checking at all use "string" which will cover all possiblities.
By *forcing* the type-check, many of the programming
errors and double-checks in the procedure itself can be saved.

Also, all people needing no extra checking can always use plain Tcl-proc
calls. Others who'd need checking would use exteneded ns_proc syntax.
This is pretty clear and I see no problems with that at all.

For example:
    ns_parseargs {{-retries int 0} {-blockcopy flag} {device string}  
args}

or
    ns_proc dosomething {{-retries int 0} {-blockcopy flag} {device  
string}} {
       # ...
    }

The most natural Tcl-way is to use lists. We've seen that the
desperate try to keep the Tcl-proc syntax leads to too complicated
nested lists which humans are not easily assembling.
So, this is a plain list which is easy to write and read by humans.

We can evolve typecheck over time to include more elaborate
tests, but a handful of those (string, int, flag, bool, list) are most
useful and complete for about all the needs I can think of
at the moment.

What is wrong with that (except that it perhaps does not follow
openacs model)?

Zoran

Re: [Re: Re: [naviserver-devel] ns_parseargs example]

[Zoran V. <zv...@ar...>]

Am 30.05.2005 um 12:23 schrieb Stephen Deasey:

> On 5/28/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
>
>
> Well, the above example is wrong.  It should be:
>
>     ns_proc connect {{{-eightbit flag}} {{-speed list ...
>
> Nested lists are clearly unusable.

They are, indeed, but this is the price for being flexible.

>
>
>
>> I'm afraid we do not have very much options about all that.
>> Either we drop the type checking and therefore accept no
>> flag-type arguments or similar, or we do something like above
>> and be faced with more complex writing.
>>
>
>
> The other option is -eightbit:flag syntax.  It's not perfect, but it's
> better than nested lists and has ~5 years of usage in the ACS behind
> it.

Hm... what I do not like about that is: this is a usage convention
and this convention is against the syntax of the language.
Maybe this "against" is too harsh word for it, but still...

As I said many times before, I think that we are going little bit
too far with that. If we just accept the status-quo then we'd have
to live without optional flags (and other types of checking),
that is, one should/could use:

   ns_parseargs {{-option true} args}

and write like this:

   -option true
   -option 1
   -option yes
   -option no
   etc

and internally typecheck: [string is boolean $option].
Nothing will prevent somebody to "-option foo" but
this is then in the responsibility of the programmer.

Since Tcl does not enforce type-checking, we should not do
this either, if it can't coexist peacefully with the language-syntax.

Personally, I'd have nothing against if you add this type of
parsing but I believe I will not be able to use it. If you
think this is widely accepted (I have no such experience so far)
then go ahead; I won't stay in the way.

Cheer's
Zoran

>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by Yahoo.
> Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
> Search APIs Find out how you can build Yahoo! directly into your own
> Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg- 
> q22005
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>