state-threads-devel — State Threads developers' list

[ST-devel] Re: Patch to get st/linux to build on most archs

[Gene <ge...@pa...>]

> 
> The patch attached makes libst work on linux for:
>         sparc, arm, alpha, powerpc, ia64, i386, mipsel, mips.
> 

This is great!


> libst is quite broken on hppa. I have no idea what's wrong. If someone could
> look at this, I'd really appriciate it.
> 

One issue I can think about is that on hppa stack grows up.
I believe it's the only architecture currently in use where
stack grows up.


Also, the following comment should be changed from

/* IA-64 architecture -- needs a faster context switcher since gcc sucks */

to something like that:

/*
 * Besides traditional memory call stack, IA-64 uses general register
 * stack. Thus each thread needs a backing store for register stack in
 * addition to memory stack. Standard setjmp()/longjmp() cannot be used
 * for thread context switching because their implementation implicitly
 * assumes that only one register stack exists.
 */


--Gene

[ST-devel] Patch to get st/linux to build on most archs

[Wesley W. T. <we...@te...>]

The patch attached makes libst work on linux for:
	sparc, arm, alpha, powerpc, ia64, i386, mipsel, mips.

libst is quite broken on hppa. I have no idea what's wrong. If someone could
look at this, I'd really appriciate it.

Finally, libst is now in debian/woody main for all the architectures listed
above. Sorry it took so long!

I've also have prebuilt potato .debs which could be made available on
sourceforge if you think people would find them useful. I haven't attached
them however since it totals to 161k. :-)

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] [PATCH] 1.4a

[Gene <ge...@pa...>]

This is a small patch to 1.4a that adds to the Andreas' NetBSD
port.  It adds Alpha support on NetBSD and SPARC support on
OpenBSD (file md.h).

--Gene

Re: [ST-devel] 1.4a

[Claus A. <sta...@es...>]

On Wed, Feb 20, 2002, Mike Abbott wrote:
> Ok, I misunderstood.  Looks like the file was erroneously dos2unix'd
> during upload (^M's removed).  Should be OK now, try again.

Thanks, it works now.

> I thought you were talking about the automagic download-and-build that
> Wesley added, which works only for final releases.

I don't even know about that...

PS: any plans to use PGP signatures for the distribution?

Re: [ST-devel] 1.4a

[Mike A. <mj...@at...>]

Ok, I misunderstood.  Looks like the file was erroneously dos2unix'd
during upload (^M's removed).  Should be OK now, try again.

I thought you were talking about the automagic download-and-build that
Wesley added, which works only for final releases.

Re: [ST-devel] 1.4a

[Claus A. <sta...@es...>]

On Wed, Feb 20, 2002, Mike Abbott wrote:
> > This seems to be broken:
> > 
> > $ fetch http://prdownloads.sourceforge.net/state-threads/st-1.4a.tar.gz
> 
> Right.  Since 1.4a is a pre-release version of 1.4 this won't work, but
> it will work once 1.4 is released.  This is the same as it was with the
> last couple of 1.3 pre-releases.  I'm not sure but I think it has to be
> this way for some numbering scheme to work on some platform.  Perhaps
> Wesley can elaborate, or correct my thinking so that future pre-releases
> support this feature.

Sorry, I don't follow you... :-(

I can't download the software in a version such that gunzip can
actually unpack it. Is this intended?

As I wrote: I tried fetch (FreeBSD), ftp (OpenBSD), lynx, and just
now also w3m and NetScape.  How am I supposed to download the
pre-release? Not at all?

Re: [ST-devel] 1.4a

[Mike A. <mj...@at...>]

> This seems to be broken:
> 
> $ fetch http://prdownloads.sourceforge.net/state-threads/st-1.4a.tar.gz

Right.  Since 1.4a is a pre-release version of 1.4 this won't work, but
it will work once 1.4 is released.  This is the same as it was with the
last couple of 1.3 pre-releases.  I'm not sure but I think it has to be
this way for some numbering scheme to work on some platform.  Perhaps
Wesley can elaborate, or correct my thinking so that future pre-releases
support this feature.

Re: [ST-devel] 1.4a

[Claus A. <sta...@es...>]

On Mon, Feb 18, 2002, Mike Abbott wrote:
> Experimental version 1.4a of the State Threads Library is now available.
> It adds Darwin and NetBSD support and plays nice with strict
> prototyping.  As a side effect of the latter it resolves the
> long-standing conflict between the public and private typedefs (although
> not in a very pretty way).

This seems to be broken:

$ fetch http://prdownloads.sourceforge.net/state-threads/st-1.4a.tar.gz
Receiving st-1.4a.tar.gz (76447 bytes): 100%
76447 bytes transfered in 2.5 seconds  (30.21 Kbytes/s)
$ gunzip st-1.4a.tar.gz 
 incomplete literal tree

gunzip: st-1.4a.tar.gz: invalid compressed data--format violated
$ file st-1.4a.tar.gz 
st-1.4a.tar.gz: gzip compressed data, deflated, last modified: Mon Feb 18 23:07:58 2002, max compression, os: Unix

I tried this from three different systems using three different
download methods (fetch (FreeBSD), ftp (OpenBSD), lynx (SunOS)).

PS: could you add a PGP signature for those tar files?

[ST-devel] 1.4a

[Mike A. <mj...@at...>]

Experimental version 1.4a of the State Threads Library is now available.
It adds Darwin and NetBSD support and plays nice with strict
prototyping.  As a side effect of the latter it resolves the
long-standing conflict between the public and private typedefs (although
not in a very pretty way).
	http://state-threads.sourceforge.net/
-- 
Mike Abbott    ma...@us...
State Threads Project co-administrator

[ST-devel] building without gmake?

[Claus A. <sta...@es...>]

Does anyone have a Makefile(.in ?) that doesn't require GNU make?
I would like to minimize the requirements to configure/compile
statethreads. I want to use statethreads for my MTA which uses
configure (generated by autoconf, which I'm just learning to use).
This currently allows me to use the make program that comes with
the OS instead of requiring GNU make.

[ST-devel] NetBSD support

[<gs...@no...>]

The following patch adds support for NetBSD/i386 and NetBSD/sparc to
state-threads.  I already uploaded it to the state-threads patch
repository on sourceforge, but apparently no one is actually using
that so I'm resending it to the state-threads-devel list.
-- 
Andreas Gustafsson, gs...@no...

diff -u -r st-1.3d.orig/Makefile st-1.3d/Makefile
--- st-1.3d.orig/Makefile	Tue Jan 15 22:34:48 2002
+++ st-1.3d/Makefile	Fri Feb  8 17:17:02 2002
@@ -46,6 +46,7 @@
 #OS         = IRIX_64
 #OS         = LINUX
 #OS         = LINUX_IA64
+#OS         = NETBSD
 #OS         = OPENBSD
 #OS         = OSF1
 #OS         = SOLARIS
@@ -86,6 +87,7 @@
               irix-64-debug irix-64-optimized       \
               linux-debug linux-optimized           \
               linux-ia64-debug linux-ia64-optimized \
+              netbsd-debug netbsd-optimized         \
               openbsd-debug openbsd-optimized       \
               osf1-debug osf1-optimized             \
               solaris-debug solaris-optimized
@@ -148,6 +150,12 @@
 OTHER_FLAGS = -Wall
 endif
 
+ifeq ($(OS), NETBSD)
+SFLAGS      = -fPIC
+LDFLAGS     = -shared -soname=$(SONAME) -lc
+OTHER_FLAGS = -Wall
+endif
+
 ifeq ($(OS), OPENBSD)
 SFLAGS      = -fPIC
 LDFLAGS     = -shared -soname=$(SONAME) -lc
@@ -326,6 +334,11 @@
 	$(MAKE) OS="LINUX_IA64" BUILD="DBG"
 linux-ia64-optimized:
 	$(MAKE) OS="LINUX_IA64" BUILD="OPT"
+
+netbsd-debug:
+	$(MAKE) OS="NETBSD" BUILD="DBG"
+netbsd-optimized:
+	$(MAKE) OS="NETBSD" BUILD="OPT"
 
 openbsd-debug:
 	$(MAKE) OS="OPENBSD" BUILD="DBG"
diff -u -r st-1.3d.orig/examples/Makefile st-1.3d/examples/Makefile
--- st-1.3d.orig/examples/Makefile	Sat Nov  3 13:18:42 2001
+++ st-1.3d/examples/Makefile	Fri Feb  8 17:17:43 2002
@@ -38,6 +38,7 @@
 # IRIX_64
 # LINUX
 # LINUX_IA64
+# NETBSD
 # OPENBSD
 # OSF1
 # SOLARIS
diff -u -r st-1.3d.orig/md.h st-1.3d/md.h
--- st-1.3d.orig/md.h	Tue Jan 15 22:20:40 2002
+++ st-1.3d/md.h	Fri Feb  8 17:13:40 2002
@@ -245,6 +245,40 @@
   (void) gettimeofday(&tv, NULL); \
   return (tv.tv_sec * 1000000LL + tv.tv_usec)
 
+#elif defined (NETBSD)
+
+#define MD_STACK_GROWS_DOWN
+#define MD_USE_BSD_ANON_MMAP
+#define MD_ACCEPT_NB_INHERITED
+#define MD_ALWAYS_UNSERIALIZED_ACCEPT
+#define MD_HAVE_SOCKLEN_T
+
+#define MD_SETJMP(env) _setjmp(env)
+#define MD_LONGJMP(env, val) _longjmp(env, val)
+
+#if __sparc__
+#define MD_INIT_CONTEXT(_thread, _sp, _main) \
+  ST_BEGIN_MACRO                             \
+  (void) MD_SETJMP((_thread)->context);      \
+  (_thread)->context[0] = (long) (_sp);      \
+  (_thread)->context[1] = (long) (_main) - 8; \
+  ST_END_MACRO
+#elif __i386__
+#define MD_INIT_CONTEXT(_thread, _sp, _main) \
+  ST_BEGIN_MACRO                             \
+  (void) MD_SETJMP((_thread)->context);      \
+  (_thread)->context[0] = (long) _main;      \
+  (_thread)->context[2] = (long) (_sp);      \
+  ST_END_MACRO
+#else
+#error Unsupported architecture
+#endif
+
+#define MD_GET_UTIME()            \
+  struct timeval tv;              \
+  (void) gettimeofday(&tv, NULL); \
+  return (tv.tv_sec * 1000000LL + tv.tv_usec)
+
 #elif defined (OPENBSD)
 
 #define MD_STACK_GROWS_DOWN
diff -u -r st-1.3d.orig/osguess.sh st-1.3d/osguess.sh
--- st-1.3d.orig/osguess.sh	Tue Jan 15 22:34:20 2002
+++ st-1.3d/osguess.sh	Fri Feb  8 17:18:43 2002
@@ -32,6 +32,7 @@
   *-sgi-irix6*    ) OS=IRIX       ;;
   ia64-*-linux*   ) OS=LINUX_IA64 ;;
   *-linux*        ) OS=LINUX      ;;
+  *-netbsd*       ) OS=NETBSD     ;;
   *-openbsd*      ) OS=OPENBSD    ;;
   *-dec-osf*      ) OS=OSF1       ;;
   *-solaris2*     ) OS=SOLARIS    ;;

[ST-devel] Re: ST darwin support

[Wesley W. T. <we...@te...>]

On Tue, Jan 29, 2002 at 06:37:03PM -0800, Gene Shekhtman wrote:
> I wonder if DARWIN can be combined with OPENBSD in md.h.
> Something like that:
>
> #elif defined (OPENBSD) || defined (DARWIN)
> 
> ....
> 
> #if defined(__i386__)
> #define MD_JB_SP   2
> #elif defined(__alpha__)
> #define MD_JB_SP  34
> #elif defined(__ppc__)
> #define MD_JB_SP   0
> #else
> #error Unknown CPU architecture
> #endif
> 
> Unfortunately, I don't have OpenBSD on PPC to test it.

I tried exactly that when I first made the patch - it does work.

However, there is a few reasons I choose not to do that: 

-- What if OpenBSD runs on a ppc at some point? 
	This will probably mess up - 0 will not be right.
	Just look at AIX; it uses 3 for the same CPU.

-- Conversely, Darwin already does run on i386.
	I have no idea what MD_JP_SP to use there though.
	However, I doubt it's safe to assume it's 2.

On the other hand, if someone who runs darwin/i386 tries it and finds that
the value is 2, then merging them is probably smart.

PS. The thing that really annoys me is the poll.h problem. (!!)

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] Re: ST darwin support

[Gene S. <gs...@ab...>]

"Wesley W. Terpstra" wrote:

> I ported libst to Darwin (MacOS X) yesterday (wanted my application to run
> on it). Here's the patch.
>

This is very cool!

I wonder if DARWIN can be combined with OPENBSD in md.h.
Something like that:

#elif defined (OPENBSD) || defined (DARWIN)

....

#if defined(__i386__)
#define MD_JB_SP   2
#elif defined(__alpha__)
#define MD_JB_SP  34
#elif defined(__ppc__)
#define MD_JB_SP   0
#else
#error Unknown CPU architecture
#endif


Unfortunately, I don't have OpenBSD on PPC to test it.

[ST-devel] ST darwin support

[Wesley W. T. <we...@te...>]

I ported libst to Darwin (MacOS X) yesterday (wanted my application to run
on it). Here's the patch.

Crazy linker options hey? They work and appear to be consistent with the
MacOS linker documentation and porting web pages I looked at.

I'm not actually sure about the serialized accept, but it is roughly bsd, so
it's probably right. Also, there is no 'poll.h' for Darwin so I ripped the
needed defitions and wrapped them in an ifdef.

Finally, I also fixed a dependency problem in the Makefile that was my
fault. :-)

Advantages:
	Compiles fine as a native MacOS X library
Disadvantages:
	There might be other platforms where
		#if defined(__MACH__) && defined(__APPLE__)
	comes out true and thus the headers will conflict with the system.

	I'm don't run MacOS myself; I did this port while visiting a friend
	and the test programs and my own app ran so it seems to work, but it
	is hardly well tested.

Documentation:
	CONFIG_GUESS_PATH=...; make default
	or
	make darwin-optimized

PS. A friend of mine is looking into making a 'fink' package out of it.

-- 
Wesley W. Terpstra <we...@te...>

Re: [ST-devel] libst & large files

[Wesley W. T. <we...@te...>]

On Tue, Jan 22, 2002 at 11:47:12AM -0800, Lev Walkin wrote:
> Wesley W. Terpstra wrote:
> >On Mon, Jan 21, 2002 at 12:14:19PM -0800, Lev Walkin wrote:
> >>Wesley W. Terpstra wrote:
> >>>Has anyone tried to use libst with 64bit large file support on a 32bit
> >>>platform?
> >>
> >>lseek(fd, 0, SEEK_END);
> >>while(1) {
> >>	ssize_t rsize;
> >>	rsize = read(fd, buf, sizeof(buf));
> >>	if(rsize > 0) {
> >>		/* Do things */
> >>	} else {
> >>		usleep(250000);
> >>	}
> >>}
> >>
> >>This will work on 32-bit OS'es supporting 64-bit file systems. The off_t 
> >>parameter is being used only in lseek().
> >
> >However, I want to use libst, so where you have read(...) I'd have to put
> >st_read(...). That way libst could context switch and work on other tasks
> >while I was waiting for data---the whole point of using it here.
> >
> >Suppose I open the file with proper lfs support and do the lseek. This will
> >work as you describe above. However, consider this:
> >
> >My program is compiled with _FILE_OFFSET_BITS=64 which makes my read
> >lfs-aware. Unfortunately, libst does not have this toggle when compiled.
> 
> What you probably missing is what the hell is "lfs-aware". 
> _FILE_OFFSET_BITS=64 does not change the _read()_ call in any way, so
> it should be compatible.

The X/OPEN standard strongly implies that the implementations can and should
redirect the IO functions when LFS is enabled. This same standard defines
how normal 32bit calls should fail on large files if they have not declared
that they know who to deal with lfs.

On linux, at least, pread is redirected as are many other IO functions. read
appears to be an exception, but that is surely not a portable assumption to
make.

features.h tests:
#if defined _FILE_OFFSET_BITS && _FILE_OFFSET_BITS == 64
# define __USE_FILE_OFFSET64    1
#endif

Then unistd.h does:
# ifndef __USE_FILE_OFFSET64
extern ssize_t pread (int __fd, void *__buf, size_t __nbytes, __off_t
__offset)
     __THROW;
extern ssize_t pwrite (int __fd, __const void *__buf, size_t __n,
                       __off_t __offset) __THROW;
# else
#  ifdef __REDIRECT
extern ssize_t __REDIRECT (pread, (int __fd, void *__buf, size_t __nbytes,
                                   __off64_t __offset) __THROW,
                           pread64);
extern ssize_t __REDIRECT (pwrite, (int __fd, __const void *__buf,
                                    size_t __nbytes, __off64_t __offset)
                           __THROW,
                        pwrite64);
#  else
#   define pread pread64
#   define pwrite pwrite64
#  endif
# endif

> Probably, you won't use the st_read instead of read. Why do you need
> to do it? st_read() will never return EWOULDBLOCK or EAGAIN on hardware
> filesystems, so the thread-switch logic has no meaning on plain files.

Ahh, I thought select(...) would not claim a file ready for input if it was
at EOF. However, a test program does not behave this way---as you indicated.
That's really too bad, it would have been an elegant way of following
several files at once.

I suppose I can use st_sleep to yield control and thus never let libst even
see the file descriptors for the lfs files. That will certainly resolve any
concerns I have.

Well, out of all this I have a workable solution---even if it is polling.

Thanks.

-- 
Wesley W. Terpstra <we...@te...>

Re: [ST-devel] libst & large files

[Lev W. <vl...@ne...>]

Wesley W. Terpstra wrote:

> On Mon, Jan 21, 2002 at 12:14:19PM -0800, Lev Walkin wrote:
> 
>>Wesley W. Terpstra wrote:
>>
>>>Has anyone tried to use libst with 64bit large file support on a 32bit
>>>platform?
>>>
>>>I assume it won't work, but wanted to check. I notice that the read/write
>>>apis all use size_t instead of off_t, so I doubt they'll work.
>>>
>>FreeBSD has the 64 bit large file support on 32 bit code. read/write 
>>returns obvious ssize_t, and it is 32 bit in size. So particularly this
>>will work right.
>>
> 
> What the heck. You're right: I just checked my headers, read/write use
> ssize_t even in LFS mode. I suppose that makes sense -- you can't read/write
> more than 2Gb at once even if LFS is enabled. Who'd want to anyways?
> 
> So the API in libst for read/write is fine. I still have concerns though.
> 
> 
>>>Before you say 'why would you need 64bit support for sockets', it's not for
>>>sockets... I want to have the equivalent of 'tail -f' on several files at
>>>the same time I'm watching sockets for incoming requests. These files could
>>>be >> 2Gb.
>>>
>>lseek(fd, 0, SEEK_END);
>>while(1) {
>>	ssize_t rsize;
>>	rsize = read(fd, buf, sizeof(buf));
>>	if(rsize > 0) {
>>		/* Do things */
>>	} else {
>>		usleep(250000);
>>	}
>>}
>>
>>This will work on 32-bit OS'es supporting 64-bit file systems. The off_t 
>>parameter is being used only in lseek().
>>
> 
> However, I want to use libst, so where you have read(...) I'd have to put
> st_read(...). That way libst could context switch and work on other tasks
> while I was waiting for data---the whole point of using it here.
> 
> Suppose I open the file with proper lfs support and do the lseek. This will
> work as you describe above. However, consider this:
> 
> My program is compiled with _FILE_OFFSET_BITS=64 which makes my read
> lfs-aware. Unfortunately, libst does not have this toggle when compiled.


What you probably missing is what the hell is "lfs-aware". 
_FILE_OFFSET_BITS=64 does not change the _read()_ call in any way, so
it should be compatible.


> Hence it's read function (which st_read will call) is not lfs aware and will
> return EOVERFLOW when I call it (section 2.2.1.25 of the X/OPEN LFS spec
> <http://ftp.sas.com/standards/large.file/x_open.20Mar96.html>).
> 
> On the other hand, maybe the spec means that I get EOVERFLOW if I hadn't
> opened it as an lfs-fildes? Perhaps the read is the same and it checks some
> flag on the fildes?
> 
> This is why I was wondering if I could use libst with an lfs system.

Probably, you won't use the st_read instead of read. Why do you need
to do it? st_read() will never return EWOULDBLOCK or EAGAIN on hardware
filesystems, so the thread-switch logic has no meaning on plain files.

-- 
Lev Walkin
vl...@ne...

Re: [ST-devel] libst & large files

[Wesley W. T. <we...@te...>]

On Mon, Jan 21, 2002 at 12:14:19PM -0800, Lev Walkin wrote:
> Wesley W. Terpstra wrote:
> >Has anyone tried to use libst with 64bit large file support on a 32bit
> >platform?
> >
> >I assume it won't work, but wanted to check. I notice that the read/write
> >apis all use size_t instead of off_t, so I doubt they'll work.
> 
> FreeBSD has the 64 bit large file support on 32 bit code. read/write 
> returns obvious ssize_t, and it is 32 bit in size. So particularly this
> will work right.

What the heck. You're right: I just checked my headers, read/write use
ssize_t even in LFS mode. I suppose that makes sense -- you can't read/write
more than 2Gb at once even if LFS is enabled. Who'd want to anyways?

So the API in libst for read/write is fine. I still have concerns though.

> >Before you say 'why would you need 64bit support for sockets', it's not for
> >sockets... I want to have the equivalent of 'tail -f' on several files at
> >the same time I'm watching sockets for incoming requests. These files could
> >be >> 2Gb.
> 
> lseek(fd, 0, SEEK_END);
> while(1) {
> 	ssize_t rsize;
> 	rsize = read(fd, buf, sizeof(buf));
> 	if(rsize > 0) {
> 		/* Do things */
> 	} else {
> 		usleep(250000);
> 	}
> }
>
> This will work on 32-bit OS'es supporting 64-bit file systems. The off_t 
> parameter is being used only in lseek().

However, I want to use libst, so where you have read(...) I'd have to put
st_read(...). That way libst could context switch and work on other tasks
while I was waiting for data---the whole point of using it here.

Suppose I open the file with proper lfs support and do the lseek. This will
work as you describe above. However, consider this:

My program is compiled with _FILE_OFFSET_BITS=64 which makes my read
lfs-aware. Unfortunately, libst does not have this toggle when compiled.

Hence it's read function (which st_read will call) is not lfs aware and will
return EOVERFLOW when I call it (section 2.2.1.25 of the X/OPEN LFS spec
<http://ftp.sas.com/standards/large.file/x_open.20Mar96.html>).

On the other hand, maybe the spec means that I get EOVERFLOW if I hadn't
opened it as an lfs-fildes? Perhaps the read is the same and it checks some
flag on the fildes?

This is why I was wondering if I could use libst with an lfs system.

-- 
Wesley W. Terpstra <we...@te...>

Re: [ST-devel] libst & large files

[Lev W. <vl...@ne...>]

Wesley W. Terpstra wrote:

> Has anyone tried to use libst with 64bit large file support on a 32bit
> platform?
> 
> I assume it won't work, but wanted to check. I notice that the read/write
> apis all use size_t instead of off_t, so I doubt they'll work.


FreeBSD has the 64 bit large file support on 32 bit code. read/write 
returns obvious ssize_t, and it is 32 bit in size. So particularly this
will work right.

> However,
> what I want to know is whether or not sticking them in a st_poll would work:
> There's that new errno value for if you use a 32bit call on a 64bit file
> and I am concerned that they could break even those st functions without
> offset paremeter.
> 
> Has anyone gotten or failed to get this to work?
> If it doesn't work, should this be fixed?
> 
> Before you say 'why would you need 64bit support for sockets', it's not for
> sockets... I want to have the equivalent of 'tail -f' on several files at
> the same time I'm watching sockets for incoming requests. These files could
> be >> 2Gb.

lseek(fd, 0, SEEK_END);
while(1) {
	ssize_t rsize;
	rsize = read(fd, buf, sizeof(buf));
	if(rsize > 0) {
		/* Do things */
	} else {
		usleep(250000);
	}
}

This will work on 32-bit OS'es supporting 64-bit file systems. The off_t 
parameter is being used only in lseek().

-- 
Lev Walkin
vl...@ne...

[ST-devel] libst & large files

[Wesley W. T. <we...@te...>]

Has anyone tried to use libst with 64bit large file support on a 32bit
platform?

I assume it won't work, but wanted to check. I notice that the read/write
apis all use size_t instead of off_t, so I doubt they'll work. However,
what I want to know is whether or not sticking them in a st_poll would work:
There's that new errno value for if you use a 32bit call on a 64bit file
and I am concerned that they could break even those st functions without
offset paremeter.

Has anyone gotten or failed to get this to work?
If it doesn't work, should this be fixed?

Before you say 'why would you need 64bit support for sockets', it's not for
sockets... I want to have the equivalent of 'tail -f' on several files at
the same time I'm watching sockets for incoming requests. These files could
be >> 2Gb.

Thanks.

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] 1.3d

[Mike A. <mj...@at...>]

Experimental version 1.3d of the State Threads Library is now available.
It fixes some bugs, runs on Alpha, and makes rpms and debs.
	http://state-threads.sourceforge.net/

I intend to release this as 1.3 in about a week so holler if you see
something broken.
-- 
Mike Abbott    ma...@us...
State Threads Project co-administrator

[ST-devel] Re: please send a final clean patch

[Wesley W. T. <we...@te...>]

On Mon, Jan 07, 2002 at 10:57:40PM -0800, Mike Abbott wrote:
> I'm going to release 1.3 in the next week or so.  Please send me your
> best candidate patch or patches that include all the changes you want,
> modified as desired to incorporate the recent feedback.  

Alrighty, here's my final version of the patch:
	It no longer uses '-fpic' -- only '-fPIC'
	It still compiles twice - with and without large branch pic
	It uses a variant on standard unix versioning that stays
	  in sync with the st version, but addresses the api concerns.

To apply: you need a clean tree, Gene's config.guess patch (re-attached),
and finally my patch: (note the different -px options)

tar xvzf st-1.3c.tar.gz
cd st-1.3c
patch -p0 < ~/st-1.3-guess.patch
patch -p1 < ~/st-1.3-pkg.patch

If you don't want the scripts to package the project in the upstream
version, follow this with:

rm -rf debian st.spec

Leaving the spec file in the tar.gz is probably a good plan. 
redhat/mandrake users can then use: 'rpm -ta st-1.3.tar.gz' which is much
more convenient than having to hunt for the spec file on the net.

The debian/ directory is not really necessary if st gets into debian main,
since then it could be recreated by maintainer patches, but it's kinda nice
to have it available: 'cd st-1.3; debuild'

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] Re: Patches to 1.3c to produce rpm/debs

[Wesley W. T. <we...@te...>]

On Mon, Jan 07, 2002 at 11:19:49AM -0800, Gene Shekhtman wrote:
> "Wesley W. Terpstra" wrote:
> 
> > Problem 1:
> >
> > The libraries built right now are not relocatable. I'm not entirely sure
> > what this does, but from packaging libraries before I know that it is
> > supposed to be done. :-)
> >
> 
> A few points on Problem 1.
> 
> - As far as I know, on many platforms (e.g., Solaris, Irix, and *Linux*)
>   compiler emits position-independent code by default.  So there is no
>   need for -fpic/-fPIC flag.  Often these flags are used "just in case",
>   but it doesn't mean they are always needed.

Ok, I've gone and read up on this now to be more sure of things... You are
correct: on linux '-fpic' is implied. However, '-fPIC' is *not*.

This is because it is slower and larger to do position-independent large
displacement branches. So, gcc does some optimization for you: generally
you aren't writing a shared library. Unfortunately, if you are the code must
be fully relocatable to be shared.

For concrete evidence: if the option were truly redundant, the file size
with and without this option would remain the same. It does not: -fPIC makes
the objects bigger. Hence the option does do something in linux. libtool
chooses to use them. Also, most other well-scrutinised packages (even those
that don't use libtool) also use them. eg: libc, libdb3, etc

libc:

/usr/i486-linuxlibc1/bin/gcc -V 2.7.2.3 -b i486-linuxlibc1 -m486
-malign-loops=2 -malign-jumps=2 -malign-functions=2 -O1 -funroll-loops -I.
-I.. -I../libio -I../libio/stdio -DNLS -I../nls  -DYP -DNO_SHADOW
-D_GNU_SOURCE -DSTDC_HEADERS -DUSG -DDIRENT -DSYSV -DUSE_BSD_REGEX -D_LIBC
-DINTERNAL_LINUX_C_LIB -D_REENTRANT  -Wall -Wstrict-prototypes
-Wmissing-prototypes -funsigned-char -I../internal -nostdinc -I../include
-I/usr/lib/gcc-lib/i486-linuxlibc1/2.7.2.3/include -DNIS -DUSE_OPTIONS_H=1
-c inet_ntoa.c -o ../elfstatic/libc/inet_ntoa.o

** Note: elfstatic does not have the -fPIC or -fpic options. They later 'ar'
these .o files up.

/usr/i486-linuxlibc1/bin/gcc -V 2.7.2.3 -b i486-linuxlibc1 -m486
-malign-loops=2 -malign-jumps=2 -malign-functions=2 -fPIC -O6 -funroll-loops
-fomit-frame-pointer -g1 -I. -I.. -I../libio -I../libio/stdio -DNLS -I../nls
-DYP -DNO_SHADOW  -D_GNU_SOURCE -DSTDC_HEADERS -DUSG -DDIRENT -DSYSV
-DUSE_BSD_REGEX -D_LIBC -DINTERNAL_LINUX_C_LIB -D_REENTRANT  -Wall
-Wstrict-prototypes -Wmissing-prototypes -funsigned-char -I../internal
-nostdinc -I../include -I/usr/lib/gcc-lib/i486-linuxlibc1/2.7.2.3/include
-DNIS -DUSE_OPTIONS_H=1  -c inet_ntoa.c -o ../elfshared/libc/inet_ntoa.o

** Note: elfshared uses -fPIC, they compiled inet_ntoa twice, and they later
use 'ld' on these objects.

> - According to gcc man page, -fPIC is a superset of -fpic, so you don't
>   need both of them (if you need them at all).

I agree, -fPIC is probably sufficient since libc uses it alone. I just put
both in because that's what libtool does and in all things library I defer
to it just in case. :-)

> - It's OK to have relocatable objects in an ar archive (.a).  So there
>   is no need to compile twice.

It is indeed ok, but it's slower and larger. Generally people want the
static library to not have pic code. Since I was repeatedly told that speed
is important, I stuck with that. According to one of the links below the
impact of position independent code is about 5%.

You may be interested to read:
http://www.io.com/help/linux/ELF-HOWTO-1.html
which explains that -fPIC is needed and the pic speed impact.

Also, item 4.2 in:
http://chandra.ph1.uni-koeln.de/linux/debian/debian-policy/ch4.html
which mentions that one should compile libraries twice.

-- Aside:

Interestingly, this item also mentions that for a library you should use
-D_REENTRANT so that it will work with multithreaded applications
(pthreaded I presume)... This probably counts doubly for libst!

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] Re: Patches to 1.3c to produce rpm/debs

[Gene S. <gs...@ab...>]

"Wesley W. Terpstra" wrote:

>
> Problem 1:
>
> The libraries built right now are not relocatable. I'm not entirely sure
> what this does, but from packaging libraries before I know that it is
> supposed to be done. :-)
>

A few points on Problem 1.

- As far as I know, on many platforms (e.g., Solaris, Irix, and *Linux*)
  compiler emits position-independent code by default.  So there is no
  need for -fpic/-fPIC flag.  Often these flags are used "just in case",
  but it doesn't mean they are always needed.

- According to gcc man page, -fPIC is a superset of -fpic, so you don't
  need both of them (if you need them at all).

- It's OK to have relocatable objects in an ar archive (.a).  So there
  is no need to compile twice.

[ST-devel] Patches to 1.3c to produce rpm/debs

[Wesley W. T. <ter...@te...>]

On Fri, Jan 04, 2002 at 07:21:37PM -0800, Gene Shekhtman wrote:
> +case "$sys_info" in
> +  *-ibm-aix4*     ) OS=AIX        ;;
> +  *-freebsd*      ) OS=FREEBSD    ;;
> +  hppa*-hp-hpux11*) OS=HPUX       ;;
> +  *-sgi-irix6*    ) OS=IRIX       ;;
> +  ia64-*-linux*   ) OS=LINUX_IA64 ;;
> +  *-linux*        ) OS=LINUX      ;;
> +  *-openbsd*      ) OS=OPENBSD    ;;
> +  *-dec-osf*      ) OS=OSF1       ;;
> +  *-solaris2*     ) OS=SOLARIS    ;;
> +  *               ) OS=
> +                    echo "Sorry, unsupported OS"
> +                    exit 1        ;;
> +esac

Are you sure this case statement will work? In my own patch this was the
part that worried me the most.

Anyways, since you're the upstream maintainer and I but a lowly contributor,
I've decided to abandon my first packaging attempt and build my next off of
this patch.

This email is quite long, but it's because I explain why I made the changes
to the Makefile I had to make. These changes should be merged upstream even
if the packaging files are not; they constitute bugs at present.

---

Problem 1:

The libraries built right now are not relocatable. I'm not entirely sure
what this does, but from packaging libraries before I know that it is
supposed to be done. :-) 

I believe it has something to do with being able to share the library in
memory: if it's not relocatable you have to modify the addresses at load
time for each program that needs the library and so it can't be shared. A
relocatable library looks at some register for this modification and thus it
doesn't need to be modified and can be shared by all programs. I could be
wrong however; like I said, I don't know exactly what it does. :-)

Anyways, of the things I know for sure: -fPIC -fpic turn it on, relocatable
libraries are slightly larger than normal, it is supposed to be done for .so
libraries and not for .a libraries, and libtool does it.

Unfortunately, this means you have to compile the library objects twice.

I believe this is also true for FreeBSD and OpenBSD so I've changed them as
well. I'm fairly positive that something similar exists for the other
platforms, but I don't know what it is. This is why I advocated libtool
earlier: it knows all about these issues and correctly deals with them.

I've attached a patch that fixes it for those platforms I know about - it is
incremental to your Makefile changes above. (libst-1.3-fpic.patch)

Problem 2:

Also, under linux/bsd systems (and unixs in general I believe), it is common
practice to adopt the following library versioning system:
	libst.so.x.y.z
	libst.so.x -> libst.so.x.y.z
	libst.so -> libst.so.x.y.z
	a soname of libst.so.x

Here is an example situtation:
	libst.so.1.3.1	(soname libst.so.1)
	libst.so.0.6.2	(soname libst.so.0)
	libst.so.1 -> libst.1.3.1
	libst.so.0 -> libst.0.6.2
	libst.so -> libst.1.3.1

	st.h (v1.3.1)

** NOTE: x.y.z in some situations does not match the package version.
This is because there are restrictions x.y.z must obey and package versions
sometimes mean things like: x=rewrite, y=major change, z=bug fix. For a
small library like libst, it would make more sense to keep the two in sync.
Hence, I strongly advocate applying the restrictions to x.y.z to the libst
version in general. My patch assumes that this is what is being done
(x.y.z=1.3.0).

These are the constraints that apply:
1	The version of the header and libst.so symlink should match
2	The soname of a library should be libst.so.x
3	The actual binary should use the full version information
4	Each version of the API that removes features or otherwise breaks
compatability requires x++
5	Each change to the API that adds features requires y++
6	Each bug fix / optimization requires z++
7	The libst.so.x link should point to the newest libst.so.x.y.z
available.

The reason for this setup is:
	When compiling with '#include <st.h>' and -lst, a matching pair of
the library and header are used [1]. The program will also remember what
library version it needs for later [2].
	The sysadmin knows exactly at a glance which version of the library
is which and what programs are using (by looking at libst.so.x -> ?) [3]
	Any program that was compiled against a given interface will
continue to work if that library is present. Even the removal of a library
feature or change to a functions API will not break the program since the
library maintainer will have increased x++ [4]. Hence, the new library can
coexist with the old (which the program needs). (as in example above)
	When a library is upgraded due to bugfix or feature add only,
programs linked against it will benefit from the new version. [5,6,7]

One of the nice things about libtool is that it will check at runtime that
*y* is sufficiently large for an application as well. That is, the dynamic
loader already makes sure to use the right *x* (no features removed), but
libtool ensures that the program has all the new features it needs (y).

However, this problem (version check y) can for the most part be ignored.
Any decent packaging format (rpm/deb) will automatically deal with it. For
example: 'foo' depends on libst1 (>= 1.3.2) ensures that the feature
requirement y>=3 is met.

Note: libst1 vs. libst0 as packages make sure that the libraries can coexist
in the package management tool. However, libst-dev can only be installed
once, and it include libst.so -> ? and st.h that match.

Presently libst violates constraints: 2, 3, 7 and by omision 4,5,6
For 4,5,6 this is simply a versioning policy it's easy to adopt and adhere
to.

A common problem with packaging libraries is that the upstream maintainer is
not aware of these constraints on versioning. Then the packager gets in
trouble when his package breaks the distribution. This is a problem if, say,
the libdb upstream maintainers removed or changed a feature from 3.0.2 ->
3.1.0. Suddenly every program linked to libdb.so.3 is broken on the system!
(don't laugh, this happens!) 

libst probably hasn't had to worry about this yet because it's not widely
used as a shared library by vital system tools. However, we want to change
that right? :-) So we have to play by the rules.

Here's my patch that address 2,3,7. The upstream maintainers will just have
to be careful to follow 4,5,6 and all will be well.

(libst-1.3-version.patch) is incremental on libst-1.3-fpic.patch

One caveat: I added -lc to the link line so that package tools detect that
it depends on libc (which it does).

--

Finally! Now libst is able to be properly packaged. Here is a patch that
adds those files to build it as a deb and rpm. Not sure if you want this in
the upstream version or not... Sometimes it's nice, but it's not crucial.

(libst-1.3-packaged.patch) is incremental on libst-1.3-version.patch

As before:
	cd st-1.3c; debuild -> makes debs
	rpm -ta st-1.3c.tar.gz -> makes rpms

--

Summary:
	Patches libst-1.3-(fpic&version).patch should be applied to the tree
after the patch you already made. (they are bug fixes)
	The versioning policy constraints 4,5,6 should be adhered to.
	libst-1.3-packaged.patch may be optionally applied.

A tarball with all the patches applied is at:
<http://www.pez.ca/terpstra/st-1.3.0.tar.gz>

-- 
Wesley W. Terpstra <we...@te...>

[ST-devel] packaging libst: patch

[Gene S. <gs...@ab...>]

Claus Assmann wrote:

> What about the software that uses state-threads?  I would like to
> use it (even though I haven't figured out the license issues) and
> I would prefer if the entire system just "automagically" builds on
> many OS. Hence I appreciate the efforts to use autoconf. However,
> I agree that there should be no performance impact, i.e., runtime
> decisions what to use. Maybe some "use these switches on this OS"
> cases can be integrated? Currently that's done by hand by selecting
> the correct OS target in the Makefile.

I'm attaching a small patch to the 1.3c version which is supposed
to facilitate building ST as a part of a larger software project.
This patch allows to eliminate manual selection of target OS (if
so desired).  It doesn't affect current mechanism, it just adds
new targets.


CHANGES

- 3 lines in Makefile. New targets have been added: "default",
  "default-debug", and "default-optimized" (same as "default").
  Also, a symlink to the output directory is created at build
  time. That allows to use the library and header file without
  knowing the output directory name (which is OS-dependent) in
  advance.

- New small script "osguess.sh" to automatically guess target OS.
  This script requires the "config.guess" utility (a part of GNU
  Autoconf) to do actual system info guessing.
  "config.guess" itself is not included with ST.  You can use one
  from a larger "main" software project (if it uses Autoconf) or
  just use any config.guess available on your system.  You can
  also get it directly from GNU: ftp://ftp.gnu.org/gnu/autoconf/
  (same approach as with any other GNU tools).


USAGE

Specify path to the "config.guess" utility -- set the
CONFIG_GUESS_PATH variable either by editing "osguess.sh" or via
environment.  Use "default" as a make target (make default).

Hope it solves some of the problems.