I think PTlib has a bug in P_fd_set when running with a raised file limit ( above FD_SET_SIZE ) on glibc with _FORTIFY_SOURCE=2 ( which seems to be the default now ). Description is a bit lengthy as it's a complex issue.
Note I found the problem in PTLIB 2.10.9 with other programs / libraries, but I've checked the trunk to the best of my limited knowledge and the problem seems to be there too and I think it will affect opal users too, so I'm reporting it here.
I encountered it with a program ( yate ) which uses h323plus and PTlib 2.10.9 running under Gentoo Linux with glibc 2.15 and under Ubuntu 1304 with glibc 2.17, with the process file limit raised to 8192 ( it's a high traffic instance ). As soon as I had about 130 simultaneous calls my program died with the following stack trace:
Program terminated with signal 6, Aborted.
#0 0x00007fd155b60f35 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007fd155b60f35 in raise () from /lib64/libc.so.6
#1 0x00007fd155b623b8 in abort () from /lib64/libc.so.6
#2 0x00007fd155b9fc2b in ?? () from /lib64/libc.so.6
#3 0x00007fd155c28147 in __fortify_fail () from /lib64/libc.so.6
#4 0x00007fd155c26130 in __chk_fail () from /lib64/libc.so.6
#5 0x00007fd155c280f7 in __fdelt_warn () from /lib64/libc.so.6
#6 0x00007fd15488d227 in P_fd_set::operator=(int) () from /home/h323plus/lib/libpt.so.2.10.9
#7 0x00007fd154873a23 in PThread::PXBlockOnIO(int, int, PTimeInterval const&) ()
from /home/h323plus/lib/libpt.so.2.10.9
#8 0x00007fd15486de84 in PChannel::PXSetIOBlock(PChannel::PXBlockType, PTimeInterval const&) ()
from /home/h323plus/lib/libpt.so.2.10.9
#9 0x00007fd154869b5f in PTCPSocket::Read(void*, int) () from /home/h323plus/lib/libpt.so.2.10.9
#10 0x00007fd15486cae0 in PIndirectChannel::Read(void*, int) ()
from /home/h323plus/lib/libpt.so.2.10.9
#11 0x00007fd15486c054 in PChannel::ReadChar() () from /home/h323plus/lib/libpt.so.2.10.9
#12 0x00007fd154f4f6b7 in H323TransportTCP::ReadPDU(PBYTEArray&) ()
from /home/h323plus/lib/libh323_linux_x86_64_.so.1.25.0
#13 0x00007fd154f291b6 in H323Connection::HandleControlChannel() ()
from /home/h323plus/lib/libh323_linux_x86_64_.so.1.25.0
#14 0x00007fd154873819 in PThread::PX_ThreadStart(void*) () from /home/h323plus/lib/libpt.so.2.10.9
#15 0x00007fd156a43f4a in start_thread () from /lib64/libpthread.so.0
#16 0x00007fd155c1298d in clone () from /lib64/libc.so.6
(gdb)
After lots of digging thorugh PTlib's sources and system include files I think this problem is because P_fd_set uses the usual idiom for enlarging the fd_set:
void P_fd_set::Construct()
{
max_fd = PProcess::Current().GetMaxHandles();
set = (fd_set *)malloc((max_fd+7)>>3);
}
but then uses FD_SET to set bits, which, when FORTIFY_SOURCE is active, checks the descriptor index with the following code ( from /usr/include/bits/select2.h ):
extern long int __fdelt_warn (long int __d)
__warnattr ("bit outside of fd_set selected");
#undef __FD_ELT
#define __FD_ELT(d) \
__extension__ \
({ long int __d = (d); \
(__builtin_constant_p (__d) \
? (0 <= __d && __d < __FD_SETSIZE \
? (__d / __NFDBITS) \
: __fdelt_warn (__d)) \
: __fdelt_chk (__d)); })
Leading to my problem as soon as I have and fd > FD_SETSIZE.
P_fd_set::Zero() uses a memset, which is not catched with FORTIFY_SOURCE because GCC does not know the chunk size int it.
I have a couple of workarounds, one is rebuilding PTlib without FORTIFY_SOURCE, the other, for which I'm developing and testing a patch, is using an array of fd_sets in construct ( using set = calloc(sizeof(fd_set), (max_fd+FD_SET_SIZE-1)/FD_SETSIZE ) and using FD_SET(fd % FD_SETSIZE) on element fd / FD_SETSIZE, which I think should work with FORTIFY_SOURCE. This can lead to some extra used space, but in my experience FD_SETSIZE tends to be 1024 or a similar power of two, and when the limit is raised it's typically done to a large value, and also to a multiple of it, so it shouldn't be a big issue. If there is interest in it I can contribute it as soon as I have it tested.
Also, I think the initialization in ::Construct uses an incorrect size, as normally fd_sets are arrays of int or longs, in fact in my /usr/include/sys/select.h file I have:
/* The fd_set member is required to be an array of longs. */
typedef long int __fd_mask;
And the FD_SET/CLEAR/ISSET macros use __fd_mask elements, so the byte aligned malloc
set = (fd_set *)malloc((max_fd+7)>>3);
in construct allocates less than needed, specially in big-endian architectures, although this problem is masked because normally malloc pads & aligns for long or greater. Problem will typically be solved by using ((max_fd+31)>>5)<<2, but as long must be at least 32 bits, but can be greater, I'll suggest to play it safe and use 64 or even 128 bits chunks.
