#1911 The interpreter is using too small of a stacksize.

[Rick McGuire]

The build of the inerpreter uses a 500Kb stack size that is hardcoded into the PlatformDefinitions.h file for both Windows and *ix platforms. This size is also specified as the /STACK option for linking the .exe files on Windows. This value is actually smaller than the default 1Mb limit that Windows uses. This should be reworked so that the value used comes from the CMakeList.txt definitions and is inline with more modern limits.

[Rick McGuire]

Fix committed [12696]

[Per Olov Jonsson]

This commit triggered some errors on Linux platforms (Ubuntu, Debian) there might be more coming but this is essentially where it breaks

[ 63%] Building CXX object CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o
/home/osboxes/workspace/ooRexx-Debian11-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function ‘static char SysActivity::getStackBase()’:
/home/osboxes/workspace/ooRexx-Debian11-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:149:19: error: ‘pthread_get_stacksize_np’ was not declared in this scope; did you mean ‘pthread_getname_np’?
149 | size_t size = pthread_get_stacksize_np(pthread_self());
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getname_np
/home/osboxes/workspace/ooRexx-Debian11-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:152:20: error: ‘pthread_get_stackaddr_np’ was not declared in this scope; did you mean ‘pthread_getattr_np’?
152 | return (char )pthread_get_stackaddr_np(thread) - size;
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getattr_np
/home/osboxes/workspace/ooRexx-Debian11-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function ‘static size_t SysActivity::getStackSize()’:
/home/osboxes/workspace/ooRexx-Debian11-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:179:12: error: ‘pthread_get_stacksize_np’ was not declared in this scope; did you mean ‘pthread_getname_np’?
179 | return pthread_get_stacksize_np(pthread_self());
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getname_np
make[2]: *** [CMakeFiles/rexx.dir/build.make:2565: CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:504: CMakeFiles/rexx.dir/all] Error 2
make: *** [Makefile:171: all] Error 2
Build step 'Execute shell' marked build as failure
Archiving artifacts
Finished: FAILURE

[Rick McGuire]

Linux build break fixed [r12697]

[Rick McGuire]

Additional fix [r12698]

[Per Olov Jonsson]

This now works on all Windows (7,8,10,11) and all macOS (10,11,12,13) including macOS Ventura (=13) on M1 hardware. Unfortunately all Linux platforms fail at

[ 63%] Building CXX object CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o
/home/osboxes/workspace/ooRexx-Ubuntu22-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function ‘static char SysActivity::getStackBase()’:
/home/osboxes/workspace/ooRexx-Ubuntu22-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:153:19: error: ‘pthread_get_stacksize_np’ was not declared in this scope; did you mean ‘pthread_getname_np’?
153 | size_t size = pthread_get_stacksize_np(pthread_self());
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getname_np
/home/osboxes/workspace/ooRexx-Ubuntu22-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:156:20: error: ‘pthread_get_stackaddr_np’ was not declared in this scope; did you mean ‘pthread_getattr_np’?
156 | return (char )pthread_get_stackaddr_np(thread) - size;
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getattr_np
/home/osboxes/workspace/ooRexx-Ubuntu22-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function ‘static size_t SysActivity::getStackSize()’:
/home/osboxes/workspace/ooRexx-Ubuntu22-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:183:12: error: ‘pthread_get_stacksize_np’ was not declared in this scope; did you mean ‘pthread_getname_np’?
183 | return pthread_get_stacksize_np(pthread_self());
| ^~~~~~~~~~~~~~~~~~~~~~~~
| pthread_getname_np
make[2]: *** [CMakeFiles/rexx.dir/build.make:2750: CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:370: CMakeFiles/rexx.dir/all] Error 2
make: *** [Makefile:156: all] Error 2
Build step 'Execute shell' marked build as failure

I think/guess that pthread_get_stacksize_np does not exist for Linux and that another function should be used instead. I read about pthread_attr_getstacksize() that appear to be for the same purpose, could that do the trick on Linux? see

https://unix.stackexchange.com/questions/127602/default-stack-size-for-pthreads

If it helps I can provide access to the Linux VMs using ssh, please send me a mail off the list for details or ask Erich.

[Rick McGuire]

Yes, that section of the code should only be used for the Mac. For some reason, the define HAVE_PTHREAD_GETATTR_NP does not seem to be getting set, which was why I was asking about the contents of config.h. The log does not show the details of that part of the build, unfortunately, so I want to know if the Cmake configuration is detecting if pthread_getattr_np() is available or not. I do not want to access any of these machines directly.

[Erich]

Hi Rick,
the compile issues are with these two functions in the non-Apple (ie. Linux) code path
pthread_get_stacksize_np()
pthread_get_stackaddr_np()

I assume what's meant to be used here is
pthread_attr_getstacksize()
pthread_attr_getstackaddr()

Also the pthread_get_stacksize_np() issue on the main thread in Mac OS X seems to be rather old.
This worked in 10.6, 10.7 and 10.8, but didn't in 10.9 (see https://mail.openjdk.org/pipermail/hotspot-dev/2013-October/011369.html)
I cannot reproduce the issue in our current MacOS 11.2.2 and 12.6.5 clients.
A printf("stack %zd\n", pthread_get_stacksize_np(pthread_self())); in the main thread shows stack 8388608 for both.

I'd vote to remove the workaround for MacOS.

[Rick McGuire]

On Wed, Jul 5, 2023 at 1:37 PM Erich erich_st@users.sourceforge.net wrote:

Hi Rick,
the compile issues are with these two functions in the non-Apple (ie.
Linux) code path
pthread_get_stacksize_np()
pthread_get_stackaddr_np()

Yes, but that's only a symptom of the problem. The conditional compile
should be triggering off of the HAVE_PTHREAD_GETATTR_NP define,
which should be getting set in config.h. I'm trying to figure out why that
isn't working. I suspect it's not getting set for some reason, but can't
figure out why. I did find this Stackoverflow item on a similar problem:

https://stackoverflow.com/questions/55317131/cmake-check-function-exists-for-a-function-requiring-define

But this didn't work, and I noticed today, it was already being used for
the HAVE_PTHREAD_MUTEX_TIMEDLOCK definition immediately preceding the ones
I added. This is why I would like to see the config.h and maybe the CMake
log to see if there's any additional information in there.

Rick

I assume what's meant to be used here is
pthread_attr_getstacksize()
pthread_attr_getstackaddr()

Also the pthread_get_stacksize_np() issue on the main thread in Mac OS X
seems to be rather old.
This worked in 10.6, 10.7 and 10.8, but didn't in 10.9 (see
https://mail.openjdk.org/pipermail/hotspot-dev/2013-October/011369.html)
I cannot reproduce the issue in our current MacOS 11.2.2 and 12.6.5
clients.
A printf("stack %zd\n", pthread_get_stacksize_np(pthread_self())); in
the main thread shows stack 8388608 for both.

I'd vote to remove the workaround for MacOS.

---

**[bugs:#1911] The interpreter is using too small of a stacksize. **

Status: pending
Group: 5.1.0
Created: Tue Jun 27, 2023 12:06 AM UTC by Rick McGuire
Last Updated: Wed Jul 05, 2023 01:26 PM UTC
Owner: nobody

The build of the inerpreter uses a 500Kb stack size that is hardcoded into
the PlatformDefinitions.h file for both Windows and *ix platforms. This
size is also specified as the /STACK option for linking the .exe files on
Windows. This value is actually smaller than the default 1Mb limit that
Windows uses. This should be reworked so that the value used comes from the
CMakeList.txt definitions and is inline with more modern limits.

---

Sent from sourceforge.net because you indicated interest in <
https://sourceforge.net/p/oorexx/bugs/1911/>

To unsubscribe from further messages, please visit <
https://sourceforge.net/auth/subscriptions/>

[Erich]

Additional fix and test committed with [r12701].
This should cover Linux, MacOS, and the BSDs.

I noticed that the default stacksize on FreeBSD is 512 MB. For large stacksizues like this the interpreter becomes slower and slower and cannot exhaust the stack in any reasonable time.
Also unwinding the stack as in the newly committed CALL testgroup test gets exponentially slower.

[Erich]

fix typo with revision [r12702]

[Per Olov Jonsson]

There are still two platforms, OpenBSD and OpenIndiana where this bug has not been fixed. The errors seems a bit different, I found a link that might help resolve the issue

https://www.openwall.com/lists/musl/2013/03/31/3

OpenBSD

[ 63%] Building CXX object CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o
/home/osboxes/workspace/ooRexx-OpenBSD7_2-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:153:19: error: use of undeclared identifier 'pthread_get_stacksize_np'
size_t size = pthread_get_stacksize_np(pthread_self());
^
/home/osboxes/workspace/ooRexx-OpenBSD7_2-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:156:20: error: use of undeclared identifier 'pthread_get_stackaddr_np'
return (char *)pthread_get_stackaddr_np(thread) - size;
^
/home/osboxes/workspace/ooRexx-OpenBSD7_2-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:183:12: error: use of undeclared identifier 'pthread_get_stacksize_np'
return pthread_get_stacksize_np(pthread_self());
^

OpenIndiana (OS=SunOS)

[ 63%] Building CXX object CMakeFiles/rexx.dir/interpreter/platform/unix/SysActivity.cpp.o
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:164:2: error: #error no code for getStackBase()
164 | #error no code for getStackBase()
| ^~~~~
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:199:2: error: #error no code for getStackSize()
199 | #error no code for getStackSize()
| ^~~~~
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function 'static char* SysActivity::getStackBase()':
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:166:1: warning: no return statement in function returning non-void [-Wreturn-type]
166 | }
| ^
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp: In static member function 'static std::size_t SysActivity::getStackSize()':
/export/home/osboxes/workspace/ooRexx-OpenIndiana-build/oorexxSVN/interpreter/platform/unix/SysActivity.cpp:201:1: warning: no return statement in function returning non-void [-Wreturn-type]
201 | }
| ^
*** Error code 1

[Per Olov Jonsson]

Possibly as a side effect of this fix ALL Windows 64 bit (W7, W8.1, W10 and W11) all break the execution at this testgroup:

Executing ...\ooRexx\base\keyword\CALL.testGroup
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE

The testing really breaks out without completing the summary .

<off topic="">
32 bit Windows pass all test but the VMs hang at the end of executing the test suite, probably a lost "ticker" due to time jitter in the VM preventing the suite to exit properly. This is a known problem (without a solution until now) but manageable since the physical Windows machine does not show this behavior. </off>

[Erich]

Committed test group workaround/fix with revision [r12704]