Menu ▾ ▴

#5 error running SCO coff on RHEL8

1.0
open
Russell Stuart
None
2021-10-06
2020-08-04
Rod Sheppard
No

on RHEL 8 I am getting the below error when trying to execute a SCO coff binary?

[root@rhel80t21 bin]# ibcs-us /u/catcom/bin/csum.coff
<3>mm->brk does not lie within mmap/u/catcom/bin/csum.coff: Exec format error.

Related

Tickets: #5

Discussion

  • Russell Stuart

    Russell Stuart - 2020-08-04

    Can you give me a copy of the binary (eg, email), csum.coff?

     

  • Russell Stuart

    Russell Stuart - 2020-08-04
    • assigned_to: Russell Stuart
     

  • Rod Sheppard

    Rod Sheppard - 2020-08-04

    csum.coff (attached) Usage: csum filename
    It generates a check sum of filename or stdin of no param
    I also get the error with all other coff binaries
    the ibcs-us version is 4.1.5

    ibcs-us csum.coff works if run on rhel 7
    Linux rhel72t21 3.10.0-1127.13.1.el7.x86_64 #1 SMP Fri Jun 12 14:34:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux

    the kernel for rhel 8 is
    Linux rhel80t21 4.18.0-193.14.3.el8_2.x86_64 #1 SMP Mon Jul 20 15:02:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

     
    csum.coff

  • Sidao

    Sidao - 2020-09-06

    Hello. Some news about this topic ? I am having the some trouble.

     

  • Russell Stuart

    Russell Stuart - 2020-10-07

    On 7/10/20 3:57 pm, Rodney Sheppard wrote:

    > FYI
> I've tried compiling on RHEL 8 with the below result....
> In file included from ./misc.c:9:
> ./misc.c: In function 'xnx_eaccess':
> ./misc.c:311:31: error: 'AT_FDCWD' undeclared (first use in this function)
>      return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
> ^~~~~~~~
> ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of 
> macro 'SYS'
>   #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
> ^~~~
> ./misc.c:311:31: note: each undeclared identifier is reported only once 
> for each function it appears in
>      return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
> ^~~~~~~~
> ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of 
> macro 'SYS'
>   #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
> ^~~~
> ./misc.c:311:53: error: 'AT_EACCESS' undeclared (first use in this 
> function); did you mean 'EACCES'?
>      return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
> ^~~~~~~~~~
> ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of 
> macro 'SYS'
>   #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
> ^~~~
> ./misc.c:334:1: error: control reaches end of non-void function 
> [-Werror=return-type]

    That looks like it would not be too difficult to fix, but I doubt it would help as it looks unrelated to the reported problem.

    I'll have to do the same thing I did last time: reproduce it in the RHEL 8 environment, somehow.

     

  • Rod Sheppard

    Rod Sheppard - 2020-10-07

    If you need access to a dev/test RHEL 8 system, I can setup a user rstuart with your pub key from last time, and open a port to it.

    Rod
    -- reply above this line --

    Rodney Sheppard
    Software Engineer - Development Manager
    Trans-Mit Pty. Ltd.
    7 Wendy Court Hampton Park,
    Vic 3976, Australia
    Phone... +61 3 87878744
    Email...... rod@transmit.com.au
    Web....... www.transmit.com.au

    ----- Original Message -----
    From: Russell Stuart
    To: [ibcs-us:tickets]
    Sent: Wednesday, October 07, 2020 5:12 PM
    Subject: [ibcs-us:tickets] #5 error running SCO coff on RHEL8

    On 7/10/20 3:57 pm, Rodney Sheppard wrote:

    FYI
    I've tried compiling on RHEL 8 with the below result....
    In file included from ./misc.c:9:
    ./misc.c: In function 'xnx_eaccess':
    ./misc.c:311:31: error: 'AT_FDCWD' undeclared (first use in this function)
    return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
    ^~~~~~~~
    ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of
    macro 'SYS'
    #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
    ^~~~
    ./misc.c:311:31: note: each undeclared identifier is reported only once
    for each function it appears in
    return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
    ^~~~~~~~
    ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of
    macro 'SYS'
    #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
    ^~~~
    ./misc.c:311:53: error: 'AT_EACCESS' undeclared (first use in this
    function); did you mean 'EACCES'?
    return (int)SYS(faccessat, AT_FDCWD, path, mode, AT_EACCESS);
    ^~~~~~~~~~
    ../include/ibcs-us/ibcs/linux26-compat.h:44:59: note: in definition of
    macro 'SYS'
    #define SYS(name, args...) linux26_syscall(__NR_##name, ##args)
    ^~~~
    ./misc.c:334:1: error: control reaches end of non-void function
    [-Werror=return-type]
    That looks like it would not be too difficult to fix, but I doubt it would help as it looks unrelated to the reported problem.

    I'll have to do the same thing I did last time: reproduce it in the RHEL 8 environment, somehow.

    [tickets:#5] error running SCO coff on RHEL8

    Status: open
    Milestone: 1.0
    Created: Tue Aug 04, 2020 06:28 AM UTC by Rod Sheppard
    Last Updated: Sun Sep 06, 2020 09:08 PM UTC
    Owner: Russell Stuart

    on RHEL 8 I am getting the below error when trying to execute a SCO coff binary?

    [root@rhel80t21 bin]# ibcs-us /u/catcom/bin/csum.coff
    <3>mm->brk does not lie within mmap/u/catcom/bin/csum.coff: Exec format error.

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/ibcs-us/tickets/5/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #5

    alternate

  • mlthati

    mlthati - 2020-11-06 
        Dear Russell Stuart Sir,
    Please can you guide or provide any document to compile ibcs-us on RHEL 7.8 i.e 3.10 kernel to run sco based foxplus.
    Thanks in advance
     

  • Russell Stuart

    Russell Stuart - 2021-01-06

    At long last I've had a chance to have a look at this. An mmap() of a file to address 0 is returning EPERM on RHEL 8, whereas it succeeds on every other Linux if you are root, or have the executable has CAP_SYS_RAW_IO.

    The next step is normally to look at the kernel source to see if its possible to work around this. (That's how I learnt ibcs-us needs CAP_SYS_RAW_IO). Failing that with the source you could even patch the kernel and recompile. However, in a change as of RHEL 8, RedHat doesn't appear to make the kernel source, or indeed any source (ie srpm) available to non-customers. Apparently customers can download the sources from https://access.redhat.com/

    I am not a customer, so no source for me. Unless someone can get a me copy this is where this bug investigation stops.

    I hadn't really considered it before, but this makes me realise when open source ceases to be completely open, it loses some of its usefulness.

     

    • Russell Stuart

      Russell Stuart - 2021-01-08

      An mmap() of a file to address 0 is returning EPERM on RHEL 8, whereas it succeeds on every other Linux if you are root, or have the executable has CAP_SYS_RAW_IO.

      For anyone following along:

      Turns out that isn't what's going on. Instead the mmap() is failing because it's asking for both write and execute permissions on the mapped memory, ie asking PROT_EXEC|PROT_WRITE. That works on a standard kernel, but a program with those permissions can be a little easier to exploit easier so there are kernel patches out there such as PaX that disallow it. I guessing RHEL 8 has added one of these patches (but as I don't have the source it's impossible to be sure.)

      However, unlike a "you may not mmap() to address 0" prohibition this may not be so hard to work around, albeit at the expense of slowing things down. I'll have a look at some point.

       

  • Rod Sheppard

    Rod Sheppard - 2021-01-06

    Thanks Russell, I will see if I can get access.

    -- reply above this line --

    Rodney Sheppard
    Software Engineer - Development Manager
    Trans-Mit Pty. Ltd.
    7 Wendy Court Hampton Park,
    Vic 3976, Australia
    Phone... +61 3 87878744
    Email...... rod@transmit.com.au
    Web....... www.transmit.com.au

    ----- Original Message -----
    From: Russell Stuart
    To: [ibcs-us:tickets]
    Sent: Wednesday, January 06, 2021 9:21 PM
    Subject: [ibcs-us:tickets] #5 error running SCO coff on RHEL8

    At long last I've had a chance to have a look at this. An mmap() of a file to address 0 is returning EPERM on RHEL 8, whereas it succeeds on every other Linux if you are root, or have the executable has CAP_SYS_RAW_IO.

    The next step is normally to look at the kernel source to see if its possible to work around this. (That's how I learnt ibcs-us needs CAP_SYS_RAW_IO). Failing that with the source you could even patch the kernel and recompile. However, in a change as of RHEL 8, RedHat doesn't appear to make the kernel source, or indeed any source (ie srpm) available to non-customers. Apparently customers can download the sources from https://access.redhat.com/

    I am not a customer, so no source for me. Unless someone can get a me copy this is where this bug investigation stops.

    I hadn't really considered it before, but this makes me realise when open source ceases to be completely open, it loses some of its usefulness.

    [tickets:#5] error running SCO coff on RHEL8

    Status: open
    Milestone: 1.0
    Created: Tue Aug 04, 2020 06:28 AM UTC by Rod Sheppard
    Last Updated: Fri Nov 06, 2020 10:29 AM UTC
    Owner: Russell Stuart

    on RHEL 8 I am getting the below error when trying to execute a SCO coff binary?

    [root@rhel80t21 bin]# ibcs-us /u/catcom/bin/csum.coff
    <3>mm->brk does not lie within mmap/u/catcom/bin/csum.coff: Exec format error.

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/ibcs-us/tickets/5/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #5

    alternate

  • Rob Pomeroy

    Rob Pomeroy - 2021-10-06

    Commenting to subscribe to issue - same (?) scenario, "Exec format error" with SCO/80386 COFF executable on RHEL 8. Working fine on Ubuntu 20.04.

     

Log in to post a comment.

MongoDB Logo MongoDB