#3571 sdcpp preprocessor doesn't check it's working dir for cc1, only checks system PATH

[bbbbbr]

SDCC on Linux x64 built from source at revision 13911

When sdcc invokes sdcpp it appears to include it's current executable path. That has worked well and continues to.

However in the new behavior where sdcpp invokes cc1 it does not include the current executable path and so it can only find and run cc1 if the path for it is part of the system PATH.

It would be preferable for cc1 to get invoked with the same path behavior as sdcc uses for sdcpp.

$ strace "../../../"bin/sdcc -msm83 --no-std-crt0 --fsigned-char --use-stdout -D__PORT_sm83 -D__TARGET_gb -Wa-pogn -I"../../../"lib/gb -DINT_16_BITS -I"../../../"include -c main.c -o main.o

execve("../../../bin/sdcc", ["../../../bin/sdcc", "-msm83", "--no-std-crt0", "--fsigned-char", "--use-stdout", "-D__PORT_sm83", "-D__TARGET_gb", "-Wa-pogn", "-I../../../lib/gb", "-DINT_16_BITS", "-I../../../include", "-c", "main.c", "-o", "main.o"], 0x7fff700237e0 /* 54 vars */) = 0
...
access("../../../bin/sdcpp", X_OK)      = 0
...
read(4, sdcpp: fatal error: cannot execute 'cc1': execvp: No such file or directory

compilation terminated.
"", 8192)                       = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=794286, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---
ioctl(4, TCGETS, 0x7ffd20d31140)        = -1 ENOTTY (Inappropriate ioctl for device)
write(2, "at 1: warning 190: ISO C forbids"..., 59at 1: warning 190: ISO C forbids an empty translation unit
) = 59
close(4)                                = 0
wait4(794286, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 794286
write(2, "subprocess error 256\n", 21subprocess error 256
)  = 21
exit_group(1)                           = ?
+++ exited with 1 +++

[Felix]

Thanks for the bug report.

However in the new behavior where sdcpp invokes cc1 it does not include the current executable path and so it can only find and run cc1 if the path for it is part of the system PATH.

This does not seem to be the case for me.

Somewhat replicating the test, I get the following.

$ strace -f sdcc file.c  |& grep execve
execve("/usr/local/bin/sdcc", ["sdcc", "file.c"], 0x7ffddfd05fe0 /* 61 vars */) = 0
[pid 29873] execve("/bin/sh", ["sh", "-c", "/usr/local/bin/sdcpp -nostdinc -"...], 0x7ffc75e84c80 /* 61 vars */ <unfinished ...>
[pid 29873] <... execve resumed>)       = 0
[pid 29874] execve("/usr/local/bin/sdcpp", ["/usr/local/bin/sdcpp", "-nostdinc", "-Wall", "-std=c11", "--obj-ext=.rel", "-D__SDCC_CHAR_UNSIGNED", "-D__SDCC_MODEL_SMALL", "-D__SDCC_FLOAT_REENT", "-D__SDCCCALL=0", "-D__SDCC=4_2_14", "-D__SDCC_VERSION_MAJOR=4", "-D__SDCC_VERSION_MINOR=2", "-D__SDCC_VERSION_PATCH=14", "-DSDCC=4214", "-D__SDCC_REVISION=13909", "-D__SDCC_mcs51", "-D__STDC_NO_COMPLEX__=1", "-D__STDC_NO_THREADS__=1", "-D__STDC_NO_ATOMICS__=1", "-D__STDC_NO_VLA__=1", "-D__STDC_ISO_10646__=201409L", "-D__SIZEOF_FLOAT__=4", "-D__SIZEOF_DOUBLE__=4", "-D__SDCC_BITINT_MAXWIDTH=64", "-isystem", "/usr/local/bin/../share/sdcc/inc"..., "-isystem", "/usr/local/share/sdcc/include/mc"..., "-isystem", "/usr/local/bin/../share/sdcc/inc"..., "-isystem", "/usr/local/share/sdcc/include", ...], 0x55cae11162d8 /* 61 vars */ <unfinished ...>
[pid 29874] <... execve resumed>)       = 0
[pid 29875] execve("/usr/local/libexec/sdcc/x86_64-pc-linux-gnu/12.1.0/cc1", ["/usr/local/libexec/sdcc/x86_64-p"..., "-E", "-quiet", "-nostdinc", "-D", "__SDCC_CHAR_UNSIGNED", "-D", "__SDCC_MODEL_SMALL", "-D", "__SDCC_FLOAT_REENT", "-D", "__SDCCCALL=0", "-D", "__SDCC=4_2_14", "-D", "__SDCC_VERSION_MAJOR=4", "-D", "__SDCC_VERSION_MINOR=2", "-D", "__SDCC_VERSION_PATCH=14", "-D", "SDCC=4214", "-D", "__SDCC_REVISION=13909", "-D", "__SDCC_mcs51", "-D", "__STDC_NO_COMPLEX__=1", "-D", "__STDC_NO_THREADS__=1", "-D", "__STDC_NO_ATOMICS__=1", ...], 0x1727830 /* 63 vars */ <unfinished ...>
[pid 29875] <... execve resumed>)       = 0

My interpretation is that cc1 is found in /usr/local/libexec/sdcc/x86_64-pc-linux-gnu/12.1.0/. This is neither part of the system PATH, nor meant to be. IIRC, PATH will be used as a fallback if cc1 is not where it should be. (But perhaps the fallback should be removed?)

FWIW, this behaviour is inerited from gcc/cpp. With sdcpp replaced by cpp in the above, it looks similar, as follows.

$ strace -f cpp file.c |& grep execve
execve("/usr/bin/cpp", ["cpp", "file.c"], 0x7ffc068ddb60 /* 61 vars */) = 0
[pid 14792] execve("/usr/lib/gcc/x86_64-linux-gnu/12/cc1", ["/usr/lib/gcc/x86_64-linux-gnu/12"..., "-E", "-quiet", "-imultiarch", "x86_64-linux-gnu", "file.c", "-mtune=generic", "-march=x86-64", "-fasynchronous-unwind-tables", "-dumpbase", "file.c", "-dumpbase-ext", ".c"], 0x896ba0 /* 65 vars */ <unfinished ...>
[pid 14792] <... execve resumed>)       = 0

It would be preferable for cc1 to get invoked with the same path behavior as sdcc uses for sdcpp.

The cc1 executable is not "next to" sdcpp in the way that sdcpp is expected to be in the same directory as sdcc. It is not even meant to be a public executable. "The same" approach does not apply, but a modification might work.

[bbbbbr]

Thanks for the response.

My interpretation is that cc1 is found in /usr/local/libexec/sdcc/x86_64-pc-linux-gnu/12.1.0/. This is neither part of the system PATH, nor meant to be. IIRC, PATH will be used as a fallback if cc1 is not where it should be.

If I understand right, "where it should be" means an absolute system path for finding cc1 is hardwired into the sdcpp executable. Looking at the snapshot build packages I do see they have various different os/architecture dependent paths for where cc1 is found.

Maybe based off this from the /cpp/gcc/ Makefile:
libexecsubdir = $(libexecdir)/sdcc/$(real_target_noncanonical)/$(version)$(accel_dir_suffix)?

Anyhow - with previous versions of SDCC it has been very convenient to be able to bundle all sdcc binaries into a self-contained single folder and have it work in a arbitrary location (without having to modify anything outside that location in the system).

It seems this usage style might be possible again if sdcpp could also check it's own working path (or a relative sub-folder of that path) when trying to locate cc1.

(For context, I have various versions of SDCC bundled with differing versions of GBDK which rely on those specific SDCC versions. They are not installed to formal system-wide locations.)

(But perhaps the fallback should be removed?)

Not sure - it does seem that having path be used as a fallback might lead to unpredictable behavior. If it did not emit a warning and was sourcing an alternate CC1 with some substantive difference between the version expected and the version found / used.

[bbbbbr]

Update- Looks like there are some paths relative to sdcpp which it will use to search for cc1.

One of them looks suitable for our purposes so far:

• "[PATH TO SDCPP]../libexec/sdcc/cc1"

We'll check across different OS platforms to see if that is uniformly available.

Edit: Tested on Windows and sdcpp only finds cc1 under the relative path if the .exe is not part of the executable name. That's kind of funny. :D
Works: ..\libexec\sdcc\cc1
Does not work: ..\libexec\sdcc\cc1.exe

[pid 939992] stat("gbdk-2020-git/build/gbdk/bin/../libexec/sdcc/x86_64-pc-linux-gnu/12.1.0/cc1", 0x7fff72b6ba40) = -1 ENOENT (No such file or directory)
[pid 939992] stat("gbdk-2020-git/build/gbdk/bin/../libexec/sdcc/cc1", 0x7fff72b6ba40) = -1 ENOENT (No such file or directory)
[pid 939992] stat("gbdk-2020-git/build/gbdk/bin/../lib/sdcc/x86_64-pc-linux-gnu/12.1.0/../../../../x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu/12.1.0/cc1", 0x7fff72b6ba40) = -1 ENOENT (No such file or directory)
[pid 939992] stat("gbdk-2020-git/build/gbdk/bin/../lib/sdcc/x86_64-pc-linux-gnu/12.1.0/../../../../x86_64-pc-linux-gnu/bin/cc1", 0x7fff72b6ba40) = -1 ENOENT (No such file or directory)

[Tony Pavlov]

i agree with @bbbbbr, ability to make it run from the arbitrary folder is very important. any fixed place means it will require "installation" and not "portable" anymore, but that is crucial for the projects like GBStudio.

[Felix]

One of them looks suitable for our purposes so far: * "[PATH TO SDCPP]../libexec/sdcc/cc1"

Well spotted. Looking closer (strace .. | grep stat), it appears that all those "hardwired" cc1 locations are relative to the sdcpp binary executable -- whereever you put it.

Tony: This ought to be sufficient for relocation as part of GBStudio. Can you confirm?

[bbbbbr]

I can't speak for Tony with certainty, but usually if it works for normal GBDK installs then it will work for GBStudio (which bundles GBDK). So it should be ok.

We can do further testing and see if issues arise.

For the cc1 ".exe" extension behavior on Windows- I think my earlier testing showed it worked ok to have the binary in that relative path without the ".exe" extension.

[Maarten Brock]

Can the .exe problem please be fixed? How does gcc/cpp handle this? Does it use a cc1 without the .exe extension?

[Felix]

According to bbbbbr, this is working ok, in particular the .exe case. I'm not sure what else we could do here.

[Deqing Sun]

I just realized SDCC is going to release a new version and I downloaded the new version.
Here is my test with sdcc-snapshot-i586-mingw32msvc-20230531-14093.zip:
I unziped ths snapshot and put it in c:\TEMP

c:\TEMP>echo > empty.c

c:\TEMP>sdcc\bin\sdcc empty.c
sdcpp.exe: fatal error: cannot execute 'cc1': CreateProcess: No such file or directory
compilation terminated.
at 1: warning 190: ISO C forbids an empty translation unit
subprocess error 1

c:\TEMP>sdcc\bin\sdcc.exe empty.c
sdcpp.exe: fatal error: cannot execute 'cc1': CreateProcess: No such file or directory
compilation terminated.
at 1: warning 190: ISO C forbids an empty translation unit
subprocess error 1

So on my machine, the CC1 bug is not fixed.

[Felix]

On Thu, Jun 01, 2023 at 01:53:27PM -0000, Deqing Sun wrote:

I just realized SDCC is going to release a new version and I downloaded the new version.
Here is my test with sdcc-snapshot-i586-mingw32msvc-20230531-14093.zip:
I unziped ths snapshot and put it in c:\TEMP
~~~
c:\TEMP>echo > empty.c

c:\TEMP>sdcc\bin\sdcc empty.c
sdcpp.exe: fatal error: cannot execute 'cc1': CreateProcess: No such file or directory
compilation terminated.
at 1: warning 190: ISO C forbids an empty translation unit
subprocess error 1

Thanks Deqing for testing.

I have looked into the zip file [1]. It seems there is no cc1, but a
cc1.exe. I do not know at which stage the file (and its extension) was
added there. Please point me to the script so I can examine the process.

The easy way: Sdcpp might work after renaming cc1.exe back (?) to cc1.
Could you please try that?

If this fails, here is a more involved attempt: cc1 should have been
installed to where sdcpp(.exe) is looking for it. Perhaps strace can
help revealing this path. It should be somehow similar to
../../libexec[host_spec?]/sdcc/cc1 (relative to sdcpp).

(Strictly, this is a new bug, let's leave it at #3571 for now.)

cheers
felix

[1] https://sourceforge.net/projects/sdcc/files/snapshot_builds/i586-mingw32msvc/sdcc-snapshot-i586-mingw32msvc-20230531-14093.zip/download

[Deqing Sun]

Unfortunately, no:

c:\TEMP>dir sdcc\bin\cc1*
 Volume in drive C has no label.
 Volume Serial Number is EA3A-B14F

 Directory of c:\TEMP\sdcc\bin

06/01/2023  09:49 AM         1,611,343 cc1.exe
               1 File(s)      1,611,343 bytes
               0 Dir(s)  676,570,300,416 bytes free

c:\TEMP>copy sdcc\bin\cc1.exe sdcc\bin\cc1
        1 file(s) copied.

c:\TEMP>sdcc\bin\sdcc empty.c
sdcpp.exe: fatal error: cannot execute 'cc1': CreateProcess: No such file or directory
compilation terminated.
at 1: warning 190: ISO C forbids an empty translation unit
subprocess error 1

And it didn't even work in the path

c:\TEMP\sdcc\bin>sdcc ..\..\empty.c
sdcpp.exe: fatal error: cannot execute 'cc1': CreateProcess: No such file or directory
compilation terminated.
at 1: warning 190: ISO C forbids an empty translation unit
subprocess error 1

There is a cc1.exe in the libexec

c:\TEMP\sdcc\bin>dir ..\..\sdcc\libexec\sdcc\i586-mingw32msvc\12.1.0\cc1*
 Volume in drive C has no label.
 Volume Serial Number is EA3A-B14F

 Directory of c:\TEMP\sdcc\libexec\sdcc\i586-mingw32msvc\12.1.0

06/01/2023  09:49 AM         1,611,343 cc1.exe
               1 File(s)      1,611,343 bytes
               0 Dir(s)  676,561,514,496 bytes free

[bbbbbr]

If sdcc.exe is located and named as follows:
c:\TEMP\sdcc\bin\sdcc.exe

Then it should work if cc1 is located and named as follows (without the .exe extension) under:
c:\TEMP\sdcc\libexec\sdcc\cc1

This is working with my current testing (on Linux, and Windows 32 and 64bit. Have not checked on MacOS yet)

[Deqing Sun]

Yes I can confirm this works.
But this is so inconvenient.

[Felix]

On Thu, Jun 01, 2023 at 04:54:34PM -0000, bbbbbr wrote:

If sdcc.exe is located and named as follows:
c:\TEMP\sdcc\bin\sdcc.exe

Then it should work if cc1 is located and named as follows (without the .exe extension) under:
c:\TEMP\sdcc\libexec\sdcc\cc1

This is working with my current testing (on Linux, and Windows 32 and 64bit. Have not checked on MacOS yet)

Thanks, this sounds promising.

I have added a workaround in r14114 and r14115. It might take some time
to show effect.

[Aoineko]

It might take some time to show effect.

You mean the fix will be in the next snapshot?

[Felix]

On Sun, Jun 04, 2023 at 05:42:11PM -0000, Aoineko wrote:

It might take some time to show effect.

You mean the fix will be in the next snapshot?

Yes, there is some delay with binary archives, due to hardware failure.

Thanks for your patience.

(In fact, the cc1 vs cc1.exe confusion should not be addressed in
bootstrap.mk. The additional libexec/sdcc/cc1 has introduced just
another bug. Let's fix this properly after the release.)

[Deqing Sun]

I've tested the sdcc-snapshot-i586-mingw32msvc-20230623-14184. It compiles. And it won't compile if I remove the libexec folder. So I con confirm the workaround works.

[Roy Qu]

I've tried the newest windows installer ( "sdcc-20230923-14356-x64-setup.exe" ). cc1 is still installed to sdcc/bin, not sdcc/libexec/sdcc .

[Deqing Sun]

I've tried the newest windows installer ( "sdcc-20240729-14932-setup.exe" ). cc1 is still installed to sdcc/bin, not sdcc/libexec/sdcc .