#110 CRTL lstat() with dangling symbolic link fails.

[John Malmberg]

This problem first showed up on OpenVMS/IA64 8.4 in running the diffutils no-dereference check which tests diffutils with various valid and dangling symbolic links.

This problem has been traced down to the CRTL lstat() function. The original problem stated is below.
The problem has now been duplicated with out search lists involved.

If the POSIX format pathame to a dangling symbolic link does not have a directory component, it needs
to have a "/." appended to it or a "./" prepended to it or lstat() may fail. This failure may corrupt internal CRTL memory resulting in other failures.

On OpenVMS/IA64 8.4, the internal corruption caused an access violation in CRTL that looked like heap
corruption. Initialy the "/." was appended filenames with no directory components to fix, as that was the
same fix used for GNV Bug 46 and GNV Bug 50.

This code is common to most of the updated GNV packages that need to deal with symbolic links.

This fix caused __main() function to terminate with an access violation occasionally witth OpenVMS/AXP 8.4, as was seen in trying to remove the non-existant files:

rm -f a.out conftest.exe conftest a.exe a_out1.ex

Removing the above fix for OpenVMS/AXP fixed the access violation in the above case.

The problem then showed up on OpenVMS/AXP in the diffutils no-dereference test,

The fix was modified to pre-pend "./" to filenames with out a directory component.

At this point, one of the coreutils utilities is issuing system unwind messages when that test is run with coreutils 8.25 on OpenVMS/AXP 8.4.

In a previous attempt to fix this, the vms_terminal_io.c was modified. This code is common to grep, coreutils, and bash. This code also had additional null pointer and invalid file descriptor checks added to it.

• Start with a search list VMS_ROOT: = LCL_ROOT:,VMS_ROOT:,SRC_ROOT:.
• Create a subdirectory in LCL_ROOT: that does not exist in VMS_ROOT: or SRC_ROOT
• chdir() into that subdirectory may and probably will fail with an access violation.
• If you use another tool to set the default directory there, other routines in the CRTL will access violate.

Where this code is tripped is in the GNU lib openat() related routines, which internally cache the path that the file is opened on and then chdir() to it. When the routine returns, it restores the pervious directory with another chdir().

So we tried to fix this by having the chdir() call used check to see:

• Is the path a search list?
• Is the path present on the device at the end of the search list.
• If not, then check the next device in the list going backwards.
• When the device is found with the directory chdir to it.
• If the directory is not found, chdir to the directory at the beginning of the search list and let it fail.

[John Malmberg]

For now, this is just going to be noted in the release notes.

In the future, a possible fix might be to use RMS system services to verify if the file is a symlink or non-existant. Remembering that the filename is a POSIX filename and may need ODS-5 escape characters added to convert it to a VMS file specification.