On the latest snapshot f3a869cf88abc5651977973f7f63 rEFInd hangs after selecting partition number from gdisk.efi. Also it does not load Shell.efi from rEFIt. Log is attached.
Basically, gdisk will invalidate the DeviceHandle of all LoadedImage protocols that were loaded from disk because gdisk causes the DeviceHandle to be disposed and the device gets a new DeviceHandle at a different location when gdisk exits. It's possible that the old address of the DeviceHandle gets reallocated for some other purpose. RefindPlus or any other LoadedImage might try to use that old address as a DeviceHandle which should give an invalid parameter error but if that old address gets a DeviceHandle that isn't invalid, then RefindPlus could be doing work on a wrong device (I haven't seen that possibility happen).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just re-ran rEFInd and now gdisk.efi works as designed there as well. Shell.efi from rEFIt returns back to the main screen after unsupported message (why unsupported?), but does not hang.
Also I have to note RP and RPE do not log after exiting from any EFI application wheras rEFInd does log everyhing.
We'll discuss my RefindPlus in the PM. If something related to rEFInd is discovered then we can come back here. One question: did you try a release version? The debug version's stack tracing or leak detection or debug logging could be issues.
Regarding rEFInd, did you try gdisk in the shell and check if the devicehandle for the disk containing rEFInd changed? I see in your RefindPlus log that you used gdisk successfully but it was only changing the device handles of a different disk device handle changed. I see no message where the device handle was changed for the disk containing RefindPlus Self device handle changed. If you did the same gdisk test in rEFInd then it probably would also not have a problem. You have to use gdisk to look at the disk that rEFInd is on to see a problem.
Regarding logging after exitingg from a EFI application, can you cite examples (log file name and line number) where RP or RPE fails and rEFInd succeeds? The RefindPlus logs show it returning from gdisk (21n30t1808.log:2279)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Looking at this table this SHELL would never work on non-apple x86 machine:
"Regarding rEFInd, did you try gdisk in the shell and check if the devicehandle for the disk containing rEFInd changed? "
Now gdisk.efi works fine everywhere. Only rEFIt shell is not loading in rEFind and in RP is hanging.
I loaded HermitShell_X64.efi and then loaded as a driver edit.efi. It came with an error, but afterwards I was able to issue edit command, which previously failed on any Shell.efi I tried.
This only worked for the load.efi and probably will work the other tools from rEFIt package.
Last edit: Startergo 2021-07-31
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Regarding fat binaries: they can work on non-apple x86 with enough code to handle them. Or you can split the fat binary into its parts. The FAT binary format is super simple. You could make a script to do it or there's a python script already created for the job https://github.com/snare/efitools
but you may want to change the default names of the split files so that the architecture type appears before the .efi extension instead of afterward. I think x86 is the same as ia32.
Regarding gdisk: you haven't determined if the device handle of the loaded image of RefindPlus or rEFInd is changing. Did you use gdisk on every disk for the tests?
Regarding edit.efi: it is not a driver so it cannot load as a driver or it doesn't do anything as a driver. Are you sure HermitShell_X64.efi doesn't have the edit command built-in (like most EFI Shells) and therefore doesn't require the seperate edit.efi? Also, please describe where HermitShell_X64.efi is from.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
"Regarding edit.efi: it is not a driver so it cannot load as a driver or it doesn't do anything as a driver. Are you sure HermitShell_X64.efi doesn't have the edit command built-in (like most EFI Shells) and therefore doesn't require the seperate edit.efi? Also, please describe where HermitShell_X64.efi is from."
This is the one I linked in the PM. I extracted it from the firmware of my quo motherboard. Edit function worked on that board, but not on the HP 8570p, but after loading edit.efi it works.
gdisk.efi works on all disks now. No hanging, just as expected. Actually as I said the application gdisk cannot handle live sealed snapshot like Big Sur, but has no problems in EFI shell.
Last edit: Startergo 2021-07-31
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Which edit.efi are you using? The one from rEFIt is FAT (144K) so you couldn't be loading that. There's one from tianocore-edk2-UDK2018/EdkShellBinPkg that is 48416 bytes (not FAT).
Maybe edit.efi loads a protocol that makes the edit built-in command work?
To be sure, do these commands:
1. cd directory_that_does_not_contains_edit_efi
2. edit to see if the built-in command works.
3. cd directory_that_contains_edit_efi
4. dh -d -v > dh_d_v_beforeloadedit.txt
5. edit.efi to load the edit.efi. Does it run or exit immediately or what?
6. dh -d -v > dh_d_v_afterloadedit.txt
7. cd directory_that_does_not_contains_edit_efi
8. edit to see if the built-in command now works.
Then we can compare the txt files to see if any handles changed.
Did you check if gdisk is changing device handle of the disk that contains RefindPlus or rEFInd? Maybe it's working because the old device handle is still usable (hasn't been overwritten yet)? Do a dh -d -v > dh_d_v_before or after_gdisk.txt before and after running gdisk on all the disks to compare.
gdisk application in macOS or Linux is very different than gdisk.efi because they live in totally different environments. gdisk in Linux should have no problem with the macOS disk. It's only macOS that has a problem with the Big Sur APFS disk/partition. The same problem does not occur in Catalina.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Obviously a bug in that Shell's version of dh. Use a diferent Shell? Try without -d. Try without -v. Try without -d -v. Maybe they'll work but maybe they won't have all the info to show the problem.
Compile your own dh.efi and fix the bug?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The bug is that it's trying to show a device path at address 0 (null pointer). There probably isn't a device path at address 0. The code should stop trying to interpret a device path when the type/size is weird. It should definitely not try to do anything with a null pointer.
But how did that handle 71 get a image device path protocol of null? UEFI Spec v2.8 section 9.2 says it's valid to add a null Loaded Image Device Path Protocol.
The Loaded Image Device Path Protocol must be installed onto the image handle of a PE/COFF image loaded through the EFI Boot Service LoadImage(). A copy of the device path specified by the DevicePath parameter to the EFI Boot Service LoadImage() is made before it is installed onto the image handle. It is legal to call LoadImage() for a buffer in memory with a NULL DevicePath parameter. In this case, the Loaded Image Device Path Protocol is installed with a NULL interface pointer."
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
On the latest snapshot f3a869cf88abc5651977973f7f63 rEFInd hangs after selecting partition number from gdisk.efi. Also it does not load Shell.efi from rEFIt. Log is attached.
Last edit: Startergo 2021-07-28
Those logs appear to be for RefindPlus. RefindPlus is at https://github.com/dakanji/RefindPlus
I did fix one issue with RefindPlus related to gdisk which might affect rEFInd (but I haven't checked)
https://github.com/joevt/RefindPlus/commit/0166aed246f5e11b5b27d5c0f5926ac9fb5124d0
Basically, gdisk will invalidate the DeviceHandle of all LoadedImage protocols that were loaded from disk because gdisk causes the DeviceHandle to be disposed and the device gets a new DeviceHandle at a different location when gdisk exits. It's possible that the old address of the DeviceHandle gets reallocated for some other purpose. RefindPlus or any other LoadedImage might try to use that old address as a DeviceHandle which should give an invalid parameter error but if that old address gets a DeviceHandle that isn't invalid, then RefindPlus could be doing work on a wrong device (I haven't seen that possibility happen).
It is true your fork fixed gdisk.efi, but unfortunately broke everything else as described in the "The Quest PM". Attached are the logs
I just re-ran rEFInd and now gdisk.efi works as designed there as well. Shell.efi from rEFIt returns back to the main screen after unsupported message (why unsupported?), but does not hang.
Also I have to note RP and RPE do not log after exiting from any EFI application wheras rEFInd does log everyhing.
Last edit: Startergo 2021-07-30
We'll discuss my RefindPlus in the PM. If something related to rEFInd is discovered then we can come back here. One question: did you try a release version? The debug version's stack tracing or leak detection or debug logging could be issues.
Regarding rEFInd, did you try gdisk in the shell and check if the devicehandle for the disk containing rEFInd changed? I see in your RefindPlus log that you used gdisk successfully but it was only changing the device handles of a different disk
device handle changed. I see no message where the device handle was changed for the disk containing RefindPlusSelf device handle changed. If you did the same gdisk test in rEFInd then it probably would also not have a problem. You have to use gdisk to look at the disk that rEFInd is on to see a problem.Regarding rEFIt, did you check if it was a FAT EFI binary? The command
file refit.efiin macOS will tell you.http://refit.sourceforge.net/info/fat_binary.html
I don't think PCs have the code to load FAT EFI binaries. OpenCore has code to deal with FAT EFI binaries. I don't know if it gets used.
https://github.com/acidanthera/OpenCorePkg/blob/master/Library/OcMachoLib/Fat.c
Regarding logging after exitingg from a EFI application, can you cite examples (log file name and line number) where RP or RPE fails and rEFInd succeeds? The RefindPlus logs show it returning from gdisk (21n30t1808.log:2279)
Looking at this table this SHELL would never work on non-apple x86 machine:
"Regarding rEFInd, did you try gdisk in the shell and check if the devicehandle for the disk containing rEFInd changed? "
Now gdisk.efi works fine everywhere. Only rEFIt shell is not loading in rEFind and in RP is hanging.
Last edit: Startergo 2021-07-31
So here is what I did and it worked on non Apple x86 machine:
I loaded
HermitShell_X64.efiand then loaded as a driver edit.efi. It came with an error, but afterwards I was able to issue edit command, which previously failed on any Shell.efi I tried.This only worked for the load.efi and probably will work the other tools from rEFIt package.
Last edit: Startergo 2021-07-31
Regarding fat binaries: they can work on non-apple x86 with enough code to handle them. Or you can split the fat binary into its parts. The FAT binary format is super simple. You could make a script to do it or there's a python script already created for the job https://github.com/snare/efitools
but you may want to change the default names of the split files so that the architecture type appears before the .efi extension instead of afterward. I think x86 is the same as ia32.
Regarding gdisk: you haven't determined if the device handle of the loaded image of RefindPlus or rEFInd is changing. Did you use gdisk on every disk for the tests?
Regarding edit.efi: it is not a driver so it cannot load as a driver or it doesn't do anything as a driver. Are you sure HermitShell_X64.efi doesn't have the edit command built-in (like most EFI Shells) and therefore doesn't require the seperate edit.efi? Also, please describe where HermitShell_X64.efi is from.
"Regarding edit.efi: it is not a driver so it cannot load as a driver or it doesn't do anything as a driver. Are you sure HermitShell_X64.efi doesn't have the edit command built-in (like most EFI Shells) and therefore doesn't require the seperate edit.efi? Also, please describe where HermitShell_X64.efi is from."
This is the one I linked in the PM. I extracted it from the firmware of my quo motherboard. Edit function worked on that board, but not on the HP 8570p, but after loading edit.efi it works.
gdisk.efi works on all disks now. No hanging, just as expected. Actually as I said the application gdisk cannot handle live sealed snapshot like Big Sur, but has no problems in EFI shell.
Last edit: Startergo 2021-07-31
Which edit.efi are you using? The one from rEFIt is FAT (144K) so you couldn't be loading that. There's one from tianocore-edk2-UDK2018/EdkShellBinPkg that is 48416 bytes (not FAT).
Maybe edit.efi loads a protocol that makes the edit built-in command work?
To be sure, do these commands:
1.
cd directory_that_does_not_contains_edit_efi2.
editto see if the built-in command works.3.
cd directory_that_contains_edit_efi4.
dh -d -v > dh_d_v_beforeloadedit.txt5.
edit.efito load the edit.efi. Does it run or exit immediately or what?6.
dh -d -v > dh_d_v_afterloadedit.txt7.
cd directory_that_does_not_contains_edit_efi8.
editto see if the built-in command now works.Then we can compare the txt files to see if any handles changed.
Did you check if gdisk is changing device handle of the disk that contains RefindPlus or rEFInd? Maybe it's working because the old device handle is still usable (hasn't been overwritten yet)? Do a
dh -d -v > dh_d_v_beforeorafter_gdisk.txtbefore and after running gdisk on all the disks to compare.gdisk application in macOS or Linux is very different than gdisk.efi because they live in totally different environments. gdisk in Linux should have no problem with the macOS disk. It's only macOS that has a problem with the Big Sur APFS disk/partition. The same problem does not occur in Catalina.
Even after creating a 4GB FAT32 partition I get
write error: volume fullafter
dh -d v.Handle 71 (B6104598) Image (B6103C40) File: ParentHandle..: B6104918 SystemTable...: B93FEF18 DeviceHandle..: 0 FilePath......: ImageBase.....: B9C2F000 - B9C30600 ImageSize.....: 1600 CodeType......: BS_code DataType......: BS_data ImageDpath (0) Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device Type Device Path Error - Unknown Device TypeDevice Path Error - Unknown Device TypeLast edit: Startergo 2021-08-01
Obviously a bug in that Shell's version of dh. Use a diferent Shell? Try without -d. Try without -v. Try without -d -v. Maybe they'll work but maybe they won't have all the info to show the problem.
Compile your own dh.efi and fix the bug?
The bug is that it's trying to show a device path at address 0 (null pointer). There probably isn't a device path at address 0. The code should stop trying to interpret a device path when the type/size is weird. It should definitely not try to do anything with a null pointer.
But how did that handle 71 get a image device path protocol of null? UEFI Spec v2.8 section 9.2 says it's valid to add a null Loaded Image Device Path Protocol.
Joe I am not sure if it is a bug of the shell version. Just used the gdisk.efi on a HP8570P and it worked fine.
Last edit: Startergo 2021-12-29