Oddly, the issue only occurs on two of my three Archlinux machines (laptop, home desktop, work desktop). I've compared config files and I don't see any differences that might suggest why two machines have this issue and one not...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
rEFInd 0.14.2 introduced some significant changes to the way it scans for disk-based tools. I did this for various reasons -- partly to simplify the code, partly to deal with changes in the names and locations of some tools (most notably, various memtest tools). One consequence, however, is that rEFInd may detect a slightly different set of tools, potentially increasing or decreasing the number of tools, depending on what's available on a system. That said, some of what you've reported seems strange, such as the replication of shutdown and reboot options. These are both built-in options, rather than disk-based tools, and so should not have been affected by changes in scanning code for disk-based tools.
My suspicion is therefore that you may have multiple showtools options somewhere, such as in the main refind.conf file and in the configuration file for a theme, and you're looking at the definition that's overridden by a later one. You may want to look for that and, if there are multiple such lines, comment one of them out. Certainly I have not seen the symptoms you describe, or that are described in the Arch thread to which you linked. I even tried reproducing your showtools line, and the empty showtools line described in the thread. rEFInd 0.14.2 produced exactly the results it should have in my tests.
As a general debugging tip, you may want to be sure that the hideui option is not active and check the paths to the tools that are reported. This may provide a clue about what's going on; the new code lists the path to each on-disk tool, which can help you figure out where it's located. This may help you locate duplicates.
It would also be helpful for you to enable logging (setting log_level 1 should be adequate) and post the resulting refind.log file, or at least the "Scanning for Tools" section of that file. This should provide some clues about what's going on.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Here is the log file. (The log file produced had every second character as a null character.... I had to strip them all with tr before it was readable.)
==========Systeminformation==========14:46:14-rEFInd0.14.2builtwithGNU-EFI14:46:14-rEFIndbootpartitionGUID:D4C510E1-AE84-4938-A898-BEFDD74A4F7514:46:14-Platform:x86-64/X64/AMD64(64-bit)14:46:14-Loglevelis114:46:14-Menutimeoutis1014:46:14-Firmware:HP264.0014:46:14-EFIRevision2.7014:46:14-SecureBootactive14:46:14-Shimisavailable14:46:14-CSMisnotavailable14:46:14-Error2;UnabletoretrieveEFIvariablecapacity14:46:14-Systemdoesnotsupporttextmode14:46:14-SystemdoesnotsupportUGADrawgraphicsmode14:46:14-SystemdoessupportGOPgraphicsmode14:46:14-SettingupSecureBoot(ifapplicable)==========Loadingdrivers====================Initializingbasicfeatures==========14:46:14-Adjustingdefault_selectionwithPreviousBootvalues14:46:14-EnteringInitScreen()14:46:14-Settingwatchdogtimer14:46:14-Settingscreenresolutionandmode14:46:14-Settingvolumeicons....==========Scanningforbootloaders==========----------Scanningforuser-configuredbootstanzas----------14:46:14-Addingmanualloaderfor'\vmlinuz-linux'14:46:14-Searchingforaninitrdtomatch'\vmlinuz-linux'on'SYSTEM'14:46:14-Locatedinitrdis'\initramfs-linux.img'14:46:14-Addingmanualloaderfor'\EFI\Microsoft\Boot\bootmgfw.efi'==========Scanningfortools==========14:46:14-Scanningfortools'MokManager.efi,HashTool.efi,HashTool-signed.efi,KeyTool.efi,KeyTool-signed.efi,mmx64.efi'in'EFI\tools,EFI\refind'14:46:14-'EFI\refind\mmx64.efi'isavalidloaderfile14:46:14-Addingtagfor'mmx64.efi'on'SYSTEM'14:46:14-Scanningfortools'memtest86.efi,memtest86_x64.efi,memtest86x64.efi,memtest86+x64.efi'in'EFI\tools,EFI\refind'14:46:14-Scanningfortools'MokManager.efi,HashTool.efi,HashTool-signed.efi,KeyTool.efi,KeyTool-signed.efi,mmx64.efi'in'EFI\tools,EFI\refind'14:46:14-'EFI\refind\mmx64.efi'isavalidloaderfile14:46:14-Addingtagfor'mmx64.efi'on'SYSTEM'14:46:14-Scanningfortools'fwupx64.efi'in'EFI\tools,EFI\refind'==========Enteringmainloop====================Launching'Boot Arch Linux from SYSTEM'==========14:46:25-Startingvmlinuz-linux14:46:25-Usingloadoptions'root=LABEL=root ro rootfstype=ext4 initrd=\initramfs-linux.img'14:46:25-'\vmlinuz-linux'isavalidloaderfile14:46:25-Launching'vmlinuz-linux'
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I suspect what you are seeing is out of a deep lying memory management issue , most likely what I call El Gordo that I think is a buffer overrun of sorts.
Basically, the SetMem item is not taking and the original default 12 items are still in place. So while your new showtools list changes the first five default items, TAG_SHELL, TAG_MEMTEST, TAG_GDISK, TAG_APPLE_RECOVERY, TAG_WINDOWS_RECOVERY
... the remaining seven are still in play for you: TAG_MOK_TOOL, TAG_ABOUT, TAG_HIDDEN, TAG_SHUTDOWN, TAG_REBOOT, TAG_FIRMWARE, TAG_FWUPDATE_TOOL
As with many memory conflict issues, it might show up on one unit and not on another.
Yes, this is almost certainly the issue. I edited the showtools code in config.c by commenting out the SetMem call and then adding a loop to "manually" zero out the rest of the GlobalConfig.ShowTools array, and it fixed the issue.
Couple of side notes:
* On my two machines where the issue occurs, secure boot is enabled and managed by rEFInd. My machine where the issue doesn't occur is older and does not use secure boot. So maybe something about secure boot is triggering the memory issue.
* NUM_TOOLS is defined to be 24 in global.h, but the default showtools array in main.c is only zeroed out to a total of 21 items. Might want to add a few more zeros to that initial showtools array in main.c.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Also, isn't there an off-by-one error in the bound of that for loop in config.c ? Since i-1 is used as the index variable to set items in the GlobalConfig.ShowTools array, shouldn't
for (i = 1; (i < TokenCount) && (i < NUM_TOOLS); i++) {
instead be
for (i = 1; (i < TokenCount) && (i <= NUM_TOOLS); i++) {
?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry, last thing. Should note that my "fixed" code at https://github.com/loserMcloser/rEFInd/blob/86c7ab03db42001bfbe5c7e72654e568572e063f/refind/config.c#L669-L718
is not actually an appropriate fix, because of the way that the TokenList index variable and the GlobalConfig.ShowTools index variable are tied together. If the SetMem command is commented out and one of the TokenList items triggers the final "unknown showtools flag" else block, the entry at position GlobalConfig.ShowTools[i-1] is not going to get overwritten with anything and will remain as it was initialized in the default GlobalConfig.ShowTools array in main.c.
You might want to de-couple the GlobalConfig.ShowTools array index variable from the TokenList index variable in that for loop in config.c.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I handled the iteration as well as invalid entries differently in RefindPlus. The i-1 setup is not used. and there are separate iterators for TokenList and GlobalConfig.ShowTools.
Additionally, a variable counts invalid items and this is added to the TokenList iterator. There is an issue otherwise as you noted indeed where if a user enters an invalid entry, things go out of shape.
I have also dropped SetMem (not yet pushed) and now just zero things out manually there as well. This is done right off the top though. That is, everything is set to zero before assigning anything.
PS: The link to rEFInd I posted is a mirror of the repo here without edits. Just more convenient on GitHub for linking to specific blocks of code
Last edit: dakanji 2024-04-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So the sizing should not be an issue.
That is, ShowTools by itself is OK and something else is manifesting.
The issues with the iteration probably could be sorted but they are not quite critical.
Last edit: dakanji 2024-04-19
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The MR works around the issue but does not fix the root cause.
From my original comment, it is:
Basically, the SetMem item is not taking ...
Luckily, it seems someone else with an unrelated issue has stumbled on the root issue of SetMem being broken on recent GNUEFI builds and come up with a fix.
SetMem basically needs to be refit_call3_wrapper(gBS->SetMem, ...) to work with recent GNUEFI builds.
I posted a comment on the MR pointing to the thread with the fix.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Confirming the issue here, on 2 different machines.
Issue which didn't exist in previous rEFInd releases.
Strangely enough, if I have in refind.conf :
showtools reboot, shutdown, memtest, bootorder, hidden_tags, windows_recovery, shell, firmware, about
...then the reboot and poweroff icons appear duplicated, and the firmware_upgrade icon appears even though it is not listed in wanted tools.
But if I comment out the “showtools” line completely, then only the tools actually present on the system appears, AND there are no duplicates for reboot and poweroff.
Same on the 2 machines (a several years old Asus and a 2 years old Lenovo Thinkpad)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I tried the patch from the PR that was originally sent and I confirm too that I can see only one tool. In the PR it was mentioned that maybe this is not the optimal way to handle this and the patch was cancelled. I don't know if there's a better patch (I couldn't find it), neither in pending patches or accepted. So I think no more progress has been done. If the"proper" patch is not available yet, I could provide the binary that I compiled with the "wrong" patch, since it works for me -- although I strongly believe that properly solving it is the way to go.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The patch works nicely. My worry is that this issue hasn't had any response by the refind author, even when there's a working patch that could be just merged and released with little effort. Is the project abandoned?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Since upgrading to 0.14.2 on Archlinux, the tools are duplicated.
My
showtoolsentry inrefind.confisshowtools mok_tool,reboot,shutdown,firmware,memtestbut since upgrading, what shows up on the refind screen when booting is
mok_tool,reboot,shutdown,memtest,mok_tool,about,shutdown,reboot,memtestI'm not the only archuser to have this issue:
rEFInd - showtools option no longer working after updating to 0.14.2-1
Oddly, the issue only occurs on two of my three Archlinux machines (laptop, home desktop, work desktop). I've compared config files and I don't see any differences that might suggest why two machines have this issue and one not...
rEFInd 0.14.2 introduced some significant changes to the way it scans for disk-based tools. I did this for various reasons -- partly to simplify the code, partly to deal with changes in the names and locations of some tools (most notably, various memtest tools). One consequence, however, is that rEFInd may detect a slightly different set of tools, potentially increasing or decreasing the number of tools, depending on what's available on a system. That said, some of what you've reported seems strange, such as the replication of shutdown and reboot options. These are both built-in options, rather than disk-based tools, and so should not have been affected by changes in scanning code for disk-based tools.
My suspicion is therefore that you may have multiple
showtoolsoptions somewhere, such as in the mainrefind.conffile and in the configuration file for a theme, and you're looking at the definition that's overridden by a later one. You may want to look for that and, if there are multiple such lines, comment one of them out. Certainly I have not seen the symptoms you describe, or that are described in the Arch thread to which you linked. I even tried reproducing yourshowtoolsline, and the emptyshowtoolsline described in the thread. rEFInd 0.14.2 produced exactly the results it should have in my tests.As a general debugging tip, you may want to be sure that the
hideuioption is not active and check the paths to the tools that are reported. This may provide a clue about what's going on; the new code lists the path to each on-disk tool, which can help you figure out where it's located. This may help you locate duplicates.It would also be helpful for you to enable logging (setting
log_level 1should be adequate) and post the resultingrefind.logfile, or at least the "Scanning for Tools" section of that file. This should provide some clues about what's going on.Only one
showtoolsoption in my configs:After commenting out
hideuithe two instances ofmok_toolreported identical paths. The other tools did not report any paths. (Sorry for the glare.)Here is my current config:
Here is the log file. (The log file produced had every second character as a null character.... I had to strip them all with
trbefore it was readable.)The
GlobalConfig.ShowToolsitem has a default of 12 items:https://github.com/dakanji/rEFInd/blob/94ba63b502c65933f4d8ef8871c5914d6dae4495/refind/main.c#L154-L156
When you set
showtoolsin your config, the defaults are supposed to be cleared here:https://github.com/dakanji/rEFInd/blob/94ba63b502c65933f4d8ef8871c5914d6dae4495/refind/config.c#L670
I suspect what you are seeing is out of a deep lying memory management issue , most likely what I call
El Gordothat I think is a buffer overrun of sorts.Basically, the
SetMemitem is not taking and the original default 12 items are still in place. So while your new showtools list changes the first five default items,TAG_SHELL, TAG_MEMTEST, TAG_GDISK, TAG_APPLE_RECOVERY, TAG_WINDOWS_RECOVERY... the remaining seven are still in play for you:
TAG_MOK_TOOL, TAG_ABOUT, TAG_HIDDEN, TAG_SHUTDOWN, TAG_REBOOT, TAG_FIRMWARE, TAG_FWUPDATE_TOOLAs with many memory conflict issues, it might show up on one unit and not on another.
More on El Gordo:
https://github.com/dakanji/RefindPlus/blob/a9f45080c651de2d6d2129c7996366bcc4e1f57d/BootMaster/screenmgt.c#L94-L119
https://github.com/dakanji/RefindPlus/blob/a9f45080c651de2d6d2129c7996366bcc4e1f57d/BootMaster/menu.c#L3312-L3328
Yes, this is almost certainly the issue. I edited the
showtoolscode inconfig.cby commenting out theSetMemcall and then adding a loop to "manually" zero out the rest of theGlobalConfig.ShowToolsarray, and it fixed the issue.See here:
https://github.com/loserMcloser/rEFInd/blob/86c7ab03db42001bfbe5c7e72654e568572e063f/refind/config.c#L669-L718
Couple of side notes:
* On my two machines where the issue occurs, secure boot is enabled and managed by rEFInd. My machine where the issue doesn't occur is older and does not use secure boot. So maybe something about secure boot is triggering the memory issue.
*
NUM_TOOLSis defined to be24inglobal.h, but the defaultshowtoolsarray inmain.cis only zeroed out to a total of 21 items. Might want to add a few more zeros to that initialshowtoolsarray inmain.c.Also, isn't there an off-by-one error in the bound of that
forloop inconfig.c? Sincei-1is used as the index variable to set items in theGlobalConfig.ShowToolsarray, shouldn'tinstead be
?
Sorry, last thing. Should note that my "fixed" code at
https://github.com/loserMcloser/rEFInd/blob/86c7ab03db42001bfbe5c7e72654e568572e063f/refind/config.c#L669-L718
is not actually an appropriate fix, because of the way that the
TokenListindex variable and theGlobalConfig.ShowToolsindex variable are tied together. If theSetMemcommand is commented out and one of theTokenListitems triggers the final "unknown showtools flag"elseblock, the entry at positionGlobalConfig.ShowTools[i-1]is not going to get overwritten with anything and will remain as it was initialized in the defaultGlobalConfig.ShowToolsarray inmain.c.You might want to de-couple the
GlobalConfig.ShowToolsarray index variable from theTokenListindex variable in thatforloop inconfig.c.I handled the iteration as well as invalid entries differently in RefindPlus. The
i-1setup is not used. and there are separate iterators forTokenListandGlobalConfig.ShowTools.Additionally, a variable counts invalid items and this is added to the TokenList iterator. There is an issue otherwise as you noted indeed where if a user enters an invalid entry, things go out of shape.
I have also dropped
SetMem(not yet pushed) and now just zero things out manually there as well. This is done right off the top though. That is, everything is set to zero before assigning anything.PS: The link to rEFInd I posted is a mirror of the repo here without edits. Just more convenient on GitHub for linking to specific blocks of code
Last edit: dakanji 2024-04-18
Good catch!
This could well be the mysterious memory issue as
SetMemis used with a larger size.EDIT:
ShowTools is set to
NUM_TOOLSsize here:https://github.com/dakanji/rEFInd/blob/94ba63b502c65933f4d8ef8871c5914d6dae4495/refind/global.h#L394
So the sizing should not be an issue.
That is, ShowTools by itself is OK and something else is manifesting.
The issues with the iteration probably could be sorted but they are not quite critical.
Last edit: dakanji 2024-04-19
Hi, I just registered at sourceforge to say this bug is a very sad thing. Please help.
@Felipe I have submitted a merge request to fix the issue, hopefully the author accepts it.
The MR
works aroundthe issue but does notfixthe root cause.From my original comment, it is:
Luckily, it seems someone else with an unrelated issue has stumbled on the root issue of
SetMembeing broken on recent GNUEFI builds and come up with a fix.SetMem basically needs to be
refit_call3_wrapper(gBS->SetMem, ...)to work with recent GNUEFI builds.I posted a comment on the MR pointing to the thread with the fix.
Can confirm that
refit_call3_wrapper(gBS->SetMem, ...)fixes the duplicate tools bug. I've closed my original merge request and opened a simpler one.I'm also affected by this, maybe we can get a new release with the fix?
I see the same issue as well.
Confirming the issue here, on 2 different machines.
Issue which didn't exist in previous rEFInd releases.
Strangely enough, if I have in refind.conf :
showtools reboot, shutdown, memtest, bootorder, hidden_tags, windows_recovery, shell, firmware, about
...then the reboot and poweroff icons appear duplicated, and the firmware_upgrade icon appears even though it is not listed in wanted tools.
But if I comment out the “showtools” line completely, then only the tools actually present on the system appears, AND there are no duplicates for reboot and poweroff.
Same on the 2 machines (a several years old Asus and a 2 years old Lenovo Thinkpad)
Can confirm having this issue aswell
I tried the patch from the PR that was originally sent and I confirm too that I can see only one tool. In the PR it was mentioned that maybe this is not the optimal way to handle this and the patch was cancelled. I don't know if there's a better patch (I couldn't find it), neither in pending patches or accepted. So I think no more progress has been done. If the"proper" patch is not available yet, I could provide the binary that I compiled with the "wrong" patch, since it works for me -- although I strongly believe that properly solving it is the way to go.
@Panayotis Katsaloulis
Both patches work, but the second version of the fix is much simpler and required less editing of the code:
https://sourceforge.net/p/refind/code/merge-requests/55/
The patch works nicely. My worry is that this issue hasn't had any response by the refind author, even when there's a working patch that could be just merged and released with little effort. Is the project abandoned?