#96 ipmi_sim: segfault on bad auth message log (2.0.35)

[Fabio Massimo Di Nitto]

Hi guys,

I recently updated lanserver to 2.0.35 and noticed a new segfault that was not present in 2.0.34.

The application (based on ipmitools) attempts to connect to the BMC with a default password and if it fails, it will attempt to set the password and connect again. In the process, ipmi_sim segfaults. Here is the gdb traceback:

Program received signal SIGSEGV, Segmentation fault.
isim_log (sys=<optimized out>, logtype=9, msg=0x7fffffffc710, format=<optimized out>, ap=<optimized out>, len=<optimized out>) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/ipmi_sim.c:768
768                 pos += sprintf(str + pos, " %2.2x", msg->data[i]);
(gdb) bt
#0  isim_log (sys=<optimized out>, logtype=9, msg=0x7fffffffc710, format=<optimized out>, ap=<optimized out>, len=<optimized out>) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/ipmi_sim.c:768
#1  0x000055555555d490 in sim_log (sys=0x7fffffffd2d0, logtype=9, msg=0x7fffffffc710, format=0x7ffff7f89c40 "LAN msg failure: Invalid authtype") at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/ipmi_sim.c:802
#2  0x00007ffff7f84088 in ipmi_handle_lan_msg (lan=0x555555584990, data=<optimized out>, len=<optimized out>, from_addr=<optimized out>, from_len=<optimized out>)
    at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/lanserv_ipmi.c:3042
#3  0x000055555555c8bc in lan_data_ready (lan_fd=9, cb_data=0x555555584990, id=<optimized out>) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/ipmi_sim.c:285
#4  0x00007ffff7fae54e in handle_selector_call (sel=sel@entry=0x55555557ce50, fdc=fdc@entry=0x5555555acc30, fdset=fdset@entry=0x0, enabled=<optimized out>, handler=0x7ffff7fad790 <fd_handler>)
    at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/unix/selector.c:1064
#5  0x00007ffff7fb0826 in process_fds_epoll (isigmask=<optimized out>, tvtimeout=0x7fffffffca60, sel=0x55555557ce50) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/unix/selector.c:1218
#6  sel_select_intr_sigmask (sel=sel@entry=0x55555557ce50, send_sig=send_sig@entry=0x0, thread_id=thread_id@entry=0, cb_data=cb_data@entry=0x0, timeout=timeout@entry=0x0, sigmask=sigmask@entry=0x0)
    at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/unix/selector.c:1304
#7  0x00007ffff7fb09ce in sel_select (sel=sel@entry=0x55555557ce50, send_sig=send_sig@entry=0x0, thread_id=thread_id@entry=0, cb_data=cb_data@entry=0x0, timeout=timeout@entry=0x0)
    at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/unix/selector.c:1356
#8  0x00007ffff7fb0ad4 in sel_select_loop (sel=0x55555557ce50, send_sig=0x0, thread_id=0, cb_data=0x0) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/unix/selector.c:1379
#9  0x000055555555b94d in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/OpenIPMI-2.0.35-1.el9.x86_64/lanserv/ipmi_sim.c:1830
(gdb)

"#1" in the gdb backtrace points at this commit:

`
commit b52e8e2538b2b48ef6b63bff12b5cc9e2d52eff1
Author: Corey Minyard minyard@acm.org
Date: Mon Apr 29 12:46:23 2024 -0500

lanserv: Check some bounds on incoming messages

Signed-off-by: Corey Minyard <minyard@acm.org>

diff --git a/lanserv/lanserv_ipmi.c b/lanserv/lanserv_ipmi.c
index 9ef81e30..8d460276 100644
--- a/lanserv/lanserv_ipmi.c
+++ b/lanserv/lanserv_ipmi.c
@@ -880,6 +880,12 @@ handle_temp_session(lanserv_data_t lan, msg_t msg)
}

 auth = msg->data[0] & 0xf;

• if (auth >= MAX_IPMI_AUTHS) {
• lan->sysinfo->log(lan->sysinfo, NEW_SESSION_FAILED, msg,
• "Activate session failed: Invalid auth: 0x%x", auth);
• return;

• }
  +
  user = &(lan->users[user_idx]);
  if (! (user->valid)) {
  lan->sysinfo->log(lan->sysinfo, NEW_SESSION_FAILED, msg,
  @@ -3032,6 +3038,11 @@ ipmi_handle_lan_msg(lanserv_data_t *lan,
  }

  msg.authtype = data[4];

• if (msg.authtype >= MAX_IPMI_AUTHS) {

• lan->sysinfo->log(lan->sysinfo, LAN_ERR, &msg,
• "LAN msg failure: Invalid authtype");
• return;
• }
  msg.data = data+5;
  msg.len = len - 5;
  msg.channel = lan->channel.channel_num;
  `

I wish I could propose a patch but I am not familiar enough with the whole code base. Reverting this commit workarounds the crash.

Just a hunch tells me that msg.len should probably be initialized before calling into log().

Hope it helps.

Cheers
Fabio

[Corey Minyard]

Very helpful, and you are correct, it was calling the log with parts of the message uninitialized. And it was a bug even before that change, because the check above it would have caused a crash if a too-short message came in.

I've pushed up a patch to fix this, can you test it? I'll do a release if it's correct.

[Fabio Massimo Di Nitto]

Hi Corey,

thanks for the fast response. The fix appears to be working fine. No crashes and I see the expected message in the logs.

Signed-off-by: Fabio M. Di Nitto fdinitto@redhat.com

[Corey Minyard]

Also, if you don't mind me asking, what are you using ipmi_sim for? I wasn't aware many people were using it. If people are using it for serious purposes, I need to be more serious about security on it.

[Fabio Massimo Di Nitto]

Not at all. We use ipmi_sim in Linux HA / ClusterLabs CI https://clusterlabs.org/

Specifically: https://github.com/ClusterLabs/anvil/
expects to deploy on real bare-metal hardware with BMC/ipmi and other goodies.

We can´t have enough BM to run those tests, so we simulate the whole HW environment, including UPS, APC, PDUs via a fork of Simengine: https://github.com/Alteeve/simengine

Simengine uses and configures ipmi_sim to work with a normal libvirt+qemu VM + a bunch of other stuff.

CI is here: https://ci.kronosnet.org/job/anvil/

As for security, our CI does not expose the BMC/ipmi_sim to the internet, it´s all internal to the nodes running the simulations for obvious reasons :)

Cheers
Fabio

[Corey Minyard]

Closing, since this is resolved. Thanks again.