Hi,
after using pam_mount for several years I encountered segfaulting in pam_mount.so in a new JupyterHub related setup some days ago. See JupyterHub issue 22 for context. I think I've found the reason for segfaulting, but fixing the bug is not trivial and beyond my very limited C programming skills. Maybe someone more skilled finds time to fix this. Many thanks in advance!
System information
tested on two systems:
Logs below are from the Debian system.
How to reproduce?
Create a PAM service /etc/pam.d/pamtest containing
auth required pam_permit.so
auth optional pam_mount.so
session optional pam_mount.so
Open two PAM transactions in parallel within one (!) process. In each transaction open and close a PAM session. Ending the second transaction segfaults. Minimal C example:
#include <security/pam_appl.h>
#include <security/pam_misc.h>
static struct pam_conv conv1 = {misc_conv, NULL};
static struct pam_conv conv2 = {misc_conv, NULL};
int main(int argc, char *argv[])
{
pam_handle_t *pamh1 = NULL;
pam_handle_t *pamh2 = NULL;
pam_start("pamtest", "username", &conv1, &pamh1);
pam_authenticate(pamh1, 0);
pam_open_session(pamh1, 0);
pam_start("pamtest", "username", &conv2, &pamh2);
pam_authenticate(pamh2, 0);
pam_open_session(pamh2, 0);
pam_close_session(pamh1, 0);
pam_close_session(pamh2, 0);
pam_end(pamh1, 0);
pam_end(pamh2, 0);
return 0;
}
Compile with
gcc -o binary_name source.c -lpam -lpam_misc
Run with
./binary_name
Prompts for password two times., then segfaults. May be run as non-root or root, but pam_mount complains about running as non-root.
** Logs **
journalctl yields (first line is newest message, possibly sensitive output replaced by XXXX...):
Feb 07 05:54:25 kernel: Code: 74 e0 48 8b 45 00 48 8b 78 08 e8 ab 8f ff ff eb d1 66 0f 1f 84 00 00 00 00 00 49 8b bc 24 b8 00 00 00 49 8d ac 24 b8 00 00 00 <48>>
Feb 07 05:54:25 kernel: pamtest[121250]: segfault at 0 ip 00007ff507d7c370 sp 00007ffd1bbba690 error 4 in pam_mount.so[7ff507d75000+9000]
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:116): Clean global config (0)
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:133): clean system authtok=0xXXXXXXXXXXXX (0)
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:116): Clean global config (0)
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:743): pam_mount execution complete
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:441): pmvarrun says login count is 0
Feb 07 05:54:25 pamtest[121250]: command: 'pmvarrun' '-u' 'username' '-o' '-1'
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:709): No volumes to umount
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:706): received order to close things
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:743): pam_mount execution complete
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:734): username seems to have other remaining open sessions
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:441): pmvarrun says login count is 1
Feb 07 05:54:25 pamtest[121250]: command: 'pmvarrun' '-u' 'username' '-o' '-1'
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:709): No volumes to umount
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:706): received order to close things
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:660): done opening session (ret=0)
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:441): pmvarrun says login count is 2
Feb 07 05:54:25 pamtest[121250]: command: 'pmvarrun' '-u' 'username' '-o' '1'
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:629): no volumes to mount
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:568): pam_mount 2.18: entering session stage
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:133): clean system authtok=0xXXXXXXXXXXXX (XXXXXXXXX)
Feb 07 05:54:25 pamtest[121250]: (pam_mount.c:365): pam_mount 2.18: entering auth stage
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:365): pam_mount 2.18: entering auth stage
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:660): done opening session (ret=0)
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:441): pmvarrun says login count is 1
Feb 07 05:54:20 pamtest[121250]: command: 'pmvarrun' '-u' 'username' '-o' '1'
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:629): no volumes to mount
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:568): pam_mount 2.18: entering session stage
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:133): clean system authtok=0xXXXXXXXXXXXX (XXXXXXXXX)
Feb 07 05:54:20 pamtest[121250]: (pam_mount.c:365): pam_mount 2.18: entering auth stage
Feb 07 05:54:17 pamtest[121250]: (pam_mount.c:365): pam_mount 2.18: entering auth stage
valgrind yields
Invalid read of size 8
at 0x4857370: ??? (in /usr/lib/x86_64-linux-gnu/security/pam_mount.so)
by 0x48779BB: ??? (in /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1)
by 0x487847F: pam_end (in /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1)
by 0x10925D: main (pamtest2.c:24)
Address 0x0 is not stack'd, malloc'd or (recently) free'd
Process terminating with default action of signal 11 (SIGSEGV)
Access not within mapped region at address 0x0
at 0x4857370: ??? (in /usr/lib/x86_64-linux-gnu/security/pam_mount.so)
by 0x48779BB: ??? (in /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1)
by 0x487847F: pam_end (in /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1)
by 0x10925D: main (pamtest2.c:24)
Reason for segfaulting
PAM explicitly supports parallel transactions, man pam:
The transaction state is contained entirely within the structure identified by this handle, so it is possible to have multiple transactions in parallel.
But pam_mount.so stores session configuration in a global structure Config (see line 61 in pam_mount.c (sourceforge.net)).
Each process using a shared library gets its own copy of the library's global variables. So the global Config structure is not commonly used by different processes. But if a process opens multiple PAM sessions, than pam_mount.so stores configuration of all those sessions at the same memory location. Or: opening a second session overwrites the configuration of the first session!
Communication between libpam and pam_mount is done via pam_set_data. This function allows to store transaction related data. In line 587 of pam_mount.c (sourceforge.net) the function is called to store a pointer to the global Config structure. So for each transaction the same pointer is stored.
Segfault occurs when ending transactions. The first call to pam_end calls cleanup callbacks associated with each data element of the transaction. The cleanup callback for pam_mount config data frees the memory of global Config. Thus, ending a second transaction (or maybe also doing some other stuff with the second transaction) segfaults when freeing (or otherwise accessing) the Config again.
I suspect line 161 of libpam's pam_data.c (github.com) to be the segfault position. Or pam_mount's cleanup function called there (see line 114 in pam_mount.c (sourceforge.net)). Don't know how to use a debugger... Thus, only guessing from reading the code.
** How to repair?**
From my limited understanding of C programming, linux processes and so on I would say that for each PAM transaction there has to be a separate Config structure in pam_mount.c. Implementing this would require major changes in the code...
At least pam_mount shouldn't segfault, but show some (log) message that multiple PAM transactions within in one process are not supported.
Many thanks and best regards,
Jens
Log in to post a comment.
