Thread: [SSI-devel] [ ssic-linux-Bugs-945628 ] NFS server export problem
Brought to you by:
brucewalker,
rogertsang
From: SourceForge.net <no...@so...> - 2004-04-30 21:42:22
|
Bugs item #945628, was opened at 2004-04-30 14:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 Category: Filesystem Group: None Status: Open Resolution: None Priority: 5 Submitted By: David B. Zafman (dzafman) Assigned to: David B. Zafman (dzafman) Summary: NFS server export problem Initial Comment: Hi - I'm experiencing NFS server problems under RC4 similar to those described in the thread starting at http://article.gmane.org/gmane.linux.cluster.ssic.user/1260 -- am I right in thinking that RC4 was supposed to incorporate the mountd fix described by David Zafman in http://article.gmane.org/gmane.linux.cluster.ssic.user/1263 ? What I'm seeing is that a non-cluster machine can NFSmount *either* initnode-local disks or clustermounted disks, but not both -- and furthermore, once a non-cluster machine has mounted one partition, it is limited to that type of partition until the next cluster reboot. An example... Say that A and B are mounted on the initnode. A is local hardware, B is a CFS filesystem from node 2. Both are exported via the CVIP interface to hostX and hostY. On hostX, I do this: [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp [root@hostX ]# mount cvip:B /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp and on hostY, I do this: [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp [root@hostY ]# mount cvip:A /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp And in /var/log/messages on the initnode, this is what I see: initnode rpc.mountd: authenticated mount request from hostX:945 for A (A) initnode rpc.mountd: authenticated unmount request from hostX: 948 for A (A) initnode rpc.mountd: authenticated mount request from hostX:958 for B (B) initnode rpc.mountd: getfh failed: Operation not permitted initnode rpc.mountd: authenticated mount request from hostY:668 for B (B) initnode rpc.mountd: authenticated unmount request from hostY: 673 for B (B) initnode rpc.mountd: authenticated mount request from hostY:671 for A (A) initnode rpc.mountd: getfh failed: Operation not permitted However, I can reboot the cluster and reverse the mount order for A and B (above) with exactly the same results -- if a remote host mounts a cfs disk from the CVIP, it will not be able to mount an initnode disk from the CVIP until the cluster reboots, and vice- versa. Since at present I have only a two-node cluster, I couldn't test whether the restriction is binary between (initnode-local disks) and (initnode-remote disks) or is in fact that a given external host can mount only one cluster node's disks, that being whichever node hosts the first disk mounted. ? Ole -- Ole Craig * UNIX, linux, SMTP-ninja; news, web; SGI martyr * CS Computing Facility, UMass * <www.cs.umass.edu/~olc/pgppubkey.txt> for public key Where are the missing deficit-reduction program-related activities? ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Ssic-linux-users mailing list Ssi...@li... https://lists.sourceforge.net/lists/listinfo/ssic-linux-users ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 |
From: SourceForge.net <no...@so...> - 2004-04-30 21:43:47
|
Bugs item #945628, was opened at 2004-04-30 14:42 Message generated for change (Comment added) made by dzafman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 Category: Filesystem Group: None Status: Open Resolution: None Priority: 5 Submitted By: David B. Zafman (dzafman) Assigned to: David B. Zafman (dzafman) Summary: NFS server export problem Initial Comment: Hi - I'm experiencing NFS server problems under RC4 similar to those described in the thread starting at http://article.gmane.org/gmane.linux.cluster.ssic.user/1260 -- am I right in thinking that RC4 was supposed to incorporate the mountd fix described by David Zafman in http://article.gmane.org/gmane.linux.cluster.ssic.user/1263 ? What I'm seeing is that a non-cluster machine can NFSmount *either* initnode-local disks or clustermounted disks, but not both -- and furthermore, once a non-cluster machine has mounted one partition, it is limited to that type of partition until the next cluster reboot. An example... Say that A and B are mounted on the initnode. A is local hardware, B is a CFS filesystem from node 2. Both are exported via the CVIP interface to hostX and hostY. On hostX, I do this: [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp [root@hostX ]# mount cvip:B /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp and on hostY, I do this: [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp [root@hostY ]# mount cvip:A /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp And in /var/log/messages on the initnode, this is what I see: initnode rpc.mountd: authenticated mount request from hostX:945 for A (A) initnode rpc.mountd: authenticated unmount request from hostX: 948 for A (A) initnode rpc.mountd: authenticated mount request from hostX:958 for B (B) initnode rpc.mountd: getfh failed: Operation not permitted initnode rpc.mountd: authenticated mount request from hostY:668 for B (B) initnode rpc.mountd: authenticated unmount request from hostY: 673 for B (B) initnode rpc.mountd: authenticated mount request from hostY:671 for A (A) initnode rpc.mountd: getfh failed: Operation not permitted However, I can reboot the cluster and reverse the mount order for A and B (above) with exactly the same results -- if a remote host mounts a cfs disk from the CVIP, it will not be able to mount an initnode disk from the CVIP until the cluster reboots, and vice- versa. Since at present I have only a two-node cluster, I couldn't test whether the restriction is binary between (initnode-local disks) and (initnode-remote disks) or is in fact that a given external host can mount only one cluster node's disks, that being whichever node hosts the first disk mounted. ? Ole -- Ole Craig * UNIX, linux, SMTP-ninja; news, web; SGI martyr * CS Computing Facility, UMass * <www.cs.umass.edu/~olc/pgppubkey.txt> for public key Where are the missing deficit-reduction program-related activities? ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Ssic-linux-users mailing list Ssi...@li... https://lists.sourceforge.net/lists/listinfo/ssic-linux-users ---------------------------------------------------------------------- >Comment By: David B. Zafman (dzafman) Date: 2004-04-30 14:43 Message: Logged In: YES user_id=297844 I reproduced the problem you are having. Let's see if I found your problem. I have two filesystems in my fstab like this: /dev/1/sda1 on /boot type ext3 (rw) /dev/2/sda1 on /cluster/node2/introot type ext3 (rw) I get the exact behavior if I export both of these. The problem is that both filesystems have the same device major/minor. I need to fix our NFS server code not to get confused by what appears to be the same major/minor. You can't export two filesystems in the cluster if they have the same device major/minor. The kernel function exp_get() called from exp_export() will find the first export entry when the second is being added. The base linux code assumes you are just updating options to an existing export, not adding a new one. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 |
From: SourceForge.net <no...@so...> - 2004-04-30 21:55:43
|
Bugs item #945628, was opened at 2004-04-30 14:42 Message generated for change (Comment added) made by dzafman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 Category: Filesystem Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: David B. Zafman (dzafman) Assigned to: David B. Zafman (dzafman) Summary: NFS server export problem Initial Comment: Hi - I'm experiencing NFS server problems under RC4 similar to those described in the thread starting at http://article.gmane.org/gmane.linux.cluster.ssic.user/1260 -- am I right in thinking that RC4 was supposed to incorporate the mountd fix described by David Zafman in http://article.gmane.org/gmane.linux.cluster.ssic.user/1263 ? What I'm seeing is that a non-cluster machine can NFSmount *either* initnode-local disks or clustermounted disks, but not both -- and furthermore, once a non-cluster machine has mounted one partition, it is limited to that type of partition until the next cluster reboot. An example... Say that A and B are mounted on the initnode. A is local hardware, B is a CFS filesystem from node 2. Both are exported via the CVIP interface to hostX and hostY. On hostX, I do this: [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp [root@hostX ]# mount cvip:B /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostX ]# mount cvip:A /mnt/tmp [root@hostX ]# ls /mnt/tmp list of files local to initnode [root@hostX ]# umount /mnt/tmp and on hostY, I do this: [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp [root@hostY ]# mount cvip:A /mnt/tmp mount: cvip:B failed, reason given by server: Permission denied [root@hostY ]# mount cvip:B /mnt/tmp [root@hostY ]# ls /mnt/tmp list of files local to node 2 [root@hostY ]# umount /mnt/tmp And in /var/log/messages on the initnode, this is what I see: initnode rpc.mountd: authenticated mount request from hostX:945 for A (A) initnode rpc.mountd: authenticated unmount request from hostX: 948 for A (A) initnode rpc.mountd: authenticated mount request from hostX:958 for B (B) initnode rpc.mountd: getfh failed: Operation not permitted initnode rpc.mountd: authenticated mount request from hostY:668 for B (B) initnode rpc.mountd: authenticated unmount request from hostY: 673 for B (B) initnode rpc.mountd: authenticated mount request from hostY:671 for A (A) initnode rpc.mountd: getfh failed: Operation not permitted However, I can reboot the cluster and reverse the mount order for A and B (above) with exactly the same results -- if a remote host mounts a cfs disk from the CVIP, it will not be able to mount an initnode disk from the CVIP until the cluster reboots, and vice- versa. Since at present I have only a two-node cluster, I couldn't test whether the restriction is binary between (initnode-local disks) and (initnode-remote disks) or is in fact that a given external host can mount only one cluster node's disks, that being whichever node hosts the first disk mounted. ? Ole -- Ole Craig * UNIX, linux, SMTP-ninja; news, web; SGI martyr * CS Computing Facility, UMass * <www.cs.umass.edu/~olc/pgppubkey.txt> for public key Where are the missing deficit-reduction program-related activities? ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Ssic-linux-users mailing list Ssi...@li... https://lists.sourceforge.net/lists/listinfo/ssic-linux-users ---------------------------------------------------------------------- >Comment By: David B. Zafman (dzafman) Date: 2004-04-30 14:55 Message: Logged In: YES user_id=297844 BUG FIX #945628: NFS server export problem cluster/ssi/cfs/cfs_mnthooks.c Set s_ssidev during mount discovery or notification fs/nfsd/export.c Instead of using physical device in export entries use ssidev Fix places that check against i_dev or output i_dev to use s_ssidev exp_unexport() lookup and replace user specified ex_dev with s_ssidev openssi/kernel/fs/nfsd/export.c 1.2.2.6 openssi/kernel/cluster/ssi/cfs/cfs_ipcshm.c 1.4.2.3 openssi/kernel/cluster/ssi/cfs/cfs_mnthooks.c 1.6.2.4 ---------------------------------------------------------------------- Comment By: David B. Zafman (dzafman) Date: 2004-04-30 14:43 Message: Logged In: YES user_id=297844 I reproduced the problem you are having. Let's see if I found your problem. I have two filesystems in my fstab like this: /dev/1/sda1 on /boot type ext3 (rw) /dev/2/sda1 on /cluster/node2/introot type ext3 (rw) I get the exact behavior if I export both of these. The problem is that both filesystems have the same device major/minor. I need to fix our NFS server code not to get confused by what appears to be the same major/minor. You can't export two filesystems in the cluster if they have the same device major/minor. The kernel function exp_get() called from exp_export() will find the first export entry when the second is being added. The base linux code assumes you are just updating options to an existing export, not adding a new one. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=405834&aid=945628&group_id=32541 |