1) Node won't boot.
The NIM master will not respond to the bootp request. This seems to be related to the fact that when we define alternate NIM machine defs we do not include the mac addresses. This is/was supported by NIM and did work for our rolling update support demo. I tried defining the NIM client without the mac and no boot - I defined the client with the mac and it booted. This needs further debug.
2) We have new code that manages some files in the .../...shared_root/etc/.client_data directory. Our assumption was that the file names would include the hostnames (ex. hosts.compute14) but in fact they actually include the NIM name which could be something like "hosts.compute14_71dskls". This is a problem when xCAT code copies /etc/hosts files and /etc/filesystems files. These files will not be updated correctly.
3) Process issue: When using the "-n" option to create alternate NIM machine defs you can eventually wind up with a bunch of old useless machine defs (along with NIM resources) that are no longer being used. To clean up the old defs you can run rmdsklsnode.
However, I just noticed that good old NIM will actually remove the entries from the tftpboot and bootptab files based on hostname. This means that the entries for the currently running node will be removed. So, if you need to reboot the node you would have to redo the NIM dkls_init operation - Ugh.
What makes this worse is that you will not be able to remove the old resources until they are deallocated from the old machine def.