xCAT / Bugs / #1607 postscript error when installing a CN using SN on rhels6

An extreme cluster/cloud administration toolkit

#1607 postscript error when installing a CN using SN on rhels6

Milestone: 2.6

Status: closed

Owner: Lissa Valletta

Labels: Deployment-linux (161)

component:

Priority: 5

Updated: 2015-01-04

Created: 2010-12-21

Creator: fang zheng

Private: No

Using rhels6 SN install a rhels6 CN :

error when run postscript and got message in /vat/log/message on MN:
*start*
Dec 20 21:50:39 xcatn06 xCAT: xcatd: possible BUG encountered by xCAT DB worker Describe failed during DBI::st=HASH(0x1000864a648)->FETCH(NAME) at /opt/xcat/lib/perl/xCAT/Table.pm line 2682, <gen154> line 8.
Dec 20 21:50:39 xcatn06 xCAT: xcatd: getpostscript plugin bug, pid 9533, process description: 'xCATd SSL: getpostscript for xcatn05: getpostscript instance' with error 'Describe failed during DBI::st=HASH(0x1000864a648)->FETCH(NAME) at /opt/xcat/lib/perl/xCAT/Table.pm line 2682, <gen154> line 8.#012'
Dec 20 20:53:16 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:19 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:22 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:25 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:27 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:30 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 21:50:57 xcatn06 xCAT: xCAT: Allowing getpostscript from xcatn05
Dec 20 21:50:57 xcatn06 xCAT: xcatd: possible BUG encountered by xCAT DB worker Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen199> line 8.
Dec 20 21:50:57 xcatn06 xCAT: xcatd: getpostscript plugin bug, pid 9547, process description: 'xCATd SSL: getpostscript for xcatn05: getpostscript instance' with error 'Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen199> line 8.#012'
Dec 20 20:53:33 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:36 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:39 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:42 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:45 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:47 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 21:51:14 xcatn06 xCAT: xCAT: Allowing getpostscript from xcatn05
Dec 20 21:51:15 xcatn06 xCAT: xcatd: possible BUG encountered by xCAT DB worker Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen244> line 8.
Dec 20 21:51:15 xcatn06 xCAT: xcatd: getpostscript plugin bug, pid 9562, process description: 'xCATd SSL: getpostscript for xcatn05: getpostscript instance' with error 'Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen244> line 8.#012'
Dec 20 20:53:53 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:53:56 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:00 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:03 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:06 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 21:51:31 xcatn06 xCAT: xCAT: Allowing getpostscript from xcatn05
Dec 20 21:51:31 xcatn06 xCAT: xcatd: possible BUG encountered by xCAT DB worker Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen289> line 8.
Dec 20 21:51:31 xcatn06 xCAT: xcatd: getpostscript plugin bug, pid 9565, process description: 'xCATd SSL: getpostscript for xcatn05: getpostscript instance' with error 'Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen289> line 8.#012'
Dec 20 20:54:08 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:11 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:14 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:17 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 21:51:44 xcatn06 xCAT: xCAT: Allowing getpostscript from xcatn05
Dec 20 21:51:44 xcatn06 xCAT: xcatd: possible BUG encountered by xCAT DB worker Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen334> line 8.
Dec 20 21:51:44 xcatn06 xCAT: xcatd: getpostscript plugin bug, pid 9580, process description: 'xCATd SSL: getpostscript for xcatn05: getpostscript instance' with error 'Can't call method "execute" on an undefined value at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen334> line 8.#012'
Dec 20 20:54:20 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:23 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:26 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:28 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:54:31 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0</gen334></gen334></gen289></gen289></gen244></gen244></gen199></gen199></gen154></gen154>

Dec 20 20:56:16 hv32s5fp16 dhcpd: DHCPOFFER on 192.168.5.67 to 0a:91:a7:ca:af:06 via eth1
Dec 20 20:56:16 hv32s5fp16 dhcpd: Unable to add forward map from xcatn05.ppd.pok.ibm.com to 192.168.5.67: bad DNS key
Dec 20 20:56:16 hv32s5fp16 dhcpd: DHCPREQUEST for 192.168.5.67 (192.168.5.37) from 0a:91:a7:ca:af:06 via eth1
Dec 20 20:56:16 hv32s5fp16 dhcpd: DHCPACK on 192.168.5.67 to 0a:91:a7:ca:af:06 via eth1
Dec 20 20:56:17 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases
Dec 20 20:56:20 hv32s5fp16 dhcpd: DHCPDISCOVER from 00:14:5e:b5:57:8d via eth0: network eth0: no free leases

**end*
error when run postbootscript :
begin*

Starting sshd: [ OK ]
Starting postfix: [ OK ]
Starting abrt daemon: [ OK ]
Starting xcatpostinit1++ uname

'[' '!' Linux == Linux ']'
let SLI=5170%10
sleep 0
grep 'rw /rw tmpfs ' /proc/mounts
cd /xcatpost
PATH=/xcatpost:/sbin:/usr/sbin:/bin:/usr/bin
export PATH
chmod +x /xcatpost/addsiteyum /xcatpost/aixlitesetup /xcatpost/aixremoteshell /xcatpost/allowcred.awk /xcatpost/ca /
xcatpost/confGang /xcatpost/configeth /xcatpost/confignimsh /xcatpost/createFS /xcatpost/cvs_template.pl /xcatpost/cvs
_template.sh /xcatpost/db2install /xcatpost/db2install.new /xcatpost/db2install.org /xcatpost/enabledebug /xcatpost/en
ablekdump /xcatpost/enablesysreq /xcatpost/getcredentials.awk /xcatpost/getpostscript.awk /xcatpost/hardeths /xcatpost
/hostkeys /xcatpost/HPCbootstatus /xcatpost/HPCbootstatus.aix /xcatpost/HPCbootstatus.linux /xcatpost/HPCbootstatus.li
nuxinit /xcatpost/joindomain /xcatpost/locallibs /xcatpost/locktftpdir.awk /xcatpost/makesshgkh /xcatpost/mkhyperv /xc
atpost/mkresolvconf /xcatpost/mountpost /xcatpost/my /xcatpost/nfsmounts /xcatpost/odbcsetup /xcatpost/ospkgs /xcatpos
t/otherpkgs /xcatpost/reboot /xcatpost/remoteshell /xcatpost/repos /xcatpost/resyslog /xcatpost/serialconsole /xcatpos
t/servicenode /xcatpost/servicenode.new /xcatpost/servicenode.org /xcatpost/setbootfromdisk /xcatpost/setbootfromnet /
xcatpost/setiscsiparms.awk /xcatpost/setupesx /xcatpost/setupLDAP /xcatpost/setupntp /xcatpost/_ssh /xcatpost/startsyn
cfiles.aix /xcatpost/startsyncfiles.awk /xcatpost/syncclock /xcatpost/syncfiles /xcatpost/syslog /xcatpost/umountpost
/xcatpost/unlocktftpdir.awk /xcatpost/updateflag.awk /xcatpost/upflag /xcatpost/upflagx64.exe /xcatpost/upflagx86.exe
/xcatpost/uploadboot /xcatpost/_xcat /xcatpost/xcataixpost /xcatpost/xcataixscript /xcatpost/xcatclient /xcatpost/xcat
dsklspost /xcatpost/xcatdsklspost.aix /xcatpost/xcatinstallpost /xcatpost/xcatpostinit /xcatpost/xcatpostinit1 /xcatpo
st/xcatserver /xcatpost/xcatserver.new /xcatpost/xcatserver.org
'[' -x /usr/bin/openssl ']'
++ grep '^MASTER=' /tmp/mypostscript.post
++ cut -d= -f2
SIP=
XCATSERVER=:3001
export XCATSERVER
USEOPENSSLFORXCAT=1
export USEOPENSSLFORXCAT
++ sed 's/(DHCPINTERFACES=)(.*)$/\1"\2"/' /tmp/mypostscript.post
DHCP_TMP='getpostscript plugin bug, pid 4467, process description: '\''xCATd SSL: getpostscript for xcatn05: getpost
script instance'\'' with error '\''Can'\''t call method "execute" on an undefined value at /opt/xcat/lib/per
l/xCAT/Table.pm line 2681, <GEN3836> line 8.
'\''
1'
echo 'getpostscript plugin bug, pid 4467, process description: '\''xCATd SSL: getpostscript for xcatn05: getpostscri
pt instance'\'' with error '\''Can'\''t call method "execute" on an undefined value at /opt/xcat/lib/perl/xC
AT/Table.pm line 2681, <GEN3836> line 8.
'\''
1'
echo 'updateflag.awk $MASTER 3002 "installstatus booted"'
chmod +x /tmp/mypostscript.post
'[' -x /tmp/mypostscript.post ']'
/tmp/mypostscript.post
/tmp/mypostscript.post: line 1: getpostscript: command not found
/tmp/mypostscript.post: line 1: quot: command not found
/tmp/mypostscript.post: line 1: quot: command not found
/tmp/mypostscript.post: line 1: execute: command not found
/tmp/mypostscript.post: line 1: lt: command not found
/tmp/mypostscript.post: line 1: on: command not found
/tmp/mypostscript.post: line 1: gt: command not found
/tmp/mypostscript.post: line 1: GEN3836: command not found
/tmp/mypostscript.post: line 1: line: command not found
/tmp/mypostscript.post: line 2: unexpected EOF while looking for matching `''
/tmp/mypostscript.post: line 5: syntax error: unexpected end of file
chkconfig xcatpostinit1 off
[ OK ] crond: [ OK ]
[ OK ] atd: [ OK ]

Red Hat Enterprise Linux Server release 6.0 (Santiago)
Kernel 2.6.32-71.el6.ppc64 on an ppc64

*end*

minjun found that if use xcatd -f this error will not appear and installation will be successfully.
and he will probe this problem.

Discussion

fang zheng - 2010-12-23

This problem found on AIx too.
using aix71B MN and aix71B SN install a aix71B CN:

get postscript error on CN when the bos installation finished:
begin
AIX Version 7
Copyright IBM Corporation, 1982, 2010.
Console login: 0513-059 The ctrmc Subsystem has been started. Subsystem PID is 4915364.
Thu Dec 23 02:12:36 CST 2010 xcataixpost: run: cat /xcatpost/myxcatpost_hv32s38p21 | grep 'INSTALLDIR=' 2>&1 -
Use of uninitialized value $installdir in scalar chomp at /xcatpost/xcataixpost line 169.
Use of uninitialized value $installdir in substitution (s///) at /xcatpost/xcataixpost line 170.
Thu Dec 23 02:12:36 CST 2010 xcataixpost: run: cat /xcatpost/myxcatpost_hv32s38p21 | grep 'TIMEZONE=' 2>&1 -
end
On SN got the message:
begin
bash-3.2# xCAT: Allowing getpostscript from hv32s38p21
DBD::DB2::db prepare failed: [IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "192.168.5.112". Communication function detecting the error: "recv". Protocol specific error code(s): "", "", "0". SQLSTATE=08001
DBD::DB2::st execute failed: [IBM][CLI Driver] CLI0108E Communication link failure. SQLSTATE=40003 at /opt/xcat/lib/perl/xCAT/Table.pm line 2681, <gen478> line 8.
xcatd: possible BUG encountered by xCAT DB worker Describe failed during DBI::st=HASH(0x23210de8)->FETCH(NAME) at /opt/xcat/lib/perl/xCAT/Table.pm line 2682, <gen478> line 8.</gen478></gen478>

xcatd: getpostscript plugin bug, pid 6422630, process description: 'xCATd SSL: getpostscript for hv32s38p21: getpostscript instance' with error 'Describe failed during DBI::st=HASH(0x23210de8)->FETCH(NAME) at /opt/xcat/lib/perl/xCAT/Table.pm line 2682, <gen478> line 8.
end
and the CN is on booting status at last.</gen478>

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

minjun xi - 2010-12-29

Lissa,
I spent a lot of time on debugging on the RHELs6 environment. Here is what I found:
If the "xcatd" daemon on the service node is running foreground, "getpostscript.awk" will be OK without any errors. But if the xCAT service is running background, "getpostscript.awk" will be failed like Fang Zheng described here.

I can recreate this issue without re-installing the compute node. Here is what I have done:
1) login the compute node;
2) running the command:
USEOPENSSLFORXCAT=1 XCATSERVER=192.168.5.78:3001 /xcatpost/getpostscript.awk
3) check the command's output; you will see the error message.

besides, I also added several debug spots in xCAT code, the error is triggered in the line 318 of perl/xCAT/Postage.pm, where "xCAT::Utils->isSN($node)" is called.

then, I tracked into the "isSN()" subroutine, the error is triggered in line 3861 of perl/xCAT/Utils.pm, where "$servicenodetab->getAllNodeAttribs(['tftpserver']);" is called. Following this thread, I guess it is related with the code to handle the database.

I am curious about it because when "xcatd -f" is running, there's no issue for any xCAT commands.

Thank you!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Lissa Valletta - 2011-01-05

Is this still a problem because xcatn05 seems to be installed.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

fang zheng - 2011-01-06

yes, this problem still exist on xcatn05 which is a cn.
after the installation :
Starting xcatpostinit1/tmp/mypostscript.post: line 1: quot: command not found
/tmp/mypostscript.post: line 1: getpostscript: command not found
/tmp/mypostscript.post: line 1: quot: command not found
/tmp/mypostscript.post: line 1: execute: command not found
/tmp/mypostscript.post: line 1: lt: command not found
/tmp/mypostscript.post: line 1: on: command not found
/tmp/mypostscript.post: line 1: GEN822: command not found
/tmp/mypostscript.post: line 1: gt: command not found
/tmp/mypostscript.post: line 1: line: command not found
/tmp/mypostscript.post: line 2: unexpected EOF while looking for matching `''
/tmp/mypostscript.post: line 5: syntax error: unexpected end of file

you can run
nodeset xcatn05 install and rnetboot xcatn05 on hv32s5fp16(9...68) to make this problem.

On aix the problem is still exist too.
run nimnodeset -i 1042A71BCNimage hv32s38p21 and rnetboot hv32s38p21 on Mn(hv32s38p01 9...85). you can try to install the cn and see the result, but please don't do any modify for this is my test environment.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Bruce - 2011-01-31

Lissa, Fang Zheng,

Project mgmt is now tracking sev 1 (priority 9 & 8) and sev 2 (priority 7) bugs. Lissa, do you know what is causing this problem? Fang Zheng, is this still an important enough bug for it to be priority 7? Is it blocking your testing?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Lissa Valletta - 2011-02-01

I believe this is the same as defect 3167038 , which we know the cause as of yesterday, but I am waiting for a response from Jarrod. Since the change was to fix a memory leak/performance proble, I don't just want t change it.
This is the same machine Linda and I were debuggin on so I am pretty confident , this will fix the problem.
To prove we can fix the code temporarily in NodeRange.pm

Just remove the unless around the new function this lie

Try to match groups?

unless ($grptab) { $grptab = xCAT::Table->new('nodegroup'); }
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Lissa Valletta - 2011-02-15

I believe if you get the latest snapshot of 2.6, you will not hit this problem.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

SourceForge Robot - 2011-03-15

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 28 days (the time period specified by
the administrator of this Tracker).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

postscript error when installing a CN using SN on rhels6

An extreme cluster/cloud administration toolkit

Milestone

Searches

Help

#1607 postscript error when installing a CN using SN on rhels6

Discussion

Try to match groups?