vEMan - VMware ESX/ESXi Manager / Community Support / vEMan - Support Forum: Error checking authentication or ... Return message was: Server version unavailable

John Lumby - 2012-05-17

I am running current vEMan v0.8.4 on debian amd64 squeeze,
trying to connect to an ESX V4.1.0 build 348481 server.
I finally got all the pre-reqs in place including the ovftool V2.0.1,
and the vEMan client window comes up, but after I enter the server login, vEMan gives me

meine data: /home/lumby/.vEMan/sessiondata --server www.xxx.yyy.zzz --username root --password ***
DEBUG: Error checking authentication or creating cookie file (lib F_CRTCOOKIE)
DEBUG: Return message was: Server version unavailable at 'https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl' at /usr/local/share/perl/5.14.2/VMware/VICommon.pm line 545.
DEBUG: aborted by user

is there any workaround or fix for this?

John

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- secure diversITy - 2012-05-17
  
  first of all thx coming over here from the GB ;o)
  
  Hm normally this error should not occur when you can reach your server.. Can you please provide the "function.log" inside the vEMan installation directory?
  
  Please ensure in 'etc/sysvars_vEMan.cfg' that the variable is set:
  
  PERL_LWP_SSL_VERIFY_HOSTNAME=0
  
  Can you also please provide:
  
  $> grep export libs/runFUNCTION.vEMan
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - John Lumby - 2012-05-18
    
    yes PERL_LWP_SSL_VERIFY_HOSTNAME=0 in sysvars
    
    grep export libs/runFUNCTION.vEMan
    export PERL_LWP_SSL_VERIFY_HOSTNAME
    . $CFG && echo DEBUG: Setting file included.. && export VUSER SRV
    . $SOURCES && echo "DEBUG: Source file included.." && export LASTSOURCE
    export AANS
    
    function.log:
    
    (Note - yesterday the user running vEMan did not have write permission to the vEMan_v0.8.4 directory as it was installed by root. so no function.log created,, today I chowned recursive that dir to the runtime user and now same problem and there is a function.log :)
    
    DEBUG: Argument given is valid (F_MAINLOOP)
    DEBUG: User variable file ./etc/uservars_vEMan.cfg included successfully.
    DEBUG: System variable file ./etc/sysvars_vEMan.cfg included successfully.
    DEBUG: Res before:
    DEBUG: TARGETVM is , AANS is:
    DEBUG: F_ESXMGR starting
    DEBUG: checking config...
    DEBUG: checking config reqs..
    DEBUG: Setting file included..
    DEBUG: No source file detected.
    DEBUG: config reqs checks (F_CHKCFG) finished.
    DEBUG: config ok.
    HASH is U2FsdGVkX19E6txiFWvA1Ad2Y4IoXMbwa0BgYviAveg=
    DEBUG: checking config...
    DEBUG: checking config reqs..
    DEBUG: Setting file included..
    DEBUG: No source file detected.
    DEBUG: config reqs checks (F_CHKCFG) finished.
    DEBUG: config ok.
    tail: /home/lumby/.vEMan/progress.tmp: file truncated
    No session file detected. Creating one..
    HASH is U2FsdGVkX19E6txiFWvA1Ad2Y4IoXMbwa0BgYviAveg=
    meine data: /home/lumby/.vEMan/sessiondata --server www.xxx.yyy.zzz --username root --password **
    DEBUG: Error checking authentication or creating cookie file (lib F_CRTCOOKIE)
    DEBUG: Return message was: Server version unavailable at 'https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl' at /usr/local/share/perl/5.14.2/VMware/VICommon.pm line 545.
    DEBUG: aborted by user
    DEBUG: reached exit function
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-21

hm strange. What VMware Perl toolkit version do you use?
And do you have special characters in your username and/or password?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Lumby - 2012-05-21
  
  Last week I was using VMware-vSphere-CLI-4.1.0-254719
  Today I tried with VMware-vSphere-CLI-5.0.0-615831 but still same result.
  No special chars in uname or passwd
  
  I am not familiar with the VMware Perl toolkit - I simply unpacked and then ran
  perl Makefile.PL PREFIX='/usr/local'
  make
  make install
  
  and then set
  XVMCMD=/mnt/soltbakp/sysbuild/vmware-vsphere-cli-distrib/bin/vmware-cmd # Part of VMware Toolkit for Perl. Tested with version: 4.1,5.0
  XESXCLI=/mnt/soltbakp/sysbuild/vmware-vsphere-cli-distrib/lib/bin/esxcli/esxcli
  in the uservars.
  Is that all that is needed? It seems from the log that it found my /usr/local/share/perl/5.14.2/VMware/VICommon.pm ok.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - John Lumby - 2012-05-21
    
    One other thing I've noticed that just might be relevant:
    
    since the error message stated
    
    Server version unavailable at 'https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl' at ...
    
    I typed that url into my (firefox) browser just to see what would happen,
    and it came back with the message indicating that this is an untrusted site (no recognized certificate, do you accept the risk, etc etc),
    so I replied yes I accept and then it brought up the page (as raw xml,
    with the message about
    "This XML file does not appear to have any style information associated with it. The document tree is shown below"
    above it)
    
    Could the fact it required me to interactively accept the risk be the cause of failure when vEMan tries to access it? If so, can I somehow copy the cert exception from my firfox profile into somewhere for vEMan??? (I am just wild-guessing here ...)
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
    - secure diversITy - 2012-05-22
      
      Well... You're right that the certificate verification causes problems in vEMan but they were solved in v0.8.3...
      
      Can you try this both things please:
      
      1) Execute: vmapps/general/connect.pl --server xxxxxxxx --username xxxxxx
      2) Download vEMan v0.8.2 and test to login with that version
      
      Thanks
      Thomas
      
      If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
      - John Lumby - 2012-05-22
        
        1a) Execute: vmapps/general/connect.pl --server xxxxxxxx --username xxxxxx
        ==>> same error as always "Server version unavailable at ..."
        
        1b) I also tried this one
        /mnt/soltbakp/sysbuild/vmware-vsphere-cli-distrib/bin/vmware-cmd -H www.xxx.yyy.zzz -U uuuuu /vmfs/volumes/4ac267b4-9a834b39-585a-001ec955b85d/rest-of-path-to-vmx getstate
        ==>> same error as always "Server version unavailable at ..."
        
        but ---
        1c)
        ssh www.xxx.yyy.zzz -l uuuu 'vmware-cmd /vmfs/volumes/4ac267b4-9a834b39-585a-001ec955b85d/rest-of-path-to-vmx getstate'
        works ok and tells me the state
        
        2) Download vEMan v0.8.2 and test to login with that version
        ==>> exactly same behaviour as 0.8.4
        
        Based on 1c) I wonder - could vEMan have an option to run its commmands remotely via openssl rather than running command locally using the local vm perl tk???
        
        If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-22

Strange. Well that ssh workaround could be an option but I believe that needs many code changes.. :( But if you want to help me you are free to contribute ;)

Can you pls provide:

dpkg -l perl openssl

and optional:

dpkg -l libcrypt-ssleay-perl

thx
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Lumby - 2012-05-22
  
  dpkg -l perl openssl
  
  Desired=Unknown/Install/Remove/Purge/Hold
  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name Version Description
  +++-=============================-=============================-==========================================================================
  ii openssl 0.9.8o-4squeeze4 Secure Socket Layer (SSL) binary and related cryptographic tools
  ii perl 5.14.2-10 Larry Wall's Practical Extraction and Report Language
  
  dpkg -l libcrypt-ssleay-perl
  
  Desired=Unknown/Install/Remove/Purge/Hold
  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name Version Description
  +++-=============================-=============================-==========================================================================
  un libcrypt-ssleay-perl <none> (no description available)</none>
  
  Let me know if that optional one should be installed.
  
  I will think about your offer to contribute but as my purpose in switching from using windows client to vEMan is to save me time ....
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-23

ok I have tested with an older openssl version (0.9.8k-7ubuntu8.11) and with a newer than yours (1.0.1-4ubuntu5) , the same for Perl so it seems(!) to be not perl/ssl related. At least for those both packages. The optional package is not installed in my both test systems so it should be ok without it.

Hm I believe I need a ESX v4.1 because it could be a problem with that specific version maybe. Could you test another ESX server with a 4.0 or 5.x?
And is it an ESX or ESXi system?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-23

Well I looked around and found an ESXi v4.1 here and tested vEMan v0.8.4 from my both test machines against it.
Working fine :o(

Nevertheless would be fine if you could test against another ESX with another version if you have the possibility to do so..

If that is a certificate problem the PERL variable should have fixed that but it may not working as expected on your system. Please try also this:

PERL_LWP_SSL_VERIFY_HOSTNAME=0 https_proxy="" vmapps/general/connect.pl --server XXXXX

Besides that could you try to do the following:

curl -o - https://servername:443/sdk/vimService.wsdl

--> Should give an error about a certificate verify problem. That is normal but I want to see the result here nevertheless

Then try this:

curl -ko - https://servername:443/sdk/vimService.wsdl

--> This should give you an XML output

Another thing I want to know is do you need a Proxy to reach your ESX?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Lumby - 2012-05-23
  
  PERL_LWP_SSL_VERIFY_HOSTNAME=0 https_proxy="" ./vmapps/general/connect.pl --server www.xxx.yyy.zzz --username uuuuuuuu; }
  /mnt/soltbakp/sysbuild/vEMan_v0.8.4
  Wed May 23 10:36:57 EDT 2012
  Enter password:
  Server version unavailable at 'https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl' at /usr/local/share/perl/5.14.2/VMware/VICommon.pm line 545, <stdin> line 1.</stdin>
  
  curl -o - https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl
  curl -o - https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl
  curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
  error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
  More details here: http://curl.haxx.se/docs/sslcerts.html
  
  curl performs SSL certificate verification by default, using a "bundle"
  of Certificate Authority (CA) public keys (CA certs). If the default
  bundle file isn't adequate, you can specify an alternate file
  using the --cacert option.
  If this HTTPS server uses a certificate signed by a CA represented in
  the bundle, the certificate verification probably failed due to a
  problem with the certificate (it might be expired, or the name might
  not match the domain name in the URL).
  If you'd like to turn off curl's verification of the certificate, use
  the -k (or --insecure) option.
  
  curl -ko - https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl
  curl -ko - https://www.xxx.yyy.zzz:443/sdk/vimService.wsdl
  
  <definitions targetNamespace="urn:vim25Service"
  xmlns="http://schemas.xmlsoap.org/wsdl/"
  xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  xmlns:interface="urn:vim25"
  
  <import location="vim.wsdl" namespace="urn:vim25">
  <service name="VimService">
  <port binding="interface:VimBinding" name="VimPort">
  <soap:address location="https://localhost/sdk/vimService">
  </soap:address></port>
  </service>
  </import>
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - John Lumby - 2012-05-23
    
    in my previous post, somerthing corrupted the xml output that I pasted in but anyway the curl -ko returned the xml text correctly.
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
    - John Lumby - 2012-05-23
      
      I forgot to say - no proxy
      
      If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ok..
is it possible for you to test another ESX?

Next thing to check:

openssl s_client -connect yourservername:443 -showcerts -debug -state -prexit

STRG + C to exit

--> that should give the certificate of your esx.
check at least if the time values are valid.

The next will be a change on your ESX server:

1) Login by SSH on your ESX
2) Ensure you're user "root" or use "su" to become root
3) Backup your current cert and CA:

   tar cvf oldsslcerts.tar /etc/vmware/ssl/

4) Generate a new ESX cert (will be autofilled based on the ESX hostname):

   /sbin/generate-certificates.sh

5) Restart your management agents

   on ESX:
   service mgmt-vmware restart
   service vmware-vpxa restart

   on ESXi:
   /sbin/services.sh restart

6) Rename the old certificate in vEMan:

   mv certs/yourservercert.crt certs/yourservercert.crt_OLD

7) Ensure the new certificate is displayed:

   For this the easiest way would be to simply start vSphere on Windows -->
   You should see a certificate error message because the cert has changed
   Click on display certificate and double check if the cert is ok 
   (dates, name == DNS name)

   If you get no certificate error here you either have the CA of VMware installed 
   (but I don't believe that) or you need to restart the ESX...

   Another way to check would be starting vEMan and then displaying the new
   fetched cert under ./certs/yourserver.crt with openssl.

8) If 7 goes ok simply start vEMan as usual

9) If that is not working do the previous connect check again:

   PERL_LWP_SSL_VERIFY_HOSTNAME=0 https_proxy="" ./vmapps/general/connect.pl --verbose --server www.xxx.yyy.zzz --username uuuuuuuu

   Please give me the output of that.

   Please then install the following package in your system:
   "libcrypt-ssleay-perl"

   Then start the above connect.pl again and give me also that output
   If connect.pl is working after installing the above package try to start vEMan as usual.

Thx for your patience!

Hint: I believe the best way to ensure that all log etc output you paste here will be displayed is to use 4 tildes ( ~~~~ ) before and after your pasted text. See "Formatting Help" button above your text box if you're unsure

John Lumby - 2012-05-23

I do not have any access to any other ESX, just this one.

I ran the openssl comand and it returned a lot of data.
I can't paste it here (I cannot see anything in the formatting help to say how how to suppress prettifying, only how to prettify).

I am not that familiar with internals of certificates.
You said "check at least if the time values are valid."
I did not see any time values except near the end, a session start time in epoch-seconds which was valid.

The only thing I noticed which might be interesting :

verify error:num=18:self signed certificate
verify return:1

Concerning "next will be a change on your ESX server:"
unfortunately I can't do that (or not now anyway) as many users all use this system.

Thanks for your patience too. However I think mine has run out!
I think I will fall back on using ssh of vmware_cmd for poeroff/poweron plus vmplayer of remote system to bring up basic gui console.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-23

Well it makes me not happy ;) but I can fully understand you! If your workaround is working so far it will be the fastest solution of course.
The change on the ESX server will disconnect active vSphere(!) connections only but should not affect the running VMs itself. In all my tests I can say that it is true (http://www.vmware.com/webmaint/kb_maintenance.html?id=988).

Well maybe some day you come back and we can proceed ;o)

If you have 2 more seconds you can simply try to install the package "libcrypt-ssleay-perl" and try connecting vEMan again - would be very nice getting your feedback for that - it may be enough...

But if not it's also ok, of course.

Thx again
Thomas

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Lumby - 2012-05-24
  
  I apt-get install 'ed libcrypt-ssleay-perl and now it works! At last!
  
  But something strange and maybe that (libcrypt-ssleay-perl) was not the cause of earlier probs :
  After installing libcrypt-ssleay-perl, on my first attempt to run the vEMan, up came a new window I had never seen before from yad telling me my ovftool was unsupported v 2.1. This was strange since I had installed ovftool 2.0. But it seems (I can't be quite sure) that when installing one or other of the vmware/vmware-related products, that install created symlink
  /usr/bin/ovftool -> /usr/lib/vmware-ovftool/ovftool
  I erased that symlink and now it finds my ovftool-2.0 and everything works.
  
  I suggest you maybe make the libcrypt-ssleay-perl mandatory!
  
  Thanks again John
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-24

Hi John

I can't believe that ;o)) Great to hear and thx for trying out the package installation!

Keeps strange for me too because it means it has something to do with that package but on my both test machines it is not installed?! That package was just a try because of the Crypt::SSL module within.

Well I will think about an error checking mechanism which may point to that package/module install maybe.. I'm not sure if that is really needed or not but.. well it's strange :o)

Thanks again for your time and reporting back!
Thomas

PS: What may would happen if you uninstall "libcrypt-ssleay-perl".... :o))

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Lumby - 2012-05-24
  
  I dpkg -r libcrypt-ssleay-perl uninstalled libcrypt-ssleay-perl
  
  and then vEMan comes with
  
  DEBUG: Error (1) checking authentication or creating session cookie file (F_CRTCOOKIE)
  DEBUG: Return message was: Crypt::SSLeay is required for https connections, but could not be loaded: Can't locate Crypt/SSLeay.pm in @INC (@INC contains: /mnt/soltbakp/sysbuild/vEMan_v0.8.4/vmapps/general/../ /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl .) at /usr/local/share/perl/5.14.2/VMware/VICommon.pm line 508.
  
  which is a new (to me) and interesting result.
  
  I then apt-get install libcrypt-ssleay-perl again
  
  and then vEMan works again.
  
  Can it be that it is the combination of unverifiable (self-signed) vert AND lack of libcrypt-ssleay-perl which causes the original problem I had. You said "has something to do with that package but on my both test machines it is not installed?!" but maybe the certs on your ESX are all from a CA?
  
  But it also seems your vEMan has learning capabilities and after finding that it works when libcrypt-ssleay-perl is installed, it learns to require that package!!
  
  I think I need not run any more experiments but suggest you simply make this package mandatory always (without the learning!) since it is easy to install and maybe sometimes saves some lengthy trouble-shooting.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-24

John.. Thanks again! Your work is highly appreciated and of course I will take that on the checklist for vEMan (without the self-healing part ;) )
I believe that some people already run in that bug but no one else then you has reported it.
So thx again, I will change that as a requirement in the next version of vEMan.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

secure diversITy - 2012-05-24

I only want to mention that my ESX servers are all self signed default stuff no own CA or whatever... It is strange but I will not investigate that further if installing this Perl module fix that it will be set as required and good :o)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Error checking authentication or ... Return message was: Server version...

An unofficial VMware® ESX/ESXi Manager for Linux

Forums

Help

Error checking authentication or ... Return message was: Server version unavailable

dpkg -l perl openssl

dpkg -l libcrypt-ssleay-perl