#39 socket not work on windows

[Anonymous]

I built libvnc on ubuntu with mingw32, and I run example/example.exe on both Windows XP and Windows 7. I got the following error:
httpCheckFds: select: Unknown error
rfbCheckFds: select: Unknow error,
WSAGetLastError() return 10038.

the reason is for Windows, SOCKET was not defined int, it is unsigned int. you can find it in
/usr/i586-mingw32msvc/include/winsock2.h, (on my ubuntu) or
ftp://ftp.microsoft.com/bussys/wINSOCK/winsock2/winsock2.h

typedef unsigned int u_int;
typedef unsigned long u_long;

/*
* The new type to be used in all
* instances which refer to sockets.
*/
#ifdef _WIN64
typedef UINT_PTR SOCKET;
#else
typedef u_int SOCKET;
#endif

So, the following code will always get to run：
if (rfbScreen->httpSock >= 0) {
FD_SET(rfbScreen->httpSock, &fds);
}
even the value of httpSock is -1.
--- a/rfb/rfb.h
+++ b/rfb/rfb.h
I did the following chang:
#ifdef __MINGW32__
#undef SOCKET
#include <winsock2.h>
+#undef SOCKET
+#define SOCKET int
#ifdef LIBVNCSERVER_HAVE_WS2TCPIP_H
#undef socklen_t
#include <ws2tcpip.h>

and tested on both windows7 and windows xp. it works.
but if some new file don't include rfb.h, then it will met this bug again.
in winsock2.h
#define INVALID_SOCKET (SOCKET)(~0),
it is -1.
So we'd better to change all of the if condition test, to compare with INVALID_SOCKET. not >= 0.

and another issue is, on Windows, we can't use read to recv packets. we have to use recv. So I also got
httpProcessInput: read: No error
I did following changes:
--- a/libvncserver/httpd.c
+++ b/libvncserver/httpd.c
@@ -300,8 +300,8 @@ httpProcessInput(rfbScreenInfoPtr rfbScreen)
return;
}

- got = read (rfbScreen->httpSock, buf + buf_filled,
- sizeof (buf) - buf_filled - 1);
+ got = recv (rfbScreen->httpSock, buf + buf_filled,
+ sizeof (buf) - buf_filled - 1, 0);

I tested on both Windows XP, and Windows7, it works.

all of my testing are only run example. Thanks.

I also upload my diff.

[Anonymous]

My changes for testing

[Christian Beier]

Hi there, thanks for the testing - could you please provide patches created via git format-patch against current git master? That'd be awesome!