i get a black screen when i connect to a particular server. can anyone give me a hint on how to debug the problem?
like where to find out if data is actually coming in. perhaps various places where i can log something.
I guess this is obvious, but it's probably either something with the connection set-up or with processing the first frame. The handshaking is mostly in RFBHandshaker.m. Some of the final stages are in the RFBProtocol init message as called from start: in RFBConnection.m. Once that last function has returned, the final set up has been performed and the first frame buffer requested, so then it's just a matter of waiting for the server to return the first frame buffer. The central point for reading frame buffer updates is in FrameBufferUpdateReader.m, which dispatches to the readers for various types of rectangles.
Note that actually understanding the RFB protocol from the source is probably not very easy. You can find the official RFB documentation on the Internet, but I like http://tigervnc.org/cgi-bin/rfbproto, which is convenient because it's in HTML and also includes a little more information in places where implementations differ.
One thing which you can do without even recompiling or turning on a debugger is to disable all of the encoding other than Raw and try connecting. If this works, then you can experiment and narrow the problem down to a specific encoding. If you have problems even with the Raw encoding, then it's probably a problem with the handshaking, because the Raw encoding is so simple that I doubt that there are any major bugs in it.
thanks for the help. i was able to debug it and found the problem. my connection uses a non-standard authorization where the client sends a special string first thing after the connection is made. my string had an extra newline character at the end which threw the server off. so there was no fault in the chicken code at all, but i feel better knowing how to step through the handshake. i had tried changing the protocol the clients to 3.07 and was getting spurious info from the server. it didn't work til i went back to 3.08. also, it was only working in 32bit build, not 64bit. that must be the fault of the particular server i was connecting to since i was able to connect using the 64bit build to a remote windows vnc server.