x11vnc allows one to view remotely and interact with real X displays (i.e. a display corresponding to a physical monitor, keyboard, and mouse) with any VNC viewer. This project provides pre-built binaries, see LibVNCServer for the x11vnc source code.