I'm not an expert but it seems that a frame buffer driver
that can do what your VGA console does should not be
very hard to write.
The same console can be used for X server as well with
the frame buffer driver. At the begining it can be