I'm thinking about creating tool to markup video (select areas containing objects, add tags) for my computer vision algorithms (head tracking, human detection and the like). The core functions of such tool: split video into separate frames, possibility to markup some object by drawing rectangles, circles or other closed contours objects on each frame, play markuped video, interpolate unmarked object coordinates between frames.

It is seemed to me, that within inkscape it can be done in the following way:
1. Input extension that "convert" video frames into inkscape layers
2. Toolbar with play/stop buttons that toggle layer visibility to simulate video playback

if you catch the idea, can you tell how difficult to implement the tool on inkscape basis? If it isn't very hard task what is the best way to do it: creating extension or creating patch?

Best regards,
Alexander Gorban