You should add an icon to each window (on the left / top of each window).
In the past I attempted to figure out the FLTK api for this and could not find the right information in the online documentation.
Perhaps I missed something simple.
If the right api is pointed out to me I would be happy to add this feature.
I found this post on the fltk mailing list. Maybe it will help.