Example:
I guess the problem occurs due to not calling breakTrulyMixedBoundaries for the mixed shapes (i.e. not using slower method as given in the comment). I gave the following fix (marked >>> below) in line 243 of Diagram.java SVN revision 80 and it is working, but I am not sure if this is the correct fix.
//subtract from each of the mixed sets all the closed sets
sets = mixed.iterator();
while(sets.hasNext()){
CellSet set = (CellSet) sets.next();
Iterator<CellSet> closedSets = closed.iterator();
while(closedSets.hasNext()){
CellSet closedSet = closedSets.next();
set.subtractSet(closedSet);
}
>>> boundarySetsStep2.remove(set);
>>> boundarySetsStep2.addAll(set.breakTrulyMixedBoundaries(workGrid));
// this is necessary because some mixed sets produce
// several distinct open sets after you subtract the
// closed sets from them
if(set.getType(workGrid) == CellSet.TYPE_OPEN) {
boundarySetsStep2.remove(set);
boundarySetsStep2.addAll(set.breakIntoDistinctBoundaries(workGrid));
}
}