Island filtering should happen in graph compression stage
Brought to you by:
mcschatz
Islands are nodes which are unconnected to other nodes. Islands are currently filtered out in RemoveTipsAvro.
We should be trying to filter out islands (and other graph structures) as close as possible to the location in the processing where they are created. For an island, this means right after graph compression. The downside of moving island filtering into the compression stage is that we have two different MR jobs QuickMerge/PairMergAvro where these islands might be produced. So there two places we need to add the filtering. However, I think we could create a common class NodeFilter which gets applied in both places.