found a practical implementation of my original idea.
Now, each matched pixel-group is flood-filled, and the resulting region
is added to the search-border. Matched pixel-groups that already exist
in the search-border are not flood-filled again, saving lots of time.
The match-throttles still exist to keep pathological cases from
happening, but now regions can be removed from the search-border,
pruned of resolved pixels, and re-added to the search-border if it's no
longer the biggest match. y4mdenoise is finally able to apply the
largest matches and work its way down to the smaller matches without
losing any potentially useful information.
This is "beta" because I haven't finished all the performance
improvements I want to do, and I haven't rewritten the manual page yet,
but it's working well enough to unleash on the other mjpegtools
developers for their comments.