Bitcoin already keeps track of which nodes have seen what to avoid redundant inv announcements.
I think if you are approaching most transactions in a block matching the filter then you would just request full blocks and do all the filtering client side
RE: sharing parts of the merkle branches when returning a 'merkleblock' :
I think I agree that complicating the BIP for what should be a very
rare case (more than a handful of transactions in a block match the
transactions in your wallet) is the right decision.
I want to make sure I'm understanding this bit correctly:
"In addition, because a merkleblock message contains only a list of
transaction hashes, any transactions that the requesting node hasn't
either received or announced with an inv will be automatically sent as
well. This avoids a slow roundtrip that would otherwise be required
(receive hashes, didn't see some of these transactions yet, ask for
Requiring serving/relaying nodes to keep track of which transactions
they have or have not sent to their peers makes me nervous. I think
requiring an extra 'inv' round-trip would be simpler to implement and
less likely to lead to some kind of DoS attack.