As best I can tell there is no checking in the system for alternative mailhosts that are lower priority MX records, and subsequently they are not excluded, therefore upon not finding matches in virtusertable, or mailertable, this milter will try lower priority MX records for the domain, as well as localhost on a round-robin basis (depends on your DNS I guess), thereby intermittently causing weird problems.
If I remember my RFC's correctly, there should be no attempt to try lower priority MX records by a higher priority system.
Also, it makes no sense at all for the system to bang on its own front door, as it does not appear to recognise when a domain is local unless it points to localhost, or 127.0.0.1, when any server machine may have multiple local addresses.
I am willing to help rectify these issues, but uncertain if I wish to continue using this Milter in the face of these problems without prompt resolution of some kind.