Remove peers locally after we lost contact. Also works for nodes we got passed via the acknowledge message.
Authored by: dannybackx 2019-03-24
Parent: [r198]
Child: [r200]