LinkedIn contacts keep coming back to People Hub

I have way too many LinkedIn contacts, which I always filter out of the People Hub, in order to keep the hub actually usefull. However, every so often the contacts come back up, and I have to filter them out again. Twitter contacts which are also filtered out keep away. Anybody else experiencing this?