Actually, they never left. We have quite a few rabbits hanging around our property. We see them every morning over by the gazebo or near the house munching on clover. They are pretty used to us. We can usually walk right by them (within 10 feet) without them hopping away. During the day, you can usually find one under a boxwood by the gazebo.
We love to see the critters, but they do cause damage. We have had put small wire cages around the silver lace clematis and the peonies. As the summer comes on, we notice that that flowers on the Asiatic lillies seem to get eaten.
You can't fight nature. A little bit of wire is a lot better than not having the critters.