Do it!!
Seriously, I know how you feel and you got my full sympathy!
We live in an apartment complex and have neighbors to the left, right and below us. Then there is another house infront of ours not even 30 feet away.
Cannot wait to move either. I am just glad we do not have any one living above us, cause our neighbor downstairs tends to complain how loud our cats are running around in the middle of the night.
