I just saw an answer from a new user (1 reputation point joined today) with 3 downvotes I'd be surprised if that guy ever decided to return to the site because of the welcome he received. Only one person who downvoted decided to leave a reason for his vote.
I think it might make sense to give new users (by length on site or after a certain amount of questions asked/answered) a protected status.
Now that I'm thinking about it...what point does a negative vote have? If the idea of the system is to bubble quality answers and questions to the top, shouldn't an upvote be enough? A bad answer will remain at zero, good answers will get more points.