Honestly? I'm all for removing the downvote button. It feels like it causes more problems than helps, and it's primary purpose can easily be misinterpreted since it's such a vague system. Most any reason you can think to downvote something, can just be hit with the 'blacklist' excuse from someone else.
Don't like a topic in the art? Blacklist it. Don't like the artists style? Blacklist them. Don't like the character in the post? Blacklist it. Don't think it's a good post? Just blacklist the post itself.
If a post isn't good, just don't upvote it, don't even bother with having a downvote button. If the art is bad then just leave it with no score.
Genuine use of the downvote button can't be discerned from disingenuous use, why? Because of it's vagueness, it's just an up 1 or down 1, the reasoning behind it can be literally anything from something as serious as someone being hateful to something as benign as someone just disliking the art in question. This has unfortunately led to problems with posts that might have sensitive or protected content, you can't honestly take scores on these posts seriously due to 'us vs them' kinds of people and those who are trying to just leave their opinion on something can be unfortunately lumped into groups they do not associate with.
How would I fix it? I think the best way, for both users and others, is just to replace the downvote button with a button that just blacklists that specific post. The artist doesn't have to deal with their posts getting downvote bombed, and users can more easily avoid posts they do not like. Then again, making the blacklist more user friendly and more obvious can also help as well.
Updated by Rainbow Dash