There are lots of cases of retaliation, or 'disagreement on rewards'. The latter isn't really abuse, but it causes a lot of unhappiness among users. Making flagging more powerful would make that worse.

There needs to be a balance. I'm not saying that the current 1:1 between upvoted and downvotes is 100% for certain the right one, but since it is the current implementation the burden is on you (since you are proposing the change) to show beyond a reasonable doubt that the current system is not balanced correctly.. There does still seem to be a good argument in preserving the status quo.