Well, it won't really come out accidentally. It'll come out with humans shaping it. What's particularly scary is that the most greedy humans are generally working on this. I just hope someone will not be stupid enough to try optimising their gains without putting any "humane" protections; otherwise he'll eventually cause a rogue AI.