One aspect is holding your private keys

But then you have to be able to treat those safely so you yourself aren't exposed to somebody stealing your coins.

Or other stories about people not having a proper backup of their seed, (or worse yet, storing that backup on the same device as their wallet) and then losing all their coins when their phone or laptop dies.

For some, it's probably safer on an exchange