Gosh I feel the same way about quite a lot of other things actually. Starting from a young age feels like a cheat code because by the time you're old enough to decide whether you like it or not, you're already pretty darn good. On the other hand, one of my teachers ended up being a well revered musician despite beginning (from scratch) in her 20s. So I think it can be done if you really want!