Gold would be a lot harder to mine if we could only dig for it with spoons, would that make it worth more? Maybe, but only because there would be less of it.
Bitcoin is being minted at roughly same rate regardless of difficulty, which has been ballooned up ridiculously high due to its counterproductive collective action problem.
Ultimately, how do you value something? By the difficulty of obtaining it, or by the utility it provides? Pretending it was difficult to obtain does not make the obtained item more valuable. If I spent $1 million mining the most difficult ounce of gold in the world, could I demand $2 million for it? No. People would think I was idiot for mining it.