There are a few other letters which dropped out of use over the years. My favourite (which I occasionally use when I'm writing by hand rather than typing) is Thorn, which represents the "Th" sound. It's quicker to write and with Th being so commonly used it makes sense to have a single character to represent it.

But it fell out of use with the invention of printing, because the typesets initially used were from the continent and didn't have the thorn character, so they substituted Y (which at the time was hardly ever used in English). Hence, when you see something along the lines of "Ye Olde...." pronounced with a Y sound it's wrong, it's still pronounced "The" just that they didn't have the thorn character in their typeface tray.