Btukfyl

How does one prove that the Dirac Delta distribution is the eigenfunction of the position operator $hat{x}$? In math, why does $langle x’|xrangle = delta(x’-x)$?

This is true not just for position eigenvectors. This is true for all eigenvectors in their own (orthonormal) eigenbasis.

For an operator with discrete eigenvalues $n$ with eigenvectors $|nrangle$,
$$langle n'|nrangle=delta_{n,n'}$$
where $delta_{n,n'}$ is the Kronecker Delta that is $1$ when $n=n'$ and $0$ otherwise.

For an operator with a continuous spectrum of eigenvalues $p$ with eigenvectors $|prangle$,
$$langle p'|prangle=delta(p'-p)$$
where the $delta$ this time is the Dirac delta function.

This is all just because when expressing a vector in a basis where that vector is a basis vector, it only has a component along that one vector (itself).

This isn't part of the main question, but I think it relates to the general idea of what the above expressions really mean and why they are used in the way they are used.

Do you happen to know why it is necessary to make the eigenfunction infinitely tall when working in the continuous spectrum?

This is a poor picture of the Dirac delta function. Lets first look at the Kronecker delta. If we have this in a sum
$$sum_{n'} delta_{n,n'}c_{n'}$$
We know this sum will just evaluate to $c_n$. The Kronecker delta "kills off" all other $c_{n'}$ terms.

Now let's move to the continuous version of this example with the Dirac delta function:
$$int delta(p-p')c(p')text d p'=c(p)$$
Now the integral is a "continuous sum" of the terms $c(p')delta(p-p')text d p'$. Notice how here we have $text d p'$ in each term. Therefore, if we want to pick out just $c(p)$ from this sum, we need $delta(p-p')$ to be equal to $1/text d p'$ when $p'=p$ and $0$ otherwise. Since $text d p'$ is an infinitesimal amount, $1/text d p'$ is an infinite amount. This is why you usually see that the Dirac delta function as an infinite spike, but saying this somewhat obscures the above analogy with the discrete case.