The problem posed here is to show that

$$ \int_0^{\infty} dx \, \frac{\coth^2{x}-1}{\displaystyle (\coth{x}-x)^2+\frac{\pi^2}{4}} = \frac{4}{5} $$

The OP actually seemed to know what he was doing but could not get the correct residue that would allow him to get the stated result. In showing a link to Wolfram Alpha, the OP revealed that he was simply finding the residue of the integrand at a pole without defining a complex integral (including a contour of integration). Thus, this guy was making a very common mistake by thinking the Residue Theorem is a formula and not a procedure.

It needs to be internalized by those who wish to use the Residue Theorem that it is a complex integral which is equal to $i 2 \pi$ times the sum of the residues of the poles inside the contour of integration. The contour of integration, when parametrized, needs to produce the real integral of interest. (Then we can solve for the real integral in terms of the residues and maybe some other stuff related to contributions from others parts of the contour.) It is NOT the real integral being equal to $i 2 \pi$ times the residues!

To give the OP the benefit of the doubt, perhaps he merely thought that the contour is a semicircle in the upper half-plane. While this is a valid contour to use, the problem is that there are an infinite number of poles of the integrand within this contour. And aside from the pole at $z=i \pi/2$, the poles are not realizable in closed form and are difficult to obtain.

Better is a closed contour that contains only the pole at $z=i \pi/2$. In this case, we need to determine the form of the integrand we need in the contour integral. Remember, somehow we should get the original integral back using a direct parameterization of the contour. So, without further ado…

Consider the contour integral

$$\oint_C dz \, \tanh{\left [z-\operatorname{arctanh}{\left (z-i \frac{\pi}{2} \right )} \right ]} $$

which is equal to

$$\oint_C dz \, \frac{\displaystyle 1-z \coth{z} + i \frac{\pi}{2} \coth{z}}{\displaystyle \coth{z}-z+i \frac{\pi}{2}} $$

where $C$ is the rectangle with vertices $\pm R$ and $\pm R+i \pi$.

The contour integral is then equal to

$$\int_{-R}^R dx \frac{\displaystyle 1-x \coth{x} + i \frac{\pi}{2} \coth{x}}{\displaystyle \coth{x}-x+i \frac{\pi}{2}} + i \int_0^{\pi} dy \, \frac{\displaystyle 1-(R+i y)\coth{(R+i y)} + i \frac{\pi}{2} \coth{(R+i y)}}{\displaystyle \coth{(R+i y)} – (R+i y) + i \frac{\pi}{2}} \\ + \int_{R}^{-R} dx \frac{\displaystyle 1-x \coth{x} – i \frac{\pi}{2} \coth{x}}{\displaystyle \coth{x}-x-i \frac{\pi}{2}}\\ + i \int_{\pi}^0 dy \, \frac{\displaystyle 1-(-R+i y)\coth{(-R+i y)} + i \frac{\pi}{2} \coth{(-R+i y)}}{\displaystyle \coth{(-R+i y)} – (-R+i y) + i \frac{\pi}{2}}$$

Now consider the second and fourth integrals, i.e., those over the vertical edges of the rectangle. We consider the limit as $R \to \infty$. Note that the integrand of the second integral approaches $1$ in this limit, while the integrand of the fourth integral approaches $-1$. (Exercise for the reader.) Thus, the sum of these two integrals is $i 2 \pi$.

We can also combine the integrand of the first and third integrals to get the integrand of the integral we seek, times $i \pi$. Thus, the contour integral is equal to (after exploiting the evenness of that integrand)

$$i 2 \pi \int_0^{\infty} dx \, \frac{\coth^2{x}-1}{\displaystyle (\coth{x}-x)^2+\frac{\pi^2}{4}} + i 2 \pi$$

By the residue theorem, the contour integral is also equal to $i 2 \pi$ times the residue at the pole $z=i \pi/2$. Interestingly enough, this pole is not simple, but is instead a third-order pole. Given the integrand, I find it easier to compute the Laurent series directly and use that to find the residue. However, I will simply state the result as follows:

$$\operatorname*{Res}_{z=i \pi/2} \frac{\displaystyle 1-z \coth{z} + i \frac{\pi}{2} \coth{z}}{\displaystyle \coth{z}-z+i \frac{\pi}{2}} = \frac{9}{5} $$

so that

$$i 2 \pi \int_0^{\infty} dx \, \frac{\coth^2{x}-1}{\displaystyle (\coth{x}-x)^2+\frac{\pi^2}{4}} + i 2 \pi = i 2 \pi \frac{9}{5} $$

or

$$ \int_0^{\infty} dx \, \frac{\coth^2{x}-1}{\displaystyle (\coth{x}-x)^2+\frac{\pi^2}{4}} = \frac{4}{5} $$

**ADDENDUM**

Let’s take a look at that residue calculation. For this, it helps to know that

$$\coth{\left ( z + i \frac{\pi}{2} \right )} = \tanh{z}$$

and

$$\tanh{z} = z – \frac13 z^3 + \frac{2}{15} z^5 + O \left ( z^7 \right )$$

so that the integrand looks like, in the neighborhood of $z=i \pi/2$,

$$ \frac{1-z \tanh{z}}{\tanh{z}-z} $$

Now we can find the Laurent expansion of this function about $z=0$, which looks like

$$-\frac{3}{z^3} \frac{\displaystyle 1-z^2 +O \left ( z^4 \right )}{\displaystyle 1-\frac{2}{5} z^2+O \left ( z^4 \right )} = -\frac{3}{z^3} \left [1-z^2 +O \left ( z^4 \right ) \right ] \left [1+\frac{2}{5} z^2+O \left ( z^4 \right ) \right ]$$

The residue is the coefficient of $z^2$ in the numerator, which may simply be read off as $9/5$ as stated above.

**ADDENDUM II**

How do we show that $z=i \pi/2$ is the only pole inside the rectangle? We can use Rouche’s theorem. For example, we would just need to show that, on the rectangle,

$$\left | \coth{z}-z \pm i \frac{\pi}{2} \right | \gt |\coth{z} | $$

On the horizontal sides of the rectangle, we see that

$$\left | \coth{x}-x \pm i \frac{\pi}{2} \right |^2 – |\coth{x} |^2 = \frac{\pi^2}{4} – \left (2 x \coth{x}-x^2 \right ) $$

which is indeed $\gt 0$ for all $x \in \mathbb{R} $.

On the vertical sides of the rectangle, the inequality is obviously satisfied because $|\coth{(R + i y)}|$ approaches $1$ as $R \to \pm \infty$.

Therefore, by Rouche’s theorem, the denominator of the integrand has the same number of zeroes inside the rectangle as $\coth{z}$, which is just the one at $z=i \pi/2$ and no others. Thus, the only pole inside the rectangle is at $z=i \pi/2$.

Hey Ron, I want to thank you for this blog, I am going to highly benefit from it.