Evaluating the product $\prod\limits_{k=1}^{n}\cos\left(\frac{k\pi}{n}\right)$

The roots of the polynomial $X^{2n}-1$ are $\omega_j:=\exp\left(\mathrm i\frac{2j\pi}{2n}\right)$, $0\leq j\leq 2n-1$. We can write \begin{align} X^{2n}-1&=(X^2-1)\prod_{j=1}^{n-1}\left(X-\exp\left(\mathrm i\frac{2j\pi}{2n}\right)\right)\left(X-\exp\left(-\mathrm i\frac{2j\pi}{2n}\right)\right)\\ &=(X^2-1)\prod_{j=1}^{n-1}\left(X^2-2\cos\left(\frac{j\pi}n\right)X+1\right). \end{align} Evaluating this at $X=i$, we get $$(-1)^n-1=(-2)(-2\mathrm i)^{n-1}\prod_{j=1}^{n-1}\cos\left(\frac{j\pi}n\right),$$ hence \begin{align} \prod_{j=1}^n\cos\left(\frac{j\pi}n\right)&=-\prod_{j=1}^{n-1}\cos\left(\frac{j\pi}n\right)\\ &=\frac{(-1)^n-1}{2(-2\mathrm i)^{n-1}}\\ &=\frac{(-1)^n-1}2\cdot \frac{\mathrm i^{n-1}}{2^{n-1}}. \end{align} The RHS is $0$ if $n$ is even, and $-\dfrac{(-1)^m}{2^{2m}}=-\dfrac{\sin(n\pi/2)}{2^{n-1}}$ if $n$ is odd with $n=2m+1$.

If $n$ is even, then the term with $k=n/2$ makes the product on the left $0$ and $\sin\left(\frac{n}{2}\pi\right)=0$. So assume that $n$ is odd. $$ \begin{align} \prod_{k=1}^n\cos\left(\frac{k\pi}{n}\right) &=-\prod_{k=1}^{n-1}\cos\left(\frac{k\pi}{n}\right)\tag{1}\\ &=-\prod_{k=1}^{n-1}\frac{\sin\left(\frac{2k\pi}{n}\right)}{2\sin\left(\frac{k\pi}{n}\right)}\tag{2}\\ &=\frac{-1}{2^{n-1}}\frac{\prod\limits_{k=\frac{n+1}{2}}^{n-1}\sin\left(\frac{2k\pi}{n}\right)}{\prod\limits_{k=1}^{\frac{n-1}{2}}\sin\left(\frac{(2k-1)\pi}{n}\right)}\tag{3}\\ &=\frac{(-1)^{\frac{n+1}{2}}}{2^{n-1}}\frac{\prod\limits_{k=1}^{\frac{n-1}{2}}\sin\left(\frac{(2k-1)\pi}{n}\right)}{\prod\limits_{k=1}^{\frac{n-1}{2}}\sin\left(\frac{(2k-1)\pi}{n}\right)}\tag{4}\\ &=-\frac{\sin\left(n\frac\pi2\right)}{2^{n-1}}\tag{5} \end{align} $$ $(1)$: $\cos(\pi)=-1$

$(2)$: $\sin(2x)=2\sin(x)\cos(x)$

$(3)$: cancel $\sin\left(\frac{j\pi}{n}\right)$ in the numerator and denominator for even $j$ from $2$ to $n-1$

$(4)$: in the numerator, change variable $k\mapsto k+\frac{n-1}{2}$ and use $\sin(x+\pi)=-\sin(x)$

$(5)$: for odd $n$, $\sin\left(n\frac\pi2\right)=(-1)^{\frac{n-1}{2}}$

The monic Chebyshev polynomial of the second kind,

$$\hat{U}_n(x)=\frac{\sin((n+1)\arccos\,x)}{2^n \sqrt{1-x^2}}$$

can be easily seen to have the roots $x_k=\cos\dfrac{\pi k}{n+1}$ for $k=1,\dots,n$. By Vieta, the constant term of $\hat{U}_n(x)$ is equal to

$$\hat{U}_n(0)=\prod_{k=1}^n \cos\dfrac{\pi k}{n+1}$$

and thus

$$\prod_{k=1}^n \cos\dfrac{\pi k}{n}=-\hat{U}_{n-1}(0)=-\frac{\sin\frac{n\pi}{2}}{2^{n-1}}$$

The idea of using $\cos x=\frac{1}{2}\frac{\sin 2x}{\sin x}$ is a nice one, and works quickly. We should not use this when $x=\frac{n\pi}{n}$,because of the $\frac{0}{0}$ issue. Also, as you observe, the product is $0$ if $n$ is even, so we needn't bother. So let $n$ be odd.

Look at the product from $k=1$ to $k=n-1$. As $k$ ranges over these values, the numbers $2k$ range, modulo $n$, over all numbers from $1$ to $n-1$. So the $\cos(2k\pi/n)$ (apart from sign) range in some order over the $\cos(\pi j/n)$. Thus, apart from sign, there is cancellation and the product has absolute value $\frac{1}{2^{n-1}}$.

There is no sign issue. If $n\equiv 1\pmod {4}$, the product from $1$ to $n-1$ has an even number of negative terms. The term $\cos(\pi n/n)$ then gives us an extra $-1$, and the product is negative. For the same reason, if $n\equiv 3\pmod{4}$ then the product is positive. The $-\sin(n\pi/2)$ term captures these sign facts, and also produces the right answer of $0$ when $n$ is even.