qalc/Qalculate, 13 bytes

(ans−1)!²%ans 

Example usage in Bash:

echo "5" | qalc -t -f - "(ans−1)! ²%ans" 

The tricky part was reading the number in. It's pretty easy to insert the number into the formula two times, but that would mean modifying the "program", which is surely not allowed. There seems to also be no way to execute multiple expressions non-interactively with qalc, except by reading in a file. In this case, I used - as a "file", which is StdIn, and used that only to store the input in ans, which is implicitly done by qalc. You could also say that ans is the input of the "program", in which case the program is simply the 13 bytes seen at the top. It could also be stored in any other variable, in which case the solution could be shortened by 4 bytes.

qalc/Qalculate, 8 bytes

If reading from any variable is allowed:

p-1)!²%p 

Otherwise 12 bytes:

ans−1)!²%ans 

Example usage in Bash for the "ans" case:

echo "5" | qalc -t -f - "ans−1)! ²%ans" 

Reading the number in in qalc is difficult. It has a system of variables, but I haven't figured out how to set those in the CLI yet. In the GUI it's pretty easy. That's one of the main reasons why I also provided a solution with "ans".

Inserting the number into the formula two times would mean modifying the "program", which is surely not allowed. 

There seems to be no way to execute multiple expressions non-interactively with qalc, except by reading in a file. In this case, I used - as a "file", which is StdIn, and used that only to store the input in ans, which is implicitly done by qalc. You could also use an actual file and use the number in that file as input. Or you could even say that ans is already the input of the "program", in which case the program is simply the 12 bytes seen above, or even any variable, then the 8 byte solution.

TI-Basic, 10 bytes

fPart((Ans-1)!²/Ans 

This outputs 0 for non-prime numbers and something in the range 0 < number < 1 for prime numbers.* If you want a proper 1 for prime numbers, prepend either not(not( or 0≠ for a 12 byte solution. But this is not actually required for a "truthy" value, because for example If .5 executes the "Then" statement.

This solution is based on this Labyrinth answer, but the great thing about programming on a calculator is that you can basically just enter formulas directly. The only part of that formula that's not a builtin is a%b. I had to use fPart(a/b), which is the fractional part of the division. Newer calculator models and maybe even newer operating systems for this one (I'm using 2.43, 2.55 is available, but horrendously laggy and counterintuitive) have a modulo builtin, so this program would probably be one byte shorter with those.

The program reads from and writes to the variable "Ans", which is basically the StdIn and StdOut of the TI-84+ calculator. If the last instruction of a program results in a number, then that number is automatically printed. But you can also just run that line as an instruction in the regular calculator view.

As a program with the name "P", the file is 20 bytes large, of which one byte is the file name and nine bytes are metadata etc. I'm not quite sure whether that counts as part of the byte count for the answer, but since it can also be executed as a simple calculation statement, I would rather not count it. Writing a program is not actually required to execute this.

Breakdown:

fPart((Ans-1)!²/Ans  Ans the previous output, handled just like any variable  Ans  same  (  -1) subtract one, of course ! builtin factorial function ² builtin square/second power function fPart(  fractional part of a decimal (internally actually complex) number fPart((Ans-1)!²/Ans 

A closing bracket at the end is actually not needed, TI-Basic understands it anyway.
The optional not( I mentioned returns 1 for an input of 0 and 0 otherwise. Applying that twice leaves 0 unchanged and turns everything else into 1. 0≠ acts exactly the same.

For the first few prime numbers, the result of that modulo operation is always 1. That might be random or it might be guaranteed, I haven't checked that yet.
If it's guaranteed, you can also prepend Ans for an 11 byte solution with an output of 1 for prime numbers (because b·fPart(a/b) is the same as a%b).

The catch

*This program has one big issue, however: Due to imprecisions, this program only correctly identifies the prime numbers up to 11, afterwards it claims everything to be a non-prime number. Everything above 42 even causes an overflow error, because 42!² is bigger than 10¹⁰⁰. So the check for 9 is actually the only one within the working range that differs from a check for even/odd numbers.

Alternatively, this formula can be used in the command line calculator "qalc" that is often preinstalled on Linux and in the GUI equivalent "Qalculate":

(42−1)!²%42 returns 0.
(13-1)!²%13 returns 1.
(100000007-1)!²%100000007 takes a while and then returns 1.

But I wouldn't call inserting that number there a "programming language". Variables could be defined before the execution and then used in the calculation, which might make that a valid solution, but I haven't played around with that much yet.

qalc/Qalculate, 13 bytes

(ans−1)!²%ans 

The great thing about "programming" in a calculator is that you can basically just enter formulas directly.

Example usage in Bash:

echo "5" | qalc -t -f - "(ans−1)! ²%ans" 

Note that I added a space after the exclamation mark only because Bash otherwise tries to interpret "!²" as an "event". It's not actually required by qalc, which can be seen by for example creating this file:

5 (ans−1)!²%ans 

…and then executing it with qalc -t -f test.txt.

Other consoles or other ways of calling qalc than Bash might not have this problem.

The calculation can also be entered in "interactive mode" by simply running qalc -t, entering the number and then entering the expression on the next input. This also works in the GUI program "Qalculate".

Meta explanation:

The output is 0 for non-prime numbers and a positive integer for prime numbers. I let this run in a loop for all numbers up to 10000 and it was always 1 for prime numbers, but I haven't checked whether that is guaranteed yet.

This solution is based on this Labyrinth answer. I essentially just found something (qalc) that has all the parts of the formula as builtins instead of having to program all the parts myself on a lower level.

The -t argument hides the interpreted calculation, so that the output of the example command looks like this:

5 1 

…instead of like this:

5 = 5 rem(factorial(ans - 1)^2, ans) = 1 

The tricky part was reading the number in. It's pretty easy to insert the number into the formula two times, but that would mean modifying the "program", which is surely not allowed. There seems to also be no way to execute multiple expressions non-interactively with qalc, except by reading in a file. In this case, I used - as a "file", which is StdIn, and used that only to store the input in ans, which is implicitly done by qalc. You could also say that ans is the input of the "program", in which case the program is simply the 13 bytes seen at the top. It could also be stored in any other variable, in which case the solution could be shortened by 4 bytes.

I originally wrote this answer for TI-Basic:

fPart((Ans-1)!²/Ans 

That's just 10 bytes, but it fails on the input 13 already, due to floating point imprecisions in those large numbers (12!²=229442532802560000). So the only difference to an even/odd checker within the working range is 9 and it does not fulfil the criterion to work up to 255. For inputs above 42 it even crashed from an overflow (42!²>10¹⁰⁰).