Yes
Non-competing doesn't allow an answer to do what it wants, and not meet the questions specifications. If a question has a specific scoring system, answers whose scores can't be calculated shouldn't be marked non-competing, they should be removed from the question.
In short, if I posted an answer to the Hello, World! question that didn't display Hello, World!, is it a valid answer? No. Can it be marked non-competing? No. To quote Dennisquote Dennis,
Marking an answer as non-competing does not exempt it from being invalid.
The Mathematica answer, while being interesting, did not meet the answer specification, and, per our rules it should have been deleted. This also applies to any other answers which, while accomplishing the task at hand, can't be scored.