@dalanicolai's answer is correct but you still have a problem: the legal values of the :tangle header are the strings yes, no or a filename, (IOW, the argument always has to be a string), so your elisp snippet has to be more complicated:

#begin_src shell :tangle (if (not file-exists-p "foo") "foo" "bar") ... 

Your original snippet returns nil if the file exists, but nil is not a string. The modified snippet returns the string foo if foo does not exist, but it returns the string bar otherwise, so the :tangle header always gets a string argument and everybody is happy.

@dalanicolai's answer is correct but you still have a problem: the legal values of the :tangle header are the strings yes, no or a filename, (IOW, the argument always has to be a string), so your elisp snippet has to be more complicated:

#begin_src shell :tangle (if (not file-exists-p "foo") "foo" "bar") ... 

Your original snippet returns nil if the file exists, but nil is not a string. The modified snippet returns the string foo if foo does not exist, but it returns the string bar otherwise, so the :tangle header always gets a string argument and everybody is happy.

EDIT: The OP in a comment to the question mentions that the tangling should only happen the first time (i.e. when the file does not exist) and never again. So the header should read:

#begin_src shell :tangle (if (not file-exists-p "foo") "foo" "no") ...