I don't see the reason why you should use Invoke-Expression in this case, as was mentioned by Grok42 in a comment, the reason why the attempt failed is because you need to quote your path and, in addition to that, you'll need to add the call operator & to invoke that path:
Invoke-ExpressionWithErrorHandling "'C"& 'C:\Program Files\Git\cmd\git.exe' version" However you can simply invoke the command passed to the -Command argument using the call operator and you can add a second parameter for the arguments that takes ValueFromRemainingArguments:
function Invoke-WithErrorHandling { param ( [Parameter(Mandatory)] [string] $Command, [Parameter(ValueFromRemainingArguments)] [string[]] $Arguments ) & $Command @Arguments if ($LASTEXITCODE -ne 0) { Write-Error "'$Command' exited with exit code '$LASTEXITCODE'" } } Invoke-WithErrorHandling cmd /c 'ping -n 3 google.com & exit 2' # this way would also work but requires escaping of & with a backtick (`) in pwsh 7+ # cause otherwise its interpreted as a job and as a reserved char in pwsh 5.1 :( Invoke-WithErrorHandling cmd /c ping -n 3 google.com `& exit 2 Invoke-WithErrorHandling 'C:\Program Files\Git\cmd\git.exe' version