I don't see the reason why you should use `Invoke-Expression` in this case, as was mentioned by [Grok42][1] in a comment, the reason why the attempt failed is because you need to quote your path and, in addition to that, you'll need to add the call operator `&` to invoke that path:

```
Invoke-ExpressionWithErrorHandling "& 'C:\Program Files\Git\cmd\git.exe' version"
```

However you can simply invoke the command passed to the `-Command` argument using the [call operator][2] and you can add a second parameter for the arguments that takes [`ValueFromRemainingArguments`][3]:

```sh
function Invoke-WithErrorHandling {
 param (
 [Parameter(Mandatory)]
 [string] $Command,

 [Parameter(ValueFromRemainingArguments)]
 [string[]] $Arguments
 )

 & $Command @Arguments
 if ($LASTEXITCODE -ne 0) {
 Write-Error "'$Command' exited with exit code '$LASTEXITCODE'"
 }
}

Invoke-WithErrorHandling cmd /c 'ping -n 3 google.com & exit 2'
# this way would also work but requires escaping of & with a backtick (`) in pwsh 7+
# cause otherwise its interpreted as a job and as a reserved char in pwsh 5.1 :(
Invoke-WithErrorHandling cmd /c ping -n 3 google.com `& exit 2
Invoke-WithErrorHandling 'C:\Program Files\Git\cmd\git.exe' version
```

[1]: https://stackoverflow.com/users/20573245/grok42
[2]: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_operators?view=powershell-7.2#call-operator-
[3]: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_functions_advanced_parameters?view=powershell-7.4#valuefromremainingarguments-argument