Among other tricks (like passing `LC_*` environment variables) you can do the following:

 PARAMETER="123"
 ssh user@host PARAMETER="$PARAMETER" bash -c << __EOF
 echo \$PARAMETER
 __EOF

The advantage of the approach is absence of the requirement to `export PARAMETER` or using `env` in form:

 env PARAMETER="$PARAMETER" ssh user@host bash -s << __EOF
 echo \$PARAMETER
 __EOF