HAL9256's answer is helpful, but let me try to frame the issue differently:

To get the desired verbatim output, this is what your command would need to look like if you ran it from bash itself:

# From Bash itself $ bash -c "echo \"'hello' \\\"world\\\"!\"" 'hello' "world"! 

Bash's initial parsing of the string literal above, inside of which \ functions as the escape character, means that the new bash instance sees the following verbatim text as its -c argument:
echo "'hello' \"world\"!"
Therefore, if you were to execute this as a command directly in Bash, it would yield the desired output too.

Note how the echo argument as a whole is enclosed in "...", for robustness: without that, if world were world & space, for instance, the command would break; similarly, world class would turn into world class (normalization of whitespace due to shell expansions).

The many \ characters are needed due to 2 layers of escaping being involved:

• Escaping for the calling shell (also bash in this example), to ensure that the input string is syntactically valid: \" embeds a single ", and \\ a single \.
• Escaping for the target shell (bash), so that the resulting string is syntactically valid for it.

Translating the command so it can be called from PowerShell requires adapting the first layer above, which means using `, PowerShell's escape character, instead of \.

A direct translation is therefore:

# SHOULD work, but DOESN'T as of PowerShell 7.0 PS> bash -c "echo `"'hello' \`"world\`"!`"" hello # !! WRONG output. 

Yet, if you print string literal "echo `"'hello' \`"world\`"!`"" directly in PowerShell, you'll see that it properly yields the verbatim string that bash needs to see, as shown above
(echo "'hello' \"world\"!")

The reason it doesn't work is that PowerShell's passing of arguments to external programs has always been broken with respect to embedded double quotes:

This answer provides an overview, but the short of it is, as of PowerShell 7.0:

In addition to satisfying PowerShell's own syntax requirements - which is enough if you call PowerShell commands - you must additionally \-escape embedded " characters in arguments passed to external programs.

Therefore:

# OK, but the manual \-escaping shouldn't be necessary. PS> bash -c "echo \`"'hello' \\\`"world\\\`"!\`"" 'hello' "world"! 

---

Why the current behavior is broken:

It is the job of a shell to pass arguments that result from the shell's own parsing verbatim to the target program, doing whatever is necessary behind the scenes to ensure that - you shouldn't have to worry about any escaping requirements other than PowerShell's own:

• On Windows, out of unfortunate necessity, this means constructing a command line behind the scenes that applies double-quoting and escaping as needed; while double-quoting is applied by PowerShell on demand, it is the \-escaping of " chars. embedded in arguments that is missing.

• On Unix-like platforms, no extra effort is required: there, programs receive arguments directly as an array of verbatim values.

While the particular command discussed in this question happens to make the translation to PowerShell fairly straightforward, there are much more insidious cases, where the additional need to \-escape - which is always cumbersome - is also unexpected, because these commands work as-is in bash itself:

PS> bash -c 'echo "hi, there"' hi, # !! " were stripped, so bash only saw `echo hi,` as the command string, # !! and `there` as a separate argument. 

PS> /bin/echo '{ "foo": "bar" }' # Try to pass a JSON string { foo: bar } # !! BROKEN JSON