In traditional TeX fonts, \prime is a big glyph like the one in Latin Modern Math or Asana Math. However, there isn't such a character in Unicode. Symbol U+2032 is used as prime, but it looks like $^\prime$ i.e. $'$ in TeX. That makes the problem.
As a Unicode math font, Cambria/Minion/XITS is correct. And \prime command should be unavailable then. It seems a bug. I don't think the maintainers of unicode-math can do much. Asana Math and Latin Modern Math fonts may be modified to coincide with Unicode standard; then unicode-math can produce consistent result.
Definition in Unicode: http://www.unicode.org/charts/PDF/U2000.pdf
A temporary fix for the fonts other than LMMath and AsanaCambria Math:
\documentclass{article} \usepackage{unicode-math} \setmathfont{XITSCambria Math} \ExplSyntaxOn \group_begin: \char_set_catcode_active:N \' \char_set_catcode_active:N \` \char_set_catcode_active:n {"2032} \char_set_catcode_active:n {"2033} \char_set_catcode_active:n {"2034} \char_set_catcode_active:n {"2057} \char_set_catcode_active:n {"2035} \char_set_catcode_active:n {"2036} \char_set_catcode_active:n {"2037} \cs_gset:Nn \um_define_prime_chars: { \cs_set_eq:NN ' \um_scan_prime: \cs_set_eq:NN ^^^^2032 \um_scan_prime: \cs_set_eq:NN ^^^^2033 \um_scan_dprime: \cs_set_eq:NN ^^^^2034 \um_scan_trprime: \cs_set_eq:NN ^^^^2057 \um_scan_qprime: \cs_set_eq:NN ` \um_scan_backprime: \cs_set_eq:NN ^^^^2035 \um_scan_backprime: \cs_set_eq:NN ^^^^2036 \um_scan_backdprime: \cs_set_eq:NN ^^^^2037 \um_scan_backtrprime: } \group_end: \ExplSyntaxOff \begin{document} $A' A'' A''' A'''', A` A`` A```$A''''$ \end{document} 
