Python 2, 311 283.5 271.5258 bytes (622 567 543543 516-50%)
Here's my attempt at an answer.
R=dict(zip(list('abdhlmnostxzABDHIMN'),'eD q pP y l w uU oOGQ sS t xX zZ vV E G H I W N'.split())) V='ijlmnotuvwxyAHIMOQTUVWXY' W=dict(b='d',p='q',s='z',B='E',G='D',S='Z') H='lotxBCDEHIKOX' T='AHIMOTUVWXYilmnotuvwxy' s=raw_input() r=v=h=t=1 for i,j in R.items(): for c in j:R[c]=R.get(c,'')+i for i,j in W.items(): for c in j:W[c]=W.get(c,'')+i for i in range(len(s)): c,l=s[i],s[-i-1] if notr&=(c in R and l in R[c]):r=0 if not(cv&=c in V or(c in W and l in W[c])):v=0 ifh&=(c in H)+(l in H)<2:h=0>1 if not ct&=c in T:t=0 print 'R'*r+'V'*v+'H'*h+'T'*t Prints the possible types of ambigrams, or 0 if no ambigrams are possible.
RVHT for rotational, vertical, horizontal and totem respectively.