Using sed:
sed -E '/KEYWORD/{ :lower s/("[^"]*)[a-z]([^"]*")/\1_\2/; t lower; :upper s/("[^"]*)[A-Z]([^"]*")/\1-\2/; t upper; :digit s/("[^"]*)[0-9]([^"]*")/\1*\2/; t digit; }; y/*_-/0xX/' infile This will run the set of codes in block /KEYWORD/{...} only when a line matched with a string KEYWORD.
This ("[^"]*)[###]([^"]*") matches to a " and anything after that until first lower-case [a-z]/upper-case[A-Z]/digit[0-9] character found which flowed by anything until another quote matched.
Every part will loop over again and again until all those characters were converted lower-case to _, upper-case to - and digits to * (note: chose different characters if these may occurred in your file; the reason is we didn't replace directly with x or X or 0 because it will cause infinite loop for sed since of using sed's loops to replace every lower/upper/digit characters).
After all done, those characters *_- will translate to 0Xx0xX.
Add -i option to above command to update the changes in your input file like sed -i -E ... .
Update: The command for the revised question:
sed -E '/KEYWORD/{ :lower s/^(([^"]*("[^"]*"){0,1})*)("[^"]*)[a-z]([^"]*")/\1\4_\5/; t lower; :upper s/^(([^"]*("[^"]*"){0,1})*)("[^"]*)[A-Z]([^"]*")/\1\4+\5/; t upper; :digit s/^(([^"]*("[^"]*"){0,1})*)("[^"]*)[0-9]([^"]*")/\1\4*\5/; t digit; }; y/*_+/0xX/' infile