As a general rule, the control modifier acts on the ASCII characters by removing all by the last 5 bits. That maps several different characters into the 0..31 range used for ASCII control characters.
Based on that rule, one would expect these to map into 031, using the control modifier:
037 31 1F US (unit separator) 077 63 3F ? 137 95 5F _ 177 127 7F DEL
and these would not:
057 47 2F / 117 79 4F O 157 111 6F o
Alternatively, the control modifier could apply only to characters in the range 32..63, using the analogy of the shift key which is useful in ASCII for mapping the range 64..94 to 96..126 (upper/lowercase alphabetics). In that case, these might map to 031:
037 31 1F US (unit separator) 057 47 2F / 077 63 3F ?
and (it's been a while), @aprogrammer's observation sounds familiar. But it is not in the VT100 manual. In Chapter 3, it points to a table which documents keys and their control equivalents. That uses codes from the full 95-character range (32..126), but only documents one case for each control character. The other possible characters are all undocumented. But note that the table documents control?. On my keyboard, ? is a shifted key. The pictures which I find of the VT100 keyboard do not have it as an unshifted key. And (unsurprisingly) the unshifted code for that particular key is /. DEC may have documented these based on existing keys from an earlier terminal, but just as possibly they may have done that to keep the table short.
However, the conversion is not entirely up to xterm. It receives X events which have certain keycodes along with modifier information. Generally speaking, those keycodes correspond to the letters shown on your keyboard, while the modifiers such as shift and control are passed separately. xterm uses an X runtime function such as Xutf8LookupString or XmbLookupString to combine some events (such as shift). That is where the decision is made to translate these control characters. xterm later refines/amends the information for special keys (such as backspace), but most of the conversion is done in the X libraries using the keyboard configuration.
Further reading: