63

I'm trying to implement RC4 and DH key exchange in python. Problem is that I have no idea about how to convert the python long/int from the key exchange to the byte array I need for the RC4 implementation. Is there a simple way to convert a long to the required length byte array?

Update: forgot to mention that the numbers I'm dealing with are 768 bit unsigned integers.

1

11 Answers 11

67

With Python 3.2 and later, you can use int.to_bytes and int.from_bytes: https://docs.python.org/3/library/stdtypes.html#int.to_bytes

Sign up to request clarification or add additional context in comments.

1 Comment

It OverflowError's out on big numbers.
34

Everyone has overcomplicated this answer:

some_int = <256 bit integer> some_bytes = some_int.to_bytes(32, sys.byteorder) my_bytearray = bytearray(some_bytes) 

You just need to know the number of bytes that you are trying to convert. In my use cases, normally I only use this large of numbers for crypto, and at that point I have to worry about modulus and what-not, so I don't think this is a big problem to be required to know the max number of bytes to return.

Since you are doing it as 768-bit math, then instead of 32 as the argument it would be 96.

3 Comments

In Python 3 this solution worked really well for 2048 bit integer. It Python 2.7 it works only for int (2048 bit integer is long in Python 2.7).
In Python 2.7 some_bytes = some_int.to_bytes(32, sys.byteorder) produces error AttributeError: 'int' object has no attribute 'to_bytes' 😞
Not quite everyone... see the answer by @JackOConnor
21

I haven't done any benchmarks, but this recipe "works for me".

The short version: use '%x' % val, then unhexlify the result. The devil is in the details, though, as unhexlify requires an even number of hex digits, which %x doesn't guarantee. See the docstring, and the liberal inline comments for details.

from binascii import unhexlify def long_to_bytes (val, endianness='big'): """ Use :ref:`string formatting` and :func:`~binascii.unhexlify` to convert ``val``, a :func:`long`, to a byte :func:`str`. :param long val: The value to pack :param str endianness: The endianness of the result. ``'big'`` for big-endian, ``'little'`` for little-endian. If you want byte- and word-ordering to differ, you're on your own. Using :ref:`string formatting` lets us use Python's C innards. """ # one (1) hex digit per four (4) bits width = val.bit_length() # unhexlify wants an even multiple of eight (8) bits, but we don't # want more digits than we need (hence the ternary-ish 'or') width += 8 - ((width % 8) or 8) # format width specifier: four (4) bits per hex digit fmt = '%%0%dx' % (width // 4) # prepend zero (0) to the width, to zero-pad the output s = unhexlify(fmt % val) if endianness == 'little': # see http://stackoverflow.com/a/931095/309233 s = s[::-1] return s 

...and my nosetest unit tests ;-)

class TestHelpers (object): def test_long_to_bytes_big_endian_small_even (self): s = long_to_bytes(0x42) assert s == '\x42' s = long_to_bytes(0xFF) assert s == '\xff' def test_long_to_bytes_big_endian_small_odd (self): s = long_to_bytes(0x1FF) assert s == '\x01\xff' s = long_to_bytes(0x201FF) assert s == '\x02\x01\xff' def test_long_to_bytes_big_endian_large_even (self): s = long_to_bytes(0xab23456c8901234567) assert s == '\xab\x23\x45\x6c\x89\x01\x23\x45\x67' def test_long_to_bytes_big_endian_large_odd (self): s = long_to_bytes(0x12345678901234567) assert s == '\x01\x23\x45\x67\x89\x01\x23\x45\x67' def test_long_to_bytes_little_endian_small_even (self): s = long_to_bytes(0x42, 'little') assert s == '\x42' s = long_to_bytes(0xFF, 'little') assert s == '\xff' def test_long_to_bytes_little_endian_small_odd (self): s = long_to_bytes(0x1FF, 'little') assert s == '\xff\x01' s = long_to_bytes(0x201FF, 'little') assert s == '\xff\x01\x02' def test_long_to_bytes_little_endian_large_even (self): s = long_to_bytes(0xab23456c8901234567, 'little') assert s == '\x67\x45\x23\x01\x89\x6c\x45\x23\xab' def test_long_to_bytes_little_endian_large_odd (self): s = long_to_bytes(0x12345678901234567, 'little') assert s == '\x67\x45\x23\x01\x89\x67\x45\x23\x01' 

2 Comments

I encountered problems when the value is 0 (Python 3.5) binascii.Error: Odd-length string, quick fix for this: replace s = unhexlify(fmt % val) with s = unhexlify('00') if fmt % val == '0' else unhexlify(fmt % val)
This is more concise. pastebin.com/iQRXyxsM
14

One-liner:

bytearray.fromhex('{:0192x}'.format(big_int)) 

The 192 is 768 / 4, because OP wanted 768-bit numbers and there are 4 bits in a hex digit. If you need a bigger bytearray use a format string with a higher number. Example:

>>> big_int = 911085911092802609795174074963333909087482261102921406113936886764014693975052768158290106460018649707059449553895568111944093294751504971131180816868149233377773327312327573120920667381269572962606994373889233844814776702037586419 >>> bytearray.fromhex('{:0192x}'.format(big_int)) bytearray(b'\x96;h^\xdbJ\x8f3obL\x9c\xc2\xb0-\x9e\xa4Sj-\xf6i\xc1\x9e\x97\x94\x85M\x1d\x93\x10\\\x81\xc2\x89\xcd\xe0a\xc0D\x81v\xdf\xed\xa9\xc1\x83p\xdbU\xf1\xd0\xfeR)\xce\x07\xdepM\x88\xcc\x7fv\\\x1c\x8di\x87N\x00\x8d\xa8\xbd[<\xdf\xaf\x13z:H\xed\xc2)\xa4\x1e\x0f\xa7\x92\xa7\xc6\x16\x86\xf1\xf3') >>> lepi_int = 0x963b685edb4a8f336f624c9cc2b02d9ea4536a2df669c19e9794854d1d93105c81c289cde061c0448176dfeda9c18370db55f1d0fe5229ce07de704d88cc7f765c1c8d69874e008da8bd5b3cdfaf137a3a48edc229a41e0fa792a7c61686f1f >>> bytearray.fromhex('{:0192x}'.format(lepi_int)) bytearray(b'\tc\xb6\x85\xed\xb4\xa8\xf36\xf6$\xc9\xcc+\x02\xd9\xeaE6\xa2\xdff\x9c\x19\xe9yHT\xd1\xd91\x05\xc8\x1c(\x9c\xde\x06\x1c\x04H\x17m\xfe\xda\x9c\x187\r\xb5_\x1d\x0f\xe5"\x9c\xe0}\xe7\x04\xd8\x8c\xc7\xf7e\xc1\xc8\xd6\x98t\xe0\x08\xda\x8b\xd5\xb3\xcd\xfa\xf17\xa3\xa4\x8e\xdc"\x9aA\xe0\xfay*|aho\x1f') 

[My answer had used hex() before. I corrected it with format() in order to handle ints with odd-sized byte expressions. This fixes previous complaints about ValueError.]

9 Comments

it does not work if you don't produce a Long though. I think smt like bytearray.fromhex(hex(2**61-1).strip('0x').strip('L')) is safer
@MarioAlemi the code in your comment is wrong. strip('0x') will also strip the trailing zeros, which will result bad result (and sometimes ValueError)!
@Jess Austin: Your solution is totally wrong, because it works only when x consists of even number of hex-digits. Example: x=0x963b685edb4a8f336f624c9cc2b02d9ea4536a2df669c19e9794854d1d93105c81c289cde061c0448176dfeda9c18370db55f1d0fe5229ce07de704d88cc7f765c1c8d69874e008da8bd5b3cdfaf137a3a48edc229a41e0fa792a7c61686f1fL
@lepi can you make an example?
Right, I was just trying to point out that if you change the 192 in the format string to 191 (or to any odd number), you will get a ValueError. Just something that tripped me up.
|
8

long/int to the byte array looks like exact purpose of struct.pack. For long integers that exceed 4(8) bytes, you can come up with something like the next:

>>> limit = 256*256*256*256 - 1 >>> i = 1234567890987654321 >>> parts = [] >>> while i: parts.append(i & limit) i >>= 32 >>> struct.pack('>' + 'L'*len(parts), *parts ) '\xb1l\x1c\xb1\x11"\x10\xf4' >>> struct.unpack('>LL', '\xb1l\x1c\xb1\x11"\x10\xf4') (2976652465L, 287445236) >>> (287445236L << 32) + 2976652465L 1234567890987654321L 

2 Comments

But it won't help with big numbers (> 8 bytes), which will usually be used for cryptographic applications.
it's written not to be generic but more like fixed size solution to common problem of representing all possible ip's or similar...
7

You can try using struct:

import struct struct.pack('L',longvalue) 

2 Comments

Sadly no, error: integer out of range for 'L' format code. It's a 768 bit long, which is quite a bit bigger than the 4 byte unsigned int.
Downvoted because Python long int are arbitrarily long integers. Think of it like an array of 32 (or whatever) bits integers. A C long is a size defined datatype. With this response, you are confusing both.
7

Little-endian, reverse the result or the range if you want Big-endian:

def int_to_bytes(val, num_bytes): return [(val & (0xff << pos*8)) >> pos*8 for pos in range(num_bytes)] 

Big-endian:

def int_to_bytes(val, num_bytes): return [(val & (0xff << pos*8)) >> pos*8 for pos in reversed(range(num_bytes))] 

Comments

3

Basically what you need to do is convert the int/long into its base 256 representation -- i.e. a number whose "digits" range from 0-255. Here's a fairly efficient way to do something like that:

def base256_encode(n, minwidth=0): # int/long to byte array if n > 0: arr = [] while n: n, rem = divmod(n, 256) arr.append(rem) b = bytearray(reversed(arr)) elif n == 0: b = bytearray(b'\x00') else: raise ValueError if minwidth > 0 and len(b) < minwidth: # zero padding needed? b = (minwidth-len(b)) * '\x00' + b return b 

You many not need thereversed()call depending on the endian-ness desired (doing so would require the padding to be done differently as well). Also note that as written it doesn't handle negative numbers.

You might also want to take a look at the similar but highly optimized long_to_bytes() function in thenumber.pymodule which is part of the open source Python Cryptography Toolkit. It actually converts the number into a string, not a byte array, but that's a minor issue.

Comments

2

Python 2.7 does not implement the int.to- very slow_bytes() method.

I tried 3 methods:

  1. hex unpack/pack : very slow
  2. byte shifting 8 bits at a time: significantly faster.
  3. using a "C" module and packing into the lower (7 ia64 or 3 i32) bytes. This was about twice as fast as 2/ . It is the fastest option, but still too slow.

All these methods are very inefficient for two reasons:

  • Python 2.7 does not support this useful operation.
  • c does not support extended precision arithmetic using the carry/borrow/overflow flags available on most platforms.

Comments

0
i = 0x12345678 s = struct.pack('<I',i) b = struct.unpack('BBBB',s) 

Comments

0
def bigIntToArray_v1(value): result = [] while True: result.insert(0, value % 256) value = value // 256 if value == 0: break return result def bigIntToArray_v2(value): result = [] while True: result.insert(0, value & 0b1111_1111) value = value >> 8 if value == 0: break return result 

Test

expected = [ 30, 119, 10, 234, 151, 91, 69, 198, 85, 193, 130, 94, 213, 16, 109, 151, 14, 129, 143, 64, 171, 172, 190, 28, 152, 247, 78, 21, 94, 76, 93, 58 ] result1 = bigIntToArray_v1(13779715600237310799632842054081955307819773220415101057149384327455163833658) result2 = bigIntToArray_v2(13779715600237310799632842054081955307819773220415101057149384327455163833658) print(result1 == expected) print(result2 == expected) print( bytearray(result1) ) print( bytearray(result2) ) 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.