how to change byte order of 128 bit number

Question

I have an array of unsigned char that represents a 128 bit number in network byte order. How would I go about converting this to host byte order efficiently (in this case x86_64)?

There don't seem to be any macros available in endian.h and my attempt at converting the upper 64 bits and lower 64 bits independently didn't work. The only method I've found that definitely works is a loop like so:

unsigned __int128 num = 0; for (int i = 0; i < 16; i++) { num = (num << 8) | byte[i]; }

I ended up doing the follow:

union { unsigned char b[MD5_DIGEST_LENGTH]; uint64_t d[2]; unsigned __int128 q; } digest; MD5((const unsigned char *)str, length, digest.b); uint64_t tmp = digest.d[0]; digest.d[0] = be64toh(digest.d[1]); digest.d[1] = be64toh(tmp); /* digest.q is now in native byte order */

This isn't bad, but I'd add a semicolon after byte[i]. ;)

Joshua Green
– Joshua Green

2011-11-04 04:35:01 +00:00
Commented Nov 4, 2011 at 4:35 — Joshua Green
– Joshua Green, Commented Nov 4, 2011 at 4:35

Joshua Green · Accepted Answer · 2011-11-04 04:30:14Z

4

If you can easily get the high and low __int64s, then you can reverse and swap them. Otherwise, you'll probably have to look at every byte individually.

answered Nov 4, 2011 at 4:30

Joshua Green

1,5751 gold badge10 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

GWW Over a year ago

Using gcc's __builtin_bswap64 would make this easier too.

moshbear · Accepted Answer · 2011-11-04 07:35:54Z

union _128_as_32 { unsigned __int128 v; unsigned __int32 d[4]; } u1, u2; u1.v = num; u2.d[3] = ntohl(u1.d[0]); u2.d[2] = ntohl(u1.d[1]); u2.d[1] = ntohl(u1.d[2]); u2.d[0] = ntohl(u1.d[3]); // do something with u2.v

If your environment has betoh64/be64toh (linux/bsd endian.h), you can use

union _128_as_64 { unsigned __int128 v; unsigned __int64 q[2]; } u1, u2; u1.v = num; u2.q[1] = betoh64(u1.q[0]); u2.q[0] = betoh64(u1.q[1]); // do something with u2.v

Seeing as you're likely dealing with IN6 addresses, you should have the ntohl family of functions already available.

H.T.H.

Damon · Accepted Answer · 2011-11-04 09:53:51Z

Since you explicitly said x86_64, you can cast to __m128i and use the PSHUFB instruction, or _mm_shuffle_epi8, respectively. Take care of properly aligning your data (though the compiler should already align an __int128 properly) so the compiler can use MOVDQA (gcc is pretty good at this, if possible it will just do so). You need to include <tmmintrin.h> for _mm_shuffle_epi8.

Note that you will need a (possibly slow) fallback implementation, because some very early AMD64 processors from around 2003-2004 did not have SSE3 support.

Jens Gustedt · Accepted Answer · 2011-11-04 08:00:14Z

If you know the pattern of reshifting the bytes you can use a compound literal for that. First I'd declare a union type, just to avoid some typing.

typedef union conv { uint128_t i; uint8_t c[16]; } conv;

Then you need some monster expression

#define swapped128(n) \ ((conv const){ \ .c = { \ [0] = 0xFF & (n >> xx) \ .... \ [15] = 0xFF & (n >> xx)\ } \ })

where you'd have to replace the xx by the values corresponding to the shift value for that byte.

Now your swapped value is simply swapped(n).i and a good compiler should find out by himself what assembler instructions to use for all of that.

Alexis Wilke · Accepted Answer · 2022-10-29 22:39:34Z

Here's an implementation based on Joshua's suggestion of swapping the bytes of 64-bit blocks individually, then swapping the 64-bit blocks:

uint64_t flip(uint64_t n) { return ((n & 0xFF00000000000000u) >> 56u) | ((n & 0x00FF000000000000u) >> 40u) | ((n & 0x0000FF0000000000u) >> 24u) | ((n & 0x000000FF00000000u) >> 8u) | ((n & 0x00000000FF000000u) << 8u) | ((n & 0x0000000000FF0000u) << 24u) | ((n & 0x000000000000FF00u) << 40u) | ((n & 0x00000000000000FFu) << 56u); } unsigned __int128 swap(unsigned __int128 n) { unsigned __int128 m; const uint64_t* src = (const uint64_t*)(&n); uint64_t* dest = (uint64_t*)(&m); dest[1] = flip(src[0]); dest[0] = flip(src[1]); return m; }

If using C++, replace the C-style casts above with reinterpret_cast.

According to Godbolt, the swap function compiles down to this assembly for Clang 12, -O3:

mov rax, rsi bswap rdi bswap rax mov rdx, rdi ret

I'm impressed with how well modern compilers can "see through" your code!

Under Linux we have a bswap_64() which would replace your flip() function (see man bswap_64) and works the same way.

Onur Turhan · Accepted Answer · 2011-11-04 06:59:27Z

not so efficient but;

unsigned __int128 htonllll(unsigned __int128 v) { union { unsigned long lv[4]; unsigned __int128 llv; } u; u.lv[0] = htonl(v >> 96); u.lv[1] = htonl(v >> 64); u.lv[2] = htonl(v >> 32); u.lv[3] = htonl(v & 0xFFFFFFFFULL); return u.llv; } unsigned __int128 ntohllll(unsigned __int128 v) { union { unsigned long lv[4]; unsigned __int128 llv; } u; u.llv = v; return ((unsigned __int128)ntohl(u.lv[0]) << 96) | (unsigned __int128)ntohl(u.lv[1]) << 64) | (unsigned __int128)ntohl(u.lv[2]) << 32) | (unsigned __int128)ntohl(u.lv[3]); }

Collectives™ on Stack Overflow

how to change byte order of 128 bit number

6 Answers 6

1 Comment

Comments

Comments

Comments

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

1 Comment

Comments

Comments

Comments

1 Comment

Comments

Related