4

My application uses UDP. Typical Client-Server Communication consists of one packet (< 500 bytes payload) requests from the client and one packet answers to those requests from the server. In the case of packet loss the client has to resend its request.

To secure the "connection" an AES-128 PSK is used (previously transferred via TLS/SSL). Now each packet will be encrypted using AES-128 in CTR mode. The packet will contain the nonce/counter, the message to be delivered and a hash of the nonce plus message:

MyPayload = aes128CTR(nonce + plaintext + hash(nonce + plaintext))

Can a non cryptographic hash function (e.g. Murmur3) be used as a MAC in this case?

Furthermore do I have to send the complete nonce (16 bytes) or will a smaller counter (e.g. 4 bytes) be enough to be sure the packet was indeed sent by my client instead of being a possibly IP-spoofed packet?

Improve this question
3
  • Where is the non-cryptographic hash being used? Is it the hash in the MyPayload portion (that is encrypted), or are you computing the MAC on the ciphertext? Commented Sep 1, 2016 at 12:35
  • 1
    Also of interest: Should we MAC-then-encrypt or encrypt-then-MAC?. Commented Sep 1, 2016 at 12:44
  • @mikeazo it's indeed used in the hash within the MyPayloadportion. Commented Sep 1, 2016 at 17:08

1 Answer 1

Reset to default
3

I suppose not, because the hash serves as a bias to your plaintext. Murmur3, DJB2 etc have very small fixed domains and take collisions into account, which you cannot allow on a counter mode block cipher.

Assume an adversary is aware of the symmetric encryption parameters and block mode operation used, it would only take the size of the nonce + hash domain to decrypt the ciphertext. This can be done in polynomial time. This also answers your second question regarding the counter size.

I would argue using a hash function the way you describe can weaken the ciphertext even further since you derive a relative small hash from the plaintext. I would suggest encrypt-then-MAC either way.

Update

If you have the computational power to establish a TLS session anyway, why not go with a GCM/GHASH variant? You might even want to chain the packets together and add the packet counter as AEAD in GCM.

Improve this answer
1
  • thanks, I'm fairly new to this whole stuff. GCM seems to be exactly what I'm looking for. Commented Sep 1, 2016 at 17:04

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.