1

I am trying to create a predesigned POST request using the AWS Javascript SDK for S3, using the method createPresignedPost. The idea is to generate the URL and fields which should be used in a multipart form request, to upload a file to a protected S3 bucket. I would like to include checksum verification (whereby I pass in the expected SHA256 checksum of the file to the presigner, and the S3 server verifies that the checksum of the uploaded file corresponds to the one provided, so that only the originally specified file may be uploaded using this request) as well as to restrict the size of the upload. I originally tried to follow this tutorial which seemed pretty comprehensive. However, this did not work (I kept on getting an error regarding the Content-MD5 header) and I eventually found that MD5 does not seem to be supported by S3 anymore. I also noticed that the presigner was including the field X-Amz-Algorithm=AWS4-HMAC-SHA256 which would seem to indicate that it is defaulting to SHA-256. However, sending the SHA256 hash with the same request did not solve the problem nor could I find any explicit way to tell the presigner which algorithm to use and what value to expect. I cannot remember where I came across it, but I tried including:

{ ChecksumAlgorithm: ChecksumAlgorithm.SHA256 }, { ChecksumSHA256: hash },

in my conditions but this did not work either. I am not sure whether I am understanding this issue correctly, but it may even be that S3 does not support checksum verification for presigned POST.

So, I suppose my questions come down to the following:

  1. does S3 support checksum verification on presigned POST?
  2. does the AWS Javascript SDK support adding the parameters required for checksum verification on presigned POST (and if so, in which version)?
  3. does the AWS Javascript SDK support specifying the algorithm to use for checksum verification?
  4. does anyone have an example of code using the AWS Javascript SDK to generate the URL and fields for a presigned POST, restricting the uploaded file to correspond to the given checksum value?
Improve this question
4
  • S3 will not validate your file for you, nor will it restrict the size - you'll need something like a Lambda to read the file and calculate the hash and verify the size. Commented Dec 3, 2024 at 21:54
  • Thank you for your reply @stdunbar, but as you can see in the accepted answer, size validation is something that seems to work. Commented Dec 18, 2024 at 18:03
  • No, a Lambda is doing the verification, as I indicated. S3 is not doing the verification - it's calling code to do so. It is restricting the size as you indicate. You are asking multiple questions in one post. Commented Dec 18, 2024 at 18:08
  • @stdunbar as explicitly stated in the AWS documentation linked to by Fedi, you can add restrictions based on the content-length-range allowing you to specify a minimum and maximum file size. This does not mention the use of a lambda anywhere. The lambda provided by Fedi takes care of validating the checksum of the file. And indeed, I am explicitly asking multiple questions in the one post (indicated by the numbering), although a reply to the final question would be an answer to all 4 questions. Commented Jan 28 at 12:24

2 Answers 2

Reset to default
0

So for checksum parameters like ChecksumSHA256 or ChecksumAlgorithm are not directly supported in presigned POST requests. The SDK can handle checksums for other types of uploads, such as PutObject, where headers like x-amz-checksum-sha256 can be included, but not for presigned POST.

S3 checksum validation is designed for multipart uploads or for requests with the x-amz-checksum headers, you can't use them for the POST but checksum validation can be achieved by manually validating the uploaded object after it lands in the bucket (using Lambda for example).

this is an ecample of lambda code to check the checksum after the upload:

const validateChecksum = async (bucketName, key, expectedChecksum) => { const client = new S3Client({ region: 'eu-central-1' }); try { const response = await client.send( new HeadObjectCommand({ Bucket: bucketName, Key: key, }) ); const actualChecksum = response.ChecksumSHA256; if (actualChecksum === expectedChecksum) { console.log('Checksum validation passed!'); } else { console.error('Checksum validation failed!'); } } catch (error) { console.error('Error validating checksum:', error); } };

and you can restrict the size of the file being uploaded using the content-length-range condition in the policy of the presigned POST request. This condition enforces a size range for the file uploaded to S3. here's the documentation for that : https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

I think the above answer is not correct. I was able to get checksum validation to work with signed urls. After reading this AWS SDK Issue on the topic the underlying issue seems to be that AWS S3 server-side does not support reading headers from the query string generated by the AWS SDK.

However headers can be excluded from being hoisted to the query params by using the unhoistableHeaders option on the getSignedUrl function provided by the AWS SDK.

import { S3Client, PutObjectCommand, ChecksumAlgorithm } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; const s3Client = new S3Client() const putObjectCommand = new PutObjectCommand({ Bucket: bucketName, Key: fileKey, ChecksumCRC32: checksumCRC32, ChecksumAlgorithm: ChecksumAlgorithm.CRC32 }) const presignedUrl = await getSignedUrl(s3Client, putObjectCommand, { expiresIn: 900, // This will prevent the header from being hoisted to the query params. unhoistableHeaders: new Set(['x-amz-checksum-crc32']) })

On client side the Header will need to be included explicitly:

const response = await axios.put(presignedUrl, file, { headers: { 'Content-Type': file.type, // This needs to be added explicitly! 'x-amz-checksum-crc32': checksumCRC32_base64Encoded, } })
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.