We are writing an Object Storage Catalog web service and allowing users to optionally send a Content-MD5 header. If they do not send the header we would like to generate one for the content that was ingested.
As the InputStream is read and saved to the data source we would like to generate an MD5 checksum. The problem is that some of our data can be as large as several hundred megabytes or possibly larger so we do not want to read the InputStream into memory to generate the MD5 checksum.
Is there some way to generate an MD5 checksum on the fly as data is read into a byte[] buffer and then streamed to the data source?