1

Say I have a bucket called uploads with two directories, both of which contain images.

  1. The first directory, called catalog, has images with various extensions (.jpg, .png, etc.)

  2. The second directory, called brands, has images with no extensions.

I can request uploads/catalog/some-image.jpg and uploads/brands/extensionless-image, and they both return an image as I expect.

We're already using a third-party service, imgix, which is just an image-processing CDN that links to the S3 bucket so that we can request, say, a smaller or cropped version of the image in the bucket.

Ideally, I'd like to keep the images and objects in their current formats in the bucket, but I would like the client-side to be agnostic about which file it is requesting. In other words, I'd like to request some-image, and even though it may or may not actually have an extension in the bucket, I'd still like to somehow "intelligently guess" the image I'm requesting. We'll also assume that there are no collisions, i.e., there will never be an image some-image.jpg and some-image with both the same name (our objects are named with a collision-less algorithm).

This is what I've tried:

  • Simply request images in one directory by their extension, and the images in the other bucket without their extension (however, even though the policy is the same of requesting an image, the mechanism has to be implemented in two different ways. I would like a singular mechanism)

  • Another solution is to programmatically remove the extensions from all the images in catalog and re-sync the bucket

Anyone run into something similar before? Thoughts?

Improve this question
2
  • Just to be clear, you have these uploads/catalog/some-image.jpg and uploads/brands/extensionless-image but you'd like to request the first one as uploads/catalog/some-image? Commented Sep 19, 2015 at 0:51
  • @AdrianLynch, yes, that is correct. Commented Sep 19, 2015 at 7:02

2 Answers 2

Reset to default
2

I suspect your best bet is going to be renaming the images. Not that there aren't other solutions, but because that is probably going to be the simplest and most straightforward approach.

First, S3 will not guess. The key on an S3 object is an opaque string from S3's perspective. The extension has no meaning, and even the slashes delimiting "directories" have no intrinsic meaning to S3. (Deleting a "directory" in S3 means sending a delete request for every individual object in the directory. The console creates a convenient illusion by doing this for you.)

S3 has redirect rules, but they only match and manipulate path prefixes, not suffixes, so no help there.

It would be possible, using a reverse proxy in front of S3, to inspect requests and for any 404 or 403, the proxy could retry the request with alternate extensions, until it found one that worked, and it could potentially "learn" the right extension for use on subsequent requests, but then you'd have the added turn-around time and additional cost for multiple requests.

I have developed systems whose job it is to "find" things requested over HTTP by trying multiple back-end URLs, without the requester being aware of the "hunting" going on in the background, and it can be very useful... but that is a much more complicated solution than you would probably want to consider, particularly in light of the fact that every millisecond counts when it comes to image loading.

There is no native solution for magic guessing with S3. You pretty much have to ask it for exactly what you want. Storage in S3 is cheap enough, of course, that you could probably duplicate your content, with and without extensions, without giving too much thought to the cost. If you used a Lambda event on the bucket, you could even automate the process of copying "kitten.jpg" to "kitten" each time "kitten.jpg" was modified.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Yeah the reverse proxy idea is an interesting one, but like you said, it's probably not worth the extra time it would take making (possibly) multiple round-trips until it finds the correct object. I like the idea of the Lambda you mentioned.
0

If the content-type is set correctly in your object metadata, you should be fine regardless of extensions. If content-type header is not set, you can set it, for example using ImageMagick Identify to discover the image type and AWS CLI to set it.

Improve this answer

3 Comments

Thanks for the answer. That doesn't seem to be the case though. I have an object called 1570, which has a content type of image/jpeg. I can request https://s3.amazonaws.com/<bucket>/<directory>/1570, and this returns the image. However, if I request https://s3.amazonaws.com/<bucket>/<directory>/1570.jpg (with the extension, which doesn't actually exist on the object), I get a NoSuchKey error code. Am I missing something?
@Julio-AWSEvangelist he is trying to a request a file (object) whose key has an extension, without specifying the extension in the request. e.g GET /kitten to fetch /kitten.jpg. That, of course, will not work out of the box, since keys in S3 have no semantic meaning. Fetching an object saved with no extension, yes, that works because of Content-Type and/or browser heuristics applied to the payload.
I see... in this case a redirection rule would also not work, so i'd consider a lambda function to generate the proper image with extension and mime type when the blob is uploaded, as you mentioned.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.