2

I need to download the directory tree of a folder in a bucket recursively, with the caveat of not downloading any of the files themselves (they are large). This is what I have so far:

s3cmd --dry-run --recursive --no-delete-removed --verbose --rinclude='^[^.]*$' --rexclude='[^.]*$' sync s3://bucket_name/folder/ ~/Downloads/local_folder/

which correctly skips the files, but it isn't downloading the folders.

Here are some related questions I found

Improve this question
1
  • As noted in the answer to the other question you cited, there are no folders to download. The folders in S3 are largely an illusion, a side effect of the / delimiters in the key names... s3cmd may not have a simple way of downloading things that don't actually exist. A direct API call to the REST interface for ListObjects allows you to fetch "common prefixes" -- the "folders" -- but whether s3cmd leverages this in a useful way is unknown to me, fwiw. Commented Oct 7, 2015 at 2:36

1 Answer 1

Reset to default
3

Amazon S3 does not have folders. The closest concept is a CommonPrefix, which allows listing of files "within" a directory.

You would have to write your own program (eg in Python using boto) to retrieve these common prefixes and create a directory structure locally.

BTW, these days it is better to use the AWS Command-Line Interface (CLI) rather than s3cmd since it can work with all AWS services rather than just Amazon S3.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.