4

I'm new to AWS and specifically to the AWS CLI tool, but so far I seem to be going OK.

I'm using the following commands to connect to AWS S3 and synchronise a local directory to my S3 bucket:

set AWS_ACCESS_KEY_ID=AKIAIMYACCESSKEY set AWS_SECRET_ACCESS_KEY=NLnfMySecretAccessCode set AWS_DEFAULT_REGION=ap-southeast-2 aws s3 sync C:\somefolder\Data\Dist\ s3://my.bucket/somefolder/Dist/ --delete

This is uploading files OK and displaying the progress and result for each file. Once the initial upload is done, I'm assuming that all new syncs will just upload new and modified files and folders. Using the --delete will remove anything in the bucket that no longer exists on the local server.

I'd like to be able to output the results of each upload (or download in the case of other servers which will be getting a copy of what is being uploaded) to a .txt file on the local computer so that I can use blat.exe to email the contents to someone who will be monitoring the sync.

All of this will be put into a batch file that will be scheduled to run nightly.

Can the output to .txt be done? If so, how?

Improve this question

1 Answer 1

Reset to default
7

I haven't tested this myself, but I found some resources that indicate you can redirect output from command-line driven applications in Windows command prompt just like you would in linux.

aws s3 sync C:\somefolder\Data\Dist\ s3://my.bucket/somefolder/Dist/ --delete > output.txt

The resources I found are:

https://stackoverflow.com/a/16713357/4471711

https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/redirection.mspx?mfr=true

Once the initial upload is done, I'm assuming that all new syncs will just upload new and modified files and folders. Using the --delete will remove anything in the bucket that no longer exists on the local server.

That is correct, sync will upload either new or modified files as compared to the destination (whether it is an S3 bucket or your local machine).

--delete will remove anything in the destination (not necessarily an S3 bucket) that is not in the source. It should be used carefully so as to avoid a situation where you've downloaded, modified and sync one file and because your local machine doesn't have ALL of the files, use of the --delete flag will then delete all other files at destination.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

not concerned about using --delete unless it modifies the source. the source is the repository for our documents and will always be the correct listing/version. this server should only ever upload (and delete) data to S3. I won't be using --delete for the batch file that does downloads FROM S3 on the other servers. Using the > works!!! Thanks.
one question regarding this method though - can it output to the console as well as file?
Not to my knowledge. I don't think that's even possible in Linux with just the output redirecting to a file. You could try working within Cygwin or, as you discovered already, maybe the AWS Powershell has some better features.
Instead of trying to duplicate the output to the console and file in one step, can you write to the file and then output the contents of the file in a second step? You can output the contents of the file with "type output.txt". superuser.com/questions/256651/…
You can send the output to both the console and the file in one step using the Linux tee command. en.wikipedia.org/wiki/Tee_(command)
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.