86

When I try to use sftp to transfer a directory containing files, I get an error message:

skipping non-regular file directory_name

The directory contains a couple of files and two subdirectories.

What am I doing wrong?

Improve this question

11 Answers 11

Reset to default
91

sftp, like cp and scp, requires that when you copy a folder (and its contents, obviously), you have to explicitly tell it you want to transfer the folder recursively with the -r option.

So, add -r to the command.

Improve this answer
10
  • 2
    -r after the put command? I am getting a "put: Invalid flag -r" error message message if I do. Adding it to the original sftp command also does not work. Commented Dec 17, 2011 at 9:12
  • 6
    Add -r to the sftp command when you're connecting. sftp -r user@host Commented Dec 17, 2011 at 15:50
  • 1
    @Kevin I am getting the following error "sftp: unknown option --r" Commented Mar 27, 2014 at 20:06
  • 5
    Beware that there is a quite long standing bug in OpenSSH's SFTP: when copying recursively a directory to a remote server, the directory itself must already exist on the target server otherwise the transfer will fail with a non-explicit error message: Couldn't canonicalise: No such file or directory. Commented Dec 7, 2015 at 10:09
  • 9
    should be something like: # sftp user@remote-host #> get -r remote-folder Commented Aug 1, 2017 at 5:03
34

I can only suggest, you use rsync. It is somewhat of an industry standard, when moving files over secure connections.

rsync -alPvz ./source_dir server.com:destination_dir

It is what I've been using for years by now.

(the -a option takes care of things like directory recursion)

Improve this answer
3
  • 7
    A bit of clarification on the parameters mentioned: -a = recursion + keep symbolic links, permissions, modification times, owner, group, special files and device files (the latter requires super-user privileges); -l = keep symbolic links (already included in -a); -P = show progress during the transfer + keep partially transferred files; -v = verbose mode (however it seems mandatory when using -P to avoid a weird (bug?) protocol version mismatch error); -z = enable compression of transferred data. Commented Dec 7, 2015 at 13:52
  • @WhiteWinterWolf it seems to be version dependant whether the -l switch has to be used or not. I remember older versions didn't include -l in -a. Commented Dec 7, 2015 at 15:13
  • 3
    Using rsync is not adviced if you have very large number of files (40000+) as it might run out of memory - especially on smaller systems like Raspberry Pi 1-3. The more files you have, the more memory rsync uses (at least prior to version 3.1 - not sure if it's yet fixed in that or newer either), leading to rsync eventually using all system memory. Commented Apr 16, 2020 at 10:16
18

This works for me:

1) connect via sftp to remote host

2) change into the remote directory you wish to copy. (Example: cd Music)

3) change to the local directory you wish to copy stuff to. (Example: lcd Desktop)

4) Issue this command: get -r *

Improve this answer
1
  • You can also pass the local target path as a last argument: get -r remote/folder/*.xml local/folder/ Commented Sep 16 at 13:31
10

You may also be able to use use scp. The general format is

scp -rp sourceDirName username@server:destDirName

scp means "secure copy". The flags are

  • -r recurse into subdirectories
  • -p preserve modification times

I believe the rest of the items are self-explanatory

Improve this answer
2
  • 1
    and scp -rp -P <port> .... when using a non-standard port Commented May 7, 2016 at 9:25
  • I'd also add the -C flag for compression (both scp/sftp). btw - scp uses sftp under the hood since OpenSSH 9 - see also openssh.com/txt/release-9.0 Commented Nov 10, 2022 at 15:48
6

If you question is actually 'how do I use sftp to transfer a directory', then

 sftp -r user@server

But if you transferring a directory, I might suggest two better options, depending on your needs. sftp requires logging into the remote server in a ssh-like command prompt, but if all you want is the file(s), there are easier and quicker ways.

Scp

For a one time transfer, to upload try:

 # upload source directory to remote server scp -rp source_dir user@server:dest #download directory from remote server: scp -rp user@server:source_dir dest #specifying ssh key: scp -rp -i ~/.ssh/key user@server:source_dir dest #remote directory has spaces scp -rp -i ~/.ssh/key user@server:"source\\ dir" dest

Rsync

If you plan on syncing the directories on a regular basis, using rsync makes more sense. It performs deltas between the two directories, saving transfer time and data over the wire.

rsync -r -a -v -e ssh --delete source_dir user@server:dest

Here is a list of some of the most common rsync options: (taken from here)

  • --delete - delete files that don't exist on sender (system)
  • -v - verbose (-vv will provide more detailed information)
  • -e "ssh options" - specify the ssh as remote shell
  • -a - archive mode - it preserves permissions (owners, groups), times, symbolic links, and devices
  • -r - recurse into directories
  • -z - compress file data during transfer
  • --exclude 'foldername' – excludes the corresponding folder from transfer
  • -P – show progress during transfer

Hope that helps!

Improve this answer
5

If rsync is not an option, I would next recommend lftp:

lftp sftp://[email protected]/path/path/

To download recursively from server use:

mirror

To upload recursively to server use:

mirror -R

Note that recursion is the default behavior. The -R is for reverse — to make the mirror command upload instead of download. Use --dry-run to make sure it's doing what you expect.

To download without recursion use:

mirror --no-recursion

You have to cd into the directories you want to mirror. Works great!

Improve this answer
0
3

If you can, use sshfs. It's a FUSE filesystem, available on most modern unices, and works with any SFTP server. This is a remote filesystem: it allows you to manipulate remote files (over the SFTP protocol) with the usual utilities.

mkdir /mount/point sshfs server.example.com:/remote/path /mount/point ls /mount/point cp -Rp /mount/point/somedir /local/location fusemount -d /mount/point
Improve this answer
0

It's a bit of a kludge but what works for me is to:

  1. Use ssh to login to the remote machine
  2. Use sftp from the remote machine to your local machine
  3. Then use the get command with the -r option to move the directory and all of its files.
Improve this answer
1
  • 1
    get dont work with -r sftp> get -r data /opt/bin get: Invalid flag -r Commented Aug 9, 2017 at 10:20
0

you can get from the server to your local path by

scp -rp user@server:directoryname(full path) .
Improve this answer
2
  • Does this add any new information to what other answers posted already? Commented Aug 25, 2019 at 15:38
  • This seems to be mostly the same as Using sftp to Transfer a Directory? Commented Aug 25, 2019 at 16:32
0

It's a workaround,

  1. You need to connect via SFTP.
  2. Go to the directory where files reside.(Example: cd [remote directory name])
  3. If you have many files with the same format, you can use the get *.[file format]. Example: If you have many files with .csv format, then use,
get *.csv
Improve this answer
1
  • 1
    Welcome to this forum. Please explain how this answers the OP's question. Commented Sep 28, 2020 at 21:32
0

I had the same need when I was already connected via sftp, and I didn't want to reconnect using -r param because in my case it's a bit annoying. There's a proxy which I have to connect first and then use a 2 factor token.

So, to avoid that, I did the following workaround: I compressed the folder using another ssh session and moved the resulting file on my sftp session.

SSH

tar -czvf myfolder.tar.gz myfolder

SFTP

put myfolder.tar.gz /server-path-name

Then, in the server (SSH), I extracted the folder by using:

tar -xzvf myfolder.tar.gz

If you're transferring a big amount of data, this workaround might be rather useful because it helps reducing the amount of data transferred and thus, the total transfer time.

Improve this answer
1
  • I like your answer, as it makes it easier to see the total size, but does require space on the server to create the zip file. alternatively, you can use -C for compression with sftp, which would also reduce the amount transferred Commented Nov 10, 2022 at 15:53

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.