0

I have an Amazon Linux instance running SSH acting as an SFTP server. Clients log in, and are chrooted into an NFS-mounted directory. Users can read, write, and delete files, but renaming files fails with a non-specific "protocol error".

Here is a copy of my sshd_config file:

Port 22 Protocol 2 HostKey /etc/ssh/ssh_host_rsa_key UsePrivilegeSeparation yes KeyRegenerationInterval 3600 ServerKeyBits 1024 SyslogFacility AUTH LogLevel INFO LoginGraceTime 120 PermitRootLogin prohibit-password StrictModes yes RSAAuthentication yes PubkeyAuthentication yes IgnoreRhosts yes RhostsRSAAuthentication no HostbasedAuthentication no PermitEmptyPasswords no ChallengeResponseAuthentication no PasswordAuthentication yes X11Forwarding yes X11DisplayOffset 10 PrintMotd no PrintLastLog yes TCPKeepAlive yes #UseLogin no AcceptEnv LANG LC_* # Subsystem sftp /usr/lib/openssh/sftp-server -u 0002 Subsystem sftp internal-sftp -l DEBUG -u 002 -d %u UsePAM yes Match Group sftpusers ChrootDirectory /autohome AllowTCPForwarding no X11Forwarding no ForceCommand internal-sftp -l DEBUG -u 002 -d %u

I've seen reference to sftp rename not working when the source and destination are on separate filesystems, but that's not the case here. I've also seen reference to sftp rename not working on filesystems that do not support hard links, but I think our NFS server (AWS File Storage Gateway) should be fine. I'm at a loss, any help is appreciated.

Improve this question
3
  • If you strace the sshd instance handling the SFTP session while trying the rename operation, you should be able to see exactly what system call the sftp server logic is trying to do, and exactly what system error it's getting. Commented Feb 22, 2018 at 20:40
  • Turns out my assumption was wrong. Our NFS mount isn't allowing hard links, so that's a separate problem. Thanks for pushing me in the right direction. If you add this as an answer I'll mark it as correct. Commented Feb 22, 2018 at 22:52
  • I'm not sure a troubleshooting tip counts as an answer. You can answer your own question, and I think it'd be a good idea to describe what the problem was and how you figured it out. Commented Feb 22, 2018 at 23:07

1 Answer 1

Reset to default
2

Thanks to @Kenster's tip I found the issue. I was mistaken in assuming that the AWS File Storage Gateway NFS mount supported hard links, as the documentation clearly states that it does not.

I was so sure that was the case that I wound up tracing system calls with strace. If you attach an sftp client to your server while ssh'd into it, get the pid of the current sftp process with ps -eaf | grep sftp. Then you can trace the system calls with strace and save the output to a file with this command: strace -ff -p 2116 -o sftp_rename.log where -ff is following the child processes, -p is the pid, and -o is the output file.

That'll give you some really terrible looking output, but what I found interesting was this bit:

write(7, "\0\0\0L\0\0\0\3\0\0\0Drename old \"/testuse"..., 80) = 80 lstat("/testuser/test/asdfasdf.txt", {st_mode=S_IFREG|0664, st_size=159, ...}) = 0 link("/testuser/test/asdfasdf.txt", "/testuser/test/as.txt") = -1 ENOTSUPP (Unknown error 524)

Which I then tested with a simple link command to create a hard link, which failed.

# ln asdfasdf.txt link.txt ln: failed to create hard link ‘link.txt’ => ‘asdfasdf.txt’: Unknown error 524 #

Which led me back to AWS's documentation. That's not all though, apparently SFTP rename will work with certain clients (like Paramiko) that implement a vender specific CMD_EXTENDED protocol, as Paramiko does:

 oldpath = self._adjust_cwd(oldpath) newpath = self._adjust_cwd(newpath) self._log(DEBUG, 'posix_rename({!r}, {!r})'.format(oldpath, newpath)) self._request( CMD_EXTENDED, "[email protected]", oldpath, newpath )

There doesn't seem to be any way to force using the posix-rename option for all clients, but at least we know what happened and why.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.