Production-ready Secure Copy Protocol (aka: SCP) implemented in Go with well documentation and neat dependency.
Secure Copy Protocol uses Secure Shell (SSH) to transfer files between host on a network.
There is no RFC that defines the specifics of the protocol. This package simply implements SCP against the OpenSSH's scp tool, thus you can directly transfer files to/from *uinx system within your Go code, as long as the remote host has OpenSSH installed.
- Copy file from local to remote.
- Copy file from remote to local.
- Copy from buffer to remote file. (e.g: copy from
bytes.Reader) - Copy from remote file to buffer. (e.g: copy to
os.Stdout) - Recursively copy directory from local to remote.
- Recursively copy directory from remote to local.
- Set permission bits for transferred files.
- Set timeout/context for transfer.
- Preserve the permission bits and modification time at transfer.
- No resources leak. (e.g: goroutine, file descriptor)
- Low memory consuming for transferring huge files.
- TODO:
- Transfer speed limit.
- Performance benchmark/optimization for lots of small files.
- Won't support:
- Copy file from remote to remote.
go get github.com/povsister/scpThis package leverages golang.org/x/crypto/ssh to establish a SSH connection to remote host.
Error handling are omitted in examples!
// Build a SSH config from username/password sshConf := scp.NewSSHConfigFromPassword("username", "password") // Build a SSH config from private key privPEM, err := ioutil.ReadFile("/path/to/privateKey") // without passphrase sshConf, err := scp.NewSSHConfigFromPrivateKey("username", privPEM) // with passphrase sshConf, err := scp.NewSSHConfigFromPrivateKey("username", privPEM, passphrase) // Dial SSH to "my.server.com:22". // If your SSH server does not listen on 22, simply suffix the address with port. // e.g: "my.server.com:1234" scpClient, err := scp.NewClient("my.server.com", sshConf, &scp.ClientOption{}) // Build a SCP client based on existing "golang.org/x/crypto/ssh.Client" scpClient, err := scp.NewClientFromExistingSSH(existingSSHClient, &scp.ClientOption{}) defer scpClient.Close() // Do the file transfer without timeout/context err = scpClient.CopyFileToRemote("/path/to/local/file", "/path/at/remote", &scp.FileTransferOption{}) // Do the file copy with timeout, context and file properties preserved. // Note that the context and timeout will both take effect. fo := &scp.FileTransferOption{ Context: yourCotext, Timeout: 30 * time.Second, PreserveProp: true, } err = scpClient.CopyFileToRemote("/path/to/local/file", "/path/at/remote", fo)// Copy the file from remote and save it as "/path/to/local/file". err = scpClient.CopyFileFromRemote("/path/to/remote/file", "/path/to/local/file", &scp.FileTransferOption{}) // Copy the remote file and print it in Stdout. err = scpClient.CopyFromRemote("/path/to/remote/file", os.Stdout, &scp.FileTransferOption{})// From buffer buffer := []byte("something excited") reader := bytes.NewReader(buffer) // From fd // Note that its YOUR responsibility to CLOSE the fd after transfer. reader, err := os.Open("/path/to/local/file") defer reader.Close() // Note that the reader must implement "KnownSize" interface except os.File // For the content length must be provided before transfer. // The last part of remote location will be used as file name at remote. err := scpClient.CopyToRemote(reader, "/path/to/remote/file", &scp.FileTransferOption{})// recursively copy to remote err := scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", &scp.DirTransferOption{}) // recursively copy to remote with timeout, context and file properties. // Note that the context and timeout will both take effect. do := &scp.DirTransferOption{ Context: yourContext, Timeout: 10 * time.Minute, PreserveProp: true, } err:= scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", do) // recursively copy directory contents to remote. Does not create the // source directory on the target side, like `scp -r /path/to/dir/* ...` do := &scp.DirTransferOption{ ContentOnly: true, } err := scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", do)// recursively copy from remote. // The content of remote dir will be save under "/path/to/local". err := scpClient.CopyDirFromRemote("/path/to/remote/dir", "/path/to/local", &scp.DirTransferOption{})// Defines an anonymous function for ReceivePassthrough that conf the // Writer attribute of PassthroughCopy to track and display the upload // progress. The new Writer writes data to the original writer, calculates // the upload progress, and prints it as a percentage. type Progress struct { TotalSize int64 Writed int } func (p *Progress) Draw(n) () { p.Writed = n progress := float64(p.writed) / float64(p.TotalSize) * 100 fmt.Printf("progress: %.2f%%\n", progress) } func (p *Progress) Close() error { return nil } var bar *Progress do := &scp.DirTransferOption{ ObserverCallback: func(oet scp.ObserveEventType, ti scp.TransferInfo) { if oet == scp.ObserveEventStart { bar = &Progress{TotalSize: ti.TotalSize()} return } bar.Draw(ti.TransferredSize()) }, } err = scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", do)SCP is a light-weighted protocol which implements file transfer only. It does not support advanced features like: directory listing, resume from break-point.
So, it's commonly used for transferring some small-size, temporary files. If you heavily depend on the file transfer, you may consider using SFTP instead.
Another thing you may notice is that I didn't put context.Context as the first argument in function signature. Instead, it's located in TransferOption. This is intentional because it makes the API also light-weighted.