Interactive ssh works, but not rsync or scp, on OSX

Question

The motivation is that there's a server that I can ssh into (with or without key pairs set up ... i.e., without or with a password) but that I cannot send files to via rsync, scp, or sftp. Someone mentioned that I should check if I'm using the same port for ssh and for the file-transfer protocols, but I don't know how to check this.

If you have any other thoughts on how I could get rsync to work, I'd appreciate that too.

When I run rsync -avvvvvPt ./ [USER]@[REMOTE]:. I get:

FILE_STRUCT_LEN=24, EXTRA_LEN=4
cmd=<NULL> machine=[REMOTE] user=[USER] path=.
cmd[0]=ssh cmd[1]=-l cmd[2]=[USER] cmd[3]=[REMOTE] cmd[4]=rsync cmd[5]=--server cmd[6]=-vvvvvlogDtpre.iLsfx cmd[7]=--partial cmd[8]=. cmd[9]=. 
opening connection using: ssh -l [USER] [REMOTE] rsync --server -vvvvvlogDtpre.iLsfx --partial . .  (10 args)
msg checking charset: UTF-8

and then it hangs until ctrl-c.

Potentially interesting update?: I can use FileZilla to access this server, but can't use sftp from the command line. — dslack, Nov 09 '14 at 16:42
Sorry I was unclear. There's no error message. It just hangs until I hit `ctrl-c`. — dslack, Nov 09 '14 at 17:11
I'm running `rsync -avuPt ./ [user]@[remote]:.` Is there something else you'd recommend to make it more verbose? — dslack, Nov 09 '14 at 17:14
Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/18514/discussion-between-dslack-and-l0b0). — dslack, Nov 09 '14 at 19:50
`scp`, `sftp` and `rsync` don't use different ports unless explicitly told to on the command line. There's only one SSH service, it can't be a problem with the port number. — Gilles 'SO- stop being evil', Nov 09 '14 at 21:38
I have an idea what could be the problem: an [incorrect MTU](http://unix.stackexchange.com/questions/4261/cant-access-select-https-sites-on-linux-over-pppoe/4319#4319), which causes interactive sessions to work because they only use small packets, but file transfers don't work because they use as large packets as your machine thinks would fit — but these larger packets are getting dropped. If you copy-paste a large amount of data (2kB) in an interactive session, does it block? If so, the MTU is the problem. See the linked answer for how to fix it on Linux, I have no idea how to do it on OSX. — Gilles 'SO- stop being evil', Nov 09 '14 at 21:39
Are you trying this with your $HOME? Maybe try with a smaller directory first. `rsync` might be busy calculating checksums of a larger file. — stefan, Nov 09 '14 at 22:25

score 1 · Answer 1 · answered Dec 01 '14 at 18:05

1

Both the servers needs to have same ssh port.

If your port on one server is 22 and other one is say 44 then scp will not work.

answered Dec 01 '14 at 18:05

HCS420

11
3

There is only one server. You can see from the `rsync` debug it's using `ssh -l...` – roaima Feb 01 '18 at 14:51

score 1 · Answer 2 · answered Feb 01 '18 at 14:22

1

In my case it was a ForceCommand directive set in the remote server's sshd configuration file /etc/ssh/sshd_config. After commenting out & reloading the server it started working fine.

answered Feb 01 '18 at 14:22

ph4r05

111
2

score 0 · Answer 3 · edited Jun 11 '20 at 12:04

0

If they are all available as agrek asked then you can check the .plist files for the ports:

Open a terminal prompt and run cat /System/Library/LaunchDaemons/ssh.plist.

Scroll down a bit and you should see:

           <key>Listeners</key>
           <dict>
                  <key>SockServiceName</key>
                   <string>ssh</string>
                  <key>Bonjour</key>
                  <array>
                          <string>ssh</string>
                          <string>sftp-ssh</string>
                 </array>
          </dict>

The <string>...<string> below <Key> SockServiceName </key>is what you are after. If the text is ssh like above then its using the standard port which is port 22. If it is a number then that number is the port.

Rsync over ssh uses the same port as it sends the data through the ssh tunnel and iirc sftp and scp use the ssh tunnel so their ports are the same as ssh.

Sources: https://serverfault.com/questions/18761/how-to-change-sshd-port-on-mac-os-x

edited Jun 11 '20 at 12:04

Community

1

answered Nov 09 '14 at 16:59

OneRainbow

16
1

Okay, my `ssh.plist` looks identical to yours. – dslack Nov 09 '14 at 17:12
That means that the port is 22 so just using the ssh command normally should work – OneRainbow Nov 09 '14 at 17:13
And i see that there is no error message – OneRainbow Nov 09 '14 at 17:14
Run ssh like you normally do with `-vvv` and put the output in an answer please. ( i dont have permission for commenting on the question) – OneRainbow Nov 09 '14 at 17:20
Just added this to the question: When I run `rsync -avvvvvPt ./ [USER]@[REMOTE]:.` I get: `FILE_STRUCT_LEN=24, EXTRA_LEN=4 cmd= machine=[REMOTE] user=[USER] path=. cmd[0]=ssh cmd[1]=-l cmd[2]=[USER] cmd[3]=[REMOTE] cmd[4]=rsync cmd[5]=--server cmd[6]=-vvvvvlogDtpre.iLsfx cmd[7]=--partial cmd[8]=. cmd[9]=. opening connection using: ssh -l [USER] [REMOTE] rsync --server -vvvvvlogDtpre.iLsfx --partial . . (10 args) msg checking charset: UTF-8` and then it hangs until `ctrl-c`. – dslack Nov 09 '14 at 19:24
Ok. Formatting of that is weird but it appears to attempt to open connection via ssh then freezes at checking charset. Given the fact that standalone ssh doesn't work for you either, can you run the command and copy-paste the output please so we can view the ssh side of things? `ssh -vvv user@host`. And if possible try with another ssh server to see if its client or server issue. – OneRainbow Nov 09 '14 at 20:10
Sorry I was unclear, `ssh` *does* work. I ran it with -vvv and it gives a whole lot of output, more than fits in this comments window. Is there something in particular you'd like to know? From another server, I also *can* `ssh` in and *cannot* `rsync` in. But I can `rsync` from the remote server and pull files over. Just can't send them. – dslack Nov 09 '14 at 20:20
Ah. Then its getting stuck on charsets with rsync like the verbosity says. Ill have to do some reading about what it is doing – OneRainbow Nov 09 '14 at 20:32
Try adding this flag to it: --iconv=CONVERT_SPEC which according to the manual page: requests charset conversion of filenames (From http://linux.die.net/man/1/rsync) – OneRainbow Nov 09 '14 at 20:41
@stefan That might be it actually. Open another terminal promot and run "top" on the server and client pc to see if it is using the cpu. – OneRainbow Nov 10 '14 at 13:55
When I add --iconv=CONVERT_SPEC, it fails immediately, and the following additional output is produced: `iconv_open("UTF-8", "CONVERT_SPEC") failed [sender] _exit_cleanup(code=4, file=rsync.c, line=122): entered rsync error: requested action not supported (code 4) at rsync.c(122) [sender=3.1.1] [sender] _exit_cleanup(code=4, file=rsync.c, line=122): about to call exit(4) [sender] _exit_cleanup(code=19, file=main.c, line=1487): entered rsync error: received SIGUSR1 (code 19) at main.c(1487) [sender=3.1.1] [sender] _exit_cleanup(code=19, file=main.c, line=1487): about to call exit(19)` – dslack Nov 10 '14 at 14:52
Oops. Sorry about that. On further research, it turns out you should use it in a different way. See the new answer for an example (Formatting prevents me from doing it here) – OneRainbow Nov 10 '14 at 15:25

score 0 · Answer 4 · answered Nov 10 '14 at 10:10

0

Try and use rsync with ssh.

rsync -avvvvvPt --rsh=ssh ./ [USER]@[REMOTE]:.

answered Nov 10 '14 at 10:10

a21

354
2
4

It's just hanging. Output: `FILE_STRUCT_LEN=24, EXTRA_LEN=4 cmd=ssh machine=[REMOTE] user=[USER] path=. cmd[0]=ssh cmd[1]=-l cmd[2]=[USER] cmd[3]=[REMOTE] cmd[4]=rsync cmd[5]=--server cmd[6]=-vvvvvlogDtpre.iLsfx cmd[7]=--partial cmd[8]=. cmd[9]=. opening connection using: ssh -l [USER] [REMOTE] rsync --server -vvvvvlogDtpre.iLsfx --partial . . (10 args) msg checking charset: UTF-8` – dslack Nov 11 '14 at 16:22

score 0 · Answer 5 · answered Nov 10 '14 at 15:33

0

New answer after reading up more on the iconv flag and the added information to the question.

According to their FAQ: http://rsync.samba.org/FAQ.html

You can avoid a charset problem by passing an appropriate --iconv option to rsync that tells it what character-set the source files are, and what character-set the destination files get stored in. For instance, the above Mac OS X problem would be dealt with by using --iconv=UTF-8,UTF8-MAC (UTF8-MAC is a pseudo-charset recognized by Mac OS X iconv in which all characters are decomposed).

So you need to find out what charset the source is and the charset used by the OS on the destination computer.

Quick note: Its also possible that @stefan may be right about the files being big and its taking time to calculate, but that's related to hash generation not with charsets and shouldn't matter for this.

answered Nov 10 '14 at 15:33

OneRainbow

16
1

command: `rsync -avvvvvPt --rsh=ssh --iconv=UTF-8,UTF8-MAC ./ [USER]@[REMOTE]:.` output: `FILE_STRUCT_LEN=24, EXTRA_LEN=4 cmd=ssh machine=[REMOTE] user=[USER] path=. cmd[0]=ssh cmd[1]=-l cmd[2]=[USER] cmd[3]=[REMOTE] cmd[4]=rsync cmd[5]=--server cmd[6]=-vvvvvlogDtpre.iLsfx cmd[7]=--iconv=UTF8-MAC cmd[8]=--partial cmd[9]=. cmd[10]=. opening connection using: ssh -l [USER] [REMOTE] rsync --server -vvvvvlogDtpre.iLsfx --iconv=UTF8-MAC --partial . . (11 args) msg checking charset: UTF-8 [sender] charset: UTF-8` and it just hangs – dslack Nov 11 '14 at 16:25
Hmm. This has me stumped. Can we have more detail on the systems you are using and the files please? – OneRainbow Nov 11 '14 at 16:27
Sure. Please let me know what kind of detail would help -- i.e., what command-line commands I can use to find out. Re: files, I'm trying with a test directory that has some blank files in it. My local computer is a Macbook Pro running Yosemite. The remote is a 1and1 server, running `uname -a` gives "Linux [NAME] 3.2.60-infong-14190 #1 SMP Wed Jul 9 20:00:11 CEST 2014 x86_64 GNU/Linux" – dslack Nov 11 '14 at 18:35
Ok hmm. Does the user actually have access to that directory? And I think you may have the -iconv figuration the wrong way round. – OneRainbow Nov 12 '14 at 15:40
Also check the encoding of the files please with the command `file -I *` (mac) or `file -i *` (Linux) if the charset is different then please update the -iconv command to match it. – OneRainbow Nov 12 '14 at 15:46
Command: `file -I *`. Output: `a: application/x-empty; charset=binary` – dslack Nov 13 '14 at 15:21
When I repeat the `rsync` command with `--iconv=binary` or `--iconv=BINARY`, it fails immediately. Output: `iconv_open("UTF-8", "BINARY") failed` and other related messages etc. – dslack Nov 13 '14 at 15:23
Hmm. Could you update your version of rsync via macports or build your own up to date version? – OneRainbow Nov 15 '14 at 21:56
Wait. First make sure rysnc isnt the latest one. Run `rsync -V` to get the version. The latest one is 3.2 if i remember correctly – OneRainbow Nov 15 '14 at 21:59
Mine is version 3.1.1. (the version command is `rsync --version` for me... `rsync -V` didn't work.) – dslack Nov 16 '14 at 22:33
Oops. Sorry about that and that is up-to-date. Hmm. I'm stumped now. – OneRainbow Nov 18 '14 at 16:28

Interactive ssh works, but not rsync or scp, on OSX

5 Answers5