1

I'm trying to copy some certificates onto my server using scp.

$ scp ./cert.* [email protected]:/tmp/

cert.crt     100% 2386     0.1KB/s   00:18
packet_write_wait: Connection to 192.168.0.42 port 22: Broken pipe
lost connection

The first file is written to the server, but not completely as the hashsum does not match with the original file.

This happens every time I try to scp these files (crt, key and p12).

Tested with Ubuntu 16.10 (OpenSSH_7.3p1 Ubuntu-1, OpenSSL 1.0.2g 1 Mar 2016) and with Windows 10 (WinSCP 5.9.4). Both fail copying the files.

It may be worth mentioning that I'm connected to an OpenVPN server, in order to reach the target server (192.168.0.42) - but that shouldn't be an issue.

Why does the pipe break and how can I successfully scp the files onto the server?

edit: This, as suggested in the comments, most likely has to do with the MTU - however, I'm not quite sure how to fix this issue yet.

SaAtomic
  • 205
  • 3
  • 14
  • 1
    Have you tried lowering your MTU on the server side or clamping the MTU to take into account the OpenVPN overhead ? – davidgo Jun 02 '17 at 06:56
  • I'll give it a shot, thank you for the suggestion. – SaAtomic Jun 02 '17 at 07:17
  • @davidgo I wouldn't want to modify the server, as others use it too. What would you suggest for the client side? Lowering/increasing the OpenVPN MTU? – SaAtomic Jun 02 '17 at 07:21
  • See also https://askubuntu.com/questions/127369/how-to-prevent-write-failed-broken-pipe-on-ssh-connection – Gnudiff Jun 02 '17 at 07:29
  • @Gnudiff thank you, but most of the answers suggest that the session is not kept alive for the duration required. My transfer should be done within seconds. – SaAtomic Jun 02 '17 at 07:33
  • Does it only happen for those files, or for any files? From your post it wasn't clear. Also, can you try scp with -v option and post output? – Gnudiff Jun 02 '17 at 07:35
  • 1
    It appears that files/data over a certain size cause the pipe to break - so I assume @davidgo was quite right and this is an issue with the MTU. – SaAtomic Jun 02 '17 at 07:38
  • 1
    If the content of the files is copy/pasted in batches of a few lines it works. If the complete text is copied and pasted, the pipe breaks. – SaAtomic Jun 02 '17 at 07:41

1 Answers1

2

Lowering the MTU of the OpenVPN connection worked for me.

From the OpenVPN manual:

--mssfix max Announce to TCP sessions running over the tunnel that they should limit their send packet sizes such that after OpenVPN has encapsulated them, the resulting UDP packet size that OpenVPN sends to its peer will not exceed max bytes. The default value is 1450.

The max parameter is interpreted in the same way as the --link-mtu parameter, i.e. the UDP packet size after encapsulation overhead has been added in, but not including the UDP header itself. Resulting packet would be at most 28 bytes larger for IPv4 and 48 bytes for IPv6 (20/40 bytes for IP header and 8 bytes for UDP header). Default value of 1450 allows IPv4 packets to be transmitted over a link with MTU 1473 or higher without IP level fragmentation.

The --mssfix option only makes sense when you are using the UDP protocol for OpenVPN peer-to-peer communication, i.e. --proto udp.

--mssfix and --fragment can be ideally used together, where --mssfix will try to keep TCP from needing packet fragmentation in the first place, and if big packets come through anyhow (from protocols other than TCP), --fragment will internally fragment them.

Both --fragment and --mssfix are designed to work around cases where Path MTU discovery is broken on the network path between OpenVPN peers.

The usual symptom of such a breakdown is an OpenVPN connection which successfully starts, but then stalls during active usage.

If --fragment and --mssfix are used together, --mssfix will take its default max parameter from the --fragment max option.

Therefore, one could lower the maximum UDP packet size to 1300 (a good first try for solving MTU-related connection problems) with the following options:

--tun-mtu 1500 --fragment 1300 --mssfix

I added the following the to OpenVPN configuration:

mssfix 1200

I also assume that this value may be tweaked for better performance.

SaAtomic
  • 205
  • 3
  • 14