13

I have a large number of identical fanless PCs running debian 6 (ARM). Most of these are connected via comcast and work ok. There are some that are connected to 'WiMax' modems and are having communication issues.

Specifically: if I ssh to one of these and try a command like 'ps -ax' I'll get about 3 lines back and then the session is locked up. If I let it sit, eventually it will close with a 'session closed by peer'.

What I've tried:

  • ssh -vvv → no error messages
  • ssh <user@host> 'command' → this will sometimes return the full output of the command. Sometimes it won't connect at all.

Suggestions on other things to try?

I've found that I can execute some commands successfully: e.g. hitting return a dozen times or more is ok. cd ~ and then lf works as does df -h. I can run df many times successfully but as soon as I try something with more output (e.g. ls /etc) it locks up.

Does it make a difference that I'm trying to communicate between these two hosts using OpenVPN?

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
ethrbunny
  • 743
  • 3
  • 9
  • 24
  • Make sure your path MTU isn't lower than your configured interface MTU. SSH does not fragment so, with DF bit set, packets are dropped instead. Read [this](http://packetlife.net/blog/2008/aug/18/path-mtu-discovery/) for a much more detailed explanation. – Aaron Copley Apr 15 '14 at 19:22
  • 1
    Tried setting MTU on all interfaces to 1400 with no apparent effect. Tested with `ping -c 1 -s $((5000-28)) -M do machine-ip` which returned 1500 - same as machine – ethrbunny Apr 15 '14 at 19:26
  • Well... That was worth a shot. – Aaron Copley Apr 15 '14 at 19:31
  • 1
    `tracepath -n ` confirms this: 1500 is permitted the whole way. – ethrbunny Apr 15 '14 at 19:32
  • @AaronCopley ssh has a `-T` option. `man ssh` – Valentin Bajrami Apr 15 '14 at 19:51
  • The comment was meant for ethrbunny. – Valentin Bajrami Apr 15 '14 at 19:54
  • 1
    Forgive my ignorance, but how does `-T` help in this instance? – Aaron Copley Apr 15 '14 at 20:05
  • 2
    It's an MTU problem. Yep, MTU problem. See [this thread](http://unix.stackexchange.com/questions/4261/cant-access-select-https-sites-on-linux-over-pppoe/4319#4319) for an explanation. I'm not voting to close as a duplicate because there one point that the other thread doesn't discuss: what you need to change in your VPN configuration to fix the problem. – Gilles 'SO- stop being evil' Apr 15 '14 at 22:54

3 Answers3

12

You have the symptoms of an MTU problem: some TCP connections freeze, more or less reproducibly for a given command or URL but with no easily discernible overall pattern. A telltale symptom is that interactive ssh sessions work well as long as you don't run commands with large output. See Can't access select https sites on Linux over PPPoE for an explanation.

OpenVPN has several MTU-related option — search for “mtu” in the manual. I don't have enough experience to be confident as to which option you need to change. (It's even possible that you can change something in the Wimax modem configuration.) The most likely option to change is mssfix: try lowering the value until it fixes the problem. The default is 1450; something like around 1400 might fix your problem. Try openvpn --fragment 1200 -mssfix; if it helps, increase the value until it starts breaking.

Community
  • 1
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • Am starting by setting `mssfix 1200` on the server and restarting. So far so good. If this works I'll up it to 1300 or 1400 and see what happens. Too many clients to change all the remote configs though so server side will have to do. – ethrbunny Apr 16 '14 at 01:08
  • I *knew* it was MTU! – Aaron Copley Apr 16 '14 at 14:27
  • Gilles gave a good answer (as always). Just to update it and make things easier - OpenVPN has an `mtu-test` option. From the manual: _To empirically measure MTU on connection startup, add the --mtu-test option to your configuration. OpenVPN will send ping packets of various sizes to the remote peer and measure the largest packets which were successfully received. The --mtu-test process normally takes about 3 minutes to complete._ – likewise May 14 '22 at 18:10
3

Gilles' answer is completely correct, but there is also another potential cause for this.

There was a bug in version 2.3.0 of OpenVPN which would disconnect clients when sending large chunks of data: https://community.openvpn.net/openvpn/ticket/263

This issue only occurred when using TCP. UDP was completely unaffected.

phemmer
  • 70,657
  • 19
  • 188
  • 223
3

We had the same problem, and it was indeed a MTU issue. However for us the problem was not in openVPN configuration but on the tun0 interface.

How we solved it: first find the maximum packet size that went through, with 

ping <host> -s 1500 -M do

and reducing the 1500 value until a value was going through (for us, 1350).

Once the right value was found, change the tun0 interface with 

sudo ip link set dev tun0 mtu 1350

as was proposed by Sebastian here. After that the VPN was going smoothly.

Eino Gourdin
  • 245
  • 1
  • 3
  • 9