3
  • if I do a scp over a wired 1gbps network I consistently get 112 MB/sec
  • after getting NFS set up as best I think I can, a 548gb tar file copied in 5456 seconds = 100 MB/sec
  • the NFS copy was calculated via date; cp data.tar /bkup; date
  • there is no one else logged in to either the source or destination server, and there is no other compute load on either server
  • RHEL 7.9 x86-64, with tuned profile latency-performance power profile is set
  • NFS is vers=4.1 with proto=tcp and async; I cannot get proto to become udp to see if that helps
  • why is the NFS protocol slower than the SCP (ssh secure copy) protocol ?
  • how can I get proto=udp to happen? I've tried in /etc/nfs.conf and /etc/sysconfig/nfs with no luck.
  • on a 1gbps wired network, what is the maximum MB/sec transfer speed possible?
  • Looking for any recommended modifications to either /etc/nfs.conf or /etc/syconfig/nfs, or other? and what /etc/exports options and /etc/fstab mount options should be used to get the highest NFS transfer speed
  • Also any advice on software to use to monitor, diagnose, troubleshoot this kind of thing
ron
  • 5,749
  • 7
  • 48
  • 84
  • Interesting observation. I suggest you to measure the CPU usage during both scp and nfs copy operations too, to see the full comparison. – Nikita Kipriyanov Jan 10 '23 at 15:34
  • 1
    NFS has a lot of overhead, scp does not. scp will just stream blocks of data across. NFS uses remote procedure calls to request blocks, another block, another RPC call, overhead.... As for max rate, 1Gbps / 8 = 125MBps, less overhead, oh and did I say NFS has a lot of overhead. You could quite simply look at the tcpdump output. – Bib Jan 10 '23 at 17:19
  • 1
    scp is fast because there is no filesystem interaction over the connection during the transfer and because modern cpu's have hardware to help with the encryption. People used to recompile ssh to allow operating without encryption to get speed. – Thorbjørn Ravn Andersen Jan 10 '23 at 18:33
  • 1
    you might want to try this with `netcat` also, which should be the fastest you can go over tcp - there'll be no encryption overhead at all this way.. see https://tutorials.technology/tutorials/How-to-transfer-files-over-the-network-using-Netcat.html – Brad Jan 10 '23 at 22:18
  • a `165gb data.tar` file took `25:13` with `scp` at 111.7 MB/sec versus `nc` (netcat) which took `25:06` calculated by using `date` before and after the command which maths out as 112.2 MB/sec – ron Jan 11 '23 at 21:18
  • it seems SSH's secure copy `scp` is the way to go these days, I don't see `nc` being any faster especially with the command syntax overhead of it – ron Jan 11 '23 at 21:21
  • "_on a 1gbps wired network, what is the maximum MB/sec transfer speed possible?_" - I'm assuming that's 1 Gbps. 1 Gb/s is 1000 Mb/s, which is approximately 100 MB/s. To get a more accurate answer you need to consider whether you're talking base 2ⁿ or 10ⁿ with Gb, Mb, MB, and how much overhead you want to allow for packet frames, application overhead, etc. – roaima Feb 15 '23 at 20:13

0 Answers0