How do I measure the bandwidth of a particular SSH connection to a remote computer? For example, if I have an SSH tunnel doing port forwarding, I want to know how much data passes through that tunnel. In particular, I would like to know the transfer speeds in both directions, and the total data transferred (separately for both directions). How can I do this?
OS: Debian 11 and Ubuntu 20.04.
You could measure the amount of network traffic sent over the ssh connection.
For instance with:
iftop -i "$iface" -nPf "tcp and port $sport and port $dport"
Where $iface is the interface the packets for the ssh connection would be sent/received on, and $sport/$dport the source and the destination TCP ports. You could refine the filter further by also specifying the source and destination IP addresses.
Or using nethogs "$iface" which will give you a breakdown per process.
But those do include the overheads added by the ethernet, IP, TCP and ssh protocols, so not the amount of data before encryption that goes through the ssh tunnel.
You could remove the ethernet, IP, TCP overheads by getting tshark for instance to decode the packets and tell you the size of the TCP payload. But it can't go further and dissect the ssh protocol as it's encrypted.
To get how much data from the different channels ssh processes, we'd need ssh to tell us itself. Alternatively we could use strace to report each of the read()s and write()s system calls made by the ssh process. The reads/writes to the ssh TCP socket will tell us how much data is sent after encryption and ssh protocol overhead (which should match the TCP payload above), while most of the rest should be data it reads writes on its different channels (like the tty for an interactive session, the network sockets for port redirection, etc).
As a proof of concept, that could be done with a perl script such as:
#! /usr/bin/perl
# usage: that-script <pid-of-ssh>
my $pid = shift@ARGV;
my $tunnel_fd;
# list the fds of the process on TCP sockets
open my $lsof, '-|', qw(lsof -wnPa -i tcp -Fd -p), $pid;
while (<$lsof>) {
if (/^f(\d+)/) {
# the first fd on a TCP socket is assumed to be that of the ssh connection
$tunnel_fd = $1;
last;
}
}
close $lsof;
die "can't identify tunnel fd\n" unless defined $tunnel_fd;
my %dir;
sub pv {
open my $fh, '|-', "sh", "-c", q(exec pv "-ctrabN$0" 2>&1 > /dev/null), $_[0];
$fh->autoflush(1);
return $fh;
}
open my $strace, '-|', qw(strace -q -s0 -e trace=read,write -e status=successful -a0 -o/dev/stdout -p), $pid;
$dir{"1read"} = pv "outer IN";
$dir{"write"} = pv "inner IN";
$dir{"read"} = pv "inner OUT";
$dir{"1write"} = pv "outer OUT";
while (<$strace>) {
if (/^(read|write)\((\d+).*= (\d+)/) {
print { $dir{($2 eq $tunnel_fd) . $1} } "." x $3;
}
}
Here using pv to report bytes, elapsed time, instant rate and average rate.
Giving something like:
$ sudo ./ssh-stat 26655
outer IN: 768KiB 0:00:55 [0.00 B/s] [14.0KiB/s]
inner IN: 718KiB 0:00:55 [0.00 B/s] [13.1KiB/s]
inner OUT: 33.0 B 0:00:55 [0.00 B/s] [ 613miB/s]
outer OUT: 1.62KiB 0:00:55 [0.00 B/s] [30.2 B/s]
Here where the 33 bytes in inner OUT for instance is the few characters sent corresponding to the keys I typed on the terminal, while outer OUT is the data sent as part of the ssh protocol (to encapsulate those characters, but also I suppose to acknowledge data sent in the other direction).
To get a break down for each of the ssh channels, you can extend that to give separate reports per fd. The -yy option of strace which describes each fd can help. For instance:
$ sudo strace -yy -q -s0 -e trace=read,write -e status=successful -a0 -o /dev/stdout -p 26655
read(6</dev/pts/1<char 136:1>>, ""..., 16384) = 3
write(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 44) = 44
read(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 8192) = 52
write(7</dev/pts/1<char 136:1>>, ""..., 16) = 16
read(6</dev/pts/1<char 136:1>>, ""..., 16384) = 3
write(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 44) = 44
read(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 8192) = 60
write(7</dev/pts/1<char 136:1>>, ""..., 26) = 26
read(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 8192) = 44
write(9<TCP:[127.0.0.1:12344->127.0.0.1:43144]>, ""..., 3) = 3
read(9<TCP:[127.0.0.1:12344->127.0.0.1:43144]>, ""..., 16384) = 11
write(3<TCP:[127.0.0.1:42334->127.0.0.1:22]>, ""..., 52) = 52
in my tests shows how much data goes on the ssh connection (fd 3), how much goes from/to the tty (fds 6, 7), how much data goes from to the local port forward on port 12344 here (fd 9).
