Why do my Send-Q keep getting loaded but nothing come out of it?

Question

I'm currently diagnosing a issue related to packet forwarding. I forward link layer packets to B network namespace and I found that B network namespace didn't reply to them, but I could curl and view the site hosted in B network namespace from default namespace.

Here how I set up my namespace

sysctl -w net.ipv4.ip_forward=1 &&
sysctl -w net.ipv6.conf.all.forwarding=1 &&
ip netns add namespace1 &&
ip link add veth0 type veth peer name veth1 &&
ip link set veth0 netns namespace1 &&
ip netns exec namespace1 ip addr add 192.168.1.11/24 dev veth0 &&
ip link add name br0 type bridge &&
ip link set br0 up &&
ip link set veth1 up &&
ip netns exec namespace1 ip link set veth0 up &&
ip netns exec namespace1 ip link set lo up &&
ip link set veth1 master br0 &&
ip addr add 192.168.1.10/24 brd + dev br0 &&
ip -all netns exec ip route add default via 192.168.1.10 &&
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE

Now, when I run ss -lntp command on B network namespace, it outputs,

State           Recv-Q          Send-Q                     Local Address:Port                     Peer Address:Port          Process
LISTEN          0               511                                    *:443                                 *:*              users:(("node",pid=30798,fd=20))
LISTEN          0               511                                    *:7000

I'm unsure why Send-Q is always filled and not empty. I ran this Today too and I see same output with non empty Send Q.

And, here what TCP Dump shows when I run it on B network namespace,

16:50:11.742116 IP (tos 0x2a,ECT(0), ttl 115, id 8487, offset 0, flags [DF], proto TCP (6), length 52)
    MYHOMEIP.51202 > SERVERIP.https: Flags [SEW], cksum 0x59a4 (correct), seq 332112346, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

Here ip link output of default (host) namespace,

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether f2:3c:92:cc:a4:92 brd ff:ff:ff:ff:ff:ff
3: veth1@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP mode DEFAULT group default qlen 1000
    link/ether 5e:ea:89:32:05:b1 brd ff:ff:ff:ff:ff:ff link-netns namespace1
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5e:ea:89:32:05:b1 brd ff:ff:ff:ff:ff:ff

Here ip address output of default (host) namespace,

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether f2:3c:92:cc:a4:92 brd ff:ff:ff:ff:ff:ff
    inet MYSERVERIPHIDDENFORPRIVACY/24 brd MYSERVERIPHIDDENFORPRIVACY.25.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2600:3c04::f03c:92ff:fecc:a492/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2592000sec preferred_lft 604800sec
    inet6 fe80::f03c:92ff:fecc:a492/64 scope link
       valid_lft forever preferred_lft forever
3: veth1@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 5e:ea:89:32:05:b1 brd ff:ff:ff:ff:ff:ff link-netns namespace1
    inet6 fe80::5cea:89ff:fe32:5b1/64 scope link
       valid_lft forever preferred_lft forever
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5e:ea:89:32:05:b1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::4c9a:7dff:fee0:b45/64 scope link
       valid_lft forever preferred_lft forever

Here ip route output from default (host) namespace

default via HIDDENIPPRIVACY.25.1 dev eth0 proto static
HIDDENIPPRIVACY.25.0/24 dev eth0 proto kernel scope link src HIDDENIPPRIVACY.25.129
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.10

Here ip link output from namespace1 which I referred as B namespace above,

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
4: veth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether d6:9d:1c:c2:c9:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0

Here ip address output from namespace1 which I referred as B namespace above,

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
4: veth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether d6:9d:1c:c2:c9:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.11/24 scope global veth0
       valid_lft forever preferred_lft forever
    inet6 fe80::d49d:1cff:fec2:c928/64 scope link
       valid_lft forever preferred_lft forever

Here ip route output from namespace1 which I referred as B namespace above,

default via 192.168.1.10 dev veth0
192.168.1.0/24 dev veth0 proto kernel scope link src 192.168.1.11

An similar question has been asked before on this site but no one succeeded providing a solution.

Update: I still can't find why this happen.

Because there's no server running on that port to receive the packets? Because routing? — waltinator, Jul 15 '20 at 02:02
@waltinator There is server running on that port. I can locally curl that port and view pages, even I can view it directly from outside but this behavior starts when I forward packets to br0 interface of this namespace. — sqlbie, Jul 15 '20 at 04:46
you probably need to describe more precisely your layout and settings. If it's related to routing or bridging, details count. A sample of tcpdump in addition would help. As well as `ip link; ip address; ip route` twice, once for each network namespace — A.B, Jul 15 '20 at 18:08
@A.B I've edited question and added everything you requested. — sqlbie, Jul 16 '20 at 01:40
Settings appear to be fine so far. Are there *any* other iptables rules beside the one you add for nat? Also what's the output of `sysctl -ar net.bridge`? if it's not empty and gives a few `= 1` values there is more trouble to expect. Also I'd use socat or netcat tools to do all tests to rule out client or server programs behaviour. — A.B, Jul 16 '20 at 05:53
@A.B Output of your command is empty. The only other iptable rule I added is for blocking RSTs. — sqlbie, Jul 16 '20 at 07:39
@A.B The app that run on namespace is NodeJS and listening to two ports. — sqlbie, Jul 16 '20 at 07:41
As nothing seems wrong in what you added, I don't have any idea left. — A.B, Jul 16 '20 at 17:55

Why do my Send-Q keep getting loaded but nothing come out of it?

0 Answers0