2

Background Information

  • I'm using a Linux system to route traffic for a small block of public IPv4 addresses (by enabling IP forwarding in sysctrl.conf).

  • The router is connecting to the ISP over eth1 using PPPoE.

  • The local peer address used for ppp is a manually-configured IP address that was specified by the ISP. The local peer address is 10.0.0.10.

  • The remote peer address used for ppp is also a manually-configured IP address that was specified by the ISP. The remote peer address is 10.0.0.9.

  • The router's default route is 10.0.0.9 via 10.0.0.10.

  • The router is connected to an Ethernet switch via eth0. eth0 is configured to use one of the public addresses.

  • The switch connects all other public hosts. Each connected host uses a public IP address.

  10.0.0.9
ISP ----------+
              |   10.0.0.10                       X.X.X.X
              +------------- (eth1) ROUTER (eth0) --------------- SWTICH 
                                                                     |     
                                                                     +-- X.X.X.Y
                                                                     +-- X.X.X.Z
                                                                     ...

My Problem

Everything works as expected except for programs running on the router. Any application that I run on the router uses 10.0.0.10 as the source IP address when initiating connections to the internet. This is understandable since eth1 is where the internet is available. However, because the address is not publicly routable, apt, ping, and other programs don't work. If I explicitly set the source address on applications that support it (i.e. ping), applications do work.

My Question

How can I configure the router to route unknown packets via eth1/10.0.0.9 while also using the public IP address on eth0 as the default source when initiating new connections?

Tenders McChiken
  • 908
  • 1
  • 9
  • 24
  • Your question has some confusion between 10.0.0.9 and 10.0.0.10 so my answer (based on the schematic) might get the same confusion. Won't really matter once you get what to do. Specifically bullets #4 and #5 are not clear. You'd better add the output of `ip route` (feel free to obfuscate using [RFC 5737](https://tools.ietf.org/html/rfc5737 "IPv4 Address Blocks Reserved for Documentation ") as I did) to clear any issue and allow me to edit my answer accordingly. – A.B Mar 09 '20 at 15:50

2 Answers2

3

Note: I will consider that your router's LAN is 192.0.2.0/24 and its LAN IP on eth0 is 192.0.2.1/24, to be able to give concrete explanations and a solution.

Your ISP, to spare some public IP addresses, assigned you a private IPs for internal routing purpose. That's fine because this IP is never meant to be seen outside (and would be quickly dropped by Strict Reverse Path Forwarding along the path if ever put on the wire beyond your ISP router, or if not, since non-routable, will never get a reply). But this makes your configuration more complex to avoid having this IP to be used in all cases but connectivity to ISP, from your router.

You probably have something similar to this (it might be slightly different, doesn't matter):

# ip route
default via 10.0.0.9 dev ppp0 
10.0.0.9 dev ppp0 proto kernel scope link src 10.0.0.10 
192.0.2.0/24 dev eth0 proto kernel scope link src 192.0.2.1 

It's possible for example that you have instead default via 10.0.0.9 dev ppp0 src 10.0.0.10 , or that via 10.0.0.9 doesn't even appear, since it's a layer 3 link rather than a layer 2 link, making via not needed (but still accepted). Just adapt the settings below accordingly.

Currently, the kernel chooses the apparently best suited IP, the one set on the same side to reach Internet, since nothing tells it otherwise (or it explicitly tells to use the IP you don't want):

# ip route get 8.8.8.8
8.8.8.8 via 10.0.0.9 dev ppp0 src 10.0.0.10 uid 0 
    cache 

You just need to replace the routing behaviour when the kernel checks how to reach internet and state a specific preferred source IP address. Be aware that you might lose connectivity in case of errors and unforeseen problems. Have alternate (console) access if possible:

# ip route replace default via 10.0.0.9 dev ppp0 src 192.0.2.1
# ip route get 8.8.8.8
8.8.8.8 via 10.0.0.9 dev ppp0 src 192.0.2.1 uid 0 
    cache 

That's it, your router will route as usual, but when needing to choose a locally originated outgoing IP, will select 192.0.2.1 unless explicitly told otherwise (eg: process binding explicitly the source IP on its socket).

This route must be set again each time the link is brought down then up. It's up to you to integrate this in some pppoe script, after the link is fully established.

Note also that the interface name ppp0 might change to ppp1 or any other name. That's up to you to deal with this in your setup scripts.


Alternate methods to set this same route setting:

  1. add a lower metric, when a metric is initially set

If the original metric was set (ie it wasn't 0, let's say it was 100), you can instead add an alternate default route with a lower metric rather than replacing the previous:

    # ip route add default via 10.0.0.9 dev ppp0 src 192.0.2.1 metric 50
  1. dedicated routing rule

If you fear interaction from various tools involved in pppoe that might remove the route above, you can install this setting in an alternate routing table and give it precedence in routing rules, with a partial copy of the main table to prevent disruption. This will still have to be redone after each disconnection/reconnection, because the route will still disappear. Here iif lo is special and really means "locally originated outgoing packet" rather than incoming, and 109 is an arbitrary chosen table value:

    # ip rule add iif lo lookup 109 # needed only once
    # ip route add table 109 10.0.0.9 dev ppp0 proto kernel scope link src 10.0.0.10 # to keep using 10.0.0.10 for local link, just in case
    # ip route add table 109 192.0.2.0/24 dev eth0 src 192.0.2.1 # will disappear if eth0 is brought down
    # ip route add table 109 default via 10.0.0.9 dev ppp0 src 192.0.2.1 # will disappear if ppp0 is brought down

If you added other routing entries in the main table, chances are you'd also have to copy them above.

    # ip route get 8.8.8.8
    8.8.8.8 via 10.0.0.9 dev ppp0 table 109 src 192.0.2.1 uid 0 
        cache 
A.B
  • 31,762
  • 2
  • 62
  • 101
  • Upvoted for a good answer. But one point! How is he supposed to get an answer from any internet server to THIS address? That is what he wants. i would guess. Anyway, nice to read. – WGRM Mar 09 '20 at 18:39
  • I just read, that IPSec uses the same range. Let me precise my question: do you guess, that the ISP will perform NAT on this source? It is routable, yes. But i was talking about public servers. The ISP will NOT send out with this address, since there coulnt be any response. Also considering the peers will not route it, anyway. – WGRM Mar 09 '20 at 20:51
  • 1
    @WGRM are you talking about 192.0.2.0/24? This block is called TEST-NET-1 and is reserved for documentation as per [RFC 5737](https://tools.ietf.org/html/rfc5737): it's non routable on internet and intended only as an obfuscated example here to replace the actual routable IP block OP didn't tell about, and to distinguish from private IPs from [RFC 1918](https://tools.ietf.org/html/rfc1918 "Address Allocation for Private Internets") which are also non-routable on internet of course. As the name implies, it's often seen in documentations. I use it often in answers. – A.B Mar 09 '20 at 21:03
  • I think you have to reset. The OP wants "surfing" from the router. He states that clearly. So, although i like your answer, i think you didnt answer in the aspect of the request. :) – WGRM Mar 09 '20 at 21:06
  • @WGRM my answer allows OP to surf from the router. – A.B Mar 09 '20 at 21:07
  • Only with NAT from the ISP site, as stated. But why would the ISP do so. All other clients surf via public IP configuration. So the router should NOT be able to surf. Prove me wrong. :) – WGRM Mar 09 '20 at 21:09
  • Lets wait for some other answers and comment. – WGRM Mar 09 '20 at 21:14
  • 1
    @WGRM I explained in my answer why the ISP did so: to spare the cost of public IPs where it's not needed. Purists will yell that it's not supposed to be done, but it is done. And it was even done 10 years ago. There is no NAT involved here. 10.0.0.9 and 10.0.0.10 are just IPs used to connect two internal links. Those IPs are not needed anywhere else (and arguably not needed at all on layer 3 links) – A.B Mar 09 '20 at 21:27
  • Layer 3 is IP... – WGRM Mar 09 '20 at 22:45
  • Yes Ip instead of ethernet: no link layer resolution (ARP) needed, so every packet can be sent without stating a gateway (the via option). The interface is enough – A.B Mar 09 '20 at 22:46
  • Thank you so much for pointing me towards the right direction. I was not aware that you could set a source IP for a given route. Adding a source ip (via `src`) did indeed solve the problem. Also, the rationale behind routing traffic over private IPs is appreciated. – Tenders McChiken Mar 10 '20 at 09:44
0

Just as you said. The other clients on the switch with your router as gateway have public IP adresses. Your router doesnt. It has a LAN address which won't be routed. Since your Internet Service Provider did NOT give you a public address, the router will NOT be able to go out.

If you have another public ip, create a virtual eth configuration on eth1 and route it properly. Then your router can go out also.

WGRM
  • 787
  • 3
  • 14