2

I'm attempting to use Solaris 11's ILB to create a loadbalancer across two backend DNS servers. Here's my requirements:

Two external IPs: .XXX.YYY, .XXX.ZZZ - these are the DNS IP's that our clients hit Two ILB boxes in a HA configuration over those two external IPs Two backend DNS servers

I'd like to have the two ILB boxes (ilb1 and ilb2) each load balance across the two DNS servers (ns1 and ns2) on two different incoming IPs. I'd like ilb1 to be primary on .XXX.YYY and secondary on .XXX.ZZZ, and ilb to be the inverse of this. This way, if either of the DNS servers or the ILB servers goes down, DNS requests should continue unimpacted. However, we can't go Full NAT on this due to our requirements that the backend servers have to be able to see the actual SRC_IP of the DNS request - going full NAT makes it look to the DNS servers that all requests are coming from the ILB boxes, not the clients themselves.

Going HALF-NAT will correctly maintain the SRC_IP of the DNS request, however that means that the packet now has to be routed through the ILB box that it was requested from, so that the correct IP is on the packet when it gets to the client, otherwise the client will throw it out ("response from unexpected source" and the like). Here's the sticky issue – since there are two IPs, how do I reliably route the request back through the correct ILB box? /etc/defaultrouter on the DNS boxes only lets you use 1 IP, so roughly half of our responses would be trashed.

Is this setup possible? Is my description clear as mud?

Anthon
  • 78,313
  • 42
  • 165
  • 222
gatesphere
  • 21
  • 1

1 Answers1

0

It sounds like you've already got most of what you need, but you're missing some magic.

First of all, to handle multiple VIP addresses you want to use VRRP (or possibly anycast, but I don't think that's what you're doing). You didn't explicitly state that you have this part working, but I gather that you do so I'll just talk about ILB and routing.

Good news, you don't need to force routing back through the ILB.

What you want is called Direct Server Return and uses ILB's "stateless" mode.

Here's what it does:

  1. The LB changes the destination L2 ether address to the real server L2 address.
  2. The RS receives a packet with the real L3 source and real L3 destination.
  3. The RS responds with the correct L3 source and destination.

Oracle's documentation on how to set up DSR has a complete write up with example code for correctly configuring DSR. (Note: Oracle's documentation is accurate, but is written with poor grammar. To be clear, steps 1, 3 and 4 are performed on the ILB's while step 2 is performed on the real servers.)

To describe very shortly:

  • Use type=DSR in your ilbadm create-rule command
  • Add the VIP address(es) to the lo0 on each real server

If your real servers are Linux, you'll additionally need to disable some rout and arp spoof protection:

echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
bahamat
  • 38,658
  • 4
  • 70
  • 103