0

As noted by dirkt, I'm missing information: This is the continuation of this question, but to summarize, I'm trying to receive a lot of multicast traffic, more than one interface can handle, so in order to receive it all, I want to use multiple interfaces and join streams on each of them separately.

I have three NICs, 1 (let's call it webIf) connected to the internet, and the 2 others (let's call them lanIf1 and lanIf2) connected to a LAN (same one for both). I'm trying to receive multicast traffic on both of them, but only one interface reports when receiving IGMP queries, and only for the stream that it joined. Therefore, after the timeout period, the switch stops sending the stream that the other interface joined.

To solve that issue, I'm attempting to use policy routing: create a routing table for each lanIf, add a route on each to go to the same gateway, and create two rules to use the two new routing tables before using the main routing table

So if my addresses are:

lanIf1: 25.25.43.88 lanIf2: 25.25.43.84 default gateway of the lan switch: 25.25.43.5 igmp querier: 25.25.43.1

I'd first add the tables to /etc/iproute2/rt_tables , such as cat /etc/iproute2/rt_tables would show:

# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 2 lanIf1Table 3 lanIf2Table

Then I'd run this script as root:

ip route add 25.25.43.0/24 dev lanIf1 src 25.25.43.88 table lanIf1Table ip route add default via 25.25.43.5 dev lanIf1 table lanIf1Table ip route add 25.25.43.0/24 dev lanIf2 src 25.25.43.84 table lanIf2Table ip route add default via 25.25.43.5 dev lanIf2 table lanIf2Table ip rule add from 25.25.43.88 table lanIf1Table prio 1000 ip rule add from 25.25.43.84 table lanIf2Table prio 1001

But then nothing happens, the IGMP queries keep only being answered by one of the interfaces.

The only way I can get anything to work is to change one of the rules: ip rule add from 0.0.0.0 table lanIf1Table prio 1000. Then, the IGMP reports are done on lanIf1. But then I lose all internet access, of course, because all traffic goes through lanIf1Table.

What am I doing wrong? I've been following these tutorials: http://www.rjsystems.nl/en/2100-adv-routing.php

https://blog.scottlowe.org/2013/05/29/a-quick-introduction-to-linux-policy-routing/

https://access.redhat.com/solutions/30564

Improve this question
11
  • Having two network interfaces with the same subnet mask is broken (no matter if they are actually connected to the same subnet or not), and won't work, whatever you do. Why do you think you need two NICs connected to the same LAN? What should be the advantage of that? (If this is an XY question where your Y is "I need two NICs on the same LAN", please describe your X). Commented Jun 12, 2019 at 17:07
  • It definitely is an XY problem! I actually have posted a question that describes X, but it didn't get much traction, so I'm trying some solution avenues, hence this Y. I'll update the question with this relevant information, thanks for making me realize that! Commented Jun 12, 2019 at 17:26
  • The bottleneck for receiving network traffic is usually not the NIC, because the hardware should be able to DMA traffic in as fast as it receives it from the network, but something else. So using two NICs won't solve anything. What bandwidth does your LAN segment have, what bandwidth do you need for your multicast streams, what are you using to process the multicast streams, how do you notice you don't have enough receiving bandwith? Commented Jun 12, 2019 at 19:35
  • We're receiving uncompressed video, where one 4k uncompressed stream might require about 12Gb of bandwidth, and our use cases include receiving multiple streams, so the speed limitations of our current 25Gb/s nics quickly become a bottleneck. Commented Jun 12, 2019 at 19:55
  • So how are the devices connected? Via a switch? What bandwidth limitations does the switch have? If you are doing multicast, you probably have more than two devices. What I'm getting at: If your network infrastructure isn't supporting more then 25 GB/s, having two NICs won't help, because the medium is saturated. If the network infrastructure supports higher bandwidth, but the NICs only support 25 GB/s on the wire and will slow down everything else during send and receive, again two NICs won't help. If your switch support it, you can bond interfaces. Commented Jun 13, 2019 at 13:38

2 Answers 2

Reset to default
1

Here's how a switch works:

The switch has several network ports, and those are connected point-to-point to a NIC on a computer. If the NIC can do 25 GB/s, that means that the point-to-point connection will use a protocol that does 25 GB/s.

Now with a dumb switch, that would also mean that the incoming connection routed to that NIC is restricted to 25 GB/s. Moreoever, a dumb switch would route multicasts to all ports. So with two 25 GB/s NICs in the same computer, you'd get the same multicast packet on each NIC, for a total speed of 25 GB/s. No matter that the switch itself is capable or higher bandwidth.

But in that bandwidth segment, the switch probably is pretty smart, with internal memory. So the switch can do various things:

1) It will store and forward the packet in internal RAM. This means the incoming connection isn't throttled to the maximum speed of the outgoing connection.

2) If it can do bonding, it can bundle two point-to-point connections into a single connection with double the bandwidth.

3) If it is multicast aware, and snoops the multicast protocol, it will only forward packets to registered receivers of the multicast group. So if you have two different multicast groups (or more), and an application at one NIC registers on one group, and an application at the other NIC registers for another group, you'll actually double the bandwidth. Again: These need to be different multicast groups, just different applications is not enough.

So really the easiest way is (2): Bond the NICs and have done with it.

For (3), we haven't solved the problem that Linux isn't very well made to deal with identical subnets on different NICs. There are ways around that:

a) Put the networking interfaces in different network namespaces, and start the receiving application in the namespace.

b) Have the application bind to the network interface. Not all applications will be able to do that, and I'm not actually sure if there are not complications for multicast (it would work for unicast).

c) Use different subnets, and somewhere else in your network architecture (for example on the switch, if it can do that) route multicast between the subnets.

All this is much more difficult than just (2).

And all this will just gain you a factor of 2. Lossless video compression (if you insist on keeping the noisy pixels in the video) will also give you a factor of 2-3, and slightly lossy compression will give you a much better factor. And it doesn't take a lot of CPU for a decent factor. So in the bandwidth range where you are, I'd definitely consider compression as the very first step, before messing with the network.

Improve this answer
1

It is explained here: https://access.redhat.com/solutions/53031

And less clearly here: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt

Considering a computer with 2 net interfaces, interfaceA and interfaceB. Considering that Linux decides to use interfaceB to send packets to ip address X. Considering a packet that is received on interfaceA from ip address X. Linux will drop the packet.

Unless you run sysctl net.ipv4.conf.all.rp_filter=2 in a terminal or add that line to /etc/sysctl.conf.

It enables receiving packets from an ip address on other interfaces than the one it uses to send packets to that ip address!

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.