Access point with 2 Internet connections. Switching interfaces problem

What you want to achieve is a typical fail-over scenario. You cannot simply use two connections in the hope the second one will be used successfully if the first one fails. It is no problem to have two connections, each with an ip address. The kernel will always only use one interface as its default route to the internet. And it will use that one with the lowest metric. In Case 1 it will use eth0 with metric 202 (lower than 205 for eth1) and with its source ip address 192.168.1.254.

If eth0 fails then the kernel has no problem to dynamically switch to eth1, the next available default route. And it uses the new source ip address 192.168.8.1.

And that is the problem. Any stateful TCP connection established with source ip 192.168.1.254 to whatever destination ip address will break. These are mostly ssh, any authenticated login sessions and maybe database connections, whatever is thinkable for a stateful connection.

This problem is solved by using bonding. This defines an intermediate interface bond0 that doesn't change its ip address. Only the underlaying slave interfaces eth0 and eth1 will switch the physical connection. How it works in principle you can look at Howto migrate from networking to systemd-networkd with dynamic failover. You may be able to implement it with classic networking. Or you decide to also use systemd-networkd and configure the access point with it using Setting up a Raspberry Pi as an access point - the easy way.

Finally I managed this 'failover' to be working!

As @Milliways said, the main problem was coming from multiple default gateways.

With suggestion of @Ingo I was trying to implement bonding, however as I was told here, this technique was impossible for my case.

So I just wrote several bash scripts for making my case real.

Task

• 2 Internet providers: one by built-in ethernet connection (eth0), another one through USB modem dongle

• eth0 is the first-priority channel, while USB modem is an emergency channel

• when eth0 is available, we pass the whole traffic through it

• when eth0 is unavailable, we switch to USB modem and moving on

• as soon as eth0 is back online, we switch back.

# definitions INTERFACE_FROM="wlan0" ACCESSPOINT_IP="192.168.253" INTERFACE_TO_0="eth0" INTERFACE_TO_1="eth1" 

mkdir -p /root/failover cp "DIR_WITH_SCRIPTS (see below)" /root/failover chmod 0755 -R --quiet /root/failover sed -i "s/##INTERFACES##/$INTERFACE_TO_0 $INTERFACE_TO_1/g" /root/failover/modify_routes sed -i "s/##INTERFACE_FROM##/$INTERFACE_FROM/g" /root/failover/modify_routes sed -i "s/##INTERFACES##/$INTERFACE_TO_0 $INTERFACE_TO_1/g" /root/failover/checker sed -i "s/##INTERFACE_FROM##/$INTERFACE_FROM/g" /root/failover/checker cat > /etc/network/interfaces.d/10_$INTERFACE_TO_0 << EOF auto $INTERFACE_TO_0 allow-hotplug $INTERFACE_TO_0 iface $INTERFACE_TO_0 inet dhcp dns-nameservers 8.8.8.8 8.8.4.4 # With logging to custom log file # post-up /root/failover/save_interface_data $INTERFACE_TO_0 >> /root/failover/log 2>&1 # Logging by means of Linux post-up /root/failover/save_interface_data $INTERFACE_TO_0 metric 20 EOF cat > /etc/network/interfaces.d/20_$INTERFACE_TO_1 << EOF auto $INTERFACE_TO_1 allow-hotplug $INTERFACE_TO_1 iface $INTERFACE_TO_1 inet dhcp dns-nameservers 8.8.8.8 8.8.4.4 pre-up sleep 20 # With logging to custom log file # post-up /root/failover/save_interface_data $INTERFACE_TO_1 >> /root/failover/log 2>&1 # Logging by means of Linux post-up /root/failover/save_interface_data $INTERFACE_TO_1 metric 40 EOF # With logging to custom log file # for i in $( seq 0 5 55 ); do (crontab -l ; echo "* * * * * sleep $i; /root/failover/checker >> /root/failover/log 2>&1") | crontab -; done # Logging to /dev/null for i in $( seq 0 5 55 ); do (crontab -l ; echo "* * * * * sleep $i; /root/failover/checker > /dev/null 2>&1") | crontab -; done 

'save_interface_data' file

#!/bin/bash set -e echo -e "Save_interface_data: Start [$(date '+%Y-%m-%d %H:%M:%S')]" INTERFACE_NAME=$1 INTERFACES_DATA_DIR='/root/failover/interfaces_data' mkdir -p $INTERFACES_DATA_DIR echo "Save_interface_data: Truncate $INTERFACES_DATA_DIR/current_active_interface" > $INTERFACES_DATA_DIR/current_active_interface INTERFACE_IP='' INTERFACE_GATEWAY='' echo "Save_interface_data: Truncate $INTERFACES_DATA_DIR/$INTERFACE_NAME" > $INTERFACES_DATA_DIR/$INTERFACE_NAME # Wait for IP echo "Save_interface_data: [$INTERFACE_NAME] Wait 20 seconds for the IP address obtained" SUCCESS="false" end=$((SECONDS+20)) while [ $SECONDS -lt $end ]; do printf "." INTERFACE_IP=$((ip address show dev $INTERFACE_NAME 2>/dev/null || echo "") | (grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' || echo "") | (grep -Eo '([0-9]*\.){3}[0-9]*' || echo "") | grep -v '127.0.0.0') if [[ ! -z "$INTERFACE_IP" ]]; then SUCCESS="true" break fi sleep 1 done echo "" if [ "$SUCCESS" = "true" ]; then echo "Save_interface_data: [$INTERFACE_NAME] Ok" echo "INTERFACE_IP=$INTERFACE_IP" >> $INTERFACES_DATA_DIR/$INTERFACE_NAME else echo "Save_interface_data: [$INTERFACE_NAME] Can not get the IP address of within 20 seconds.\Aborting." exit 0 fi # Wait for Gateway echo "Save_interface_data: $INTERFACE_NAME: Wait 20 seconds for the Gateway address obtained" SUCCESS="false" end=$((SECONDS+20)) while [ $SECONDS -lt $end ]; do printf "." INTERFACE_GATEWAY=$(ip route | (grep -Eo "default(.*?)dev(.*?)$INTERFACE_NAME" 2>/dev/null || echo "") | (grep -Eo '([0-9]*\.){3}[0-9]*' || echo "") | head -1) if [[ ! -z "$INTERFACE_GATEWAY" ]]; then SUCCESS="true" break fi sleep 1 done echo "" if [ "$SUCCESS" = "true" ]; then echo "Save_interface_data: [$INTERFACE_NAME] Ok" echo "INTERFACE_GATEWAY=$INTERFACE_GATEWAY" >> $INTERFACES_DATA_DIR/$INTERFACE_NAME else echo "Save_interface_data: [$INTERFACE_NAME] Can not get the Gateway address within 20 seconds.\nAborting." exit 0 fi echo "Save_interface_data: End" 

'modify_routes' file

#!/bin/bash PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin set -e echo -e "Modify_routes: Start [$(date '+%Y-%m-%d %H:%M:%S')]" THIS_SCRIPT=`realpath $0` if [ -f "$THIS_SCRIPT-RUNNING" ]; then echo "Modify_routes: script already running.\nAborting." exit 0 fi echo "Modify_routes: Mark script as running" touch "$THIS_SCRIPT-RUNNING" WATCH_INTERFACES=( ##INTERFACES## ) # WATCH_INTERFACES=( eth0 eth1 ) INTERFACE_FROM="##INTERFACE_FROM##" # INTERFACE_FROM="wlan0" INTERFACES_DATA_DIR='/root/failover/interfaces_data' # DONT_SAVE_current_active_interface=$1 ( # Reset current_active_interface if [ ! -f "$INTERFACES_DATA_DIR/current_active_interface" ]; then echo "Modify_routes: Touch $INTERFACES_DATA_DIR/current_active_interface" touch $INTERFACES_DATA_DIR/current_active_interface else echo "Modify_routes: Truncate $INTERFACES_DATA_DIR/current_active_interface" truncate -s 0 $INTERFACES_DATA_DIR/current_active_interface fi echo "Modify_routes: Remove all 'default' gateways" while [[ ! -z $(ip route | (grep -Eo "default" 2>/dev/null || echo "")) ]]; do route delete default done FOUND="false" echo "Modify_routes: Find first available interface and set new default gateway" for INTERFACE_NAME in "${WATCH_INTERFACES[@]}" do echo "Modify_routes: Checking $INTERFACE_NAME" iptables -t nat -D POSTROUTING -o $INTERFACE_NAME -j MASQUERADE 1>/dev/null 2>&1 || echo '' > /dev/null if [ "$FOUND" = "false" ]; then echo "Modify_routes: Nothing found so far, so move on!" echo "Modify_routes: Source $INTERFACES_DATA_DIR/$INTERFACE_NAME" [ -f "$INTERFACES_DATA_DIR/$INTERFACE_NAME" ] && source "$INTERFACES_DATA_DIR/$INTERFACE_NAME" # Variables coming from "$INTERFACES_DATA_DIR/$INTERFACE_NAME" file # $INTERFACE_IP # $INTERFACE_GATEWAY echo "Modify_routes: Pinging $INTERFACE_GATEWAY through $INTERFACE_NAME" if ping -q -c 1 -W 1 -I $INTERFACE_NAME $INTERFACE_GATEWAY >/dev/null 2>&1; then echo "Modify_routes: Good!" echo "Modify_routes: Add default gateway $INTERFACE_GATEWAY" route add default gw "$INTERFACE_GATEWAY" echo "Modify_routes: Mark as found" FOUND="true" iptables -t nat -A POSTROUTING -o $INTERFACE_NAME -j MASQUERADE echo "Modify_routes: Set new active interface $INTERFACE_NAME" echo "$INTERFACE_NAME" > $INTERFACES_DATA_DIR/current_active_interface # fi fi else echo "Modify_routes: Skipping" fi done ) || echo "Modify_routes: Error occurred while running $THIS_SCRIPT.\nAborting." echo "Modify_routes: Remove running flag" rm -f "$THIS_SCRIPT-RUNNING" echo "Modify_routes: End" 

'checker' file

This script is being executed by cron every 5 seconds #!/bin/bash PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin set -e echo -e "Checker: Start [$(date '+%Y-%m-%d %H:%M:%S')]" WATCH_INTERFACES=( ##INTERFACES## ) # WATCH_INTERFACES=( eth0 eth1 ) INTERFACE_FROM="##INTERFACE_FROM##" # INTERFACE_FROM="wlan0" INTERFACES_DATA_DIR='/root/failover/interfaces_data' ACTIVE_INTERFACE=$(cat $INTERFACES_DATA_DIR/current_active_interface || echo "") # Remove all defaults except one remove_all_default_gateways_except_one() { local GW=$1 echo "Checker: Remove all defaults except $GW" for tmp_GW in $(ip route | (grep -Eo "^default(.*?)" || echo "") | (grep -Eo '([0-9]*\.){3}[0-9]*' || echo "") | grep -v '127.0.0.0' ) do echo "Checker: Found gateway $tmp_GW" if [ "$tmp_GW" != "$GW" ]; then echo "Checker: This gateway ($tmp_GW) is not $GW" route delete default gw "$tmp_GW" 2>/dev/null || echo "Checker: Remove error (unknown gateway [$tmp_GW])" echo "Checker: Gateway ($tmp_GW) removed" else echo "Checker: Skipping" fi done } # If empty $ACTIVE_INTERFACE if [[ -z "$ACTIVE_INTERFACE" ]]; then echo "Checker: Active interface is empty" /root/failover/modify_routes echo "Checker: End" exit 0 fi echo "Checker: Try to get back to 1st-priority interface" if [ "$ACTIVE_INTERFACE" != "${WATCH_INTERFACES[0]}" ]; then echo "Checker: Active interface is not the 1-st priority one (${WATCH_INTERFACES[0]})" echo "Checker: Source $INTERFACES_DATA_DIR/${WATCH_INTERFACES[0]}" [ -f "$INTERFACES_DATA_DIR/${WATCH_INTERFACES[0]}" ] && source "$INTERFACES_DATA_DIR/${WATCH_INTERFACES[0]}" echo "Checker: Pinging $INTERFACE_GATEWAY through ${WATCH_INTERFACES[0]}" if ping -q -c 1 -W 1 -I "${WATCH_INTERFACES[0]}" $INTERFACE_GATEWAY >/dev/null 2>&1; then echo "Checker: Good!" /root/failover/modify_routes echo "Checker: End" exit 0 else echo "Checker: Bad" fi else echo "Checker: Active interface is already the 1-st priority one (${WATCH_INTERFACES[0]})" fi echo "Checker: Common case for currently active interface" echo "Checker: Source $INTERFACES_DATA_DIR/$ACTIVE_INTERFACE" [ -f "$INTERFACES_DATA_DIR/$ACTIVE_INTERFACE" ] && source "$INTERFACES_DATA_DIR/$ACTIVE_INTERFACE" echo "Checker: Pinging $INTERFACE_GATEWAY through $ACTIVE_INTERFACE" if ping -q -c 1 -W 1 -I $ACTIVE_INTERFACE $INTERFACE_GATEWAY >/dev/null 2>&1; then echo "Checker: Good!" remove_all_default_gateways_except_one $INTERFACE_GATEWAY echo "Checker: End" exit 0 else echo "Checker: Bad" /root/failover/modify_routes echo "Checker: End" exit 0 fi 

Cheers!