Page MenuHomeVyOS Platform
Create Task
Maniphest T5722

Commit failure when trying to add a route in failover if the gateway is not in the same interface network
Closed, ResolvedPublicBUG
Actions

Description

Hello.

I am trying to configure failover for my main and back-up connections. My main connecion (eth0) has a /32 IP address, while the back-up one (eth1.11) has a /24 IP.

Here my the configuration snippet:

failover {
    route 0.0.0.0/0 {
        next-hop xxx.xxx.167.0 {
            check {
                target 1.1.1.1
                timeout 5
                type icmp
            }
            interface eth0
            metric 1
        }
        next-hop xxx.xxx.11.1 {
            check {
                target 1.0.0.1
                timeout 5
                type icmp
            }
            interface eth1.11
            metric 254
        }
    }
}

The xxx.xxx.167.0 is the gateway of the main connection, while xxx.xxx.11.1 is the gateway of the back-up conection.
The IP address of the main connection is 45.xxx.xxx.xxx/32.

The reported configuration does not work for eth0, failing to add the route complaining about invalid gateway.

I have also tried to add a static route for the eth0 gateway:

    static {
        route xxx.xxx.167.0/32 {
            interface eth0 {
            }
        }
    }
`

but that does not fix the issue.

The only solution I have found, is to add the onlink option to the vyos-failover.py: https://github.com/vyos/vyos-1x/blob/b7ff6f81e2bda8ff31436eced2be5be112bbd23f/src/helpers/vyos-failover.py#L210C85-L210C85

Details

Version
1.4-rolling-202304290647
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

giuavo created this task.Nov 7 2023, 10:06 AM
Viacheslav subscribed.Nov 7 2023, 11:36 AM

@giuavo I didn't test "default route", only regular routes for some prefixes, and it worked.
Could you create a PR?

Viacheslav added a project: VyOS 1.5 Circinus.Nov 7 2023, 11:37 AM
giuavo added a comment.Nov 7 2023, 11:42 AM

@Viacheslav My addition of the onlink option is really brute-force, applied blindly to everything just to see if that was a solution and give you more information. I do not think my "fix" is really ready for a PR.

giuavo updated the task description. (Show Details)Nov 7 2023, 11:43 AM
pasik subscribed.Nov 7 2023, 4:36 PM
giuavo added a comment.Nov 16 2023, 1:55 PM

I would like to contribute with a PR about this. At the same, time I would need some guidance on identifying the conditions requiring the onlink option to be added.

I was wondering whether the onlink option should just be added anytime the next_hop/gateway is not in the same sub-net as the IP of the defined interface. The corresponding code would look like:

#!/usr/bin/env python3

import json

from vyos.util import rc_cmd
from ipaddress import ip_network, ip_address

def is_in_subnet(gateway, interface):
    """Check if the gateway is in the same subnet of the interface IP"""
    try:
        rc, data = rc_cmd(f'ip -4 -detail --json address show dev {interface}')
        if rc == 0:
            d = json.loads(data)
            if len(d) > 0:
                for entry in d:
                    addrInfo = entry.get('addr_info')
                    for ip in addrInfo:
                        addr = ip.get('local')
                        prefix = ip.get('prefixlen')
                        net = ip_network(f'{addr}/{prefix}')
                        if ip_address(gateway) in net:
                            return True
    except Exception as ex:
        print(ex)

    return False

If the sub-net matches, then there is no need of the onlink option. Would that condition be enough?

Viacheslav triaged this task as Normal priority.Jan 20 2024, 1:50 PM
Viacheslav renamed this task from Failing to add route in failover to Failing to add route in failover if gateway not in the same interface network.Apr 15 2024, 11:18 AM
Viacheslav added a comment.Apr 15 2024, 11:52 AM

It is more of a feature request than a bug due to specific kernel routes.
Feature to add onlink option

set interfaces ethernet eth0 vif 10 address '10.20.30.1/32
set protocols static route 10.20.30.0/32 interface eth0.10


vyos@r1-right:~$ sudo ip route add 192.0.2.111/32 via 10.20.30.0 dev eth0.10 metric 1 proto failover
Error: Nexthop has invalid gateway.
vyos@r1-right:~$ 
vyos@r1-right:~$ 
vyos@r1-right:~$ sudo ip route add 192.0.2.111/32 via 10.20.30.0 dev eth0.10 onlink metric 1 proto failover
vyos@r1-right:~$ 
vyos@r1-right:~$

The new option should be like this:

set protocols failover route 192.0.2.1/32 next-hop 10.20.30.0 onlink
Viacheslav added a comment.EditedApr 15 2024, 3:31 PM

PR https://github.com/vyos/vyos-1x/pull/3313
Add onlink option

set interfaces ethernet eth0 vif 10 address '10.20.30.1/32'
set protocols static route 10.20.30.0/32 interface eth0.10

set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 check target '10.20.30.0'
set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 check timeout '5'
set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 check type 'icmp'
set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 interface 'eth0.10'
set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 metric '1'

set protocols failover route 192.0.2.11/32 next-hop 10.20.30.0 onlink
commit
Viacheslav changed the task status from Open to In progress.Apr 15 2024, 3:32 PM
Viacheslav claimed this task.
Restricted Repository Identity mentioned this in rVYOSONEX6825873bd1e8: Merge pull request #3313 from sever-sever/T5722.Apr 16 2024, 3:08 PM
Viacheslav mentioned this in rVYOSONEXbb832acb9788: T5722: Failover route add option onlink.Apr 16 2024, 3:09 PM
Restricted Repository Identity mentioned this in rVYOSONEX6cace2df99c7: T5722: Failover route add option onlink.Apr 16 2024, 3:10 PM
Viacheslav changed the task status from In progress to Needs testing.Apr 16 2024, 4:33 PM
Restricted Repository Identity mentioned this in rVYOSONEX5a481813c059: Merge pull request #3319 from vyos/mergify/bp/sagitta/pr-3313.Apr 16 2024, 4:40 PM
Viacheslav closed this task as Resolved.Apr 17 2024, 10:42 AM
Viacheslav edited projects, added: VyOS 1.4 Sagitta (1.4.0-epa3); removed: VyOS 1.4 Sagitta.
Viacheslav moved this task from Open to Finished on the VyOS 1.5 Circinus board.
Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta (1.4.0-epa3) board.
dmbaturin renamed this task from Failing to add route in failover if gateway not in the same interface network to Commit failure when trying to add a route in failover if the gateway is not in the same interface network.May 11 2024, 6:50 PM
dmbaturin removed a project: VyOS 1.5 Circinus.
dmbaturin changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.