"interface not ready" on XFRM network interface (Linux, IPsec)

[cluck]

There is a relatively new type of Linux network interface named "XFRM", that helps with directing network traffic to specific IPsec policies via routing, by "hiding" the policies behind such interfaces. This greatly simplifies VPN server configurations.

Unfortunately, radvd refuses to operate on such interfaces because it is fooled about the operational state of such an interface, because Linux reports it to be in an "unknown" state.

I'm not entirely sure what to propose for fixing the problem. Iproure2 effectively reports "state UNKNOWN", but then also UP,LOWER_UP, and no NO-CARRIER for the interface. And I have FRR happily proceeding with sending router-advertisements on the same interface, so that I think radvd might need some improvement in the ensure_iface_setup function in radvd/send.c.

[cluck]

For the sake of verifying the issue, such an interface can easily be instantiated with iproute2: e.g. ip link add ipsec1 type xfrm dev lo if_id 42. In real scenarios, a software like strongSwan would install matching IPsec policies (ip xfrm policy add SELECTOR dir DIR if_ID 42); without it, traffic is just discarded.

[robbat2]

I'm not sure if you're seeing the same result that I am.

Can you please try to capture the debug output of the latest master, running -d 5 -n

As to what FRR is doing differently:
Can you please capture some of those FRR RAs? ideally with tcpdump, no truncation.
And please share ALL of the other configuration on the interface in the FRR system.
Ideally ip addr show dev ... to start with, and then any FRR-specific blocks.

I did manage to make radvd trivially send RAs on the interface, despite the UNKNOWN.

The trick for me:
Set the interface as multicast ip link set ipsec1 multicast on (or configure radvd to do unicast responses)

The RA protocol fundamentally has to know where to send the announcements: either a multicast address, or specific recipients (solicited or unsolicited).

without multicast:

[May 02 21:42:44] radvd (18407): version 2.19 started
[May 02 21:42:44] radvd (18407): ipsec1 interface definition ok
[May 02 21:42:44] radvd (18407): config file, xfrm.conf, syntax ok
[May 02 21:42:44] radvd (18407): IPv6 forwarding setting is: 0, should be 1 or 2
[May 02 21:42:44] radvd (18407): IPv6 forwarding seems to be disabled, but continuing anyway
[May 02 21:42:44] radvd (18407): radvd startup PID is 18407
[May 02 21:42:44] radvd (18407): opened pid file /var/run/radvd.pid
[May 02 21:42:44] radvd (18407): locked pid file /var/run/radvd.pid
[May 02 21:42:44] radvd (18407): opened pid file /var/run/radvd.pid
[May 02 21:42:44] radvd (18407): radvd PID is 18407
[May 02 21:42:44] radvd (18407): wrote pid 18407 to pid file: /var/run/radvd.pid
[May 02 21:42:44] radvd (18407): validated pid file, /var/run/radvd.pid: 18407
[May 02 21:42:44] radvd (18407): initializing privsep
[May 02 21:42:44] radvd (18407): radvd privsep PID is 18408
[May 02 21:42:44] radvd (18407): ipsec1 if_index changed from 0 to 79
[May 02 21:42:44] radvd (18407): ioctl(SIOCGIFFLAGS) succeeded on ipsec1
[May 02 21:42:44] radvd (18408): Freeing Interfaces
[May 02 21:42:44] radvd (18407): ipsec1 is up
[May 02 21:42:44] radvd (18408): freeing interface ipsec1
[May 02 21:42:44] radvd (18407): ipsec1 is running
[May 02 21:42:44] radvd (18407): ipsec1 does not support multicast or point-to-point, forcing UnicastOnly
[May 02 21:42:44] radvd (18407): ipsec1 mtu: 1500
[May 02 21:42:44] radvd (18407): ipsec1 hardware type: ARPHRD_NONE
[May 02 21:42:44] radvd (18407): ipsec1 link layer token length: -1
[May 02 21:42:44] radvd (18407): ipsec1 prefix length: -1
[May 02 21:42:44] radvd (18407): IPv6 forwarding on interface seems to be disabled, but continuing anyway
[May 02 21:42:44] radvd (18407): ipsec1 linklocal address: fe80::d3a1:f905:3549:4e45
[May 02 21:42:44] radvd (18407): ipsec1 address: fe80::d3a1:f905:3549:4e45
[May 02 21:42:44] radvd (18407): ipsec1 is ready
[May 02 21:42:44] radvd (18407): polling for 0 second(s), next iface is ipsec1
[May 02 21:42:44] radvd (18407): timer_handler called for ipsec1
[May 02 21:42:44] radvd (18407): no client list, no destination, unicast only...doing nothing
[May 02 21:42:44] radvd (18407): ipsec1 next scheduled RA in 16 second(s)
[May 02 21:42:44] radvd (18407): polling for 16 second(s), next iface is ipsec1
^C[May 02 21:42:45] radvd (18407): poll returned early: Interrupted system call
[May 02 21:42:45] radvd (18407): exiting, 1 sigint(s) received
[May 02 21:42:45] radvd (18407): sending stop adverts
[May 02 21:42:45] radvd (18407): removing /var/run/radvd.pid
[May 02 21:42:45] radvd (18407): Freeing Interfaces
[May 02 21:42:45] radvd (18407): freeing interface ipsec1
[May 02 21:42:45] radvd (18407): returning from radvd main

Key line:

[May 02 21:42:44] radvd (18407): ipsec1 does not support multicast or point-to-point, forcing UnicastOnly

• IFF_POINTTOPOINT only applies to specific types of links all in drivers/net/usb/
• That leaves multicast or explicit unicast.

If I do enable multicast on the interface:

# ./radvd -C xfrm.conf   -d 5 -n
[May 02 21:55:00] radvd (30033): version 2.19 started
[May 02 21:55:00] radvd (30033): ipsec1 interface definition ok
[May 02 21:55:00] radvd (30033): config file, xfrm.conf, syntax ok
[May 02 21:55:00] radvd (30033): IPv6 forwarding setting is: 0, should be 1 or 2
[May 02 21:55:00] radvd (30033): IPv6 forwarding seems to be disabled, but continuing anyway
[May 02 21:55:00] radvd (30033): radvd startup PID is 30033
[May 02 21:55:00] radvd (30033): opened pid file /var/run/radvd.pid
[May 02 21:55:00] radvd (30033): locked pid file /var/run/radvd.pid
[May 02 21:55:00] radvd (30033): opened pid file /var/run/radvd.pid
[May 02 21:55:00] radvd (30033): radvd PID is 30033
[May 02 21:55:00] radvd (30033): wrote pid 30033 to pid file: /var/run/radvd.pid
[May 02 21:55:00] radvd (30033): validated pid file, /var/run/radvd.pid: 30033
[May 02 21:55:00] radvd (30033): initializing privsep
[May 02 21:55:00] radvd (30033): radvd privsep PID is 30034
[May 02 21:55:00] radvd (30033): ipsec1 if_index changed from 0 to 79
[May 02 21:55:00] radvd (30033): ioctl(SIOCGIFFLAGS) succeeded on ipsec1
[May 02 21:55:00] radvd (30033): ipsec1 is up
[May 02 21:55:00] radvd (30033): ipsec1 is running
[May 02 21:55:00] radvd (30033): ipsec1 supports multicast or is point-to-point
[May 02 21:55:00] radvd (30033): ipsec1 mtu: 1500
[May 02 21:55:00] radvd (30033): ipsec1 hardware type: ARPHRD_NONE
[May 02 21:55:00] radvd (30033): ipsec1 link layer token length: -1
[May 02 21:55:00] radvd (30033): ipsec1 prefix length: -1
[May 02 21:55:00] radvd (30034): Freeing Interfaces
[May 02 21:55:00] radvd (30033): IPv6 forwarding on interface seems to be disabled, but continuing anyway
[May 02 21:55:00] radvd (30034): freeing interface ipsec1
[May 02 21:55:00] radvd (30033): ipsec1 linklocal address: fe80::d3a1:f905:3549:4e45
[May 02 21:55:00] radvd (30033): ipsec1 address: fe80::d3a1:f905:3549:4e45
[May 02 21:55:00] radvd (30033): ipsec1 is ready
[May 02 21:55:00] radvd (30033): sending RA to ff02::1 on ipsec1 (fe80::d3a1:f905:3549:4e45), 1 options (using 16/1224 bytes)
[May 02 21:55:00] radvd (30033): ipsec1 next scheduled RA in 16 second(s)
[May 02 21:55:00] radvd (30033): polling for 16 second(s), next iface is ipsec1
[May 02 21:55:00] radvd (30033): ipsec1 recvmsg len=16
[May 02 21:55:00] radvd (30033): ipsec1 received a packet
[May 02 21:55:00] radvd (30033): ipsec1 received RA from: fe80::d3a1:f905:3549:4e45 (myself)
[May 02 21:55:00] radvd (30033): polling for 16 second(s), next iface is ipsec1
[May 02 21:55:01] radvd (30033): netlink: ipsec1, ifindex 79, flags is running
[May 02 21:55:01] radvd (30033): netlink: ipsec1, ifindex 79, flags is running
[May 02 21:55:01] radvd (30033): ipsec1 next scheduled RA in 1 second(s)
[May 02 21:55:01] radvd (30033): polling for 1 second(s), next iface is ipsec1
[May 02 21:55:02] radvd (30033): timer_handler called for ipsec1
[May 02 21:55:02] radvd (30033): ioctl(SIOCGIFFLAGS) succeeded on ipsec1
[May 02 21:55:02] radvd (30033): ipsec1 is up
[May 02 21:55:02] radvd (30033): ipsec1 is running
[May 02 21:55:02] radvd (30033): ipsec1 supports multicast or is point-to-point
[May 02 21:55:02] radvd (30033): ipsec1 mtu: 1500
[May 02 21:55:02] radvd (30033): ipsec1 hardware type: ARPHRD_NONE
[May 02 21:55:02] radvd (30033): ipsec1 link layer token length: -1
[May 02 21:55:02] radvd (30033): ipsec1 prefix length: -1
[May 02 21:55:02] radvd (30033): IPv6 forwarding on interface seems to be disabled, but continuing anyway
[May 02 21:55:02] radvd (30033): ipsec1 linklocal address: fe80::d3a1:f905:3549:4e45
[May 02 21:55:02] radvd (30033): ipsec1 address: fe80::d3a1:f905:3549:4e45
[May 02 21:55:02] radvd (30033): ipsec1 is ready
[May 02 21:55:02] radvd (30033): sending RA to ff02::1 on ipsec1 (fe80::d3a1:f905:3549:4e45), 1 options (using 16/1224 bytes)
[May 02 21:55:02] radvd (30033): ipsec1 next scheduled RA in 16 second(s)
[May 02 21:55:02] radvd (30033): polling for 16 second(s), next iface is ipsec1
[May 02 21:55:02] radvd (30033): ipsec1 recvmsg len=16
[May 02 21:55:02] radvd (30033): ipsec1 received a packet
[May 02 21:55:02] radvd (30033): ipsec1 received RA from: fe80::d3a1:f905:3549:4e45 (myself)
[May 02 21:55:02] radvd (30033): polling for 16 second(s), next iface is ipsec1
^C[May 02 21:55:08] radvd (30033): poll returned early: Interrupted system call
[May 02 21:55:08] radvd (30033): exiting, 1 sigint(s) received
[May 02 21:55:08] radvd (30033): sending stop adverts
[May 02 21:55:08] radvd (30033): stopping all adverts on ipsec1
[May 02 21:55:08] radvd (30033): sending RA to ff02::1 on ipsec1 (fe80::d3a1:f905:3549:4e45), 1 options (using 16/1224 bytes)
[May 02 21:55:08] radvd (30033): removing /var/run/radvd.pid
[May 02 21:55:08] radvd (30033): Freeing Interfaces
[May 02 21:55:08] radvd (30033): freeing interface ipsec1
[May 02 21:55:08] radvd (30033): returning from radvd main

[robbat2]

@cluck ping?

[Neustradamus]

@cluck: Have you seen @robbat2 comments?