Test last stable vpp releases and do update if no problems occurred

[denis-tingaikin]

Plan:

1. Check and test all stable releases since our current vpp version. For testing, use forwarder-vpp docker tests and NSM integration tests.
2. If no problem occurred, just do an update
3. Otherwise, provide a table with the next information: vpp version, which tests don't work
4. [Optional] Find a root cause of problems and reproduce it via pure VPP.
5. [Optional] Report found problems to vpp
6. [Optional] Commit fixes.

[NikitaSkrynnik]

NSM doesn't work with VPP v24.02 and Calico stilll uses v24.02-rc0. I think it's reasonable to wait for a new release from Calico and then update VPP in NSM

[Ex4amp1e]

Plan:

• vpp 24.06
• vpp 24.02

Decomposition:

• Run govpp docker tests without changes - 30m
• Build govpp based on vpp 24.06 - 4h
• Run cmd-forwarder-vpp tests based on vpp 24.06 - 2h
• Build govpp based on vpp 24.02 - 4h
• Run cmd-forwarder-vpp tests based on vpp 24.06 - 2h

[Ex4amp1e]

Current results:
cmd-forwarder-vpp docker tests fail on vpp v24.06:

            --- FAIL: TestForwarderTestSuite/TestCombinations/Ethernet/Memif (2.53s)
                --- FAIL: TestForwarderTestSuite/TestCombinations/Ethernet/Memif/Kernel (0.66s)
                --- FAIL: TestForwarderTestSuite/TestCombinations/Ethernet/Memif/Memif (0.92s)
                --- FAIL: TestForwarderTestSuite/TestCombinations/Ethernet/Memif/Vxlan (0.95s)

Note: ipv6 pings fail
Full docker log: 24.06.new.txt

vpp v24.02 has been tested and NSM doesn't work #24 (comment)

Possible options:
Option 1: Find the root cause why Ethernet/Memif tests do not work in vpp code
Option 2: Wait vpp v24.10 (looks like it should be released at the end of Oct 2024) and test it

[denis-tingaikin]

cc @szvincze

[elajkat]

24.10 is out, see the re-no page: https://s3-docs.fd.io/vpp/24.10/aboutvpp/releasenotes/v24.10.html
(https://gerrit.fd.io/r/c/vpp/+/41646 )

[elajkat]

I am not sure I use the govpp api diff in correct way but the API diff for v23.06 to v24.06:
https://pastebin.com/jQjPWdWN

(I used the command:
$ cd ~/temp; git clone https://github.com/FDio/vpp; git checkout v24.06; cd ~/vpp; git checkout v23.06; govpp vppapi diff --against ~/temp/vpp/
)
Perhaps worth to check first the API diff why those tests are failing.

(The API diff for 23.06=>24.10 is here: https://pastebin.com/L52EqTzX )

[NikitaSkrynnik]

Current plan:

• Update vpp to the latest release
• Run docker tests in cmd-forwarder-vpp
• If docker tests work then run all tests
• If docker tests don't work then reproduce the same setup from the failed tests on plain vpp

[NikitaSkrynnik]

We've tested vpp v24.10 on our CI and docker tests. There is a problem with memif interfaces. Particularly tests with vpp endpoint don't work.

Current plan:

• test kernel2memif scenario on plain vpp
• if it works then find the difference between our setup and the plain vpp setup

[elajkat]

Hi, you reproduced the failure (memif with VPP 24.10) with the integration-k8s-kind environment? I try to reproduce it in my local env.

[NikitaSkrynnik]

Yes, I used our integration tests. Tests work if I launch them separately with different forwarders. But if I run the whole basic suite all of them fail except the first one. It looks like we don't clean resources on forwarder properly. The same behaviour is in docker tests. Plain vpp memif tests work correctly. Current plan:

• debug docker tests

[NikitaSkrynnik]

Current state:

1. Docker tests has been fixed after disabling direct memif feature in cmd-forwarder-vpp: code
2. Integration-tests don't work even without direct memif. Debugging of Kernel2Memif has shown that the problem is with memif interfaces. cmd-nse-vpp shows error no free tx slots after several successful pings. Manual setup of Kernel2Memif with two plain vpp instances doesn't have this problem.