Skip to content

Commit

Permalink
[PULL 00/10] Functional test improvements and fixes
Browse files Browse the repository at this point in the history 
https://lore.kernel.org/qemu-devel/[email protected]

---

From: Thomas Huth <[email protected]>
To: [email protected]
Cc: Stefan Hajnoczi <[email protected]>
Subject: [PULL 00/10] Functional test improvements and fixes
Date: Thu,  2 Jan 2025 11:31:28 +0100
Message-ID: <[email protected]>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17
Received-SPF: pass client-ip=170.10.133.124; [email protected];
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -42
X-Spam_score: -4.3
X-Spam_bar: ----
X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.186,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: [email protected]
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:[email protected]?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:[email protected]>
List-Help: <mailto:[email protected]?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:[email protected]?subject=subscribe>
Errors-To: [email protected]
Sender: [email protected]

 Hi Stefan!

The following changes since commit 8b70d7f:

  Merge tag 'hw-misc-20241231' of https://github.com/philmd/qemu into staging (2025-01-01 15:17:08 -0500)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2025-01-02

for you to fetch changes up to c5efe54:

  tests/functional/test_arm_quanta_gsj: Fix broken test (2025-01-02 09:16:42 +0100)

----------------------------------------------------------------
* Update year in copyright statements
* Convert the VNC test to the functional framework
* Improve and update the ppc64_hv functional test
* Fix broken rx_gdbsim and arm_quanta_gsj functional tests

----------------------------------------------------------------
Ani Sinha (1):
      docs: update copyright date to the year 2025

Nicholas Piggin (3):
      tests/functional/test_ppc64_hv: Simplify console handling
      tests/functional/test_ppc64_hv: Update repo management
      tests/functional/test_ppc64_hv: Update to Alpine 3.21.0

Philippe Mathieu-Daudé (1):
      tests/functional/test_rx_gdbsim: Use stable URL for test_linux_sash

Thomas Huth (5):
      tests/functional: Convert the vnc test
      tests/functional/test_vnc: Do not use a hard-coded VNC port
      tests/functional/test_vnc: Remove the test_no_vnc test
      tests/functional: Extract the find_free_ports() function into a helper file
      tests/functional/test_arm_quanta_gsj: Fix broken test

 docs/conf.py                                     |  2 +-
 include/qemu/help-texts.h                        |  2 +-
 tests/functional/meson.build                     |  1 +
 tests/functional/qemu_test/ports.py              | 56 ++++++++++++++++++++++++
 tests/functional/test_arm_quanta_gsj.py          |  2 +-
 tests/functional/test_ppc64_hv.py                | 52 ++++++++++------------
 tests/functional/test_rx_gdbsim.py               | 13 +++---
 tests/{avocado/vnc.py => functional/test_vnc.py} | 55 +++++++----------------
 8 files changed, 108 insertions(+), 75 deletions(-)
 create mode 100644 tests/functional/qemu_test/ports.py
 rename tests/{avocado/vnc.py => functional/test_vnc.py} (74%)
 mode change 100644 => 100755

Signed-off-by: GitHub Actions Bot <[email protected]>
  • Loading branch information
GitHub Actions Bot committed Jan 2, 2025
1 parent 4d4f4f4 commit 348dbcf
Show file tree
Hide file tree
Showing 30 changed files with 741 additions and 801 deletions.
343 changes: 343 additions & 0 deletions .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,343 @@
on: push

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
checkapply:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# to debug container live from GitHub
# - uses: mxschmitt/action-tmate@v3
- run: bash -c '[ ! -f shazam.log ] || { cat shazam.log; exit 1; }'

checkpatch-ignore-signoff:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: git fetch -a origin --unshallow || true
- run: git remote add upstream -f https://gitlab.com/qemu-project/qemu
- run: ./scripts/checkpatch.pl --no-signoff $(git merge-base upstream/master HEAD)..HEAD

checkpatch-with-signoff:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: git fetch -a origin --unshallow || true
- run: git remote add upstream -f https://gitlab.com/qemu-project/qemu
- run: ./scripts/checkpatch.pl $(git merge-base upstream/master HEAD)..HEAD

# use docker-run to not rebuild images
# images are built daily and pushed on pbolinaro/qemu-ci:*
build-cross:
needs: checkapply
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
container: [alpine,centos9,debian,debian-all-test-cross,debian-amd64-cross,debian-arm64-cross,debian-armhf-cross,debian-hexagon-cross,debian-i686-cross,debian-legacy-test-cross,debian-loongarch-cross,debian-mips64el-cross,debian-mipsel-cross,debian-ppc64el-cross,debian-riscv64-cross,debian-s390x-cross,debian-tricore-cross,fedora,fedora-rust-nightly,opensuse-leap,ubuntu2204]
steps:
- uses: actions/checkout@v4
- run: pip install meson
- run: make docker-run J=$(nproc) RUNC=podman TEST=test-build IMAGE=docker.io/pbolinaro/qemu-ci:${{matrix.container}}

build:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure && ninja -C build install'
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin
build-cross-mingw64:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:fedora-win64-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-windows:
needs: checkapply
runs-on: windows-2025
strategy:
fail-fast: false
matrix:
sys: [UCRT64, CLANG64, MINGW64]
defaults:
run:
shell: msys2 {0}
steps:
- uses: msys2/setup-msys2@v2
with:
update: true
msystem: ${{matrix.sys}}
- run: pacman -S --noconfirm curl git
- uses: actions/checkout@v4
- run: >
pacman -S --noconfirm
base-devel binutils bison diffutils flex git grep make sed
${MINGW_PACKAGE_PREFIX}-toolchain
${MINGW_PACKAGE_PREFIX}-glib2
${MINGW_PACKAGE_PREFIX}-gtk3
${MINGW_PACKAGE_PREFIX}-libnfs
${MINGW_PACKAGE_PREFIX}-libssh
${MINGW_PACKAGE_PREFIX}-ninja
${MINGW_PACKAGE_PREFIX}-pixman
${MINGW_PACKAGE_PREFIX}-pkgconf
${MINGW_PACKAGE_PREFIX}-python
${MINGW_PACKAGE_PREFIX}-SDL2
${MINGW_PACKAGE_PREFIX}-zstd
- run: ./configure && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-macos-x86_64:
needs: checkapply
runs-on: macos-13
steps:
- uses: actions/checkout@v4
- run: brew install --quiet $(brew deps --include-build qemu) || true
# on macos, werror is not on by default
- run: ./configure --enable-werror && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-macos-aarch64:
needs: checkapply
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- run: brew install --quiet $(brew deps --include-build qemu) || true
# on macos, werror is not on by default
- run: ./configure --enable-werror && ninja -C build
- run: ./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin

build-misc:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-user --disable-system --enable-docs --enable-tools && ninja -C build install'
build-32bits:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-i686-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-big-endian:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-s390x-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS && ninja -C build install'
build-debug:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --enable-debug --enable-asan --enable-ubsan && ninja -C build install'
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
./build/qemu-system-x86_64 -nographic -plugin ./build/contrib/plugins/libstoptrigger,icount=1000000 -plugin ./build/tests/tcg/plugins/libinsn -d plugin
build-static:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-system --disable-tools --disable-guest-agent --disable-docs --static && ninja -C build install'
build-tsan:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --enable-tsan && ninja -C build install'
build-clang:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --cxx=clang++ --cc=clang --host-cc=clang && ninja -C build install'
build-clang-latest:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx 'LLVM_VERSION=19 && apt update && apt install -y lsb-release wget software-properties-common gnupg && wget https://apt.llvm.org/llvm.sh && bash llvm.sh ${LLVM_VERSION} && ./configure --cxx=clang++-${LLVM_VERSION} --cc=clang-${LLVM_VERSION} --host-cc=clang-${LLVM_VERSION} && ninja -C build install'
build-rust:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:fedora-rust-nightly
bash -cx 'cargo install [email protected] && ./configure $QEMU_CONFIGURE_OPTS --enable-rust && ninja -C build install'
build-disable-tcg:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-tcg && ninja -C build install'
build-disable-kvm:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-kvm && ninja -C build install'
build-disable-tcg-kvm-for-xen:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --disable-tcg --disable-kvm && ninja -C build install'
build-minimal:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian
bash -cx './configure --without-default-features --without-default-devices --disable-kvm --disable-tcg && ninja -C build install'
check-tcg:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-all-test-cross
bash -cx './configure $QEMU_CONFIGURE_OPTS --disable-docs --enable-debug-tcg --enable-debug-graph-lock --enable-debug-mutex --enable-asan --enable-ubsan && ninja -C build'
- run: >
podman run --init --privileged --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-all-test-cross
bash -cx "env ASAN_OPTIONS=detect_leaks=0 make -k -j $(nproc) check-tcg"
# run all meson tests, except functional.
# block tests are not ran because they don't support sanitizers:
# https://gitlab.com/qemu-project/qemu/-/blob/master/tests/qemu-iotests/meson.build
check:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --disable-docs --enable-debug-tcg --enable-debug-graph-lock --enable-debug-mutex --enable-asan --enable-ubsan && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -i -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "env ASAN_OPTIONS=detect_leaks=0 ./build/pyvenv/bin/meson test -C build --setup thorough --no-suite func-quick --no-suite func-thorough -t 5 --print-errorlogs"
check-functional:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --disable-docs --enable-debug-tcg --enable-debug-graph-lock --enable-debug-mutex --enable-asan --enable-ubsan && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -i -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "env ASAN_OPTIONS=detect_leaks=0 ./build/pyvenv/bin/meson test -C build --setup thorough --suite func-quick --suite func-thorough -j $(($(nproc) / 2)) -t 5 --print-errorlogs"
# Limit parallelism because it creates timeout.
# iotests do not support sanitizers, so we run them in their own job
check-block:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --disable-docs --enable-debug-tcg --enable-debug-graph-lock --enable-debug-mutex && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -i -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "./build/pyvenv/bin/meson test -C build --setup thorough --suite block --suite block-slow --suite block-thorough -t 5 --print-errorlogs"
check-avocado:
needs: checkapply
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
# add more time for all tests
- run: sed -i -e 's/timeout = .*/timeout = 3600/' $(find tests/avocado/ -type f)
# we use image with download cache filled. Solves servers flakiness.
- run: >
podman run --init --rm -it -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx './configure --disable-docs --enable-debug-tcg --enable-debug-graph-lock --enable-debug-mutex --enable-asan --enable-ubsan && ninja -C build'
- run: sudo chown $USER:$USER /dev/kvm
- run: >
podman run --init --privileged --rm -it -v /dev/kvm:/dev/kvm -v $(pwd):$(pwd) -w $(pwd)
docker.io/pbolinaro/qemu-ci:debian-precache-tests
bash -cx "env ASAN_OPTIONS=detect_leaks=0 make -k check-avocado V=1"
41 changes: 41 additions & 0 deletions .github/workflows/containers.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
on:
schedule:
- cron: '0 6 * * *'
workflow_dispatch:

permissions: write-all

jobs:
build_container:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
# cd tests/docker/dockerfiles/
# ls *docker | sed -e 's/.docker//' | tr '\n' ','
# remove: debian-bootstrap,debian-toolchain
container: [alpine,centos9,debian-all-test-cross,debian-amd64-cross,debian-arm64-cross,debian-armhf-cross,debian,debian-hexagon-cross,debian-i686-cross,debian-legacy-test-cross,debian-loongarch-cross,debian-mips64el-cross,debian-mipsel-cross,debian-ppc64el-cross,debian-riscv64-cross,debian-s390x-cross,debian-tricore-cross,debian-xtensa-cross,fedora,fedora-rust-nightly,fedora-win64-cross,opensuse-leap,python,ubuntu2204]
steps:
- uses: actions/checkout@v4
- run: make docker-image-${{matrix.container}} RUNC=podman V=1
- run: podman tag qemu/${{matrix.container}} docker.io/pbolinaro/qemu-ci:${{matrix.container}}
- run: podman login -u pbolinaro -p ${{secrets.DOCKERHUB_PASSWORD}}
- run: podman push docker.io/pbolinaro/qemu-ci:${{matrix.container}}

build_container_debian-precache-tests:
runs-on: ubuntu-24.04
steps:
# we clean up runner first, to get more disk space
- run: docker system prune -af && sudo rm -rf /opt/*
- uses: actions/checkout@v4
- run: make docker-image-debian RUNC=podman V=1
# fill download cache for functional and check-avocado
# running check-avocado without any qemu binary will only download data
# in /root/avocado
- run: >
podman run -it -v $(pwd):$(pwd) -w $(pwd) qemu/debian
./precache_tests.sh
# commit result as a new image. Cache will be in /root/.cache and /root/avocado
- run: podman commit "$(podman ps -aq)" docker.io/pbolinaro/qemu-ci:debian-precache-tests
- run: podman login -u pbolinaro -p ${{secrets.DOCKERHUB_PASSWORD}}
- run: podman push docker.io/pbolinaro/qemu-ci:debian-precache-tests
Loading

0 comments on commit 348dbcf

Please sign in to comment.