Fix display of waived failures: consider subject and scenario (#5397) (backport #5480) #5499
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
On the Automated Tests tab, the web UI attempts to indicate when a failed test has been waived. However, it can get this wrong in several ways. It stores the waivers only by testcase - not by subject or scenario - so if greenwave returns any waiver for the testcase, it will mark any failure of that testcase for the update as 'waived'. This can be incorrect if: * The waiver was filed against the wrong subject and is invalid * The waiver is for a different subject (different Koji build) * The waiver is for a different scenario This should fix all those problems. Instead of just parsing the waivers, we primarily parse the satisfied requirements, and when one is of type "test-result-failed-waived" - meaning it's a failure that was waived - we store it in a nested hash of waived failures. The outer hash's keys are subject identifiers, and its values are hashes. In these inner hashes, the keys are the testcase and scenario combined, and the values are the IDs of waivers for that subject/testcase/scenario combination. We keep a separate hash of waivers, the keys being the waiver IDs and the values being the waivers. So when we are constructing the result rows, we can check whether there is a waived failure for the result's testcase, scenario and subject, and only if so, we add the 'waived' marker, constructing the necessary info from the waiver retrieved from the waivers hash. Signed-off-by: Adam Williamson <[email protected]> (cherry picked from commit 8ae22c4)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an automatic backport of pull request #5480 done by Mergify.
Mergify commands and options
More conditions and actions can be found in the documentation.
You can also trigger Mergify actions by commenting on this pull request:
@Mergifyio refreshwill re-evaluate the rules@Mergifyio rebasewill rebase this PR on its base branch@Mergifyio updatewill merge the base branch into this PR@Mergifyio backport <destination>will backport this PR on<destination>branchAdditionally, on Mergify dashboard you can:
Finally, you can contact us on https://mergify.com