-
-
Notifications
You must be signed in to change notification settings - Fork 978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to filter benchmarks with TestAdapter #2662
Comments
Is that filter behavior the same without the TestAdapter? |
Without TestAdapter I also tried to filter with |
I meant if you run it the old way with |
It doesn't, because it's not the right syntax for filtering benchmarks. I can use |
May be i'm misunderstanding the purpose of TestAdapter package? My thoughts were that it allows integration with vstest and exposes proper hooks to filter / configure the test run (which i also saw in code linked in the original post, like providing category traits, etc). If that's the case - i don't see it being respected currently. It's as if |
My guess is because |
I think it gets parsed prior to BenchmarkDotNet logic, since i can't use regular BDN commands with Like |
@timcassell Hi, can you help me to find out how to debug test adapter? Because I tried several ways, but none did work |
@newmasterSG Maybe this will help? https://benchmarkdotnet.org/articles/guides/troubleshooting.html Also, I think the test adapter automatically hides benchmarks in Debug mode, so you may need to dig into the test adapter code and disable that. |
@timcassell Also I have a question about is test adapter is activated by dotnet test? |
I asked because I had these test but I couldn't see them in test explorer
} |
Maybe @AndreyAkinshin also can help |
It looks like if you apply |
Steps to reproduce
TestAdapter
package +Test.Sdk
)[BenchmarkCategory("MyTest1")]
and[BenchmarkCategory("MyTest2")]
attributesdotnet test -c Release --filter "Category=MyTest1"
Expected: 1 benchmark executed (category attribute is being added as a trait )
Actual: both benchmarks executed
Also [unrelated] - are these benchmarks supposed to show up in Test Explorer in VS? I'd assume they would, since
dotnet test
picks them up0.14.0 version of TestAdapter is used
The text was updated successfully, but these errors were encountered: