Allure.XUnit.AllureXunitFacade failure: Could not load type 'System.Web.IHttpModule' from assembly System.Web

[pauuser]

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Running an XUnit test project in Gitlab runner docker, I get the following runtime exception:

[xUnit.net 00:00:02.93] Catastrophic failure: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load type 'System.Web.IHttpModule' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Could not load type 'System.Web.IHttpModule' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Could not load type 'System.Web.IHttpModule' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Could not load type 'System.Web.HttpCookie' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Could not load type 'System.Web.HttpCookie' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Could not load type 'System.Web.HttpCookie' from assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Allure.XUnit.AllureXunitFacade.<>c.<GetReporters>b__7_1(Assembly assembly)
   at System.Linq.Enumerable.SelectManyIterator[TSource,TCollection,TResult](IEnumerable`1 source, Func`2 collectionSelector, Func`3 resultSelector)+MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
   at Allure.XUnit.AllureXunitFacade.ResolveAutoReporter()
   at Allure.XUnit.AllureXunitFacade.ResolveSecondReporter()
   at Allure.XUnit.AllureXunitFacade.CreateAllureXunitMessageHandler(IRunnerLogger logger)
   at Allure.XUnit.Reporters.AllureRunnerReporter.CreateMessageHandler(IRunnerLogger logger)
   at Xunit.Runner.VisualStudio.VsTestRunner.RunTests(IRunContext runContext, IFrameworkHandle frameworkHandle, LoggerHelper logger, TestPlatformContext testPlatformContext, RunSettings runSettings, Func`1 getRunInfos) in /_/src/xunit.runner.visualstudio/VsTestRunner.cs:line 442

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

To reproduce this issue I am providing you with my gitlab CI configuration:

stages:
  - build
  - test

###############################################################################################
#                                       Build stage                                           #
###############################################################################################

build-backend:
  image: mcr.microsoft.com/dotnet/sdk:7.0
  stage: build
  script: 
    - dotnet publish Parcorpus/src/Parcorpus.API/Parcorpus.API/Parcorpus.API.csproj -c Release
    - cp -r Parcorpus/src/Parcorpus.API/Parcorpus.API/bin/Release/net7.0/publish g-artefacts_dir/
  artifacts:
    expire_in: 30 mins
    when: on_success
    paths: 
    - g-artefacts_dir

###############################################################################################
#                                       Tests stage                                           #
###############################################################################################

unit-tests:
  image: mcr.microsoft.com/dotnet/sdk:7.0
  needs:
    - build-backend
  stage: test
  script: 
    - dotnet test Parcorpus/test/UnitTests/Parcorpus.UnitTests.Services/Parcorpus.UnitTests.Services.csproj
    - dotnet test Parcorpus/test/UnitTests/Parcorpus.UnitTests.DataAccess/Parcorpus.UnitTests.DataAccess.csproj

integration-tests:
  image: mcr.microsoft.com/dotnet/sdk:7.0
  needs:
    - build-backend
    - unit-tests
  stage: test
  services:
    - name: docker:dind
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script: 
    - dotnet test Parcorpus/test/IntegrationTests/Parcorpus.IntegrationTests/Parcorpus.IntegrationTests.csproj

e2e-tests:
  image: mcr.microsoft.com/dotnet/sdk:7.0
  needs:
    - build-backend
    - unit-tests
    - integration-tests
  stage: test
  services:
    - name: docker:dind
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:
    - dotnet test Parcorpus/test/E2ETests/Parcorpus.E2ETests/Parcorpus.E2ETests.csproj

Here is my csproj file. I am using .NET 7 and XUnit.Allure 2.10.0

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>

        <IsPackable>false</IsPackable>
        <IsTestProject>true</IsTestProject>
        <RootNamespace>Parcorpus.IntegrationTests</RootNamespace>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Allure.XUnit" Version="2.10.0" />
        <PackageReference Include="EntityFramework" Version="6.4.4" />
        <PackageReference Include="Microsoft.AspNetCore.Hosting.Abstractions" Version="2.2.0" />
        <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="7.0.13" />
        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0"/>
        <PackageReference Include="Npgsql" Version="7.0.4" />
        <PackageReference Include="Respawn" Version="6.1.0" />
        <PackageReference Include="Testcontainers.PostgreSql" Version="3.5.0" />
        <PackageReference Include="Testcontainers.RabbitMq" Version="3.5.0" />
        <PackageReference Include="xunit" Version="2.4.2"/>
        <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
        <PackageReference Include="coverlet.collector" Version="3.2.0">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="..\..\..\src\Parcorpus.API\Parcorpus.API\Parcorpus.API.csproj" />
      <ProjectReference Include="..\..\UnitTests\Parcorpus.UnitTests.Common\Parcorpus.UnitTests.Common.csproj" />
    </ItemGroup>
    ...

    <ItemGroup>
        <Content Include="..\..\UnitTests\Parcorpus.UnitTests.Common\AllureConfiguration\allureConfig.json">
            <CopyToOutputDirectory>Always</CopyToOutputDirectory>
        </Content>
    </ItemGroup>

</Project>

What is the expected behavior?

Normally the test project should not fail. If I remove the Allure package, tests pass successfully.

What is the motivation / use case for changing the behavior?

Test project does not work because of the Allure package.

Please tell us about your environment:

• Allure version: 2.10.0
• Test framework: xUnit 2.4.2

Other information

I have an impression that the package uses a .NET framework library which cannot be found and used in .NET Core applications.

[delatrie]

Hi, @pauuser !
Allure.xUnit has a feature that allows it to run an extra xUnit runner reporter automatically. It does so by scanning loaded assemblies looking for IRunnerReporter implementations. While we do some filtering on what assemblies are scanned, apparently we're still scanning too much. In your case it leads to an assembly being loaded that otherwise wouldn't be loaded and that causes the error.

You may disable the feature in allureConfig.json:

• Set xunitRunnerReporter to none if you don't want to run an extra reporter (I assume that's your case)
• Set xunitRunnerReporter to the assembly qualified class name of the reporter you want to use.

That should prevent allure from scanning the assemblies and the error should disappear.

See more info here.

[delatrie]

Allure.Xunit 2.12.0 is more restrictive on what assemblies it considers when scanning for the second reporter. Issues like this one should now appear much less frequently.