TestResult instance is not properly produced using Junit5 inside the container

[TheOnlyAl]

Issue Overview

I believe there is a problem when running Junit5 tests in the container because the TestResult instance is not set before the After phase is executed. In my opionion this leads for example to the following issue:
arquillian/arquillian-extension-transaction#31

Junit4 seems to corretly inject the result. Propably by using UpdateTestResultBeforeAfter.java

Expected Behaviour

The TestResult should be set before the After phase is executed for Junit5 tests.

Current Behaviour

The TestResult is not set.

Steps To Reproduce

I created a small project (arquillian-junit5-container-testresultfix) which uses an custom Extension to check if the TestResult is set and also provides an Extension which provides a workaround for the problem.

To check if the result is not set, you could:

1. Check out the project
2. Run the Tests

The tests will all succeed as long the the result is not set. When this is fixed at least the UnfixedTestResultIT.java should fail

Additional Information

I did a lot of debugging for this problem since it basically makes most of my tests using transactions fail. It is quite hard to really understand where the problem is. It could be a missing interceptor, missing observer, or something else.

Since there is also seems currently no other easys way for extensions to get the Result of a test after its run but before the After phase is called, i am not sure how else extensions should work.

Checked for: Arquillian 1.8.0.Final

Please just tell me if you need any more information. I am happy to try to help as much as possible.

[starksm64]

If I run the tests on the https://github.com/TheOnlyAl/arquillian-junit5-container-testresultfix.git project, they just hang with messages about java.lang.IllegalStateException: Unknown protocol: RFB

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running de.adtelligence.arquillian.junit5.container.testresultfix.tests.embedded.TestResultIT
Feb 20, 2024 6:15:12 AM com.sun.enterprise.v3.server.CommonClassLoaderServiceImpl findH2Client
INFO: Cannot find h2db client jar file, h2 jdbc driver will not be available by default.
Feb 20, 2024 6:15:12 AM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator null
Feb 20, 2024 6:15:13 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [development] [4.2.5] Using bind address: 0.0.0.0/0.0.0.0:4900
Feb 20, 2024 6:15:13 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [development] [4.2.5] Using public address: /fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:4900
Feb 20, 2024 6:15:13 AM com.hazelcast.system
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Hazelcast 4.2.5 (20220427 - 1dbec0d) starting at [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900
Feb 20, 2024 6:15:14 AM com.hazelcast.instance.impl.Node
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Using Discovery SPI
Feb 20, 2024 6:15:14 AM com.hazelcast.internal.diagnostics.Diagnostics
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
Feb 20, 2024 6:15:14 AM com.hazelcast.core.LifecycleService
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 is STARTING
Feb 20, 2024 6:15:14 AM com.hazelcast.internal.server.tcp.TcpServerConnection
WARNING: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Connection[id=1, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56827->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=1, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56827->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.compassionate_bell.IO.thread-in-0
java.lang.IllegalStateException: Unknown protocol: RFB
	at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:117)
	at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:138)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368)
	at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294)
	at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)

Feb 20, 2024 6:15:14 AM com.hazelcast.internal.server.tcp.TcpServerConnection
WARNING: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Connection[id=2, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56828->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=2, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56828->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.compassionate_bell.IO.thread-in-1
java.lang.IllegalStateException: Unknown protocol: RFB
	at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:117)
	at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:138)
	at com.hazelcast.internal.networking.nio.NioPipeline.lambda$start$0(NioPipeline.java:127)
	at com.hazelcast.internal.networking.nio.NioThread.processTaskQueue(NioThread.java:355)
	at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:290)
	at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)

[TheOnlyAl]

Very strange. Seems that this could be a problem with an already mapped port of 5900 and hazelcast for the embedded payara . If you can check if this port is already mapped and disable it, it might work, otherwise i could try to check if i can find out how to disable it.

[TheOnlyAl]

Hi @starksm64 i pushed an update where i changed the default hazelcast port to 15900. Seems there is a bug with some screen sharing software which reserves the port:
payara/Payara#3995

[starksm64]

Ok, got it

[uvula]

Sorry, can I ask a question:

I'm just a arquillian user in my project and not so deep in the internals.

Can you please have a look to the following problem:
https://stackoverflow.com/questions/78016093/how-can-i-control-transaction-handling-in-arquillian-using-junit5

Is this maybe related, so I can't control the transaction handling with junit5.
Thanks

[TheOnlyAl]

Hi @uvula.
I think your problem is not related to this problem since you do not seem to use https://github.com/arquillian/arquillian-extension-transaction/ as a dependency. If you would, your other tests would probably also fail since all transaction would roll back with this problem.

[uvula]

Hello @TheOnlyAl,
I was also dealing with the dependencies:

Gradle:

	testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-bom:1.0.6'
	testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-api:1.0.6'
	testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-jta:1.0.6'

arquillian.xml:

    <extension qualifier="transaction">
    <property name="manager">testManagerName</property>
    <property name="transactionDefaultMode">DISABLED</property>
  </extension>

and/or annotations
@transactional(value = TransactionMode.DISABLED)
at class/method level. I don't have a running example with Junit4 (yet),
so I can't check: DO I HAVE control with TransactionMode.*, somehow for my problem.
The idea is to use DISABLED (see below).

My Junit tests are NOT failing. But I see, after calling a method of the stateless
bean (without @javax.transaction.Transactional), asking entityManager em.isJoinedToTransaction() -> true.
My understanding: subsequent method calls WITH @Transactional will join.
So Envers can't see changes in between, just the final state after commit (wrapped by Arquillian/JUnit5)

Any hints?
We also can use german.