Calling FabricClient methods has bizarre effect on EventSource constructor in .NET 6 or newer

[andradf]

Describe the bug
Calling any FabricClient method leads to programs freezing for about 4 seconds when creating an EventSource. This bug has a high impact because internally HttpClient calls its own EventSource and these freezes can cause the first request to take 4 more seconds than it should.

Area/Component:
FabricClient

To Reproduce
Execute the following code in a .NET 6 or newer console app. I haven't tested it on older versions of .NET core. It works correctly in .NET Framework.

Stopwatch sw = new Stopwatch();
var fc = new FabricClient();
var apps = await fc.QueryManager.GetApplicationListAsync();
Console.WriteLine("Creating Event Source");
sw.Start();
var es = new MyEventSource();
sw.Stop();
Console.WriteLine($"Created Event Source {sw.ElapsedMilliseconds}ms");

MyEventSource is an empty class that inherits from EventSource.

Expected behavior
The code should print an insignificant delay when measuring the creation of an EventSource.

Observed behavior:
It actually takes about 4 seconds or more. If the await fc.QueryManager... line is commented, the EventSource constructor returns immediately as expected. This behavior only occurs once. Any subsequent calls to the constructor do return immediately. Also, if we await a Task.Delay(5000) before calling the constructor, the constructor also returns immediately.

Service Fabric Runtime Version:
10.0 (SDK) / 7,0 libs

Environment:

• Dev cluster
• Windows 11
• SF Version 7.0.1816
• .NET 6 or greater

Potentially helpful infomation
I also ran a procmon capture to try to see what the program is doing during those 4 seconds and I see zero activity and a Thread Exit/Thread Create just when the program unfreezes.

Also if we pause the program in VS and examine the current threads, one can see that there is a Blocked thread at the FabricEventSource constructor.

---

Assignees: /cc @microsoft/service-fabric-triage