Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose direct buffer send/receive #472

Open
gec75 opened this issue Apr 26, 2023 · 1 comment
Open

Expose direct buffer send/receive #472

gec75 opened this issue Apr 26, 2023 · 1 comment
Labels
Element: Socket Stacks Issues related to OS socket APIs, TIdStack and TIdSocketList descedants, etc Status: Accepted Issue has been accepted to be worked on Type: Enhancement Issue is proposing a new feature/enhancement

Comments

@gec75
Copy link

gec75 commented Apr 26, 2023

I have a special device which has a high data throughput... For this case, I want to bypass all internal buffering (I have my own simple line handling) and use low level socket select/send/receive. But currently Send/Receive only work via TBytes.
I can reach hundreds Mb/s... even though the PC can handle that, I want to avoid intermediate copies.

Currently I use TIdSocketHandle (after connect), but have to hack via TIdStackBSDBase.WSRecv...

Is there a reason NOT to allow direct buffer access?
It would be an easy interface addition with overloaded functions...

indy_ll_buffer.patch

@gec75 gec75 added Status: Reported Issue has been reported for review Type: Enhancement Issue is proposing a new feature/enhancement labels Apr 26, 2023
@rlebeau rlebeau added the Element: Socket Stacks Issues related to OS socket APIs, TIdStack and TIdSocketList descedants, etc label Apr 26, 2023
@rlebeau
Copy link
Member

rlebeau commented Apr 26, 2023

The whole reason that TIdBytes was introduced in Indy 10 in the first place was to support .NET, where using raw pointers is not allowed (or at least not easy/feasible to use). Rather than IFDEF'ing the code all over the place, everything was consolidated on TIdBytes to keep the code clean. Yes, that can cause some performance degradation on non-DotNet platforms.

Fortunately, support for .NET is being dropped in Indy 11, so there will be opportunities for Indy to use/optimize raw buffers again, as it did in Indy 9 and earlier.

Otherwise, one way you can use your own buffers with Indy 10 is to use the TId(ReadOnly)MemoryBufferStream classes with the IOHandler.ReadStream() and IOHandler.Write(TStream) methods. That won't completely eliminate all TIdBytes use (ie, ReadStream() reads from the IOHandler.InputBuffer which uses TIdBytes, and Write() uses a local TIdBytes), but it will reduce the usage.

That being said, if you need high-performance low-latency I/O, and if you are using Windows, then consider using WinSock's Registered I/O extensions, which will allow you to use your own buffers even inside the socket itself. While Indy itself does not use RIO, its IdWinsock2 unit does expose access to the RIO API.

@rlebeau rlebeau added Status: Accepted Issue has been accepted to be worked on and removed Status: Reported Issue has been reported for review labels Apr 26, 2023
@rlebeau rlebeau added this to the Indy 11 - Maintenance Release milestone Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Element: Socket Stacks Issues related to OS socket APIs, TIdStack and TIdSocketList descedants, etc Status: Accepted Issue has been accepted to be worked on Type: Enhancement Issue is proposing a new feature/enhancement
Projects
None yet
Development

No branches or pull requests

2 participants