Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdin io.Copy is too slow #141

Open
BZValoche opened this issue Oct 15, 2023 · 0 comments
Open

stdin io.Copy is too slow #141

BZValoche opened this issue Oct 15, 2023 · 0 comments

Comments

@BZValoche
Copy link

Hi,
My use case : I sent a huge stdin to a powershell in order to transfer a file. I send a base64 stream which is decoded on the fly at destination and outputed to a file.
In RunWithContextWithInput(...), io.Copy(cmd.String, stdin) is just fine as long as everything is inlined properly by the compiler.
But in my case, I want to have some information during the copy, so I use a struct of my own to have some feedback:

type ProgressReader struct {
    io.Reader
    Report func(int, error)
}
func (pr *ProgressReader) Read(p []bye) (n int, err error) {
    n, err := pr.Reader.Read(p)
    if pr.Report != nil {
        pr.Report(n, err)
    }
}

Using this, I get a 3 to 4 performance factor decrease, which is huge. All because the Read function cannot be inlined anymore.
io.Copy(...) used 32K buffers. By replacing the io.Copy(...) by io.CopyBuffer(...) with larger buffer size than 32K, the performance is the same than when not using my ProgressReader.
So my suggestion is to add an optional bufferSize parameter to the RunWithContextInput function, and use io.CopyBuffer is the parameter is provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant