Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved batch rpc mode #1685

Open
ripienaar opened this issue May 18, 2022 · 0 comments
Open

Improved batch rpc mode #1685

ripienaar opened this issue May 18, 2022 · 0 comments

Comments

@ripienaar
Copy link
Member

ripienaar commented May 18, 2022

Today, and since batching was first added to mcollective, batching basically picks n nodes and complete them as a batch before moving to the next batch. This ends up looking like a jigsaw kind of pattern with some nodes fast, most about the same and some slow - sometimes very slow leading to a long tail where no/few nodes are doing anything.

With go I think we can do a bit better by using a sliding window style batching where at any time we have n machines in flight, soon as we get one reply we unlock 1 new node from being called. This way the batch is always full and the impact of slow nodes is felt in the overall call rather than each batch.

We cant change --batch to work like this so probably we will add a new style of request, something like --window or WindowsRequest() or something like that.

This will look a lot more like the old mco puppet runall in that it always keeps a set of nodes busy (though there we had to make calls inbetween to check they are done, this wont do that)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant