Skip to content
Snippets Groups Projects
Verified Commit 9e3ecd39 authored by lelutin's avatar lelutin
Browse files

cross-ref cumin to direct-ssh setup and expand on effects of using batch

Using cumin's batch size is still a possibility to avoid issues, but it
is preferred to configure yourself for direct ssh connections and avoid
using the batch size if not necessary.

if direct-ssh connection is not possible, then using the batch size hack
is still possible. using it does have some side-effects that one should
be aware of though.

small correction in the text after my tests today: the limitation is
imposed by the MaxStartups setting, not MaxSessions.
parent e5bf6970
No related branches found
No related tags found
No related merge requests found
Pipeline #205599 passed with warnings
......@@ -165,16 +165,32 @@ change the path to cumin in the bash function.
## Avoiding spurious connection errors by limiting batch size
If your ssh configuration makes all hosts use the same `ProxyJump`, you might be
seeing some strange and fluctuating percentage of hosts will error out on
connection. This is because you are hitting your ssh proxy's max connections per
IP. This could be caused by the server's `MaxSessions` which defaults to 10.
In order to avoid weird issues, you should limit your batch size to 10 or lower,
e.g.:
If you use cumin to run ad-hoc commands on many hosts at once, you'll most
probably want to look into setting yourself up for [direct
connection](doc/ssh-jump-host#when-not-to-use-the-jump-host) to the hosts,
instead of passing through a jump host.
Without the above-mentioned setup, you'll quickly hit a problem where hosts give
you seemingly random ssh connection errors for a variable percentage of the host
list. This is because you are hitting ssh server limitations imposed on you on
the jump host. The ssh server uses the default value for its `MaxStartups`
option, which means once you have 10 simultaneous open connections you'll start
seeing connections dropped with a 30% chance.
Again, it's recommended in this case to set yourself up for direct ssh
connection to all of the hosts. But if you are not in a position where this is
possible and you still need to go through the jump host, you can avoid weird
issues by limiting your batch size to 10 or lower, e.g.:
cumin -b 10 'F:os.distro.codename=bookworm' 'apt update'
Note however that doing this will have the following effects:
* execution of the command on all hosts will be much slower
* if some hosts see command failures, cumin will stop processing your requested
commands after reaching the batch size. so your command will possibly only run
on 10 of all of the hosts.
## Example commands
This will run the `uptime` command on all hosts:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment