saltstack / saltstack/salt

[BUG] Unable to connect minion to master - strange timeout

Open
#62,646 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage Transport
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Hi everyone,

I've been dealing with a strange problem for a few weeks now, communication problem between a minion and the master. Communication timeout, while with a similar server it goes through ! Let me explain:

  • My SALT MASTER server is in 192.168.1.100. Debian 11, Salt Master version 3005.
  • My minion1 is on a different network in 192.168.10.0 /24. Debian 10.11, minion in version 3005. Minion1 has no problem to communicate with the Salt master, everything is ok.
  • Minion2 is on the same network as Minion1. Debian 10.12, Salt Minion in 3005 too. And impossible to communicate with the master ...

What I've done so far:

  • the network part is not the problem I think, I did pings/traceroute/nc at all times between the salt master and the minion2 and no worries. Moreover the minion1 is on the same network as the minion2 and has no problem
  • I deleted on the master side the minion that is causing problems, and I deleted everything on the minion2 related to salt (configuration, packages etc). I reinstalled everything and redid the configuration as for the minion1, it does not change anything.
  • Strange things: the minion2 communicates well with the salt master to register, I receive the key on the master. But then nothing, no more communication possible
  • Second strange thing: I have exactly the same behavior if I deploy another server from scratch. Communication timeout

From Minion2 :

$ salt-call pillar.items
Unable to sign_in to master: Attempt to authenticate with the salt master failed with timeout error

From minion2 and /var/log/salt/minion file :

2022-09-09 16:09:25,573 [salt.minion      :1143][ERROR   ][877] Error while bringing up minion for multi-master. Is master at salt.xxx.org responding?

Again from minion2

[DEBUG   ] SaltEvent PUB socket URI: /var/run/salt/minion/minion_event_1a58a185c3_pub.ipc
[DEBUG   ] SaltEvent PULL socket URI: /var/run/salt/minion/minion_event_1a58a185c3_pull.ipc
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] Closing AsyncReqChannel instance
[ERROR   ] Error while bringing up minion for multi-master. Is master at salt.ozerim.fr responding?

From master log, nothing.

I don't know where to turn to correct this :(

If anyone has an idea, I'm interested!

Thanks !

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing salt-call pillar.items on minion2 and reviewing /var/log/salt/minion alongside the master log. Compare minion1 and minion2 configurations and the registration-versus-authentication behavior. Done means the timeout cause is identified and minion2 can communicate with the master reliably.

Written by the indexing model from the issue text.

Assessment

Domain
devops, infrastructure, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.