saltstack / saltstack/salt

[BUG] network.routes module causes networking to fail on restart

Open
#66,762 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
network.routes module causes networking systemd service to fail on restart

Setup
On Premises VM salt master running 3007.1 and minion on 3007.1 both running on Debian 12

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (KVM)
  • classic packaging

Steps to Reproduce the behavior
Apply the state from the salt-master to the minion and restart the networking systemd service using systemctl restart networking

Expected behavior
Routes to be added and the service to successfully restart

Versions Report
Salt Version:
Salt: 3007.1

Python Version:
Python: 3.10.14 (main, Apr 3 2024, 21:30:09) [GCC 11.2.0]

Dependency Versions:
cffi: 1.16.0
cherrypy: unknown
dateutil: 2.8.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.4
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 23.1
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.16.0
smmap: Not Installed
timelib: 0.3.0
Tornado: 6.3.3
ZMQ: 4.3.4

Salt Package Information:
Package Type: onedir

System Versions:
dist: debian 12.6 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-21-amd64
system: Linux
version: Debian GNU/Linux 12.6 bookworm

routes:
  network.routes:
    - name: eth0
    - routes:
        - name: user-vpn
           ipaddr: 192.168.0.0
           netmask: 255.255.255.0
           gateway: 192.168.0.1
        - name: site-vpn
           ipaddr: 192.168.1.0
           netmask: 255.255.255.0
           gateway: 192.168.1.1

Additional context
i have noticed that the network.routes module in some cases fails to apply and returns with the following output:

Comment: An exception occurred in this state: Traceback (most recent call last):
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/state.py", line 2428, in call
              ret = self.states[cdata["full"]](
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 160, in call
              ret = self.loader.run(run_func, args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1269, in run
              return self._last_context.run(self._run_as, _func_or_method, *args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1284, in _run_as
              return _func_or_method(args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1317, in wrapper
              return f(*args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/states/network.py", line 717, in routes
              old = salt"ip.get_routes"
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 160, in call
              ret = self.loader.run(run_func, args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1269, in run
              return self._last_context.run(self._run_as, _func_or_method, *args, kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1284, in _run_as
              return _func_or_method(args, **kwargs)
            File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/modules/debian_ip.py", line 1894, in get_routes

but when it does successfully apply what it does is it places a small scrip in /etc/network/if-up.d in our case its routes-eth0 with the following content

#!/bin/sh

#add
test "${IFACE}" = "eth0" || exit 0

#user-vpn
ip route add 192.168.0.0/255.255.255.0 via 192.168.0.1 dev eth0

#site-vpn
ip route add 192.168.1.0/255.255.255.0 via 192.168.0.1 dev eth0

Which causes a strange bug to occur when running a systemctl restart networking as if the route already exists since the interface comes up before the systemd finishes restarting the networking service causes the RTNETLINK File exists to show up causing the systemd service to fail, i have solved this by currently opting out of the network.routes module and placing that file from our master using file managed with the following updates:

#!/bin/sh

#add
test "${IFACE}" = "eth0" || exit 0

#user-vpn
ip route | grep -q "192.168.0.0/255.255.255.0 via 192.168.0.1 dev eth0" || ip route add 192.168.0.0/255.255.255.0 via 192.168.0.1 dev eth0

#site-vpn
ip route | grep -q "192.168.1.0/255.255.255.0 via 192.168.0.1 dev eth0" || ip route add 192.168.1.0/255.255.255.0 via 192.168.0.1 dev eth0

This causes the script to first check if the route is already there and to not assign it if it is allowing the systemd service to properly restart.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/states/network.py around routes and salt/modules/debian_ip.py around get_routes, then reproduce the issue on Debian 12 using the shown state and systemctl restart networking. Compare the generated /etc/network/if-up.d/routes-eth0 script with the reported workaround. Done means routes are applied without duplicating existing entries and the networking service restarts successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, python, shell
Domain
devops, networking, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.