[BUG] salt state.apply syncs base modules irrespective of saltenv during salt runs

I have a custom salt state sideboard.py (and a custom sideboard.py module which the custom states make use of) which provides the custom state function sideboard.config.

I'm using gitfs with the following config:

gitfs_disable_saltenv_mapping: false
gitfs_base: master
gitfs_update_interval: 10
gitfs_global_lock: False

gitfs_remotes:
  - file:///srv/salt/salt-states:
     - root: salt/states
     - base: master
     - saltenv:
        - prod:
           - ref: master
        - dev:
           - ref: dev

So I have a dev branch, and I'm able to push changes to that branch and then sync them to my local minion:

# salt-call saltutil.sync_all saltenv=dev
...
local:
    ----------
    beacons:
    clouds:
    engines:
    executors:
    grains:
    log_handlers:
    matchers:
    modules:
        - modules.sideboard
    output:
    proxymodules:
    renderers:
    returners:
    sdb:
    serializers:
    states:
        - states.sideboard
    thorium:
    utils:

So far so good. However, when I run

# salt-call state.apply saltenv=dev
...
[INFO    ] Caching directory '_states/' for environment 'dev'
[INFO    ] Copying '/var/cache/salt/minion/files/dev/_states/sideboard.py' to '/var/cache/salt/minion/extmods/states/sideboard.py'
[INFO    ] Syncing states for environment 'base'
[INFO    ] Loading cache from salt://_states, for base
[INFO    ] Caching directory '_states/' for environment 'base'
[INFO    ] Copying '/var/cache/salt/minion/files/base/_states/sideboard.py' to '/var/cache/salt/minion/extmods/states/sideboard.py'
...

What seems to be happening is that the dev branch is downloaded, and then the base is also downloaded, and overwrites whatever was just downloaded from dev. This matches my observations, that the changes in the dev branch are not used in the run even though I specified saltenv=dev, and when I run

# cat /var/cache/salt/minion/files/dev/_states/sideboard.py

I can see that the master branch version is present rather than the dev branch. There doesn't seem to be a way to specify at the command line or in config to use the saltenv I provide as the version of the custom state. This used to work in prior versions of salt - I first noticed this on 3006.9 though I can't say for sure this is when this behavior started.

Versions Report

# salt-call --versions-report
Salt Version:
          Salt: 3006.9
 
Python Version:
        Python: 3.10.14 (main, Jun 26 2024, 11:44:37) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
  cryptography: 42.0.5
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.17.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: rocky 9.4 Blue Onyx
        locale: utf-8
       machine: x86_64
       release: 5.14.0-427.35.1.c9.x86_64
        system: Linux
       version: Rocky Linux 9.4 Blue Onyx

This seems to be related to https://github.com/saltstack/salt/issues/58362 and https://github.com/saltstack/salt/issues/55437 in that the same underlying issue may to be causing all 3 issues.

Reproduce the behavior with the shown gitfs configuration using salt-call saltutil.sync_all saltenv=dev and salt-call state.apply saltenv=dev; start by tracing the sync logs for the dev and base environments. Done means a dev run does not overwrite its custom state with the base version and uses the requested branch, with the behavior covered by a regression test.