saltstack / saltstack/salt

Data Transport is broken

Open
#56,166 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Core severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue
[WARNING ][2649] Data transport is broken, got: , type: str, exception: str indices must be integers, not str, attempt 1 of 3
[WARNING ][2649] Data transport is broken, got: , type: str, exception: str indices must be integers, not str, attempt 2 of 3
[WARNING ][2649] Data transport is broken, got: , type: str, exception: str indices must be integers, not str, attempt 3 of 3
[WARNING ][2649] Data transport is broken, got: , type: str, exception: str indices must be integers, not str, attempt 4 of 3
[ERROR   ][2649] Data transport is broken, got: , type: str, exception: str indices must be integers, not str, retry attempts exhausted

Also see https://github.com/saltstack/salt/issues/48265

This is only seen from the minion side. Issuing the command to run the state from the master, returns

minion_id:
----------
          ID: /etc/filename.conf
    Function: file.managed
      Result: False
     Comment: Source file 'salt://repo/files/filename.conf' not found
     Started: 22:19:26.668777
    Duration: 135.711 ms
     Changes:   
Setup

I don't know if this matters but the master is running gitfs for its fileserver roots config.
The file to transfer should not be in the minion's cache.

Steps to Reproduce Issue

Try transferring using a file.managed on a file that has this line:
# This should be at least (number_of_channels × 2) to be able to keep the live
If the file is already in the minion's cache there isn't an issue.

Versions Report

No differences except master has some pkgs minion doesn't

minion:
    Salt Version:
               Salt: 2019.2.3
     
    Dependency Versions:
               cffi: Not Installed
           cherrypy: Not Installed
           dateutil: 2.4.2
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
              ioflo: Not Installed
             Jinja2: 2.8
            libgit2: Not Installed
            libnacl: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.4.6
       mysql-python: Not Installed
          pycparser: Not Installed
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.5.2 (default, Jul  5 2016, 12:43:10)
       python-gnupg: 0.3.8
             PyYAML: 3.11
              PyZMQ: 15.2.0
               RAET: Not Installed
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.2.1
                ZMQ: 4.1.4
     
    System Versions:
               dist: Ubuntu 16.04 xenial
             locale: ANSI_X3.4-1968
            machine: x86_64
            release: 4.4.0-34-generic
             system: Linux
            version: Ubuntu 16.04 xenial
master:
    Salt Version:
               Salt: 2019.2.3
     
    Dependency Versions:
               cffi: 1.13.2
           cherrypy: 8.7.0
           dateutil: 2.4.2
          docker-py: Not Installed
              gitdb: 0.6.4
          gitpython: 1.0.1
              ioflo: Not Installed
             Jinja2: 2.8
            libgit2: Not Installed
            libnacl: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.4.6
       mysql-python: 1.3.7
          pycparser: 2.19
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.5.2 (default, Oct  8 2019, 13:06:37)
       python-gnupg: 0.3.8
             PyYAML: 3.11
              PyZMQ: 15.2.0
               RAET: Not Installed
              smmap: 0.9.0
            timelib: Not Installed
            Tornado: 4.2.1
                ZMQ: 4.1.4
     
    System Versions:
               dist: Ubuntu 16.04 xenial
             locale: ANSI_X3.4-1968
            machine: x86_64
            release: 4.4.0-34-generic
             system: Linux
            version: Ubuntu 16.04 xenial

I was able to fix the issue by removing the unicode character '×' and replacing it with an ascii 'x'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the minion-side file.managed transfer with a non-cached file containing the Unicode × character, then compare it with the ASCII version and a cached transfer. Trace the data-transport path from the file.managed entry point and use the reported retry output to locate the failure. Done means the Unicode-containing file transfers successfully without exhausted retries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.