[BUG]: WFM-to-CSV conversion fails when axis-unit fields contain padding after NUL

Open Beginner friendly
#211 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python
Domain
data

Research direction

Start in src/tm_data_types/helpers/byte_data_types.py at String.str(), then run the provided WFM-to-CSV reproduction with the String20 values. Confirm that bytes after the first NUL are ignored, write_file() completes, and the resulting axis units are s and V.

Written by the indexing model from the issue text.

Description

bug
Description of the bug

A WFM file produced by a Tektronix DPO71254C contains fixed-width axis-unit
fields with a valid null-terminated value followed by nonzero padding bytes.
For example, the meaningful horizontal unit is s\0, but the remaining bytes
in the 20-byte field are not necessarily valid UTF-8.

read_file() succeeds, but converting the resulting waveform to CSV with
write_file() raises UnicodeDecodeError. String.__str__() currently decodes
the entire fixed-width field before removing trailing NUL characters:

return self.decode("utf_8").rstrip("\x00")

Because decoding happens before interpreting the NUL terminator, invalid bytes
after the terminator cause the conversion to fail. This behavior is visible in
the
String.__str__() v0.4.1 implementation.

Expected behavior:

  • Bytes following the first NUL terminator are not decoded as part of the unit
    string.
  • The WFM converts successfully.
  • The resulting units are s and V.

Actual behavior:

  • CSV serialization raises UnicodeDecodeError.
Steps To Reproduce

When reading a .wfm file generated by a DPO71254C, read_file() returns the following axis-unit metadata. Passing that waveform to write_file() causes the CSV serialization error:

from pathlib import Path
from tempfile import TemporaryDirectory

import numpy as np
from tm_data_types import AnalogWaveform, write_file
from tm_data_types.helpers.byte_data_types import String20

with TemporaryDirectory() as directory:
    waveform = AnalogWaveform()
    waveform.trigger_index = 0
    waveform.source_name = "CH4"
    waveform.y_axis_values = np.array([1], dtype=np.int16)

    waveform.x_axis_units = String20(
        b"s\x00\x00\x00Vp\x8e\x12\xf6\x7f\x00\x00\x00\x00\x00\x00\xba\x02\x00\x00"
    )
    waveform.y_axis_units = String20(
        b"V\x00\x00\x00\xf0u~\xe3\xba\x02\x00\x00\xf0u~\xe3\xba\x02\x00\x00"
    )

    write_file(str(Path(directory) / "output.csv"), waveform)

Result:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8e in position 6: invalid start byte
Environment Information
OS: Windows 11, 64-bit
Python: 3.12.13
tm_data_types: 0.4.1
NumPy: 1.26.4

Instrument: Tektronix DPO71254C
Instrument firmware: 10.11.1 Build 30
Additional Information

No response

Dominant language
Python
Stars
8
Forks
14
Avg merge
3d 10h
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tektronix/tm_data_types

All issues in tektronix/tm_data_types

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.