[BUG]: WFM-to-CSV conversion fails when axis-unit fields contain padding after NUL
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in src/tm_data_types/helpers/byte_data_types.py at String.str(), then run the provided WFM-to-CSV reproduction with the String20 values. Confirm that bytes after the first NUL are ignored, write_file() completes, and the resulting axis units are s and V.
Written by the indexing model from the issue text.
Description
Description of the bug
A WFM file produced by a Tektronix DPO71254C contains fixed-width axis-unit
fields with a valid null-terminated value followed by nonzero padding bytes.
For example, the meaningful horizontal unit is s\0, but the remaining bytes
in the 20-byte field are not necessarily valid UTF-8.
read_file() succeeds, but converting the resulting waveform to CSV with
write_file() raises UnicodeDecodeError. String.__str__() currently decodes
the entire fixed-width field before removing trailing NUL characters:
return self.decode("utf_8").rstrip("\x00")
Because decoding happens before interpreting the NUL terminator, invalid bytes
after the terminator cause the conversion to fail. This behavior is visible in
the
String.__str__() v0.4.1 implementation.
Expected behavior:
- Bytes following the first NUL terminator are not decoded as part of the unit
string. - The WFM converts successfully.
- The resulting units are
sandV.
Actual behavior:
- CSV serialization raises
UnicodeDecodeError.
Steps To Reproduce
When reading a .wfm file generated by a DPO71254C, read_file() returns the following axis-unit metadata. Passing that waveform to write_file() causes the CSV serialization error:
from pathlib import Path
from tempfile import TemporaryDirectory
import numpy as np
from tm_data_types import AnalogWaveform, write_file
from tm_data_types.helpers.byte_data_types import String20
with TemporaryDirectory() as directory:
waveform = AnalogWaveform()
waveform.trigger_index = 0
waveform.source_name = "CH4"
waveform.y_axis_values = np.array([1], dtype=np.int16)
waveform.x_axis_units = String20(
b"s\x00\x00\x00Vp\x8e\x12\xf6\x7f\x00\x00\x00\x00\x00\x00\xba\x02\x00\x00"
)
waveform.y_axis_units = String20(
b"V\x00\x00\x00\xf0u~\xe3\xba\x02\x00\x00\xf0u~\xe3\xba\x02\x00\x00"
)
write_file(str(Path(directory) / "output.csv"), waveform)
Result:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8e in position 6: invalid start byte
Environment Information
OS: Windows 11, 64-bit
Python: 3.12.13
tm_data_types: 0.4.1
NumPy: 1.26.4
Instrument: Tektronix DPO71254C
Instrument firmware: 10.11.1 Build 30
Additional Information
No response
- Dominant language
- Python
- Stars
- 8
- Forks
- 14
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from tektronix/tm_data_types
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
tektronix/tm_data_types#212 ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
tektronix/tm_data_types#127 ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
tektronix/tm_data_types#86 ·
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 42/100
tektronix/tm_data_types#85 · 1 reaction ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 25/100
tektronix/tm_data_types#43 ·
All issues in tektronix/tm_data_types
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100