tektronix / tektronix/tm_data_types
[BUG]: float 64 conversion not working for raw samples
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 8
- Forks
- 14
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 4
Description
Description of the bug
All data types can be converted from and to, however, due to the nature of float64s close proximity to the highest and lowest bound of values, rounding errors can cause it to fail a conversion
Steps To Reproduce
No response
Environment Information
No response
Additional Information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Investigate the raw-sample conversion path for float64 values near their highest and lowest bounds. The issue provides no reproduction case, environment details, files, tests, or entry points; done should include a reproducible failing case and a verified conversion fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100