read_many_sample cannot use views
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 592
- Forks
- 199
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 10
Description
This is an improvement suggestion.
The read_many_sample methods of stream_reader classes have odd requirements for the data array. The array shape must be (n_channels, n_samples) and must be C_CONTIGUOUS. In the language of nidaq-mx, the data array must be 'non-interleaved'. This causes extra difficulties when using continuous acquisition mode. In this case, the exact number of samples may not be always known. As an example, this prevents the use of array views as an argument to read_many_sample. Let's review the root of this problem, show some example code, and introduce some possible solutions.
Numpy arrays can be either C_CONTIGUOUS (or row-major, C-order) or F_CONTIGUOUS (or column-major). This controls the layout in memory for multidimensional arrays. For 2D arrays, C_CONTIGUOUS arrays store the elements one row after the other. Ie: arr[i,j] is next to arr[i, j+1] in memory. See: https://docs.scipy.org/doc/numpy/glossary.html#term-row-major. This means that changing the number of rows can be done without reorganizing the entire array. Furthermore, a slice containing all rows but not all columns is not C_CONTIGUOUS, whereas a slice containing all columns but some rows is not. Since nidaqmx maps rows onto channels and samples onto columns, a slice onto some samples is not C_CONTIGUOUS, and cannot be used as an argument for stream_reader.AnalogMultiChannelReader.read_many_samples. This also means the transfer from the buffer to the numpy array is not in native order, thus not so efficient.
import numpy as np
n_chan, n_samp = 4, 100
c = np.zeros((n_chan, n_samp), order="C") # default order
assert c.flags.c_contiguous
c_cols = c[:,10:15]
assert not c_cols.flags.c_contiguous
As an example demonstration, let's perform continuous acquisition. This example requires a temporary buffer for each call. This exact example can be achieved using finite acquisition, but the pattern can be useful for other applications:
import numpy as np
import nidaqmx as ni
from nidaqmx.constants import VoltageUnits, AcquisitionType, READ_ALL_AVAILABLE
from nidaqmx.stream_readers import AnalogMultiChannelReader
from time import sleep
##### SETUP
n_tot = 100000
sample_rate = 200000
with ni.Task("signals") as task:
task.ai_channels.add_ai_voltage_chan(
"DevT/ai0:1",
min_val=-10, max_val=10,
)
n_channels = task.number_of_channels
task.timing.cfg_samp_clk_timing(
rate=sample_rate,
sample_mode=AcquisitionType.CONTINUOUS,
samps_per_chan=n_tot,
)
reader = AnalogMultiChannelReader(task.in_stream)
read_buffer = np.ones((n_channels, n_tot))*-1000 # impossible output
i = 0
##### START
task.start()
while not task.is_task_done() and i < n_tot:
sleep(0.01) # pretend to be busy with other tasks
n = reader._in_stream.avail_samp_per_chan
if n == 0: continue
n = min(n, n_tot-i) # prevent reading too many samples
##### READ
tmp = np.ones((n_channels, n)) * -1001
r = reader.read_many_sample(
tmp,
number_of_samples_per_channel=n
)
read_buffer[:,i:i+n] = tmp
i += r
##### STOP AND CHECK RESULTS
task.stop()
assert np.all(read_buffer > -1000)
If views could be used the intervening tmp array can be dropped. The code could then be:
# [setup as previously]
##### READ
i += reader.read_many_sample(
read_buffer[:, i:i+n], # read directly into array using a view
number_of_samples_per_channel=n
)
# [stop and check results as previously]
Enabling this compatibility would simplify client code (it would "just work") and should improve interoperability with numpy-based code, such as memory-mapped arrays, hdf5py, etc.
Here are a few possible paths for a solution, all of which have potential drawbacks:
- Change array shape to
(n_samples, n_channels); - Change requirement for numpy arrays to F-order`;
- Enable choosing between F-order and C-order. This can (maybe?) be delegated to the reader object. The underlying
DAQmxReadAnalogF64seem to support both cases through thefill_modeargument.
The tests were performed using the following configuration:
python=3.8.2
numpy=1.18.1
nidaqmx-python=0.5.7
NI-DAQmx=19.6
on windows 10. A simulated device (USB-6356) was created in NI-MAX, with name DevT.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start with the stream_reader read_many_sample methods and the DAQmxReadAnalogF64 fill_mode behavior described here; reproduce the NumPy slice and continuous-acquisition examples, then define done as a selected view-compatible approach with documented examples and trade-offs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- api, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100