Zip iterators with copy across GPUs gives error
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
I have a struct that contains device_vectors of the same size. I want to be able to use thrust copy with zip iterators for two structs that reside on two different GPUs.
```
namespace T = thrust;
struct Pair_vec
{
unsigned deviceId;
T::device_vector a;
T::device_vector b;
Pair_vec(const unsigned device,
const int length,
const int val1,
const int val2) : deviceId(device)
{
cudaSetDevice(deviceId);
T::device_vector tA(length, val1);
T::device_vector tB(length, val2);
a.swap(tA);
b.swap(tB);
}
T::zip_iterator::iterator,
T::device_vector::iterator>> begin(void)
{
return T::make_zip_iterator(T::make_tuple(
a.begin(), b.begin()));
}
T::zip_iterator::iterator,
T::device_vector::iterator>> end(void)
{
return T::make_zip_iterator(T::make_tuple(
a.end(), b.end()));
}
~Pair_vec(void)
{
cudaSetDevice(deviceId);
}
};
```
It is being used like this
```
unsigned device1 = std::stoi(argv[1]);
unsigned device2 = std::stoi(argv[2]);
const unsigned length = 10240;
Pair_vec v1(device1, length, 10, 11);
Pair_vec v2(device2, length, 20, 21);
```
When I try to copy using the `begin` and `end` functions of the struct, I get an error.
`T::copy(v1.begin(), v1.end(), v2.begin());` gives
```
terminate called after throwing an instance of 'thrust::system::system_error'
what(): cudaFree in free: an illegal memory access was encountered
Aborted
```
This works though
```
T::copy(v1.a.begin(), v1.a.end(), v2.a.begin());
T::copy(v1.b.begin(), v1.b.end(), v2.b.begin());
```
I tried to fix this by using `cudaSetDevice` in the destructor, but that does not work.
1. Is what I am trying to do even possible? How should I go about it?
2. Is it worthwhile to do so from a performance perspective? Copying with zip iters works for a single device. Should I concerned about the performance?
Contributor guide
Assessment
This issue has not been assessed yet.