NVIDIA / NVIDIA/cccl

Zip iterators with copy across GPUs gives error

Open
#703 0 comments 0 reactions 0 assignees View on GitHub
thrust
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

I have a struct that contains device_vectors of the same size. I want to be able to use thrust copy with zip iterators for two structs that reside on two different GPUs.

```
namespace T = thrust;
struct Pair_vec
{
unsigned deviceId;
T::device_vector a;
T::device_vector b;

Pair_vec(const unsigned device,
const int length,
const int val1,
const int val2) : deviceId(device)
{
cudaSetDevice(deviceId);
T::device_vector tA(length, val1);
T::device_vector tB(length, val2);
a.swap(tA);
b.swap(tB);
}

T::zip_iterator::iterator,
T::device_vector::iterator>> begin(void)
{
return T::make_zip_iterator(T::make_tuple(
a.begin(), b.begin()));
}

T::zip_iterator::iterator,
T::device_vector::iterator>> end(void)
{
return T::make_zip_iterator(T::make_tuple(
a.end(), b.end()));
}

~Pair_vec(void)
{
cudaSetDevice(deviceId);
}

};
```

It is being used like this
```
unsigned device1 = std::stoi(argv[1]);
unsigned device2 = std::stoi(argv[2]);
const unsigned length = 10240;
Pair_vec v1(device1, length, 10, 11);
Pair_vec v2(device2, length, 20, 21);
```

When I try to copy using the `begin` and `end` functions of the struct, I get an error.
`T::copy(v1.begin(), v1.end(), v2.begin());` gives

```
terminate called after throwing an instance of 'thrust::system::system_error'
what(): cudaFree in free: an illegal memory access was encountered
Aborted
```

This works though
```
T::copy(v1.a.begin(), v1.a.end(), v2.a.begin());
T::copy(v1.b.begin(), v1.b.end(), v2.b.begin());
```

I tried to fix this by using `cudaSetDevice` in the destructor, but that does not work.

1. Is what I am trying to do even possible? How should I go about it?
2. Is it worthwhile to do so from a performance perspective? Copying with zip iters works for a single device. Should I concerned about the performance?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.