Incremental Wrapper ValueError: Layer not in the HighLevelGraph's layers:

**What** happened**: I am getting value errors when I implement an incremental wrapper around my PyTorch model with skorch.

ValueError: Layer ('fit-cb2461b73d6a9abbf8a6eacb8d7c983d', 3) not in the HighLevelGraph's layers: ['original-array-ca0ac32624c3fb9bb23d2ca89c89e186', 'array-ca0ac32624c3fb9bb23d2ca89c89e186', 'transpose-c840e9da8930d4f38c6497326ee6e59d', 139711252509824, 'getitem-4f71595287eec8f0490287973829e6f2', 'reshape-fe8d57675329c0348f3a175d62f073d0']

**What you expected to happen**: I expected my neural network model to begin training with my dataset incrementally

**Minimal Complete Verifiable Example**:

```python

import dask.array as da
import torch.nn as nn
from skorch import NeuralNetRegressor
import torch.optim as optim
from dask_ml.wrappers import Incremental
from dask_ml.datasets import make_regression

class GRU(nn.Module):
def __init__(self, inputsize, outputsize):
super(GRU, self).__init__()
self.inputsize = inputsize
self.outputsize = outputsize
self.hiddenlayers = nn.GRU(self.inputsize, self.inputsize, num_layers=2, batch_first=True)
self.outputlayer = nn.Linear(self.inputsize, self.outputsize)

def forward(self, x):
output, hidden = self.hiddenlayers(x)
x = hidden[-1]
x = self.outputlayer(x)
return x

niceties = {
"callbacks": False,
"warm_start": False,
"train_split": None,
"max_epochs": 1,
}

model = NeuralNetRegressor(
module=GRU,
module__inputsize=10,
module__outputsize=1,
criterion=nn.L1Loss(),
optimizer=optim.SGD,
optimizer__lr=0.01,
optimizer__momentum=0.9,
batch_size=100,
**niceties
)

inc = Incremental(model, scoring="r2")
# Trains the model incrementally using chunked data

X, y = make_regression(n_samples=10000, n_features=10, n_targets=1, chunks=100)
# Creates data for github example

y = X[:, 1].reshape(-1, 1)
# reshapes the input for sequential model
X = da.stack([X, X, X], axis=1)

print(X.shape)
print(y.shape)

inc.fit(X, y)
print(inc.score(X, y))
```

**Anything else we need to know?**:

**Environment**: Anaconda 3

- Dask version: 1.7.0
- Python version: 3.9.7
- Operating System: Pop OS 21.10
- Install method (conda, pip, source): conda