Color Widget Borks My dataset

- [ ] **What's wrong?**

So I have a workflow with a bunch of spectra going through it and I set it up so that it would colour them nice and pretty, but I wanted to see how a previous data set looked in the same format. Suddenly all of the datasets were the exact same numeric values.

If I remove the color widget I can replot things just fine with either the old or new data. But then I don't have the plot colours I wanted.

- [ ] **How can we reproduce the problem?**

[bug.zip](https://github.com/biolab/orange3/files/6642848/bug.zip)
Just load the data in and i set it up to show how the colour widget causes this duplication both graphically in some line plots but also in data tables. If you break the connection between the multi-file widget and the rest and then remake it, the color widget will break the data.

- [ ] **What's your environment?**

- Operating system: Windows 10
- Orange version: 3.30.0.dev
- How you installed Orange: Command line

Start by loading bug.zip in Orange 3.30.0.dev and reproducing the workflow with the Color Widget and multi-file connection. Compare the data before and after reconnecting the workflow, including the graphical plots and data tables. Done means datasets retain distinct numeric values while the requested plot colors remain available.

Written by the indexing model from the issue text.