aws-samples / aws-samples/d-SNE
Unable to reproduce published results
- Dominant language
- Python
- Stars
- 67
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Thank you for a great paper and for publishing the code for d-SNE.
I am trying to reproduce the MNIST-USPS results in Table 1 in your paper but so far I have been unable to do so.

To reproduce the results I have done following this:
- [Forked](https://github.com/sheikhomar/d-SNE/tree/mnist-usps) your Git repository.
- Wrote a [script](https://github.com/sheikhomar/d-SNE/blob/mnist-usps/tools/gen_dataset.py) to download MNIST and USPS datasets.
- Attempted to run `./scripts/digits-mt-us-su-s.sh`
- Fixed several runtime errors and ran [`./scripts/digits-mt-us-su-s.sh`](https://github.com/sheikhomar/d-SNE/blob/mnist-usps/scripts/digits-mt-us-su-s.sh) again. Please see the [PR](https://github.com/aws-samples/d-SNE/pull/12) for the changes I made.
- Wrote a [script](https://github.com/sheikhomar/d-SNE/blob/mnist-usps/tools/parse_logs.py) to parse the logs and save the results to CSV files.
In my [analysis](https://github.com/sheikhomar/d-SNE/blob/mnist-usps/results.ipynb), I get the following Eval-Acc-Tgt accuracies when performing domain adaptation from MNIST to USPS:
| | 0 | 1 | 3 | 5 | 7 |
|--------|-----|------|-------|------|------|
| d-SNE | N/A | 41.1 | 38.5 | 40.1 | 43.7 |
What did I do wrong? Can you please provide some guidance?
Contributor guide
Assessment
This issue has not been assessed yet.