pytorch / pytorch/vision

LSUN caching is ugly

Open
#825 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

The cache for the LSUN dataset (136MB just for bedrooms) gets dumped into the same directory as the script you're running, with a filename that's a function of the path to the dataset itself. Generating the cache can take several hours, eg on a standard EC2 server with EBS disk. If you want to run a script in a different folder, you have to regenerate the cache (hours) or copy the cache file to the new folder. If you move the dataset to a different folder, you either regenerate the cache or try to modify the cache filename to match the new path. No matter what you have cache files side by side with your code that need special gitignore directives (they don't have an extension to use wildcards on).

It should be possible to put the cache file in the data directory or something to fix these issues.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue identifies the LSUN dataset cache as the entry point; locate its cache-generation code and any related tests. Verify that cache placement is independent of the script's working directory and dataset path, and that existing cache data can be reused without regeneration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.