meta-pytorch / meta-pytorch/data
[Linter] Ability to disable some lints
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
There are several options to disable specific linters.
Option 1. Disable with linter-ignore: code
Pros:
- Similar to known syntax of various linters
Cons:
- Need to modify code of datasets to disable something
datapipe = datapipe.sharding_filter().shuffle() # linter-ignore: shuffle-shard
Option 2. Global & Context disables
Pros:
- Can control datasets without modification of the code
Cons:
- Global might disable important errors
- Context requires additional indent
- Syntax feels weird
- Annoying to disable construct time linters (see below)
from torchdata import linter
linter.disable('shuffle-shard') # global
with linter.disable('shuffle-shard'): # context based
dl = DataLoader2(...)
Option 3. DLv2 argument / ReadingService argument
Pros:
- Local to specific DataLoader
- Can control datasets without modication of the code
Cons:
- Syntax feels weird
- Some linters might trigger/not in various ReadingServices
- Annoying to disable construct time linters (see below)
dl = DataLoader2(dp_graph, [adapter], disable_lint = ['shuffle-shard'])
Option 4. DataPipe 'attribute'
Pros:
- Can be defined by DataSet developer or by the user
- Can impact construct time error handling
Cons:
- Syntax feels weird
datapipe = datapipe.sharding_filter().shuffle().disable_lint('shuffle-shard')
and/or (as we can have an adapter to do the same job)
dl = DataLoader(dp_graph,[DisableLint('shuffle-shard')], ...)
Personally, I prefer the last variant, but I'm open to discussion.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by tracing the linter entry points and how DataPipe, DataLoader2, and ReadingService options are handled; done means agreeing on one disable mechanism and implementing it with coverage for the relevant linting paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100