meta-pytorch / meta-pytorch/data

[Linter] Ability to disable some lints

Open
#580 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.3k
Forks
179
Avg merge
6d 1h
Merged PRs (30d)
2

Description

🚀 The feature

There are several options to disable specific linters.

Option 1. Disable with linter-ignore: code

Pros:

  • Similar to known syntax of various linters

Cons:

  • Need to modify code of datasets to disable something
datapipe = datapipe.sharding_filter().shuffle()  # linter-ignore: shuffle-shard

Option 2. Global & Context disables

Pros:

  • Can control datasets without modification of the code

Cons:

  • Global might disable important errors
  • Context requires additional indent
  • Syntax feels weird
  • Annoying to disable construct time linters (see below)
from torchdata import linter
linter.disable('shuffle-shard') # global
with linter.disable('shuffle-shard'): # context based
    dl = DataLoader2(...)

Option 3. DLv2 argument / ReadingService argument

Pros:

  • Local to specific DataLoader
  • Can control datasets without modication of the code

Cons:

  • Syntax feels weird
  • Some linters might trigger/not in various ReadingServices
  • Annoying to disable construct time linters (see below)
dl = DataLoader2(dp_graph, [adapter], disable_lint = ['shuffle-shard'])

Option 4. DataPipe 'attribute'

Pros:

  • Can be defined by DataSet developer or by the user
  • Can impact construct time error handling

Cons:

  • Syntax feels weird

datapipe = datapipe.sharding_filter().shuffle().disable_lint('shuffle-shard')

and/or (as we can have an adapter to do the same job)

dl = DataLoader(dp_graph,[DisableLint('shuffle-shard')], ...)

Personally, I prefer the last variant, but I'm open to discussion.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by tracing the linter entry points and how DataPipe, DataLoader2, and ReadingService options are handled; done means agreeing on one disable mechanism and implementing it with coverage for the relevant linting paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.