MIGRATION: Handle issues exceeding size limits
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
While doing a dry run to test migration tooling, certain issues trigger errors during the import process due to excessive size of contents:
```
error: 413 Client Error: Payload Too Large for url: https://api.github.com/repos/toddfarmer/import_dry_run_2/import/issues
```
An example seems to be [this comment from ASF GitHub Bot](https://issues.apache.org/jira/browse/ARROW-1476?focusedCommentId=16254259&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16254259), which is a massive diff thrown into a comment.
We'll need to assess how to handle this. Some options include:
1. Arbitrarily truncate content exceeding thresholds during migration.
2. Skip importing content beyond threshold and leave a linked message back to the original content.
3. Identify and cleanse content before migration.
4. Filter all ASF GitHub Bot comments before importing (hopefully no human would create comments so large).
Contributor guide
Research direction
Start by locating the migration import path that posts issues and comments to the GitHub API, then reproduce the 413 error with the linked oversized ASF GitHub Bot comment. Define and document one handling policy for content over the API limit, and verify that migration completes without losing a usable reference to the original content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100