apache / apache/arrow

MIGRATION: Handle issues exceeding size limits

Open
#14,648 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

While doing a dry run to test migration tooling, certain issues trigger errors during the import process due to excessive size of contents:

```
error: 413 Client Error: Payload Too Large for url: https://api.github.com/repos/toddfarmer/import_dry_run_2/import/issues
```

An example seems to be [this comment from ASF GitHub Bot](https://issues.apache.org/jira/browse/ARROW-1476?focusedCommentId=16254259&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16254259), which is a massive diff thrown into a comment.

We'll need to assess how to handle this. Some options include:

1. Arbitrarily truncate content exceeding thresholds during migration.
2. Skip importing content beyond threshold and leave a linked message back to the original content.
3. Identify and cleanse content before migration.
4. Filter all ASF GitHub Bot comments before importing (hopefully no human would create comments so large).

Contributor guide

Open the contributing guide

Research direction

Start by locating the migration import path that posts issues and comments to the GitHub API, then reproduce the 413 error with the linked oversized ASF GitHub Bot comment. Define and document one handling policy for content over the API limit, and verify that migration completes without losing a usable reference to the original content.

Written by the indexing model from the issue text.

Assessment

Tech stack
github
Domain
tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.