Extract Merge Requests that will close an issue

In GitLab by @iroussos on Nov 12, 2019, 07:13

We currently export both the Issues and the Merge Requests for a Project.

We are not extracting the relation between Issues and Merge Requests that will close it.

That relation is important because it ties MRs to Issues and is also useful for tracking Cycle Analytics: From the moment an issue is created, to planning the issue, working on an issue in an MR, reviewing the MR and finally merging the MR and closing the issue.

This relation can be fetched in two ways:

1. API call to list merge requests that will close issue on merge
2. API call to List issues that will close on merge of an MR

We can use either one, but we have to be careful:

Similarly to fetching the Commits for a Merge Request, this operation requires one API call per Merge Request (or Issue), so going through that flow can slow down considerably the end-to-end extraction time. For example, in a project like gitlab-org/gitlab-foss, this would result to 15x more API calls than required for fetching all the other Entities supported by tap-gitlab.

The approach should be similar to what we did with fetching the Commits for a Merge Request, with a configuration parameter similar to fetch_merge_request_commits that will allow users to manually set this option for projects they know can handle that type of N+1 querying.

Useful Resources when working with Gitlab's API:

1. Gitlab API Documentation
2. Gitlab API entities
3. Entity Schemas

Start by tracing how tap-gitlab extracts Merge Requests and how the existing fetch_merge_request_commits configuration controls per-MR API calls. Review the linked GitLab API endpoints and entity schemas, then define the corresponding relation extraction and configuration behavior. Done means the relation is exported when enabled without adding these N+1 calls by default, with coverage for the new behavior.