linkedin / linkedin/ambry

Mapping datacenter name to an id.

Open
#637 2 comments 0 reactions 1 assignee View on GitHub

@xiahome is already working on this.

Since May 19, 2017.

enhancement
Dominant language
Java
Stars
1.8k
Forks
298
Avg merge
21h 43m
Merged PRs (30d)
9

Description

Each datacenter currently has a String representation, e.g., `DC1`, which is specified when making the cluster. Because of the reason specified in #636 , an id in short needs to be generated for each datacenter. Datacenter id will be used in both `put` and `get` path for a blob.

1. Put a blob: the router is the component to perform the put blob logic. When a put is successful, the router will get the id of the datacenter (usually the local datacenter), by calling `getDatacenterIdByName(dcNameForPut)`, and embed the datacenter id into the blob id as proposed in #636 .
2. Get a blob: if getting the blob in the local datacenter of the router fails, the router can extract the id of the original datacenter for the blob, and get the datacenter name by calling `getDatacenterNameById(dcIdInBlob)`. Then, the router can issue a get request to the original datacenter, avoiding fetching the blob from other datacenters.

Therefore, `getDatacenterIdByName` and `getDatacenterNameById` are needed. `ClusterMap` interface is a natural place to define these two methods, because datacenter is a cluster-wide component under the awareness of a cluster map. Each implementation, so far `StaticClusterManager`, `HelixClusterManager`, and `CompositeClusterManager`, implements the mapping between datacenter name and datacenter id.

There are two approaches to make an id for a datacenter name:
1. Have the cluster map creator to assign an id to each datacenter. For example, in the hardware layout json file, an additional field is created to specify an id for each datacenter;
2. The id is generated internally. This can be done by hashing the datacenter name into a short id.

The second approach is more favored, for the following reasons:
1. There is no need for the cluster map creator's to be aware of datacenter id. The id is purely internal, but requires to be cluster-wide unique.
2. It requires attention from the cluster map creator to make the id unique when adding or updating datacenter.

This ticket is created to track the work of adding the feature of mapping datacenter name to id, and vise versa.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.