linkedin / linkedin/dynamometer

Dynamometer does not support negative block id's

Open
#63 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
134
Forks
34
PR merge metrics
No merged PRs in 30d

Description

Dynamometer does NOT support negative block id's, which results in blocks with negative id's never being reported by any simulated DataNode.

A change has been made to XMLParser.java in our branch of Dynamometer so that negative block id's are also dealt with.

Due to the change made above, we have to change SimulatedMultiStorageFSDataset.java as well.

In Dynamometer, each DataNode has more than 1 SimulatedStorage to manage, and this following Map is maintained by each SimulatedStorage in a simulated DataNode. Moreover, a SimulatedStorage could be involved in multiple blockpools.

Map>

To access a given block (associated with a blockpool id) on a simulated DataNode, we have to
( i) determine which SimulatedStorage this given block belongs to according to its block id, and then
(ii) use the associated blockpool id to retrive Map corresponding to the block to be accessed.

The SimulatedStorage's managed by a DataNode are arranged on an ArrayList and each SimulatedStorage on the ArrayList could be accessed by a "non-negative" integer upper-bounded by the size of that ArrayList, exclusive. To determine the SimulatedStorage a given block belongs to, the original Dynamometer simply uses (block id % number of simulated storages) as the index to access the ArrayList mentioned above. Hence, once we have a negative block id, an ArrayIndexOutOfBoundsException will be triggered. Some changes have been made in SimulatedMultiStorageFSDataset.java so that a negative block id is properly taken care of.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with XMLParser.java to understand how negative block IDs are parsed, then inspect SimulatedMultiStorageFSDataset.java and its block-to-storage lookup. Verify that negative IDs select a valid simulated storage and are retrieved through the appropriate block pool map without an ArrayIndexOutOfBoundsException. Done means blocks with negative IDs are reported by simulated DataNodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.