llnl / llnl/magpie

-Xmx passed twice on command line

Open
#237 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
198
Forks
51
PR merge metrics
No merged PRs in 30d

Description

-Xmx1000m and -Xmx512m are passed twice, likely due to scripting logic errors in hadoop-env.sh (512m is set via HADOOP_CLIENT_OPS). This isn't a big deal, but would be nice to clean up.

> achutest 60276 2.5 0.0 2732480 264768 ? Sl 11:04 0:14
> | \_ /usr/lib/jvm/jre-1.8.0/bin/java -Xmx1000m
> -Djava.net.preferIPv4Stack=true
> -Djava.io.tmpdir=/tmp/achutest/hadoop/testlsf/28794/scratch/tmp
> -Dhadoop.log.dir=/tmp/achutest/hadoop/testlsf/28794/log
> -Dhadoop.log.file=hadoop.log
> -Dhadoop.home.dir=/g/g0/achutest/bigdata/hadoop-2.7.3
> -Dhadoop.id.str=achutest -Dhadoop.root.logger=INFO,console
> -Djava.library.path=/g/g0/achutest/bigdata/hadoop-2.7.3/lib/native
> -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
> -Xmx512m -Dhadoop.security.logger=INFO,NullAppender
> org.apache.hadoop.util.RunJar
> share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar teragen
> -Dmapreduce.job.maps=288 -Ddfs.datanode.drop.cache.behind.reads=true
> -Ddfs.datanode.drop.cache.behind.writes=true 10000000000
> terasort-teragen

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect hadoop-env.sh and the handling of HADOOP_CLIENT_OPS, then trace how JVM options are assembled for the Hadoop command shown in the report. Done means the generated command line contains only one -Xmx setting when these values are configured.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, shell
Domain
distributed-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.