linkedin / linkedin/dr-elephant
setup dr-elephant on centos7 with hadoop 2.7.3 and Spark Version 1.6.3
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
do you have some ideas please ?
could we setup Dr-elephant on centos7 with hadoop 2.7.3 and Spark Version 1.6.3 ?
i am trying to do it but i had some error to compile it ?
i am fallowing https://github.com/linkedin/dr-elephant/wiki/Quick-Setup-Instructions.
and the error with the compile dr-elephant is :
[info] - returns the job failure rate *** FAILED ***
[info] "0[,]400" was not equal to "0[.]400" (JobsHeuristicTest.scala:74)
[info] - returns the list of jobs with high task failure rates *** FAILED ***
[info] "...task failure rate: 0[,600)
[info] job 4, zzz (task failure rate: 0,]800)" was not equal to "...task failure rate: 0[.600)
[info] job 4, zzz (task failure rate: 0.]800)" (JobsHeuristicTest.scala:78)
[info] *** 13 TESTS FAILED ***
[error] Failed: Total 377, Failed 17, Errors 0, Passed 359, Skipped 1
[error] Failed tests:
[error] com.linkedin.drelephant.spark.heuristics.ExecutorsHeuristicTest
[error] com.linkedin.drelephant.spark.heuristics.StagesHeuristicTest
[error] com.linkedin.drelephant.util.MemoryFormatUtilsTest
[error] com.linkedin.drelephant.spark.heuristics.JobsHeuristicTest
[error] com.linkedin.drelephant.util.SparkUtilsTest
[error] com.linkedin.drelephant.util.UtilsTest
[error] com.linkedin.drelephant.math.StatisticsTest
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 572 s, completed 10 janv. 2018 23:00:46
./compile.sh: ligne 131 : cd: target/universal: Aucun fichier ou dossier de ce type
/bin/ls: impossible d'accéder à *.zip: Aucun fichier ou dossier de ce type
UnZip 6.00 of 20 April 2009, by Info-ZIP. Maintained by C. Spieler. Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.
Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]
Default action is to extract files in list, except those in xlist, to exdir;
file[.zip] may be a wildcard. -Z => ZipInfo mode ("unzip -Z" for usage).
-p extract files to pipe, no messages -l list files (short format)
-f freshen existing files, create none -t test compressed archive data
-u update files, create if necessary -z display archive comment only
-v list verbosely/show version info -T timestamp archive to latest
-x exclude files that follow (in xlist) -d extract files into exdir
modifiers:
-n never overwrite existing files -q quiet mode (-qq => quieter)
-o overwrite files WITHOUT prompting -a auto-convert any text files
-j junk paths (do not make directories) -aa treat ALL files as text
-U use escapes for all non-ASCII Unicode -UU ignore any Unicode fields
-C match filenames case-insensitively -L make (some) names lowercase
-X restore UID/GID info -V retain VMS version numbers
-K keep setuid/setgid/tacky permissions -M pipe through "more" pager
-O CHARSET specify a character encoding for DOS, Windows and OS/2 archives
-I CHARSET specify a character encoding for UNIX and other archives
See "unzip -hh" or unzip.txt for more help. Examples:
unzip data1 -x joe => extract all files except joe from zipfile data1.zip
unzip -p foo | more => send contents of foo.zip via pipe into program more
unzip -fo foo ReadMe => quietly replace existing ReadMe if archive file newer
rm: opérande manquant
Saisissez « rm --help » pour plus d'informations.
chmod: impossible d'accéder à « /bin/dr-elephant »: Aucun fichier ou dossier de ce type
sed: impossible de lire /bin/dr-elephant: Aucun fichier ou dossier de ce type
cp: opérande de fichier cible manquant après « /root/dr-elephant/app-conf »
Saisissez « cp --help » pour plus d'informations.
zip error: Nothing to do! (.zip)
mv: impossible d'évaluer « .zip »: Aucun fichier ou dossier de ce type
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with compile.sh and the listed failed test classes, especially JobsHeuristicTest.scala, to reproduce the CentOS 7 build failure with Hadoop 2.7.3 and Spark 1.6.3. Check why the tests produce comma decimal separators and why target/universal is not created. Done means the test suite passes and compile.sh produces the expected archive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- centos, hadoop, scala
- Domain
- build-system, distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100