aws-samples / aws-samples/aws-emr-advisor

[BUG] java.lang.UnsupportedOperationException: empty.minBy

Open
#9 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Scala
Stars
16
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
When running in EMR 7.10, I get this exception (run several times in different clusters):

25/12/24 08:52:15 INFO AppOptimizerAnalyzer: Generate Spark simulations with different sets of cores and max executors
25/12/24 08:52:17 INFO AppOptimizerAnalyzer: Generate application recommendations
StageID  Actual(ms)   Estimated(ms)  Tasks    AvgTask(ms)  MaxTask(ms)  Parents
-------------------------------------------------------------------------------
0        2225         1965           1        1965         1965
1        2258         2139           4        1403         2139
3        197          173            1        173          173          2
4        175          145            1        145          145
5        1358         1316           3        624          1316
6        481          422            4        347          422
8        177          147            1        147          147          7
9        4402         4333           4        1559         4333
10       469          440            4        423          440
12       57           32             1        32           32           11
13       130          110            1        110          110
14       510          486            3        210          486
Real Stage time      :   12s
Estimated Stage time :   11s
25/12/24 08:52:17 INFO AppOptimizerAnalyzer: Analyzing Optimizations for EC2 CostOpt
instance.emrPrice: 0.07348
runtimeHours: 0.015976666666666667
numInstances: 1
25/12/24 08:52:17 INFO AppOptimizerAnalyzer: Analyzing Optimizations for EC2 EfficiencyOpt
25/12/24 08:52:17 ERROR AppAnalyzer$: Failed in Analyzer AppOptimizerAnalyzer
java.lang.UnsupportedOperationException: empty.minBy
        at scala.collection.TraversableOnce.minBy(TraversableOnce.scala:305)
        at scala.collection.TraversableOnce.minBy$(TraversableOnce.scala:303)
        at scala.collection.AbstractTraversable.minBy(Traversable.scala:108)
        at com.amazonaws.emr.spark.optimizer.SparkEfficiencyOptimizer.recommend(SparkEfficiencyOptimizer.scala:57)
        at com.amazonaws.emr.spark.analyzer.AppOptimizerAnalyzer.$anonfun$analyze$8(AppOptimizerAnalyzer.scala:126)
        at scala.collection.immutable.List.foreach(List.scala:431)
        at com.amazonaws.emr.spark.analyzer.AppOptimizerAnalyzer.$anonfun$analyze$7(AppOptimizerAnalyzer.scala:117)
        at com.amazonaws.emr.spark.analyzer.AppOptimizerAnalyzer.$anonfun$analyze$7$adapted(AppOptimizerAnalyzer.scala:112)
        at scala.collection.immutable.List.foreach(List.scala:431)
        at com.amazonaws.emr.spark.analyzer.AppOptimizerAnalyzer.analyze(AppOptimizerAnalyzer.scala:112)
        at com.amazonaws.emr.spark.analyzer.AppAnalyzer.analyze(AppAnalyzer.scala:32)
        at com.amazonaws.emr.spark.analyzer.AppAnalyzer.analyze$(AppAnalyzer.scala:31)
        at com.amazonaws.emr.spark.analyzer.AppOptimizerAnalyzer.analyze(AppOptimizerAnalyzer.scala:56)
        at com.amazonaws.emr.spark.analyzer.AppAnalyzer$.$anonfun$start$1(AppAnalyzer.scala:66)
        at com.amazonaws.emr.spark.analyzer.AppAnalyzer$.$anonfun$start$1$adapted(AppAnalyzer.scala:64)
        at scala.collection.immutable.List.foreach(List.scala:431)
        at com.amazonaws.emr.spark.analyzer.AppAnalyzer$.start(AppAnalyzer.scala:64)
        at com.amazonaws.emr.spark.EmrSparkLogParser.analyze(EmrSparkLogParser.scala:33)
        at com.amazonaws.emr.SparkLogsAnalyzer$.delayedEndpoint$com$amazonaws$emr$SparkLogsAnalyzer$1(SparkLogsAnalyzer.scala:42)
        at com.amazonaws.emr.SparkLogsAnalyzer$delayedInit$body.apply(SparkLogsAnalyzer.scala:24)
        at scala.Function0.apply$mcV$sp(Function0.scala:39)
        at scala.Function0.apply$mcV$sp$(Function0.scala:39)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
        at scala.App.$anonfun$main$1$adapted(App.scala:80)
        at scala.collection.immutable.List.foreach(List.scala:431)
        at scala.App.main(App.scala:80)
        at scala.App.main$(App.scala:78)
        at com.amazonaws.emr.SparkLogsAnalyzer$.main(SparkLogsAnalyzer.scala:24)
        at com.amazonaws.emr.SparkLogsAnalyzer.main(SparkLogsAnalyzer.scala)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1150)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:200)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:223)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:92)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1246)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1255)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
25/12/24 08:52:17 INFO AppInsightsAnalyzer: Generate application insights...

To Reproduce
EMR release (e.g. emr-7.3.0) : 7.10.0
Steps to reproduce the behavior:

  1. Create the cluster (runtime role, LF enabled, no Internet access, only VPC endpoints)
  2. Run several steps
  3. Login to master using SSM
  4. Run the command:
spark-submit --class com.amazonaws.emr.SparkLogsAnalyzer   aws-emr-advisor-assembly-*.jar   --bucket xxxxx   hdfs:///var/log/spark/apps/application_xxxx_0003_1
  1. The report is generated, although no cost/efficiency/performace data is show (all report numbers are 0)

Expected behavior
I would expect no exceptions and some data show in the cost/efficiency/performance sections.

Screenshots
If applicable, add screenshots to help explain your problem.

Image Image

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at SparkEfficiencyOptimizer.scala:57, where the stack trace reports empty.minBy, then trace how AppOptimizerAnalyzer.scala lines 112-126 reaches that recommendation. Reproduce with the supplied EMR 7.10.0 spark-submit command and event-log setup. Done means the analyzer completes without the exception and the cost, efficiency, and performance sections contain data.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, scala, spark
Domain
cloud, data-engineering, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.