microsoft / microsoft/azure-tools-for-java
Error doing a remote run of a Spark Scala application on a Spark pool using IntelliJ new UI
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 262
- Forks
- 185
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 1
Description
When on classic UI, things work just fine. When I switch to the "new UI" I can still do a local run, but the remote run no longer works. I get the following error:
Error running 'Spark'
Spark Batch Job is not prepared for SparkJobRun
Using Azure Toolkit for IntelliJ 3.78.1-2023.1
Environment Info:
IntelliJ IDEA 2023.1.3 (Community Edition)
Build #IC-231.9161.38, built on June 19, 2023
Runtime version: 17.0.7+10-b829.16 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 11.0
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 16
Registry:
scala.erase.compiler.process.jdk.once=false
Non-Bundled Plugins:
org.intellij.scala (2023.1.19)
com.microsoft.tooling.msservices.intellij.azure (3.78.1-2023.1)
Kotlin: 231-1.8.21-IJ9161.38
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce a remote run of the Spark Scala application using Azure Toolkit for IntelliJ 3.78.1-2023.1 on IntelliJ IDEA 2023.1.3, comparing the classic and new UI. Done means the new UI successfully prepares and runs the remote Spark batch job without the “Spark Batch Job is not prepared for SparkJobRun” error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, scala
- Domain
- cloud, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100