deepjavalibrary / deepjavalibrary/djl
Getting java.lang.IllegalStateException: Tried to specify the thread pool when creating an OrtEnvironment, but one already exists.
- Dominant language
- Java
- Stars
- 4.9k
- Forks
- 759
- Avg merge
- 16h 15m
- Merged PRs (30d)
- 15
Description
We are seeing a
`Getting java.lang.IllegalStateException: Tried to specify the thread pool when creating an OrtEnvironment, but one already exists.`
This is our object to create the criteria
`object ModelCriteria {
def createCriteria(runTime: String,
modelFilePath: String,
modelFileName: String,
translator: Translator[Map[String, Any], Map[String, Any]]
): Criteria[Map[String, Any], Map[String, Any]] = {
// Manually setting the number of threads to be used by the Pytorch Engine
System.setProperty("ai.djl.pytorch.num_threads", sys.env.getOrElse("PYTORCH_NUM_THREADS", "1"))
System.setProperty("ai.djl.pytorch.num_interop_threads", sys.env.getOrElse("PYTORCH_NUM_INTEROP_THREADS", "1"))
// Using interOpNumThreads and intraOpNumThreads option in Criteria builder to control parallelism
val criteria: Criteria[Map[String, Any], Map[String, Any]] = runTime match {
case "OnnxRuntime" =>
Criteria.builder()
.optOption("interOpNumThreads", sys.env.getOrElse("ONNX_INTEROP_NUM_THREADS", "1"))
.optOption("intraOpNumThreads", sys.env.getOrElse("ONNX_INTRAOP_NUM_THREADS", "1"))
.setTypes(classOf[Map[String, Any]], classOf[Map[String, Any]])
.optTranslator(translator)
.optModelUrls(modelFilePath)
.optModelName(modelFileName)
.optEngine(runTime)
.optOption("mapLocation", "true")
.optProgress(new ProgressBar())
.build()
case _ =>
Criteria.builder()
.setTypes(classOf[Map[String, Any]], classOf[Map[String, Any]])
.optTranslator(translator)
.optModelUrls(modelFilePath)
.optModelName(modelFileName)
.optEngine(runTime)
.optOption("mapLocation", "true")
.optProgress(new ProgressBar())
.build()
}
criteria
}
}`
When we try to load two onnx models back to back we are seeing this issue
`Getting java.lang.IllegalStateException: Tried to specify the thread pool when creating an OrtEnvironment, but one already exists.`
I am guessing that this is due to
` .optOption("interOpNumThreads", sys.env.getOrElse("ONNX_INTEROP_NUM_THREADS", "1"))
.optOption("intraOpNumThreads", sys.env.getOrElse("ONNX_INTRAOP_NUM_THREADS", "1"))`
this being called multiple times.
Can you suggest what practice we should follow here?
Contributor guide
Assessment
This issue has not been assessed yet.