imagej / imagej/imagej-tensorflow
Add options dialog to control CPU/GPU mode, and graphics card selection
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 26
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
@frauzufall and I talked through how we could go about providing user-facing control over whether TensorFlow operates in CPU mode or GPU mode, as well as which graphics card is selected in GPU mode. I pushed the skeleton of an options plugin for ImageJ to expose those options (47f6d3382a0b199533815b46b34d81cf5bd094d6 on the `options-dialog` branch), but there are several challenges:
* For mode, we need to set `org.tensorflow.NativeLibrary.MODE=something` _before_ the `DefaultTensorFlowService` class loads. I.e.: before the SciJava `Context` is created. So: in effect, we would need a restart.
* For graphics card selection, we need to set `CUDA_VISIBLE_DEVICES` and/or `CUDA_DEVICE_ORDER` env vars, again _before_ TensorFlow initializes, i.e. before SciJava context is created. So again: restart.
But even with a restart: how do we ensure that when the JVM starts up, that these sys props and env vars are set early enough? What machinery could do this? We do not have anything in place. We could have a file containing desired key/values, that the launcher Java code reads and sets as early as possible—before instantiating the SciJava `Context`. But this would be a new feature. Perhaps it could be part of `ImageJ.cfg`?
An alternative could be to make sure the `TensorFlowService` does not reference any TensorFlow classes. E.g., similar to the `LegacyService` and `IJ1Helper` of imagej-legacy, it _might_ work to have something like `tfService.actions().loadModel(...)` as a level of indirection. Might be worth trying, but even if it works, it is quite ugly and unintuitive. Before we do that, let's verify whether creating an SJ context really initializes TF early—maybe it doesn't.
We could submit a pull request to the TensorFlow project that defers the loading of the native library until the first time any TensorFlow operation is performed. I.e.: eliminate the static initializers, since they create problems as described above. But it would be a non-trivial change to TensorFlow to do that, potentially more difficult to maintain.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the options-dialog branch and commit 47f6d3382a0b199533815b46b34d81cf5bd094d6. Verify whether creating a SciJava Context initializes TensorFlow before configuration can be applied, then compare the launcher configuration and TensorFlow deferred-loading alternatives. Done requires a decided approach for CPU/GPU mode and graphics-card selection before TensorFlow initialization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, tensorflow
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100