Speeding up inferencing ?

On our production environment we are running Nemo on the HPC using slurm. Currently inferencing ran for 72 hours when it was terminated due to running out of allocated time. We can allow it to run for longer, but it will be preferable if we could speed inferencing up since we expect a steep increase in the data we need to reason over.

As far as I know, Nemo can only run on a single node. However, if Nemo is multithreaded, we can increase the number of cpus allocated. Is Nemo inferencing multithreaded? Do you have any suggestions in this regard?

Then, regarding rules, is there any sense in splitting rules up according to some dependency graph of the rules, which can be run by different instances of Nemo? Then final reasoning can be done on the combined results.

Any suggestions on how we can speed up reasoning will be highly appreciated.

The issue names no files, tests, or entry points. Start by profiling the production inference workload on Slurm and examining Nemo's existing inference and parallelism behavior; done means a concrete, validated plan for speeding inference or splitting rule evaluation.

Written by the indexing model from the issue text.