BlueBrain / BlueBrain/Brion

Lazy loading CompartmentReport before subprocess hangs

Open
#272 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
23
Forks
23
PR merge metrics
No merged PRs in 30d

Description

## Issue
I have an issue on BB5 where I need to use `brion` in combination with `py-bluepy` modules to read some reports and then execute other process with **subprocess**.
The issue is that after using **brain** to get some information, then **subprocess** hangs.
We were able to identify that if we load `from brain import CompartmentReport` at the beginning of the `test_brion_report.py` file (so it is not being lazy loading) the issue disappears.
But BluePy uses this with lazy load and anyhow it should work the same so, can you investigate why am I getting this issue?

I past below the files examples and the script to run this test.

## Save files
**run.sh**
```
#!/bin/bash
module load brion
module load py-bluepy
python ./test_brion_report.py
```

**test_brion_report.py**
```
# from brain import CompartmentReport

t_start = 0
t_end = 300
t_step = 0.1
report_path = '/gpfs/bbp.cscs.ch/home/antonel/unicore_jobs/47f40ebc-8910-4fb5-99ad-d5fcc28ff059/5Cells_report_0.bbp'

def use_brain():
import pandas as pd
import numpy as np
import subprocess
from brain import CompartmentReport

# obtained from the original brion code
def _wrap_index(index):
""" Create pandas (Multi)Iindex from 'raw' index. """
return pd.MultiIndex.from_tuples(index)

report = CompartmentReport(report_path)
view = report.create_view([51848])

print('Calling subprocess ---------------------')
subprocess.call(['ls'])
print('Done')

use_brain()
```
## Allocate resources on BB5
`salloc -n 1 --time 2:00:00 --account proj42 -C uc3 --partition prod`

## Run script
`sh run.sh`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with run.sh and test_brion_report.py, then reproduce the difference between the commented top-level CompartmentReport import and the import inside use_brain(). Run the script with subprocess.call(['ls']) and confirm that the hang is resolved without eager loading.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.