deepdoctection / deepdoctection/deepdoctection

RuntimeError: context has already been set - Finetuning deepdoctection

Open
#313 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
3.3k
Forks
199
Avg merge
14m
Merged PRs (30d)
1

Description

Hello,
I am trying to fine-tune deepdoctection tensorpack on fintabnet dataset. I followed instructions on fine-tune notebook but I got this error :
RuntimeError Traceback (most recent call last)
Cell In[140], line 1
----> 1 temp = dd.train_faster_rcnn(path_config_yaml=path_config_yaml,
2 dataset_train=fintabnet,
3 path_weights=path_weights,
4 config_overwrite=config_overwrite,
5 log_dir="/kaggle/working/logs/",
6 build_train_config=build_train_config,
7 dataset_val=dataset_val,
8 build_val_config=build_val_config,
9 metric_name="coco",
10 pipeline_component_name="ImageLayoutService"
11 )
File /kaggle/working/deepdoctection/deepdoctection/train/tp_frcnn_train.py:253, in train_faster_rcnn(path_config_yaml, dataset_train, path_weights, config_overwrite, log_dir, build_train_config, dataset_val, build_val_config, metric_name, metric, pipeline_component_name)
File /kaggle/working/deepdoctection/deepdoctection/train/tp_frcnn_train.py:136, in get_train_dataflow(dataset, config, use_multi_proc_for_train, **build_train_kwargs)
File /kaggle/working/deepdoctection/deepdoctection/utils/file_utils.py:657, in set_mp_spawn()
655 if not _S.mp_context_set:
656 _S.freeze(False)
--> 657 mp.set_start_method("spawn")
658 _S.mp_context_set = True
659 _S.freeze()
File /opt/conda/lib/python3.10/multiprocessing/context.py:247, in DefaultContext.set_start_method(self, method, force)
245 def set_start_method(self, method, force=False):
246 if self._actual_context is not None and not force:
--> 247 raise RuntimeError('context has already been set')
248 if method is None and force:
249 self._actual_context = None
RuntimeError: context has already been set

My code :
temp = dd.train_faster_rcnn(path_config_yaml=path_config_yaml,
dataset_train=fintabnet,
path_weights=path_weights,
config_overwrite=config_overwrite,
log_dir="/kaggle/working/logs/",
build_train_config=build_train_config,
dataset_val=dataset_val,
build_val_config=build_val_config,
metric_name="coco",
pipeline_component_name="ImageLayoutService"
)

Log :
log-3.log

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with deepdoctection/utils/file_utils.py at set_mp_spawn, then trace its call from get_train_dataflow and train_faster_rcnn in deepdoctection/train/tp_frcnn_train.py. Reproduce the fine-tuning notebook call and review log-3.log alongside the traceback. Done means the reported fine-tuning call completes without the context RuntimeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.