open-mmlab / open-mmlab/mmengine

[Bug] "Failed to import mmocr.models, please check the location of the registry model is correct" when built from pipfile.lock

Open
#1,035 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.5k
Forks
464
PR merge metrics
No merged PRs in 30d

Description

Prerequisite
Environment

OrderedDict([('sys.platform', 'linux'), ('Python', '3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]'), ('CUDA available', True), ('numpy_random_seed', 2147483648), ('GPU 0,1,2,3,4,5,6', 'NVIDIA GeForce RTX 2080 Ti'), ('CUDA_HOME', '/usr/local/cuda'), ('NVCC', 'Cuda compilation tools, release 11.7, V11.7.64'), ('GCC', 'x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0'), ('PyTorch', '2.0.0+cu117'), ('PyTorch compiling details', 'PyTorch built with:\n - GCC 9.3\n - C++ Version: 201703\n - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications\n - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n - OpenMP 201511 (a.k.a. OpenMP 4.5)\n - LAPACK is enabled (usually provided by MKL)\n - NNPACK is enabled\n - CPU capability usage: AVX2\n - CUDA Runtime 11.7\n - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n - CuDNN 8.5\n - Magma 2.6.1\n - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n'), ('TorchVision', '0.15.1+cu117'), ('OpenCV', '4.7.0'), ('MMEngine', '0.7.0')])

Reproduces the problem - code sample
from mmocr.apis import TextDetInferencer
inferencer = TextDetInferencer(weights='dbnetpp3_tele/detectionEp20.pth', device='cuda:2')
Reproduces the problem - command or script

Run in a jupyter notebook

Reproduces the problem - error message

When installed manually

Loads checkpoint by local backend from path: dbnetpp3_tele/detectionEp20.pth
03/30 09:26:15 - mmengine - WARNING - The "model" registry in mmdet did not set import location. Fallback to call `mmdet.utils.register_all_modules` instead.
03/30 09:26:15 - mmengine - INFO - ModulatedDeformConvPack backbone.layer2.0.conv2 is upgraded to version 2.
03/30 09:26:15 - mmengine - INFO - ModulatedDeformConvPack backbone.layer2.1.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer2.2.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer2.3.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.0.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.1.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.2.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.3.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.4.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer3.5.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer4.0.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer4.1.conv2 is upgraded to version 2.
03/30 09:26:16 - mmengine - INFO - ModulatedDeformConvPack backbone.layer4.2.conv2 is upgraded to version 2.
03/30 09:26:19 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
03/30 09:26:19 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
03/30 09:26:19 - mmengine - WARNING - Failed to search registry with scope "mmocr" in the "function" registry tree. As a workaround, the current "function" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmocr" is a correct scope, or whether the registry is initialized.
03/30 09:26:19 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.

When built from pipenv lock

Loads checkpoint by local backend from path: dbnetpp3_tele/detectionEp20.pth
03/31 12:28:57 - mmengine - WARNING - Failed to import mmocr.models, please check the location of the registry model is correct.
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data [in a text editor](command:workbench.action.openLargeOutput?d66532d7-f8a5-46af-b08c-380fda861b3d)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[2], line 1
----> 1 inferencer = TextDetInferencer(weights='dbnetpp3_tele/detectionEp20.pth', device='cuda:2')

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmocr/apis/inferencers/base_mmocr_inferencer.py:64](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmocr/apis/inferencers/base_mmocr_inferencer.py:64), in BaseMMOCRInferencer.__init__(self, model, weights, device, scope)
     62 self.num_unnamed_imgs = 0
     63 init_default_scope(scope)
---> 64 super().__init__(
     65     model=model, weights=weights, device=device, scope=scope)

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/infer/infer.py:176](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/infer/infer.py:176), in BaseInferencer.__init__(self, model, weights, device, scope)
    173 if device is None:
    174     device = get_device()
--> 176 self.model = self._init_model(cfg, weights, device)  # type: ignore
    177 self.pipeline = self._init_pipeline(cfg)
    178 self.collate_fn = self._init_collate(cfg)

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/infer/infer.py:477](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/infer/infer.py:477), in BaseInferencer._init_model(self, cfg, weights, device)
    474 if cfg.model.get('pretrained') is not None:
    475     del cfg.model.pretrained
--> 477 model = MODELS.build(cfg.model)
    478 model.cfg = cfg
    479 self._load_weights_to_model(model, checkpoint, cfg)

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/registry.py:548](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/registry.py:548), in Registry.build(self, cfg, *args, **kwargs)
    526 def build(self, cfg: dict, *args, **kwargs) -> Any:
    527     """Build an instance.
    528 
    529     Build an instance by calling :attr:`build_func`.
   (...)
    546         >>> model = MODELS.build(cfg)
    547     """
--> 548     return self.build_func(cfg, *args, **kwargs, registry=self)

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/build_functions.py:241](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/build_functions.py:241), in build_model_from_cfg(cfg, registry, default_args)
    239     return Sequential(*modules)
    240 else:
--> 241     return build_from_cfg(cfg, registry, default_args)

File [~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/build_functions.py:100](https://vscode-remote+ssh-002dremote-002bwigpu2.vscode-resource.vscode-cdn.net/home/jonah-turner%40sandhills.int/software/text_extraction_api/~/.local/share/virtualenvs/text_extraction_api-rgzBsVBk/lib/python3.10/site-packages/mmengine/registry/build_functions.py:100), in build_from_cfg(cfg, registry, default_args)
     98     obj_cls = registry.get(obj_type)
     99     if obj_cls is None:
--> 100         raise KeyError(
    101             f'{obj_type} is not in the {registry.name} registry. '
...
    106         )
    107 elif inspect.isclass(obj_type) or inspect.isfunction(obj_type):
    108     obj_cls = obj_type

KeyError: 'DBNet is not in the model registry. Please check whether the value of `DBNet` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'
Additional information

My two environments are exactly the same, but when building the second environment from the first environment's Pipfile.lock, I get that error on model load. Is there any way to build from Pipfile.lock and avoid this error?
Here are the relevant versions from the Pipfiles.


[packages]
torch = "*"
torchvision = "*"
mmengine = "==0.7.0"
mmcv = "==2.0.0rc4"
mmdet = "==3.0.0rc5"
ipykernel = "*"
mmocr = "==1.0.0rc6"

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the TextDetInferencer call and the BaseMMOCRInferencer and BaseInferencer stack traces, then inspect how mmocr.models is imported and how the model registry is initialized for the listed MMEngine, MMCV, MMDetection, and MMOCR versions. Reproduce the failure from Pipfile.lock and compare it with the manually installed environment. Done means the same locked environment loads DBNet without the registry import warning or KeyError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.