zhanghang1989 / zhanghang1989/PyTorch-Encoding

gcc4.9 requirement

Open
#117 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

update pytorch
Dominant language
Python
Stars
2k
Forks
448
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am running into an import error with encoding.nn (the error is printed at the end).

My system is CentOS Linux release 7.4.1708 (Core) and it comes with gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16). So I followed the instructions to install gcc 4.9 for pytorch extensions (https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6), enabled the environment using scl enable devtoolset-3 zsh, confirmed new gcc version as gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6), and tried to import encoding.nn in an ipython session.

I have not modified any of the .zshrc paths, such as LD_LIBRARY_PATH or CUDA_HOME. My version of pytorch is 0.4.1 and was installed with conda install pytorch torchvision -c pytorch (since torch-encoding repo was asking for version higher than stable release v0.4.0).

I am trying to overcome the problem of training and running inference with a batchnorm heavy model using nn.DataParallel with multiple GPUs. Running into the issues listed here during inference: https://github.com/pytorch/pytorch/issues/1051

/data/QILCode/python/projects/airways_vessels_segmentation/pytorch_tiramisu_model_encoding.py in <module>()
      1 import torch
      2 import torch.nn as nn
----> 3 import encoding.nn as enc_nn
      4 
      5 

~/anaconda3/lib/python3.6/site-packages/encoding/__init__.py in <module>()
     11 """An optimized PyTorch package with CUDA backend."""
     12 from .version import __version__
---> 13 from . import nn, functions, dilated, parallel, utils, models, datasets

~/anaconda3/lib/python3.6/site-packages/encoding/nn/__init__.py in <module>()
     10 
     11 """Encoding NN Modules"""
---> 12 from .encoding import *
     13 from .syncbn import *
     14 from .customize import *

~/anaconda3/lib/python3.6/site-packages/encoding/nn/encoding.py in <module>()
     16 from torch.nn.modules.utils import _pair
     17 
---> 18 from ..functions import scaledL2, aggregate, pairwise_cosine
     19 
     20 __all__ = ['Encoding', 'EncodingDrop', 'Inspiration', 'UpsampleConv2d']

~/anaconda3/lib/python3.6/site-packages/encoding/functions/__init__.py in <module>()
      1 """Encoding Autograd Fuctions"""
----> 2 from .encoding import *
      3 from .syncbn import *

~/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py in <module>()
     12 from torch.autograd import Function, Variable
     13 import torch.nn.functional as F
---> 14 from .. import lib
     15 
     16 __all__ = ['aggregate', 'scaledL2', 'pairwise_cosine']

~/anaconda3/lib/python3.6/site-packages/encoding/lib/__init__.py in <module>()
     10         os.path.join(cpu_path, 'roi_align.cpp'),
     11         os.path.join(cpu_path, 'roi_align_cpu.cpp'),
---> 12     ], build_directory=cpu_path, verbose=False)
     13 
     14 if torch.cuda.is_available():

~/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py in load(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda)
    512         build_directory or _get_build_directory(name, verbose),
    513         verbose,
--> 514         with_cuda=with_cuda)
    515 
    516 

~/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py in _jit_compile(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda)
    688     if verbose:
    689         print('Loading extension module {}...'.format(name))
--> 690     return _import_module_from_library(name, build_directory)
    691 
    692 

~/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py in _import_module_from_library(module_name, path)
    771     # Close the .so file after load.
    772     with file:
--> 773         return imp.load_module(module_name, file, path, description)
    774 
    775 

~/anaconda3/lib/python3.6/imp.py in load_module(name, file, filename, details)
    241                 return load_dynamic(name, filename, opened_file)
    242         else:
--> 243             return load_dynamic(name, filename, file)
    244     elif type_ == PKG_DIRECTORY:
    245         return load_package(name, filename)

~/anaconda3/lib/python3.6/imp.py in load_dynamic(name, path, file)
    341         spec = importlib.machinery.ModuleSpec(
    342             name=name, loader=loader, origin=path)
--> 343         return _load(spec)
    344 
    345 else:

ImportError: /home/qiluser/anaconda3/lib/python3.6/site-packages/encoding/lib/cpu/enclib_cpu.so: undefined symbol: _ZN2at5ErrorC1ENS_14SourceLocationESs

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the import failure with encoding.nn, then inspect encoding/lib/init.py and the torch.utils.cpp_extension.load call shown in the traceback. Compare the installed PyTorch version with the compiled extension and determine whether the undefined symbol is reproducible; done means the import works or the supported version and installation requirements are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.