aws / aws/sagemaker-training-toolkit
Unable to install sagemaker-training on Windows
- Dominant language
- Python
- Stars
- 530
- Forks
- 140
- Avg merge
- 1h 12m
- Merged PRs (30d)
- 2
Description
**Describe the bug**
Whenever I try to run `pip install sagemaker-training`, the wheel step fails because there is a linker error when it is time to compile the `.c` files.
**To reproduce**
Run `pip install sagemaker-training` on a Windows 10 machine that has Visual Studo 2019 or a MSVC toolchain installed.
**Expected behavior**
Compile just fine. Or alert the user of what dependecies the user might be missing.
**Screenshots or logs**
The log is dense, so the probably most interest part of the log is the linker error unable to find `strndup` as seen near to end of log.
```
Building wheels for collected packages: sagemaker-training
Building wheel for sagemaker-training (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'c:\program files\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\matly\\AppData\\Local\\Temp\\pip-install-0ar0b3rh\\sagemaker-training_8676a0ae965040bd99685308a5519a1d\\setup.py'"'"'; __file__='"'"'C:\\Users\\matly\\AppData\\Local\\Temp\\pip-install-0ar0b3rh\\sagemaker-training_8676a0ae965040bd99685308a5519a1d\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\matly\AppData\Local\Temp\pip-wheel-q4lstx5x'
cwd: C:\Users\matly\AppData\Local\Temp\pip-install-0ar0b3rh\sagemaker-training_8676a0ae965040bd99685308a5519a1d\
Complete output (119 lines):
c:\program files\python39\lib\site-packages\setuptools\dist.py:642: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\content_types.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\encoders.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\entry_point.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\environment.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\errors.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\files.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\functions.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\intermediate_output.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\logging_config.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\mapping.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\modules.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\mpi.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\params.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\process.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\recordio.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\record_pb2.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\runner.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\smdataparallel.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\timeout.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\trainer.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\_entry_point_type.py -> build\lib.win-amd64-3.9\sagemaker_training
copying src\sagemaker_training\__init__.py -> build\lib.win-amd64-3.9\sagemaker_training
creating build\lib.win-amd64-3.9\sagemaker_training\cli
copying src\sagemaker_training\cli\train.py -> build\lib.win-amd64-3.9\sagemaker_training\cli
copying src\sagemaker_training\cli\__init__.py -> build\lib.win-amd64-3.9\sagemaker_training\cli
running build_ext
building 'gethostname' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
creating build\temp.win-amd64-3.9\Release\src
creating build\temp.win-amd64-3.9\Release\src\sagemaker_training
creating build\temp.win-amd64-3.9\Release\src\sagemaker_training\c
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc/sagemaker_training/c -Ic:\program files\python39\include -Ic:\program files\python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcsrc/sagemaker_training/c/gethostname.c /Fobuild\temp.win-amd64-3.9\Release\src/sagemaker_training/c/gethostname.obj -Wall -shared -export-dynamic -ldl
cl : Command line warning D9002 : ignoring unknown option '-shared'
cl : Command line warning D9002 : ignoring unknown option '-export-dynamic'
cl : Command line warning D9002 : ignoring unknown option '-ldl'
gethostname.c
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt.h(212): warning C4668: '__cplusplus' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(49): warning C4820: '_finddata32i64_t': '4' bytes padding added after data member 'name'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(54): warning C4820: '_finddata64i32_t': '4' bytes padding added after data member 'attrib'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(64): warning C4820: '__finddata64_t': '4' bytes padding added after data member 'attrib'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt_io.h(69): warning C4820: '__finddata64_t': '4' bytes padding added after data member 'name'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\time.h(36): warning C4820: '_timespec64': '4' bytes padding added after data member 'tv_nsec'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\time.h(43): warning C4820: 'timespec': '4' bytes padding added after data member 'tv_nsec'
c:\program files\python39\include\object.h(198): warning C4820: '': '4' bytes padding added after data member 'slot'
c:\program files\python39\include\object.h(206): warning C4820: '': '4' bytes padding added after data member 'flags'
c:\program files\python39\include\cpython/object.h(228): warning C4820: '_typeobject': '4' bytes padding added after data member 'tp_flags'
c:\program files\python39\include\cpython/object.h(272): warning C4820: '_typeobject': '4' bytes padding added after data member 'tp_version_tag'
c:\program files\python39\include\pytime.h(123): warning C4115: 'timeval': named type definition in parentheses
c:\program files\python39\include\cpython/bytesobject.h(15): warning C4820: '': '7' bytes padding added after data member 'ob_sval'
c:\program files\python39\include\cpython/bytesobject.h(65): warning C4820: '': '4' bytes padding added after data member 'small_buffer'
c:\program files\python39\include\cpython/unicodeobject.h(218): warning C4820: '': '4' bytes padding added after data member 'state'
c:\program files\python39\include\cpython/unicodeobject.h(626): warning C4820: '': '2' bytes padding added after data member 'readonly'
c:\program files\python39\include\longintrepr.h(88): warning C4820: '_longobject': '4' bytes padding added after data member 'ob_digit'
c:\program files\python39\include\memoryobject.h(45): warning C4820: '': '4' bytes padding added after data member 'flags'
c:\program files\python39\include\memoryobject.h(62): warning C4820: '': '4' bytes padding added after data member 'flags'
c:\program files\python39\include\methodobject.h(40): warning C4820: 'PyMethodDef': '4' bytes padding added after data member 'ml_flags'
c:\program files\python39\include\moduleobject.h(63): warning C4820: 'PyModuleDef_Slot': '4' bytes padding added after data member 'slot'
c:\program files\python39\include\cpython/code.h(27): warning C4820: 'PyCodeObject': '4' bytes padding added after data member 'co_firstlineno'
c:\program files\python39\include\cpython/code.h(64): warning C4820: 'PyCodeObject': '3' bytes padding added after data member 'co_opcache_size'
c:\program files\python39\include\cpython/initconfig.h(16): warning C4820: '': '4' bytes padding added after data member '_type'
c:\program files\python39\include\cpython/initconfig.h(19): warning C4820: '': '4' bytes padding added after data member 'exitcode'
c:\program files\python39\include\cpython/initconfig.h(209): warning C4820: '': '4' bytes padding added after data member 'parse_argv'
c:\program files\python39\include\cpython/initconfig.h(364): warning C4820: '': '4' bytes padding added after data member 'legacy_windows_stdio'
c:\program files\python39\include\cpython/initconfig.h(374): warning C4820: '': '4' bytes padding added after data member 'pathconfig_warnings'
c:\program files\python39\include\cpython/initconfig.h(381): warning C4820: '': '4' bytes padding added after data member 'module_search_paths_set'
c:\program files\python39\include\cpython/initconfig.h(401): warning C4820: '': '4' bytes padding added after data member 'skip_source_first_line'
c:\program files\python39\include\cpython/initconfig.h(424): warning C4820: '': '4' bytes padding added after data member '_isolated_interpreter'
c:\program files\python39\include\cpython/pystate.h(65): warning C4820: '_ts': '2' bytes padding added after data member 'recursion_critical'
c:\program files\python39\include\cpython/pystate.h(73): warning C4820: '_ts': '4' bytes padding added after data member 'use_tracing'
c:\program files\python39\include\cpython/pystate.h(96): warning C4820: '_ts': '4' bytes padding added after data member 'gilstate_counter'
c:\program files\python39\include\cpython/pystate.h(130): warning C4820: '_ts': '4' bytes padding added after data member 'coroutine_origin_tracking_depth'
c:\program files\python39\include\genobject.h(33): warning C4820: '': '7' bytes padding added after data member 'gi_running'
c:\program files\python39\include\genobject.h(52): warning C4820: '': '7' bytes padding added after data member 'cr_running'
c:\program files\python39\include\genobject.h(67): warning C4820: '': '7' bytes padding added after data member 'ag_running'
c:\program files\python39\include\genobject.h(80): warning C4820: '': '4' bytes padding added after data member 'ag_running_async'
c:\program files\python39\include\descrobject.h(29): warning C4820: 'wrapperbase': '4' bytes padding added after data member 'offset'
c:\program files\python39\include\descrobject.h(33): warning C4820: 'wrapperbase': '4' bytes padding added after data member 'flags'
c:\program files\python39\include\structseq.h(20): warning C4820: 'PyStructSequence_Desc': '4' bytes padding added after data member 'n_in_sequence'
c:\program files\python39\include\cpython/pyerrors.h(19): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(23): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(33): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(40): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(49): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(54): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\cpython/pyerrors.h(66): warning C4820: '': '7' bytes padding added after data member 'suppress_context'
c:\program files\python39\include\modsupport.h(100): warning C4820: '_PyArg_Parser': '4' bytes padding added after data member 'max'
c:\program files\python39\include\cpython/import.h(41): warning C4820: '_frozen': '4' bytes padding added after data member 'size'
c:\program files\python39\include\cpython/fileutils.h(54): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_dev'
c:\program files\python39\include\cpython/fileutils.h(56): warning C4820: '_Py_stat_struct': '2' bytes padding added after data member 'st_mode'
c:\program files\python39\include\cpython/fileutils.h(60): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_rdev'
c:\program files\python39\include\cpython/fileutils.h(63): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_atime_nsec'
c:\program files\python39\include\cpython/fileutils.h(65): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_mtime_nsec'
c:\program files\python39\include\cpython/fileutils.h(69): warning C4820: '_Py_stat_struct': '4' bytes padding added after data member 'st_reparse_tag'
src/sagemaker_training/c/gethostname.c(70): warning C4013: 'strndup' undefined; assuming extern returning int
src/sagemaker_training/c/gethostname.c(70): warning C4047: 'initializing': 'char *' differs in levels of indirection from 'int'
src/sagemaker_training/c/gethostname.c(90): warning C4100: 'self': unreferenced formal parameter
src/sagemaker_training/c/gethostname.c(124): warning C4255: 'PyInit_gethostname': no function prototype given: converting '()' to '(void)'
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdio.h(948): warning C4710: 'int printf(const char *const ,...)': function not inlined
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdio.h(948): note: see declaration of 'printf'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc/sagemaker_training/c -Ic:\program files\python39\include -Ic:\program files\python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcsrc/sagemaker_training/c/jsmn.c /Fobuild\temp.win-amd64-3.9\Release\src/sagemaker_training/c/jsmn.obj -Wall -shared -export-dynamic -ldl
cl : Command line warning D9002 : ignoring unknown option '-shared'
cl : Command line warning D9002 : ignoring unknown option '-export-dynamic'
cl : Command line warning D9002 : ignoring unknown option '-ldl'
jsmn.c
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\corecrt.h(212): warning C4668: '__cplusplus' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\program files\python39\libs /LIBPATH:c:\program files\python39\PCbuild\amd64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64 /EXPORT:PyInit_gethostname build\temp.win-amd64-3.9\Release\src/sagemaker_training/c/gethostname.obj build\temp.win-amd64-3.9\Release\src/sagemaker_training/c/jsmn.obj /OUT:build\lib.win-amd64-3.9\gethostname.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.lib
Creating library build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.lib and object build\temp.win-amd64-3.9\Release\src/sagemaker_training/c\gethostname.cp39-win_amd64.exp
gethostname.obj : error LNK2001: unresolved external symbol strndup
build\lib.win-amd64-3.9\gethostname.cp39-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30037\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120
----------------------------------------
ERROR: Failed building wheel for sagemaker-training
Running setup.py clean for sagemaker-training
Failed to build sagemaker-training
```
**System information**
A description of your system.
- SageMaker Training version: `4.0.0`.
- Windows 10 Pro N
- Python 3.9.5
- MSVC:
- Compiler: `Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30037 for x64`
- Linker: `Microsoft (R) Incremental Linker Version 14.29.30037.0`
**Additional context**
`sagemaker-training` is required by a framework to be installed locally on computer for development purposes. While it is recommended to use Linux or MacOSX due to amount of libraries only compatibile with Linux, there are a goal to expand the framework to be compatible with Windows. Any help or advice is appreciated.
Contributor guide
Assessment
This issue has not been assessed yet.