[BUG] distutils.msvccompiler does not work with any currently available VS build tools

setuptools version

setuptools == 62.3.1

Python version

Python 3.10

OS

Win10

Additional environment information

I have tried to make this work on Win10 with various combinations of:

• Python 3.10 64-bit
• Python 3.10 32-bit
• Python 3.8 32-bit
• Build Tools for Visual Studio 2019 (version 16.11)
  • this is the latest version of VS2019 (which the compiler wiki recommends)
• Build Tools for Visual Studio 2019 (version 16.0) <-- in case higher versions broke things
• Build Tools for Visual Studio 2022 (version 17.2) <-- just in case!
• Build Tools for Visual Studio 2022 (version 17.0) <-- also just in case!

With the latest VS2019 build tools, I'm installing these components (as directed by the compiler wiki):

Description

Main problem: Basic MS builds do not currently work at all on Win10 with currently available VS2019 Build Tools!

The fastest way to see/reproduce this is to try and initialize a distutils.msvccompiler.MSVCCompiler with any clean Win10 environment combination and the a VS2019 C++ Build Tools install (with any variety of combinations, as as indicated in the env section).

Specifically, this code throws a DistutilsPlatformError exception because it can't find Microsoft Visual C++ 14.2:

import distutils.msvccompiler as msvc 
mc = msvc.MSVCCompiler()
mc.initialize()  # <-- BOOM

The specific exception thrown, with message, is:

DistutilsPlatformError: Microsoft Visual C++ 14.2 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

This error seems like it is due to the following things:

1. setuptools is looking for the VC tools in a dotted-version based dir schema, while all available VS build tools install to a year-based dir schema
   • What setup tools expects: C:\Program Files (x86)\Microsoft Visual Studio 14.2\VC\Tools\MSVC
   • What is actually installed: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
2. setuptools really wants to find build tools version 14.2, but no installer is available from MS that gives the expected dir structure
   • even though the latest MSVC 142 - VS 2019 C++ x64/x86 build tools are installed, it does not install them to the version-dotted dir schema that setuptools is expecting

When installing build tools today (with the MSVC142 tools selected), the proper location for executables like cl.exe is in dirs like this:

C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\cl.exe

There are no older MSVC build tools installers available to get a 14.2-based install dir. The URL I am trying to downlad build tools from is https://visualstudio.microsoft.com/vs/older-downloads/, as given by the compiler wiki.

---

Further details:

Stepping through the setuptools.msvc implementation, the following things can be seen:

• setuptools does pick up the VS 2019 build tools dir as follows (in SystemInfo.known_vs_paths):
  • {16.11: 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools'}
  • Note that the version is 16.11, not 14.2
  • Also note that the dir has a date-based schema (with a 2019 dir in the path)
• The version that setuptools is looking for is 14.2
• With no version match, the fallback is to assume a dir structure with dotted versions (instead of the 2019 year)
  • specifically: C:\Program Files (x86)\Microsoft Visual Studio 14.2
  • the code that assumes this is here

---

For search purposes, I ran into this problem when trying to build WxPython with a simple pip install wxpython.

Expected behavior

setuptools should work with the currently available VS2019 build tools from Microsoft.

Alternatively, the compiler wiki (or other docs?) should be corrected to point to whatever location has compatible build tools.

How to Reproduce

As described in the description. Essentially:

1. Get a fresh Win10 install (no Visual Studio or Build tools installed)
2. Install the latest VS2019 C++ build tools from https://visualstudio.microsoft.com/vs/older-downloads/
   • this is the link currently provided by the python compiler wiki
3. Install python 3.10 (or down to 3.8)
4. pip install setuptools -U to get the latest (just in case)
5. Run the code below

import distutils.msvccompiler as msvc 
mc = msvc.MSVCCompiler()
mc.initialize()  # <-- BOOM

Output

In [1]: import distutils.msvccompiler as msvc
   ...: mc = msvc.MSVCCompiler()
   ...: mc.initialize()  # <-- BOOM
---------------------------------------------------------------------------
DistutilsPlatformError                    Traceback (most recent call last)
Input In [1], in <cell line: 3>()
      1 import distutils.msvccompiler as msvc
      2 mc = msvc.MSVCCompiler()
----> 3 mc.initialize()

File C:\venvs\udbs_unit_copy\lib\site-packages\setuptools\_distutils\msvc9compiler.py:371, in MSVCCompiler.initialize(self, plat_name)
    366 else:
    367     # cross compile from win32 -> some 64bit
    368     plat_spec = PLAT_TO_VCVARS[get_platform()] + '_' + \
    369                 PLAT_TO_VCVARS[plat_name]
--> 371 vc_env = query_vcvarsall(VERSION, plat_spec)
    373 self.__paths = vc_env['path'].split(os.pathsep)
    374 os.environ['lib'] = vc_env['lib']

File C:\venvs\udbs_unit_copy\lib\site-packages\setuptools\msvc.py:140, in msvc9_query_vcvarsall(ver, arch, *args, **kwargs)
    138 # If error, try to set environment directly
    139 try:
--> 140     return EnvironmentInfo(arch, ver).return_env()
    141 except distutils.errors.DistutilsPlatformError as exc:
    142     _augment_exception(exc, ver, arch)

File C:\venvs\udbs_unit_copy\lib\site-packages\setuptools\msvc.py:1740, in EnvironmentInfo.return_env(self, exists)
   1724 def return_env(self, exists=True):
   1725     """
   1726     Return environment dict.
   1727
   (...)
   1736         environment
   1737     """
   1738     env = dict(
   1739         include=self._build_paths('include',
-> 1740                                   [self.VCIncludes,
   1741                                    self.OSIncludes,
   1742                                    self.UCRTIncludes,
   1743                                    self.NetFxSDKIncludes],
   1744                                   exists),
   1745         lib=self._build_paths('lib',
   1746                               [self.VCLibraries,
   1747                                self.OSLibraries,
   1748                                self.FxTools,
   1749                                self.UCRTLibraries,
   1750                                self.NetFxSDKLibraries],
   1751                               exists),
   1752         libpath=self._build_paths('libpath',
   1753                                   [self.VCLibraries,
   1754                                    self.FxTools,
   1755                                    self.VCStoreRefs,
   1756                                    self.OSLibpath],
   1757                                   exists),
   1758         path=self._build_paths('path',
   1759                                [self.VCTools,
   1760                                 self.VSTools,
   1761                                 self.VsTDb,
   1762                                 self.SdkTools,
   1763                                 self.SdkSetup,
   1764                                 self.FxTools,
   1765                                 self.MSBuild,
   1766                                 self.HTMLHelpWorkshop,
   1767                                 self.FSharp],
   1768                                exists),
   1769     )
   1770     if self.vs_ver >= 14 and isfile(self.VCRuntimeRedist):
   1771         env['py_vcruntime_redist'] = self.VCRuntimeRedist

File C:\venvs\udbs_unit_copy\lib\site-packages\setuptools\msvc.py:1282, in EnvironmentInfo.VCIncludes(self)
   1272 @property
   1273 def VCIncludes(self):
   1274     """
   1275     Microsoft Visual C++ & Microsoft Foundation Class Includes.
   1276
   (...)
   1280         paths
   1281     """
-> 1282     return [join(self.si.VCInstallDir, 'Include'),
   1283             join(self.si.VCInstallDir, r'ATLMFC\Include')]

File C:\venvs\udbs_unit_copy\lib\site-packages\setuptools\msvc.py:840, in SystemInfo.VCInstallDir(self)
    838 if not isdir(path):
    839     msg = 'Microsoft Visual C++ directory not found'
--> 840     raise distutils.errors.DistutilsPlatformError(msg)
    842 return path

DistutilsPlatformError: Microsoft Visual C++ 14.2 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

Start with setuptools/msvc.py, especially SystemInfo.known_vs_paths and the fallback around line 810, then reproduce the failure with MSVCCompiler.initialize() on a Windows 10 installation. Done means the current VS2019 Build Tools layout is detected and the provided initialization example no longer raises DistutilsPlatformError.