pypa / pypa/setuptools

[FR] When sorting sources, move Message Compiler (`.mc`) files first

Open
#4,986 9 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Needs Triage
Dominant language
Python
Stars
2.9k
Forks
1.4k
Avg merge
1d 1h
Merged PRs (30d)
1

Description

What's the problem this feature will solve?

This is re-opening https://github.com/python/cpython/issues/86175 for setuptools.

If I'm not mistaken, https://github.com/python/cpython/pull/12341 is responsible for sorting sources in distutils to enable reproducible builds.

But since then, pywin32 has had to manually re-sort the sources by doing this:

from distutils import ccompiler
from distutils._msvccompiler import MSVCCompiler

def my_new_compiler(**kw):
    if "compiler" in kw and kw["compiler"] in (None, "msvc"):
        return my_compiler()
    return orig_new_compiler(**kw)

# No way to cleanly wedge our compiler sub-class in.
orig_new_compiler = ccompiler.new_compiler
ccompiler.new_compiler = my_new_compiler

# This import has to be delayed to AFTER the compiler hack
from setuptools.command.build_ext import build_ext  # noqa: E402

class my_compiler(MSVCCompiler):
    # Work around bpo-36302/bpo-42009 - it sorts sources but this breaks
    # support for building .mc files etc :(
    def compile(self, sources, **kwargs) -> list[str]:
        # re-sort the list of source files but ensure all .mc files come first.
        def key_reverse_mc(a):
            b, e = os.path.splitext(a)
            e = "" if e == ".mc" else e
            return (e, b)

        sources = sorted(sources, key=key_reverse_mc)
        return MSVCCompiler.compile(self, sources, **kwargs)

    # [...]

If pywin32 didn't have to do this "compiler patching", I could remove this entire compiler hack and 2 deprecated distutils imports

Describe the solution you'd like

I'd like setuptools (distutils) to sort sources in such a way that Message Compiler (.mc) files come first.

If there's an already existing better way to handle .mc files. Please let me know!

Alternative Solutions

(see the current workaround posted above)

Additional context
Code of Conduct
  • I agree to follow the PSF Code of Conduct

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read pywin32's setup.py workaround and the setuptools deprecated distutils extension-source-files documentation, then trace the source-sorting path discussed in CPython pull request 12341. Compare the existing behavior with the Message Compiler requirements and verify that .mc sources are sorted first without the compiler patching workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.